Whether you are using a laptop, a smartphone, or a server in the cloud, your machine likely contains a processor with several “cores”. Such multicore processors consist of many processing units, called cores, that can process instructions in parallel. If you want your program to use this hardware efficiently, you’ll have to learn how to write parallel programs. However, doing so is notoriously difficult: threads that run in parallel can interact in unexpected ways and cause tricky bugs such as race conditions and deadlocks.
In the “Multicore Programming” course, we will study different programming models, techniques, and patterns for parallelism and concurrency. After a general overview of the concepts behind parallelism and concurrency, we study three models in detail. First, the actor model, a technique that uses message passing, which we explore through Erlang. Second, we use Clojure to study models that rely on shared memory, in particular atomic variables and software transactional memory. Third, we will study how to get massive data parallelism by programming on Graphics Processing Units (GPUs) using OpenCL. Finally, we’ll touch upon some other models of concurrency.
In summary, we cover the following topics in the course:
- Introduction: multicore hardware (architectures, memory hierarchies and caches), the laws of concurrent programming (Amdahl’s Law, Gustafson’s Law), speedup and throughput
- Parallelism vs. concurrency, shared memory vs. message passing, task vs. data parallelism
- Traditional shared-memory models: locks, mutual exclusion, race conditions, deadlock, atomic variables
- Novel shared-memory models: software transactional memory (in Clojure)
- Message-passing models: actors (in Erlang), Communicating Sequential Processes and channels (in Clojure)
- Data parallelism: general-purpose programming on a GPU (in OpenCL)
- Benchmarking: approach, pitfalls, statistics, reporting
Contact
This course is taught by Janwillem Swalens. For any questions, contact me at janwillem.swalens@vub.be.
Taught at the Software Languages Lab, Department of Computer Science, Vrije Universiteit Brussel.
Grading
For the theoretical part of this course, students are expected to attend the lectures and are encouraged to read the recommended material. For the practical part, students are expected to complete the exercises.
Examination for this course is fully based on three projects:
- The first programming project is in Erlang. The focus is on the use of actors to manage a scalable architecture using message passing.
- The second programming project is in OpenCL. The focus is on the efficient use of the GPU to speed up a computation using a large amount of parallelism.
- The third programming project is in Clojure. The focus is on the use of techniques to manage correct access to shared memory, amongst others transactional memory.
The three projects each account for one third of your final grade. For each project, you will need to design a correct solution, implement it, evaluate it using benchmarks, and write a report. We will publish a detailed assignment for each project on Canvas.
There is no traditional written exam for this course. During the oral exam at the end of the semester, we will discuss your projects. There is no need to prepare a presentation with slides. You will be asked to sketch your design, discuss your experiments and relate your findings to the topics seen during the lectures.
You must submit a solution for all three projects, and participate in the oral examination, in order to pass this course. You need to obtain a minimal score of 8/20 for each of the projects separately. Otherwise, you will receive your lowest score as the end result for this course.
Prerequisites
When starting this course, you do not need prior knowledge on concurrency or parallelism; after all, that is the topic of this course. We do expect advanced programming skills, as taught in a Bachelor in Computer Science.
Having followed the course “Parallelism and Distribution” in the Bachelor is a definitive plus. Compared to that course, this course will delve deeper into more advanced techniques for dealing with parallelism and concurrency. Distributed programming is not a focus of this course; other electives in the Master focus on that.
In this course, we will code in Erlang and Clojure. These languages are introduced in the lectures and lab sessions, so no prior knowledge is necessary. However, as both of these languages are based on functional programming, having had prior exposure to such languages is a big plus (e.g. Scheme, Haskell, Prolog). For VUB students, having followed the SICP courses (structure and intepretation of computer programs) in the Bachelor and/or “Functional Programming” should be sufficient. For other students, it is recommended to take either “Higher-Order Programming” or “Functional Programming” as an elective, either before this course or at the same time.
In this course we will also learn OpenCL, which is based on C. Prior exposure to C (or C++) is a plus but not required.
If you have any doubts about your prior knowledge, feel free to contact me.
Academic honesty
All projects are individual. You are required to do your own work and will only be evaluated on the part of the work you did yourself. We check for plagiarism. You can check our plagiarism policy for more details.
Acknowledgements
This course was largely developed by Tom Van Cutsem and Jennifer Sartor over the years. I thank them for all their materials, including a position paper on the principles of the course by Tom.