Whether you are using a laptop, a smartphone, or a server in the cloud, your machine likely contains a processor with several “cores”. Such multicore processors consist of many processing units, called cores, that can process instructions in parallel. If you want your program to use this hardware efficiently, you’ll have to learn how to write parallel programs. However, doing so is notoriously difficult: threads that run in parallel can interact in unexpected ways and cause tricky bugs such as race conditions and deadlocks.
In the “Multicore Programming” course, we will study different programming models, techniques, and patterns for parallelism and concurrency. After a general overview of the concepts behind parallelism and concurrency, we study three models in detail. First, the actor model, a technique that uses message passing, which we explore through Erlang. Second, we use Clojure to study models that rely on shared memory, in particular atomic variables and software transactional memory. Third, we will study how to get massive parallelism by programming on Graphics Processing Units (GPUs) using OpenCL. Finally, we’ll touch upon some other models of concurrency.
In summary, we cover the following topics in the course:
- Introduction: multicore hardware (architectures, memory hierarchies and caches), the laws of concurrent programming (Amdahl’s Law, Gustafson’s Law), speedup and throughput
- Parallelism vs. concurrency, shared memory vs. message passing
- Traditional shared-memory models: locks, mutual exclusion, race conditions, deadlock, atomic variables
- Novel shared-memory models: software transactional memory (in Clojure)
- Message-passing models: actors (in Erlang), Communicating Sequential Processes and channels (in Clojure)
- General-purpose programming on a GPU (in OpenCL)
- Benchmarking: approach, pitfalls, statistics, reporting
About
| Instructor: | Janwillem Swalens (janwillem.swalens@vub.be) |
| Teaching assistant: | Samuel Ngugi (samuel.ngugi@vub.be) |
For general questions and questions about the lectures or projects, contact Janwillem. For questions about the exercises and projects, you can contact Samuel.
Taught at the Software Languages Lab, Department of Computer Science, Vrije Universiteit Brussel.
Grading and assessment
For the theoretical part of this course, students are expected to attend the lectures and are encouraged to read the recommended material. For the exercises, students are expected to attend the lab sessions.
Examination for this course is fully based on three projects:
- The first programming project is in Erlang. The focus is on the use of actors to manage a scalable architecture using message passing.
- The second programming project is in Clojure. The focus is on the use of techniques to manage correct access to shared memory, amongst others transactional memory.
- The third programming project is in OpenCL. The focus is on the efficient use of the GPU to speed up a computation using a large amount of parallelism.
The three projects each account for one third of your final grade. For each project, you will need to design a correct solution, implement it, evaluate it using benchmarks, and write a report. We will publish a detailed assignment for each project on Canvas.
There is no traditional written exam for this course. During the oral exam at the end of the semester, we will discuss your projects. There is no need to prepare a presentation with slides. You will be asked to sketch your design, discuss your experiments and relate your findings to the topics seen during the lectures.
You must submit a solution for all three projects, and participate in the oral examination, in order to pass this course. You need to obtain a minimal score of 8/20 for each of the projects separately. Otherwise, you will receive your lowest score as the end result for this course.
Prerequisites
When starting this course, you do not need prior knowledge on concurrency or parallelism; after all, that is the topic of this course. We do expect advanced programming skills, as taught in a Bachelor in Computer Science.
In the first two parts of this course, we will code in Erlang and Clojure. These languages are introduced in the lectures and lab sessions, so no prior knowledge is necessary. However, as both of these languages are based on a functional programming paradigm, having had prior exposure to a language with first-class functions and higher-order programming is a big plus (e.g. Scheme, Haskell, Prolog). For VUB students, having followed the SICP courses (structure and intepretation of computer programs) in the Bachelor and/or “Functional Programming” should be sufficient. For other students, it is recommended to take either “Higher-Order Programming” or “Functional Programming” as an elective, either before this course or at the same time.
In the third part of this course, we will code in OpenCL. In the lectures, we look at example code in C, but you can program largely in Python for the project. Some prior exposure to these languages is a plus but not necessary.
If you have any doubts about your prior knowledge, feel free to contact me.
Academic honesty
All projects are individual. You are required to do your own work and will only be evaluated on the part of the work you did yourself. We check for plagiarism. More information about our plagiarism policy can be found here.
Acknowledgements
This course was largely developed by Tom Van Cutsem and Jennifer Sartor over the years. I thank them for all their materials, including a position paper on the principles of the course by Tom.