Whether you are using a laptop, a smartphone, or a server in the cloud, your machine likely contains a processor with several “cores”. Such multicore processors consist of many processing units, called cores, that can process instructions in parallel. If you want your program to use this hardware efficiently, you’ll have to learn how to write parallel programs. However, doing so is notoriously difficult: threads that run in parallel can interact in unexpected ways and cause tricky bugs such as race conditions and deadlocks.
In the “Multicore Programming” course, we will study different programming models, techniques, and patterns for parallelism and concurrency. After a general overview of the concepts behind parallelism and concurrency, we study three models in detail. First, the actor model, a technique that uses message passing, which we explore through Erlang. Second, we use Clojure to study models that rely on shared memory, in particular atomic variables and software transactional memory. Third, we will study how to get massive data parallelism by programming on Graphics Processing Units (GPUs) using OpenCL. Finally, we’ll touch upon some other models of concurrency.
In summary, we cover the following topics in the course:
- Introduction: multicore hardware (architectures, memory hierarchies and caches), the laws of concurrent programming (Amdahl’s Law, Gustafson’s Law), speedup and throughput
- Parallelism vs. concurrency, shared memory vs. message passing, task vs. data parallelism
- Traditional shared-memory models: locks, mutual exclusion, race conditions, deadlock, atomic variables
- Novel shared-memory models: software transactional memory (in Clojure)
- Message-passing models: actors (in Erlang), Communicating Sequential Processes and channels (in Clojure)
- Data parallelism: general-purpose programming on a GPU (in OpenCL)
- Benchmarking: approach, pitfalls, statistics, reporting
Contact
This course is taught by Janwillem Swalens. For any questions, contact me at janwillem.swalens@vub.be.
Taught at the Software Languages Lab, Department of Computer Science, Vrije Universiteit Brussel.
Grading
For the theoretical part of this course, students are expected to attend the lectures and are encouraged to read the recommended material. For the practical part, students are expected to complete the exercises.
This section will be updated before the start of the semester.
Prerequisites
When starting this course, you do not need prior knowledge on concurrency or parallelism; after all, that is the topic of this course. We do expect advanced programming skills, as taught in a Bachelor in Computer Science.
Having followed the course “Parallelism and Distribution” in the Bachelor is a definitive plus. Compared to that course, this course will delve deeper into more advanced techniques for dealing with parallelism and concurrency. Distributed programming is not a focus of this course; other electives in the Master focus on that.
In this course, we will code in Erlang and Clojure. These languages are introduced in the lectures and lab sessions, so no prior knowledge is necessary. However, as both of these languages are based on functional programming, having had prior exposure to such languages is a big plus (e.g. Scheme, Haskell, Prolog). For VUB students, having followed the SICP courses (structure and intepretation of computer programs) in the Bachelor and/or “Functional Programming” should be sufficient. For other students, it is recommended to take either “Higher-Order Programming” or “Functional Programming” as an elective, either before this course or at the same time.
In this course we will also learn OpenCL, which is based on C. Prior exposure to C (or C++) is a plus but not required.
If you have any doubts about your prior knowledge, feel free to contact me.
Academic honesty
All projects are individual. You are required to do your own work and will only be evaluated on the part of the work you did yourself. We check for plagiarism. You can check the plagiarism policy for more details.
Acknowledgements
This course was largely developed by Tom Van Cutsem and Jennifer Sartor over the years. I thank them for all their materials, including a position paper on the principles of the course by Tom.