Dept. of Computer Science, Vrije Universiteit Brussel
Instructor: Jennifer B. Sartor
When: | Lectures: Fridays 13:00 - 15:00
(from 16 February until 25 May)
Labs: Fridays 15:00 - 17:00 |
|
Where: | SOFT at VUB lectures: room F.5.206, labs: room E.1.5 | |
Instructor Email: | jsartor@soft.vub.ac.be | |
Teaching Assistants: | Janwillem Swalens (Erlang and Clojure): jswalens@vub.ac.be and Christophe De Troyer (OpenCL for GPU): cdetroye@vub.ac.be |
Processors are no longer growing faster. Instead, they are growing ever more parallel. Hardware architects are no longer using the additional transistors afforded by Moore's Law to make sequential processors faster, but instead to put more processing cores on a single chip, leading to so-called chip multiprocessors, or simply multicore processors. Unfortunately, this creates a problem for us, programmers. Before the turn of the century, you could simply wait another year and your program would run faster on next-generation processors. This free lunch is now officially over: you'll have to learn how to successfully write concurrent and parallel programs if you want your programs' performance to continue to scale with the hardware capacity.
In this course, we will study different paradigms that make parallelism/concurrency tradeoffs. The first is message-passing, which we will explore through Erlang. The second is using software transactional memory on a shared-memory machine, which we will explore using Clojure. We will touch on shared-memory programming using Java's threads. Then we will study how to get massive parallelism by programming on Graphics Processing Units (GPUs) using OpenCL. Here is a preliminary list of topics to be discussed:
Date | Lecture Material | Assignment |
---|---|---|
23 Feb | Introduction, Laws of Parallel Programming, Hardware/Software Models | N/A |
26 Feb at 9:00 in F.4.103 | Erlang: introduction, sequential programming, concurrent programming | N/A |
2 Mar | Erlang: distributed programming, fault tolerance, simple chat, multicore support | Erlang Project due 16 April |
9 Mar | Java Concurrency: Introduction to concurrency in Java | Erlang Project due 16 April |
16 Mar | Clojure: introduction, sequential programming, sequences | Erlang Project due 16 April |
23 Mar in D.3.11 | Clojure: persistent data structures, state and concurrency (refs, atoms, agents) | Erlang Project due 16 April |
23 Mar at 15:00 in D.3.11 | Clojure: software-transactional memory, meta-circular implementation in Clojure | Erlang Project due 16 April |
30 Mar | No lecture, but 2 lab sessions at 13:00 and at 15:00 | Erlang Project due 16 April |
Easter break (31 March to 15 April) | N/A | Erlang Project due 16 April |
20 Apr | Clojure: software-transactional memory, meta-circular implementation in Clojure | Clojure Project due 14 May |
27 April | Intro to GPU Programming with OpenCL | Clojure Project due 14 May |
4 May in D.3.11 | GPU Programming with OpenCL | Clojure Project due 14 May |
4 May at 15:00 in D.3.11 | GPU Programming with OpenCL | Clojure Project due 14 May |
11 May | No lecture, but 2 lab sessions at 13:00 and 15:00 | Clojure Project due 14 May, GPU Project due 11 June |
18 May | Concurrency: comparison of Java against message passing (Actors vs Threads) + channels | GPU Project due 11 June |
25 May | OpenMP and MPI or MapReduce | GPU Project due 11 June |
I don't expect students entering this course to have any prior knowledge on concurrency or parallelism (after all, that is the topic of this course). I do expect advanced programming skills. In particular, I will not be explaining standard sequential programming in Java (i.e. I assume you have basic knowledge of object-oriented programming in a mainstream OO language such as C++/Java/C#).
I will introduce standard sequential programming in Erlang and Clojure, so no prior knowledge of these languages is necessary. However, as both of these languages are primarily functional, having had prior exposure to a language with first-class functions and higher-order programming is a big plus. For VUB students, having followed the SICP courses (structure and intepretation of computer programs) should be sufficient. For ULB students, it is highly recommended to first take this course as an elective.
The main study material used in this course are slides, which can be downloaded from Pointcarre. For each lecture, I also provide relevant pointers to the literature (mostly to articles, books or papers freely available online). Reading material is listed below, which you are encouraged to read.
Slides available on Pointcarre.
Recommended Reading:
Slides available on Pointcarre.
Good, compact overview of the core language:
Also, a good paper on how to collect and measure experimental results correctly, and another interesting one:
Slides available on Pointcarre.
Reading:
Slides available on Pointcarre.
Recommended Reading:
Slides available on Pointcarre.
Recommended Reading:
Slides available on Pointcarre.
Lab exercises are available for the 3 OpenCL labs, with solutions links included.
Useful Links:
Slides available on Pointcarre.
Recommended Reading:
Slides available on Pointcarre.
Recommended Reading:
Slides available on Pointcarre.
Recommended Reading:
For the theoretical part of this course, students are expected to attend the lectures and are encouraged to read the reading material. For the exercises, students are expected to attend the lab sessions.
Examination for this course is fully project-driven:
There is no traditional written exam for this course. During the oral exam (~30 minutes), we will discuss your projects. There is no need to prepare a formal presentation. You will be asked to sketch your design, discuss your experiments and relate your findings to the topics seen during the lectures.
You are required to do your own work. Don't copy code. It's okay to talk about general concepts or algorithms, or even benchmark sets for the projects, but don't share pseudocode or code. You can talk about problems you are having on assignments, but do not show code to classmates to get debugging help. Either use debugging tools or ask the instructor or teaching assistants for more specific help. The best way to do this is to avoid talking to others about the program while you are at the computer. If you have questions on what constitutes cheating or questions about this policy, please talk to the instructor.
This course was largely developed by Tom Van Cutsem over the years. I have made small adaptations to it, but it largely remains the class he developed. I thank him for all his materials, including a position paper over the principles of the course, and the slides of his presentation at the Workshop on Curricula on Concurrency and Parallelism.