2 Project Logistics

In this course you will build an optimizing ahead-of-time compiler from a high-level language with functional and imperative features to x86-64. While a single piece of software, the project is presented through a series of iteration in which you will gradually build and refine your compiler. After each iteration, you will have a complete compiler from some source language to machine code.

The project proceeds bottom-up. We start with the "lowest" level of abstraction, assembly language, and gradually design and implement new layers of abstraction as languages.

Each iteration specifies a set of readings that are necessary to read in order to complete the exercises. Make sure to read and understand them!

2.1 Project Submission

The project is preferably in teams of 2 students or otherwise individual. Please remember that sending code or sharing code between teams is strictly not allowed.

We use GitHub for code management and code submission. You can find the link in the announcements on Canvas. Please create a repository for your team after accepting the invitation to the Github Classrom. At this point Github will automatically create a repository for your team and provision it with template code. Note that it ideally you stay in the same team during the semester. If you do wish to change the team structure, then please talk to the teaching assistant.

Note that the template code is licensed under a restricted CC BY-ND license and per agreeing to this license you are not allowed to share your solutions, either publically or with other students.

2.2 Grading and Feedback

A snapshot of the main branch of your repository will be made on the day and time indicated by the due date. You may create other branches during development, but please make sure that all relevant code ends up in the main branch before the deadline.

We will provide feedback on each of the project iterations in a Github issue that is automatically created when your repository is provisioned. Please do not close this issue.

The main criteria for the grading of the project is correctness, i.e. semantic preserving compilation of each of the passes. In addition, we will grade you on design and code quality. In particular, we will look at the following points:

• Implementation of unit tests. The template adds to each file of a compiler pass a test submodule. See Main and Test Submodules for more details. We do expect you to implement some original tests on your own. Some passes have intricate edge cases an may require more thorough testing. Others are simple traversals in which case some simple tests are sufficient. Use your own judgement! The RackUnit library provides detailed examples on how to write various kinds of tests.

• Traversing or decomposing abstract syntax trees using car and cdr is very tedious and you should avoid that. Instead, use pattern matching to do so. Not everything has to use pattern matching. If a semantic function returns a pair of two things, then of course using car and cdr is absolutely fine.

• Similarly, abstract syntax trees contain a lot of lists internally and many compiler passes need associative datastructures like dictionaries. For this to be economic proper use of higher-order functions like map is vital. The Racket standard library provides a wealth of functionality that you can reuse. Have a look at the documentation on pairs and lists, streams, finite sets, dictionaries, and iteration.

• Compositionality refers to the principle that the meaning of a structure (such as a program or expression) can be understood in terms of the meanings of its constituent parts. Specifically, the meaning of expressions should be definable by the meaning of it’s immediate sub-expressions. This helps in reading, understanding and reasoning about code. In practice, this means that you should follow the Appendix: Compiler Design Recipe when implementing your semantic functions. In particular, you should follow the natural recursion of the given grammar and perform recursive calls on (immediate) sub-expressions only.

2.3 Custom packages

The skeletons require the following Racket packages: cpsc411/compiler-lib and cpsc411/2c-run-time. Here is how to install them:

2.4 Operating System Support

Our run-time system aims to support multiple operating systems, but this has not been tested thoroughly. We therefore assume you are running on a Linux machine. Your compiler must produce code for x64 using Linux system calls. We encourage you to try to support other operating systems by modifying your compiler to generate operating system specific parts based on the host machine, but all assignments must run on x64 linux for the grading. Make sure your code runs with the expected Racket and nasm versions, for instance by running it on the wilma server (see below) before submitting it. The template also contains definitions for Github action that you can enable to let Github build and test your code.

VUB students can access a Linux server named wilma (wilma.vub.ac.be) where racket and nasm are properly installed. You can claim a wilma account here (same credentials than VUB).

Once you have an acount you can:

• ssh into wilma with ssh <vub-net-id>@wilma.vub.ac.be. There you can debug/test your code

• transfer files using the scp protocol

  $ scp username@from_host:file.txt /local/directory/ #remote-->local

  $ scp file.txt username@to_host:/remote/directory/  #local-->remote

  This should be possible on non Linux machines as well, using equivalents of the scp command.

2.5 Organizing Code

The template code contains a subdirectory for each project iteration. In each subdirectory you can find one file per compiler pass, and a main file compiler.rkt that combines all of the passes into a full compilation pipeline. Some of the passes are already fully implemented. You should read the code for these passes, but obviously you do not have to implement or test anything for them.

You should keep the testing code that is specific to a compilation pass in the same file in a test submodule. Additionally, you add tests for the complete compilation pipeline in the compiler.rkt file.

You may create additional files to structure your code, e.g. utility function or testing harness etc. There is already a common.rkt file present that is imported by each compilation pass. Units of Code provides advice on organizing Racket code.

2.6 Tool Support

The language diff tool allows you to view differences between languages that aren’t typeset in the book. You can view both, diffs between different versions of the same language and diffs between source and target language of a compiler pass: https://soft.vub.ac.be/compilers/differ.

The online interrogator at https://soft.vub.ac.be/compilers/interrogator gives you access to a reference solution without exposing the source code. It provides the implementation of each compilation pass that you can either call individually, or you can run the full compilation pipeline. You can use the interrogator, for instance, if you have doubts about the correct transformation or to create correct outputs for unit tests.

For each of the project iterations, except for the first one, you will have to merge your solution of the previous iteration. Easy open source visual tools to do so are KDiff3 and Meld which are both available for Linux, Windows and OS X (via Homebrew).