CPD 2014-2015

Parallel and Distributed Computing

What. The primary objective of the course is to introduce students to te concepts of parallel and distributed computing.

Why. Ever heard of cloud computing, or map-reduce? Well, have you wondered how things are really under the hood? That’s why.

How. Distributed and parallel algorithms are harder to design, analyze, and implement with respect to their serial counterparts. Let’s be honest: there is no silver bullet. Not everything can be a map and a reduce, not everything can (or should) be parallelized. In this course we will learn how to think about parallelization, showing limits, problems, and hands-on solutions.

Who. These algorithms play central role in modern computing, e.g., cloud computing, and applied in several fields, from bioinformatics to web services, from structural engineering to entertainment. Needless to say: almost every aspect of modern computing will be distributed.

Class hours

  • Tuesday & Friday, 11.30am–1.00pm, room DS3A.


Lectures will be introducing the following topics.

  • Principles of Parallel and Distributed Computing
  • Synchronization and Time
  • Distributed Security
  • Vectors, Matrices, and their Operations
  • Distributed Graphs and Algorithms
  • Distributed File Systems
  • Modern and safe parallel programming

The programming technologies that will be introduced in this course will be the following:

  • MPI: Message Passing Interface
  • OpenMP: Open Multi Processing
  • CUDA: NVidia’s Parallel Computing
  • CBE: The Cell Broadband Engine
  • OpenCL: Open Computing Language

Laboratory Sessions

This class will require laboratory sessions (25% circa). Bring your laptop!


  • D. P. Bertsekas and John N. Tsitsiklis, “Parallel and Distributed Computation: Numerical Methods”, Athena Scientific, 1997. (free download)
  • A. Grama, A. Gupta, G. Karypis, and V. Kumar, ”Introduction to Parallel Computing”, Addison Wesley (2nd edition), 2003.


Students will be assigned a group project and will present their results. They are expected to use a git repository to store all of their material (source code, a paper, and presentation notes), and contribute to the group project equally: be honest when presenting.