3 ECTS credits
75 h study time

Offer 1 with catalog number 4022631DNR for all students in the 1st semester at a (D) Master - preliminary level.

Semester
1st semester
Enrollment based on exam contract
Impossible
Grading method
Grading (scale from 0 to 20)
Can retake in second session
Yes
Taught in
English
Faculty
Faculteit Ingenieurswetenschappen
Department
Industriƫle ingenieurswetenschappen
Educational team
Jan Lemeire (course titular)
Activities and contact hours
12 contact hours Lecture
18 contact hours Seminar, Exercises or Practicals
Course Content

This course is a low-entry introductory course into parallel computing with the focus on vectorization (modern CPUs), multithreading (shared-memory systems) and message-passing (distributed memory systems).

The course consists of a theoretical and a practical part.
The theoretical part consists of 5 chapters.
The practical part consists of practical sessions and a programming project. During the practical sessions, the constructs for vectorization, multithreading and message-passing in C++ are explored. The programming project happens in groups of 2 to 3 persons.
All information can be found on the website http://parallel.vub.ac.be/education/ppp

The design of simple, generic and performant parallel programs relies on insight in all aspects of modern hardware and software technology.
The course therefore offers a wider scope than just dealing with the 'parallel issues'.

1. Overview of Parallel Systems
Parallel systems constitute of processors that are connected in some way. They are more present than ever. Parallel processing is the only answer to the ever-increasing demand for more computational power. This chapter presents an overview and explains why parallel resources are not for free.

2. Code optimization and vectorization
We explore how sequential code is optimized by the compiler and by the programmer (loop unrolling, inlining, …). Furthermore, modern CPUs have vector units allowing SIMD processing through vectorization.

3. Shared-memory paradigm: Multithreading
Multicores are becoming the standard in PCs. They allow for multithreaded parallel programs, which are based on the shared-memory paradigm. In this chapter all aspects are discussed that should be taken into consideration when creating multi-threaded applications. The following programming constructs are explained: Pthreads, Java, Python and C++ 11.

4. Message-passing paradigm: MPI
Processors that do not have access to each other's memory communicate by sending messages to each other. This is called the message-passing paradigm, of which MPI is the leading standard. It is discussed how MPI should be used and MPI is compared to more special parallel languages.

5. Performance Analysis of Parallel Processing
The main goal of parallel processing is shortening the runtime of programs. This chapter presents the performance analysis or parallel programs.

Additional info

Study material:

Slides of lectures.

Documentation and practical information: on the website http://parallel.vub.ac.be/teaching/ppp

'Introduction to Parallel Programming'

A. Grama et al.

ISBN 0-201-64865-2

'Principles of Parallel Programming'

Calvin Lin and Larry Snyder

ISBN 0321487907

 

Prerequisites:

Basic knowledge of computer systems (processor and network), some programming experience, elementary knowledge of algorithms.

 

Additional questions can be directed to jan.lemeire@vub.be

Learning Outcomes

General learning outcomes

The student will understand the architecture of the various modern parallel systems (except for GPUs which is considered in the course ‘GPU Computing’). He will be able to conceive, implement and analyze parallel and distributed algorithms. He will understand why the utilization of parallel resources is not for free, that parallel processing out-of-the-box does not exist. The student will be able to utilize modern parallel technology autonomously, in the design of efficient parallel solutions for all kinds of sequential algorithms.

He will be able to apply his understanding of low-level processes and hardware characteristics to devise high-performant, scalable solutions.
The student will have learned to look for twofold-generic parallel solutions. On the one hand, the solution should be appropriate to a wide variety of parallel systems, such as a dedicated cluster, a GRID environment, a multiprocessor. On the other hand, it should be easy-to-use and efficient for a broad class of similar algorithms. The student will understand the trade-off between genericity and performance. This integration of high-level software issues and low-level hardware characteristics will have sharpened his general IT-skills.

By the practicum, the student will have demonstrated that he can make good judgments about complex situations and communicate his conclusions. The student will be able to participate to discussions about exploiting parallelism and the proper use of modern technology.

Grading

The final grade is composed based on the following categories:
Oral Exam determines 40% of the final mark.
PRAC Practical Assignment determines 60% of the final mark.

Within the Oral Exam category, the following assignments need to be completed:

  • Mond. ex. theory with a relative weight of 40 which comprises 40% of the final mark.

    Note: Oral exam for the theoretical part.

Within the PRAC Practical Assignment category, the following assignments need to be completed:

  • Evaluation Project with a relative weight of 40 which comprises 40% of the final mark.

    Note: Evaluation of the project for the practical part.
  • Mini-project optimization with a relative weight of 20 which comprises 20% of the final mark.

    Note: Evaluation of the mini-project on optimization of a simple program

Additional info regarding evaluation

Oral exam for the theoretical part (40%).

Evaluation of the mini-project on optimization of a simple program (20%).

Evaluation of the project for the practical part (40%).

 

Allowed unsatisfactory mark
The supplementary Teaching and Examination Regulations of your faculty stipulate whether an allowed unsatisfactory mark for this programme unit is permitted.

Academic context

This offer is part of the following study plans:
Master in Applied Sciences and Engineering: Applied Computer Science: Standaard traject