3 ECTS credits
75 u studietijd
Aanbieding 1 met studiegidsnummer 4022631DNR voor alle studenten in het 1e semester met een inleidend master niveau.
This course is a low-entry introductory course into parallel computing with the focus on vectorization (modern CPUs), multithreading (shared-memory systems) and message-passing (distributed memory systems).
The course consists of a theoretical and a practical part.
The theoretical part consists of 5 chapters.
The practical part consists of practical sessions and a programming project. During the practical sessions, the constructs for vectorization, multithreading and message-passing in C++ are explored. The programming project happens in groups of 2 to 3 persons.
All information can be found on the website http://parallel.vub.ac.be/education/ppp
The design of simple, generic and performant parallel programs relies on insight in all aspects of modern hardware and software technology.
The course therefore offers a wider scope than just dealing with the 'parallel issues'.
1. Overview of Parallel Systems
Parallel systems constitute of processors that are connected in some way. They are more present than ever. Parallel processing is the only answer to the ever-increasing demand for more computational power. This chapter presents an overview and explains why parallel resources are not for free.
2. Code optimization and vectorization
We explore how sequential code is optimized by the compiler and by the programmer (loop unrolling, inlining, …). Furthermore, modern CPUs have vector units allowing SIMD processing through vectorization.
3. Shared-memory paradigm: Multithreading
Multicores are becoming the standard in PCs. They allow for multithreaded parallel programs, which are based on the shared-memory paradigm. In this chapter all aspects are discussed that should be taken into consideration when creating multi-threaded applications. The following programming constructs are explained: Pthreads, Java, Python and C++ 11.
4. Message-passing paradigm: MPI
Processors that do not have access to each other's memory communicate by sending messages to each other. This is called the message-passing paradigm, of which MPI is the leading standard. It is discussed how MPI should be used and MPI is compared to more special parallel languages.
5. Performance Analysis of Parallel Processing
The main goal of parallel processing is shortening the runtime of programs. This chapter presents the performance analysis or parallel programs.
Study material:
Slides of lectures.
Documentation and practical information: on the website http://parallel.vub.ac.be/teaching/ppp
'Introduction to Parallel Programming'
A. Grama et al.
ISBN 0-201-64865-2
'Principles of Parallel Programming'
Calvin Lin and Larry Snyder
ISBN 0321487907
Prerequisites:
Basic knowledge of computer systems (processor and network), some programming experience, elementary knowledge of algorithms.
Additional questions can be directed to jan.lemeire@vub.be
The student will understand the architecture of the various modern parallel systems (except for GPUs which is considered in the course ‘GPU Computing’). He will be able to conceive, implement and analyze parallel and distributed algorithms. He will understand why the utilization of parallel resources is not for free, that parallel processing out-of-the-box does not exist. The student will be able to utilize modern parallel technology autonomously, in the design of efficient parallel solutions for all kinds of sequential algorithms.
He will be able to apply his understanding of low-level processes and hardware characteristics to devise high-performant, scalable solutions.
The student will have learned to look for twofold-generic parallel solutions. On the one hand, the solution should be appropriate to a wide variety of parallel systems, such as a dedicated cluster, a GRID environment, a multiprocessor. On the other hand, it should be easy-to-use and efficient for a broad class of similar algorithms. The student will understand the trade-off between genericity and performance. This integration of high-level software issues and low-level hardware characteristics will have sharpened his general IT-skills.
By the practicum, the student will have demonstrated that he can make good judgments about complex situations and communicate his conclusions. The student will be able to participate to discussions about exploiting parallelism and the proper use of modern technology.
De beoordeling bestaat uit volgende opdrachtcategorieën:
Examen Mondeling bepaalt 40% van het eindcijfer
WPO Praktijkopdracht bepaalt 60% van het eindcijfer
Binnen de categorie Examen Mondeling dient men volgende opdrachten af te werken:
Binnen de categorie WPO Praktijkopdracht dient men volgende opdrachten af te werken:
Oral exam for the theoretical part (40%).
Evaluation of the mini-project on optimization of a simple program (20%).
Evaluation of the project for the practical part (40%).
Deze aanbieding maakt deel uit van de volgende studieplannen:
Master of Applied Sciences and Engineering: Applied Computer Science: Standaard traject (enkel aangeboden in het Engels)