Logo ČVUT
CZECH TECHNICAL UNIVERSITY IN PRAGUE
STUDY PLANS
2020/2021

Parallel and Distributed Programming

Login to KOS for course enrollment Display time-table
Code Completion Credits Range Language
NI-PDP Z,ZK 6 2P+2C Czech
Lecturer:
Pavel Tvrdík (guarantor)
Tutor:
Pavel Tvrdík (guarantor), Ivan Šimeček, Michal Šoch
Supervisor:
Department of Computer Systems
Synopsis:

21st century in computer architectures is primarily influenced by the shift of the Moore's law into parallelization of CPUs at the level of computing cores. Parallel computing systems are becoming a ubiquitous commodity and parallel programming becomes the basic paradigm of development of efficient applications for these platforms. Students get acquainted with architectures of parallel and distributed computing systems, their models, theory of interconnection networks and collective communication operations, and languages and environments for parallel programming of shared and distributed memory computers. They get acquianted with fundamental parallel algorithms and on selected problems, they will learn the techniques of design of efficient and scalable parallel algorithms and methods of performance evaluation of their implementations. The course includes a semester project of practical programming in OpenMP and MPI for solving a particular nontrivial problem.

Requirements:

Basic sequential algorithmics, programming, the C/C++programming language, complexity theory, computer architecture, graph theory.

Syllabus of lectures:

1. Introduction into parallel and distributed programming.

2. Introduction into OpenMP.

3. Parallel algorithms for the state space search.

4. Prpgramming and performance tuning in OpenMP.

5. Parallel sorting in OpenMP.

6. Introduction into MPI.

7. Interconnection networks of parallel computers I.

8. Interconnection networks of parallel computers II.

9. Collective communication operations.

10. Parallel reduction and parallel scan. Parallel I/O.

11. Parallel algorithms in OpenMP/MPI I.

12. Parallel algorithms in OpenMP/MPI II.

Syllabus of tutorials:

1. Design and implementation of a sequential algorithm in C/C++.

2. Design and implementation of a parallel algorithm using OpenMP task parallel constructs.

3. Design and implementation of a parallel algorithm using OpenMP data parallel constructs.

4. Design and implementation of a parallel algorithm using MPI on a cluster.

5. Analysis of parallel performance and scalability of the resulting program and writing a technical report.

Study Objective:

21st century in computer architectures is primarily influenced by the shift of the Moore's law into parallelization of CPUs at the level of computing cores. Parallel computing systems are becoming a ubiquitous commodity and parallel programming becomes the basic paradigm of development of efficient applications for these platforms. Students get acquainted with architectures of parallel and distributed computing systems, their models, theory of interconnection networks and collective communication operations, and languages and environments for parallel programming of shared and distributed memory computers. They get acquianted with fundamental parallel algorithms and on selected problems, they will learn the techniques of design of efficient and scalable parallel algorithms and methods of performance evaluation of their implementations. The course includes a semester project of practical programming in OpenMP and MPI for solving a particular nontrivial problem.

Study materials:

[1] Mattson, T.G. - Sanders, B.A. - Massingill, B.L.: Patterns for Parallel Programming. Addison-Wesley Professional. 2004. 978-0321940780.

[2] Kumar, V. - Grama, A. - Gupta, A. - Karpis, G.: Introduction to Parallel Computing: Design and Analysis of Parallel Algorithms. Benjamin-Cummings. 1994. 0805331700.

[3] Miller, R. - Boxer, L.: Algorithms Sequential and Parallel: A Unified Approach. Pearson Education. 1999. 0130863734.

[4] Wilkinson, B. - Allen, M.: Parallel Programming: Techniques and Applications Using Networked Workstations and Parallel Computers. Prentice Hall. 1998. 0136717101.

[5] Jaja, J.: An Introduction to Parallel Algorithms. Addison-Wesley. 1992. 0201548569.

Note:
Further information:
https://courses.fit.cvut.cz/MI-PDP/
Time-table for winter semester 2020/2021:
Time-table is not available yet
Time-table for summer semester 2020/2021:
06:00–08:0008:00–10:0010:00–12:0012:00–14:0014:00–16:0016:00–18:0018:00–20:0020:00–22:0022:00–24:00
Mon
roomTH:A-942
Tvrdík P.
14:30–16:00
(lecture parallel1)
Thákurova 7 (FSv-budova A)
roomTK:BS
Tvrdík P.
14:30–16:00
(lecture parallel1)
Dejvice
NTK Ballingův sál
Tue
Fri
roomT9:350
Šimeček I.
09:15–10:45
(lecture parallel1
parallel nr.101)

Dejvice
NBFIT PC ucebna
roomT9:350
Šimeček I.
11:00–12:30
(lecture parallel1
parallel nr.102)

Dejvice
NBFIT PC ucebna
roomT9:350
Šimeček I.
12:45–14:15
(lecture parallel1
parallel nr.103)

Dejvice
NBFIT PC ucebna
roomT9:350
Šoch M.
16:15–17:45
(lecture parallel1
parallel nr.104)

Dejvice
NBFIT PC ucebna
roomT9:350
Šoch M.
18:00–19:30
(lecture parallel1
parallel nr.105)

Dejvice
NBFIT PC ucebna
Thu
roomT9:351
Šoch M.
07:30–09:00
(lecture parallel1
parallel nr.106)

Dejvice
NBFIT PC ucebna
roomT9:351
Šoch M.
09:15–10:45
(lecture parallel1
parallel nr.107)

Dejvice
NBFIT PC ucebna
roomT9:351
Šoch M.
11:00–12:30
(lecture parallel1
parallel nr.108)

Dejvice
NBFIT PC ucebna
roomT9:351
Šoch M.
14:30–16:00
(lecture parallel1
parallel nr.109)

Dejvice
NBFIT PC ucebna
roomT9:351
Šoch M.
16:15–17:45
(lecture parallel1
parallel nr.110)

Dejvice
NBFIT PC ucebna
roomT9:351
Šoch M.
18:00–19:30
(lecture parallel1
parallel nr.111)

Dejvice
NBFIT PC ucebna
Fri
The course is a part of the following study plans:
Data valid to 2021-04-16
For updated information see http://bilakniha.cvut.cz/en/predmet6099506.html