Parallel and Distributed Programming

Code	Completion	Credits	Range	Language
MI-PDP.16	Z,ZK	5	2P+2C	Czech

Lecturer:

Pavel Tvrdík (guarantor)

Tutor:

Pavel Tvrdík (guarantor), Ivan Šimeček, Michal Šoch

Supervisor:

Department of Computer Systems

Synopsis:

Due to the development of cloud, web, and communication technologies and due to the shift of the Moore law into parallelization of CPUs, parallel and distributed applications are becoming dominant. Students get acquainted with architectures of parallel and distributed computing systems and their models and with languages and environments for their programming. They learn the pattern designs for parallel and distributed programming and important parallel algorithms.

Requirements:

Basic sequential algorithmics, programming, the C/C++programming language, complexity theory, computer architecture and graph theory.

Syllabus of lectures:

1. Introduction into parallel and distributed programming

2. Introduction into OpenMP

3. Parallel algorithms for the state space search

4. Performance tuning in OpenMP

5. Parallel algorithms in OpenMP

6. Parallel sorting in OpenMP

7. Introduction into MPI

8. Parallel algorithms in MPI I

9. Distributed parallel architecture and interconnection networks.

10. Communication algorithms

11. Parallel algorithms in OpenMP/MPI II

12. MapReduce frameworks (principles, Hadoop, Spark,..)

Syllabus of tutorials:

1. Sequential algorithm implementation in C/C++.

2. Parallel algorithm implementation using OpenMP task parallel constructs.

3. Parallel algorithm implementation using OpenMP data parallel constructs.

4. Parallel algorithm implementation using MPI on cluster STAR.

5. Measurements of the parallel implementation performance

Study Objective:

Study materials:

[1] Mattson, T.G. - Sanders, B.A. - Massingill, B.L.: Patterns for Parallel Programming. Addison-Wesley Professional. 2004. 978-0321940780.

[2] Kumar, V. - Grama, A. - Gupta, A. - Karpis, G.: Introduction to Parallel Computing: Design and Analysis of Parallel Algorithms. Benjamin-Cummings. 1994. 0805331700.

[3] Miller, R. - Boxer, L.: Algorithms Sequential and Parallel: A Unified Approach. Pearson Education. 1999. 0130863734.

[4] Wilkinson, B. - Allen, M.: Parallel Programming: Techniques and Applications Using Networked Workstations and Parallel Computers. Prentice Hall. 1998. 0136717101.

[5] Jaja, J.: An Introduction to Parallel Algorithms. Addison-Wesley. 1992. 0201548569.

Note:

Further information:

https://courses.fit.cvut.cz/MI-PDP/

Time-table for winter semester 2019/2020:

Time-table is not available yet

Time-table for summer semester 2019/2020:

	06:00–08:0008:00–10:0010:00–12:0012:00–14:0014:00–16:0016:00–18:0018:00–20:0020:00–22:0022:00–24:00
Mon	roomTK:BS Tvrdík P. 14:30–16:00 (lecture parallel1) Dejvice NTK Ballingův sál
Tue	roomT9:351 Šoch M. 09:15–10:45 EVEN WEEK (lecture parallel1 parallel nr.101) Dejvice NBFIT PC ucebnaroomT9:351 Šoch M. 11:00–12:30 EVEN WEEK (lecture parallel1 parallel nr.101) Dejvice NBFIT PC ucebnaroomT9:351 Šoch M. 12:45–14:15 EVEN WEEK (lecture parallel1 parallel nr.101) Dejvice NBFIT PC ucebnaroomT9:351 Šoch M. 14:30–16:00 EVEN WEEK (lecture parallel1 parallel nr.101) Dejvice NBFIT PC ucebnaroomT9:351 Šoch M. 16:15–17:45 ODD WEEK (lecture parallel1 parallel nr.101) Dejvice NBFIT PC ucebna roomT9:351 Šoch M. 11:00–12:30 ODD WEEK (lecture parallel1 parallel nr.101) Dejvice NBFIT PC ucebnaroomT9:351 Šoch M. 12:45–14:15 ODD WEEK (lecture parallel1 parallel nr.101) Dejvice NBFIT PC ucebnaroomT9:351 Šoch M. 14:30–16:00 ODD WEEK (lecture parallel1 parallel nr.101) Dejvice NBFIT PC ucebna
Fri	roomT9:350 Šimeček I. 09:15–10:45 EVEN WEEK (lecture parallel1 parallel nr.101) Dejvice NBFIT PC ucebnaroomT9:350 Šimeček I. 11:00–12:30 EVEN WEEK (lecture parallel1 parallel nr.101) Dejvice NBFIT PC ucebnaroomT9:350 Šimeček I. 12:45–14:15 EVEN WEEK (lecture parallel1 parallel nr.101) Dejvice NBFIT PC ucebnaroomT9:350 Šimeček I. 14:30–16:00 EVEN WEEK (lecture parallel1 parallel nr.101) Dejvice NBFIT PC ucebna roomT9:350 Šoch M. 09:15–10:45 ODD WEEK (lecture parallel1 parallel nr.101) Dejvice NBFIT PC ucebnaroomT9:350 Šoch M. 11:00–12:30 ODD WEEK (lecture parallel1 parallel nr.101) Dejvice NBFIT PC ucebnaroomT9:350 Šoch M. 12:45–14:15 ODD WEEK (lecture parallel1 parallel nr.101) Dejvice NBFIT PC ucebnaroomT9:350 Šoch M. 14:30–16:00 ODD WEEK (lecture parallel1 parallel nr.101) Dejvice NBFIT PC ucebna
Thu	roomT9:155 Tvrdík P. 14:30–16:00 ODD WEEK (lecture parallel1 parallel nr.101) Dejvice Posluchárna
Fri

The course is a part of the following study plans:

Knowledge Engineering, in Czech, Presented in Czech, Version 2016 and and 2017 (compulsory course in the program)
Computer Security, Presented in Czech, Version 2016 to 2019 (compulsory course in the program)
Computer Systems and Networks, Presented in Czech, Version 2016 to 2019 (compulsory course in the program)
Design and Programming of Embedded Systems, in Czech, Version 2016 to 2019 (compulsory course in the program)
Specialization Web and Software Engineering, in Czech, Version 2016 to 2019 (compulsory course in the program)
Specialization Software Engineering, in Czech, Version 2016 to 2019 (compulsory course in the program)
Specialization Web Engineering, Presented in Czech, Version 2016 to 2019 (compulsory course in the program)
Master Informatics, Presented in Czech, Version 2016 to 2019 (compulsory course in the program)
Specialization System Programming, Presented in Czech, Version 2016 to 2019 (compulsory course in the program)
Specialization Computer Science, Presented in Czech, Version 2016-2017 (compulsory course in the program)
Specialization Computer Science, Presented in Czech, Version 2018 to 2019 (compulsory course in the program)
Knowledge Engineering, in Czech, Presented in Czech, Version 2018 to 2019 (compulsory course in the program)