Programming of parallel systems

Course guarantor:

Lecturer:

Tutor:

Supervisor:

Department of Computer Systems

Synopsis:

Nowadays, multi-core processors and GPU accelerators have become common components of computing clusters and high-performance computing systems, so knowledge and skills related to parallel programming are essential for every computer scientist. The aim of this course is to introduce students to the architectures and programming methods of parallel computers with shared memory, GPU accelerators, or with distributed memory. To effectively use these modern computing systems, it is essential to combine parallelization techniques at all three levels. Students will gain knowledge of the relevant programming models, languages and environments. They will become familiar with fundamental parallel algorithms and be able to analyze the limitations, efficiency, and scalability of parallel solutions to selected problems on high-performance computing systems. In addition to the necessary theory in lectures, students will gain practical experience and skills in programming in OpenMP, CUDA and MPI environments.

Requirements:

Syllabus of lectures:

1. Introduction to parallel systems.

2.OpenMP: Parallel programming for shared memory systems.

3. GPU programming.

4. MPI: Programming for distributed memory systems.

5. Efficiency of parallel algorithms.

6. Parallel reduction and prefix sum.

7. Parallel sorting I.

8. Parallel sorting II.

9. Parallel multiplication of polynomials and matrices.

10. Parallel dense solvers.

11. Parallel matrix-vector multiplication.

12. Parallel sparse solvers.

13. Synchronization and parallel IO.

Syllabus of tutorials:

Not filled yet.

Study Objective:

Nowadays, multi-core processors and GPU accelerators have become common components of computing clusters and high-performance computing systems, so knowledge and skills related to parallel programming are essential for every computer scientist. The aim of this course is to introduce students to the architectures and programming methods of parallel computers with shared memory, GPU accelerators, or with distributed memory. To effectively use these modern computing systems, it is essential to combine parallelization techniques at all three levels. Students will gain knowledge of the relevant programming models, languages and environments. They will become familiar with fundamental parallel algorithms and be able to analyze the limitations, efficiency, and scalability of parallel solutions to selected problems on high-performance computing systems. In addition to the necessary theory in lectures, students will gain practical experience and skills in programming in OpenMP, CUDA and MPI environments.

Study materials:

1. Barlas C.: Multicore and GPU Programming, 2nd Edition

Elsevier 2023

ISBN 978-0-12-814120-5

2. Kumar, V., Grama, A., Gupta, A., Karpis, G.: Introduction to Parallel Computing, 2nd Edition

Adison Wesley 2003

ISBN 0-201-64865-2

3. Mattson, T. G., Sanders, B. A., Massingill, B. L.: Patterns for Parallel Programming

Addison-Wesley Professional 2004

ISBN 978-0321940780

4. Miller, R., Boxer, L.: Algorithms Sequential and Parallel: A Unified Approach, 2nd Edition

Charles River Media 2005

ISBN 978-1-58450-412-2

5. Wilkinson, B., Allen, M.: Parallel Programming: Techniques and Applications Using Networked Workstations and Parallel Computers, 2nd Edition

Pearson 2004

ISBN 0136717101

6. Whu, W. W., Kirk, D. B., Hajj, I. E.: Programming Massively Processors, 4th Edition

Elsevier 2023

ISBN 978-0-323-91231-0

7. Zecevic, A. I.: Quantum and Parallel Computing

Scripta Aesthetica 2022

ISBN 979-8788504056

Note:

Information about the course and teaching materials can be found at https://courses.fit.cvut.cz/QNI-PPS

Further information:

https://courses.fit.cvut.cz/QNI-PPS

No time-table has been prepared for this course

The course is a part of the following study plans:

• Quantum Informatics (compulsory course in the program)