Course details

Parallel Computations on GPU

PCG Acad. year 2024/2025 Winter semester 5 credits

The course covers the architecture and programming of graphics processing units by the NVidia and partially AMD. First, the architecture of GPUs is studied in detail. Then, the model of the program execution using hierarchical thread organisation and the SIMT model is discussed. Next, the memory hierarchy and synchronization techniques are described. After that, the course explains novel techniques of dynamic parallelism and data-flow processing concluded by practical usage of multi-GPU systems in environments with shared (NVLink) and distributed (MPI) memory. The second part of the course is devoted to high level programming techniques and libraries based on the OpenACC technology.

Guarantor

Jaroš Jiří, doc. Ing., Ph.D. (DCSY)

Course coordinator

Vašíček Zdeněk, doc. Ing., Ph.D. (DCSY)

Language of instruction

Czech

Completion

Credit+Examination (written)

Time span

  • 26 hrs lectures
  • 12 hrs pc labs
  • 14 hrs projects

Assessment points

  • 60 pts final exam (written part)
  • 10 pts mid-term test (written part)
  • 30 pts projects

Department

Department of Computer Systems (UPSY)

Lecturer

Bayer David, Ing. (DCSY)
Jaroš Jiří, doc. Ing., Ph.D. (DCSY)

Instructor

Bayer David, Ing. (DCSY)
Jaroš Jiří, doc. Ing., Ph.D. (DCSY)

Learning objectives

To familiarize yourself with the architecture and programming of graphics processing unit in the area of general purpose computuing using the NVidia libraries and OpenACC standard. To learn how to design and implement accelerated programs exploiting the potential of GPUs. To gain knowledge about the available libraries for programming on GPUs.
Knowledge of the parallel programming on GPUs in the area of general purpose computing, orientation in the area of accelerated systems, libraries and tools.  
Understanding of hardware limitations having impact on the efficiency of software solutions. 

Prerequisite knowledge and skills

Knowledge gained in courses AVS and partially in PRL and PPP.

Study literature

  • Current PPT slides for lectures
  • Nvidia CUDA documentation: https://docs.nvidia.com/cuda/
  • OpenACC documentation: https://www.openacc.org/
  • Kirk, D., and Hwu, W.: Programming Massively Parallel Processors: A Hands-on Approach, Elsevier, 2010, s. 256, ISBN: 978-0-12-381472-2. download.
  • Sanders, J., & Kandrot, E: CUDA by Example: An Introduction to General-Purpose GPU Programming. Review Literature And Arts Of The Americas. Addison-Wesley, 2010. download.
  • Storti,D., and Yurtoglu, M.: CUDA for Engineers: An Introduction to High-Performance Parallel Computing, Addison-Wesley Professional; 1 edition, 2015. ISBN 978-0134177410. link.
  • Chandrasekaran, S., and Juckeland, G.: OpenACC for Programmers: Concepts and Strategies,  Addison-Wesley Professional, 2017, ISBN 978-0134694283. link.

Syllabus of lectures

  1. Architecture of graphics processing units.
  2. CUDA programming model, tread execution.
  3. CUDA memory hierarchy.
  4. Synchronization and reduction.
  5. Dynamic parallelism and unified memory.
  6. Design and optimization of GPU algorithms.
  7. Stream processing, computation-communication overlapping.
  8. Multi-GPU systems.
  9. Nvidia Thrust library.
  10. OpenACC basics.
  11. OpenACC memory management.
  12. Code optimization with OpenACC.
  13. Libraries and tools for GPU programming.

Syllabus of computer exercises

  1. CUDA: Memory transfers, simple kernels
  2. CUDA: Shared memory
  3. CUDA: Texture and constant memory
  4. CUDA: Dynamic parallelism and unified memory.
  5. OpenACC: basic techniques.
  6. OpenACC: advanced techniques.

Syllabus - others, projects and individual work of students

  • Development of an application in Nvidia CUDA
  • Development of an application in OpenACC

Progress assessment

Assessment of two projects, 14 hours in total and, computer laboratories and a midterm examination.

  • Missed labs can be substituted in alternative dates.

Schedule

DayTypeWeeksRoomStartEndCapacityLect.grpGroupsInfo
Tue exam 2025-02-04 E104 13:0014:50 3. termín
Tue exam 2025-01-21 E104 15:0016:50 2. termín
Wed lecture 1., 2., 3., 4., 5., 6., 7., 8., 9., 11., 12., 13. of lectures E104 08:0009:5070 1MIT 2MIT NHPC xx Jaroš
Wed lecture 2024-11-20 E104 08:0009:5070 1MIT 2MIT NHPC xx Bayer
Wed exam 2025-01-08 E104 12:0013:50 1. termín
Wed comp.lab 3., 4., 5., 6., 10., 11. of lectures O204 12:0013:5018 1MIT 2MIT xx Bayer
Wed comp.lab 3., 4., 5., 6., 10., 11. of lectures O204 14:0015:5018 1MIT 2MIT xx Bayer

Course inclusion in study plans

Back to top