Introduction to parallel I/O and distributed file systems

NHR Lecture (Online)

Thursday, 08/04/2022, 9:00 am - 3:00 pm

Speaker: Sebastian Oeste

The lecture gives an introduction to dealing with parallel I/O and distributed file systems in high performance computing. The first part deals with the basics of parallel I/O and working with distributed file systems. The participants learn which hardware and software components are involved in parallel I/O and how they work together. In particular, the differences between working with local and distributed parallel file systems as well as their significance for development and performance are dealt with. In the second part, possibilities are presented how users can analyze the I/O of their application. Furthermore, attention is drawn to typical pitfalls and initial possibilities for I/O optimization are shown.

Agenda

What is I/O?
Local I/O and local file systems
Parallel I/O
Introduction of network and parallel file systems
- Differences and peculiarities to local file systems
- What does this mean for my I/O performance?
- What do I have to consider when developing?
I/O performance analysis
Best practice for parallel I/O

Handouts

The course material (slides) will be made available to the class participants.

HPC-Certification Forum Links

Pre-Knowledge

Safe handling of the Unix command line (bash), USE1.3-B UNIX File System Tree
Read and understand C and Python source code
Good to know: Architecture of computers/clusters, K1.2-B Hardware Architectures

Post-Knowledge

Understanding of I/O patterns in distributed systems
Working with parallel file systems from the user's point of view, K1.3.2.5.1-B MPI-IO

Registration

Link:https://event.zih.tu-dresden.de/nhr/io
Registration is closing on 07/29/2022.
You will receive the access data shortly before the event by email to your registered email address.

Useful Information

Course language: English
Target group: HPC Basics / HPC User / HPC Dev

If you have any further questions, please contact Anja Gerbes ().