Introduction to parallel I/O and distributed file systems
NHR Lecture (Online)
Wednesday, November 6th 2024, 1:00 pm - 5:00 pm
Speaker: Sebastian Oeste
The lecture gives an introduction to dealing with parallel I/O and distributed file systems in high performance computing. The first part deals with the basics of parallel I/O and working with distributed file systems. The participants learn which hardware and software components are involved in parallel I/O and how they work together. In particular, the differences between working with local and distributed parallel file systems as well as their significance for development and performance are dealt with. In the second part, possibilities are presented how users can analyze the I/O of their application. Furthermore, attention is drawn to typical pitfalls and initial possibilities for I/O optimization are shown.
Agenda
- What is I/O?
- Local I/O and local file systems
- Parallel I/O
- Introduction of network and parallel file systems
- Differences and peculiarities to local file systems
- What does this mean for my I/O performance?
- What do I have to consider when developing?
- I/O performance analysis
- Best practice for parallel I/O
Handouts
The course material (slides) will be made available to the class participants.
HPC-Certification Forum Links
Pre-Knowledge
- Safe handling of the Unix command line (bash), USE1.3-B UNIX File System Tree
- Read and understand C and Python source code
- Good to know: Architecture of computers/clusters, K1.2-B Hardware Architectures
Post-Knowledge
- Understanding of I/O patterns in distributed systems
- Working with parallel file systems from the user's point of view, K1.3.2.5.1-B MPI-IO
Registration
Link: https://events.gwdg.de/event/898/
You will receive the access data shortly before the event by email to your registered email address.
Useful Information
Course language: English
Target group: HPC Basics / HPC User / HPC Dev
If you have any further questions, please contact Anja Gerbes ().