Project Application for using the HPC Systems

For high-performance computing (HPC), the CIDS/ZIH, as a center of the National High-Performance Computing (NHR) initiative, operates various HPC-Systems that are available at both the national level (Tier 2) and the regional level (Tier 3).

Prerequisites

A prerequisite for using the high-performance computers is an HPC project proposal. Due to the two levels mentioned above, there are generally two different application tracks, each of which includes various project categories. Which one is most suitable depends on the following factors:

• Quantitative Demand for computing resources

• Affiliation

• Intended use

• Initial or follow-up proposal

The application process, which is carried out via an online form, is essentially the same for all project categories; however, its content differs in terms of focus and extent, as a separate document may be required for the detailed project description.

There are two key roles: that of the applicant (Person of Contact, PC) and that of the principal investigator (PI), which are further detailed in the HPC Terms of Use. Only researchers with a Ph.D. from German universities are eligible to serve as Principal Investigators (PI). This must be indicated by the provided contact information (email address) and electronic signature. Applications from external collaboration partners in joint appointments must accordingly feature a member of a higher education institution as the PI.

Typically, a computing time project is approved for one year following review; however, depending on the funding period of the research project, a shorter or, if necessary, longer duration may be selected.

National High-Performance Computing (NHR, Tier 2)

Computing resources at the national level (Tier 2) are allocated through a standardized process within the framework of the National High-Performance Computing (NHR) initiative. All relevant information—including the procedure, requirements, project categories, deadlines, computing time estimates, required details, templates for the project description (including an example), and access to the online application—is provided by NHR on the following website:

nhr-verein.de/access-support/hpc-access/

Tier 3

TU Dresden also offers computing resources at the state level (Tier 3) to applicants from TU Dresden and members of higher education institutions in Saxony whose institutions do not have their own computing resources. Computing resources at this level are also allocated for TU Dresden teaching courses, including both courses spanning across the semester and block courses.
Please apply for resources at this level via the

Tier 3 Application Process 

Institution-Owned HPC Resources 

If your institution owns its own hardware resources or has funded resources operated at ZIH, please use the Tier 3 application process (see above) and indicate your institution and hardware claims accordingly. An exception to this applies to ScaDS.AI-affiliated applicants, who should use the separate ScaDS application process.

Project Categories Overview

For a detailed description of the NHR project categories, see the NHR website.

Getting Started with HPC

For applicants without HPC experience or for proposals where the required computing time cannot yet be estimated, there are three options for submitting a proposal:

• Starter Project (once per PI; see above NHR)
• Test Project (earmarked for this purpose to facilitate a better estimate; see above NHR)
• Tier 3 Project (with low resource requirements and corresponding designation (see above Tier 3)

No scientific review is required for these, which simplifies the application process.

New Compute Project

For each research project, an accompanying HPC project in the Tier 3, NHR Normal, or NHR Large category may be applied for (initial application). The choice depends on the amount of resources required (see project categories), which must be estimated in advance and explained in the application. To make this estimate, the following metrics must be derived either from tests on the applicant's own hardware or, with the help of an NHR test project, from project statistics and recorded job statistics:

Key Metrics

• Number of (effectively) used CPU cores or GPUs (depending on problem size)
• CPU: Computation time required per job, iteration, or simulated time
• GPU: Size of training data, number of training steps, training throughput in tokens/s, samples/s, time per epoch or step (depending on batch size)
  ⇒ Computation time per GPU job

Compute Time Estimate

CPU [CPU-core h]: (number of CPU-cores) × (runtime of job in hours) × (number of calculations)
GPU [GPU-h]: (number of GPUs) × (runtime of job in hours) × (number of calculations)

Examples

Low-Res-Sim: 50 simulations × 48 CPU-cores × 1,5 h/sim = 3,600 CPU core-h
High-Res-Sim: 5 simulations × 104 CPU-cores × 40 h sim-time × 3 (h per sim. h) = 62,400 CPU core-h
Model training: 10 models × 50 trainings × 3 epochs × 30,000 samples ÷ 60 samples/s × 4 GPUs ÷ 3,600 s/h = 1,000 GPUh
Inference: 15,260 steps (4 B tokens ÷ (128 samples á 2048 tokens per step)) × 30 s/step × 4 GPUs ÷ 3,600 s/h ≈ 508 GPUh

HPC Project Extension

In order to extend a HPC project, a HPC compute time application is required as it is for the initial application. This follow-up application must include the existing project ID (e.g., p_project). Unlike the initial application, the focus here is not on the scientific project’s eligibility for HPC funding, but rather on the progress and status of the HPC project. For the follow-up application, the options are Tier 3, NHR Normal, or NHR Large, regardless of the project’s previous or initial category. The choice depends on the amount of resources required (see project categories), which must be estimated in advance and explained in the application.

Compute time utilization can serve as a basis for this assessment. In particular, key metrics of your own calculations must be determined based on preliminary or current project usage for further estimation. These can be derived from from project statistics and recorded job statistics, which are accessible to PI and PC. The procedure for determining the appropriate project category for a follow-up application is therefore as follows:

• If necessary, run tests for planned calculation types that have not yet been performed
• Derive key metrics of your own calculations; see
  • project statistics
  • job statistics
• Perform compute time estimation
• Select project category

Further Notes

Projects formally start on the first of each month. Preliminary access may be granted where applies.

Funding / Simplification: No additional scientific review is required for NHR-Normal projects if funded by a designated institution (e.g., provide a link to GEPRIS or CORDIS, see NHR). For NHR-Normal or NHR-Large projects, no proof of scalability is required for known parallel codes.

Project Members: Additional users can be integrated into HPC projects via their PC or PI through the HPC-Managementsystem.
Project leaders are required to notify the ZIH of any changes affecting project members, particularly resignations, changes in affiliation or email contact information, etc.

Access: A TU Dresden account (login) is required to access the HPC. If you are an external user and do not yet have one, all intended project members should request a personal guest login naming their HPC project. The guest account is valid for one year.

Terms of Use: In order to use the HPC systems, it is required to accept the current HPC Terms of Use. 

Specialized Support: We offer dedicated support for applications related to our key NHR focus areas.

Publications: Applicants are required to acknowledge the ZIH in the acknowledgments section of their publications regarding results to which the use of the HPC contributed (NHR: Examples, Tier 3: Examples).

Export Controls: High performance computers are considered dual-use goods and are subject to export control regulations. Applicants and project managers are responsible for ensuring compliance with these regulations.  BAFA - Export Controls in Research and Science

Usage Statistics: Project managers and admins have access to the  HPC-Managementsystem (access via ZIH login). Here, they can view current project's usage by month, year, and user (more information). For statistics based on HPC jobs, the PIKA job monitoring and visualization tool is available to all users.

Not a Windows-System: For HPC alternatives with Windows-support please consider our Research-Cloud or Enterprise-Cloud offers.