Detailed information about the central file service

The ZIH provides central disk storage space for all members of the TU Dresden. Part of this is the central home file service (home directory or home drive), in which personal space is reserved for all registered users. Additional storage space can be provided in the form of group drives for individuals or workgroups.

Features & options for home directories

The central home directory can be used by the owner to store files that should not be made accessible to third parties. Home directories are automatically set up for all members of the TU Dresden and are automatically activated after confirmation of the "End User License Agreement" (EULA). Change requests can be formulated via ticket and will be considered depending on technical feasibility.

The availability of the home directory ends automatically with the expiration or blocking of the associated login. The data is kept for up to 26 weeks according to the availability rules.

The exception is a subspace (in the subdirectory "public_html") which serves as storage space for a personal WWW appearance within the WWW service of the TU Dresden. The owner of the files is responsible for the content of the personal WWW appearance.

Access rights, size and quality of service of the storage space

The home directories are subject to quotas whose upper limits cannot be exceeded. They currently amount to 20 GB. If more is needed, the quota can be increased via the Self-Service portal. The maximum quota is set at 200 GB for employees and 50 GB for students.

The access rights for the home directory are set according to the Linux rule "0701".

The owner login has full access
The owner group has no rights
The "Execute" right for "Everyone" is necessary to access the personal WWW site on the WWW server without third parties being able to see the rest of the data.

Home directories are highly available and protected against failure or data loss. They are mirrored asynchronously to a second site and protected against accidental deletion of data with backup and snapshots. The recovery period is 26 weeks.

Access to home directories

SSH connection to the central login server (login.zih.tu-dresden.de).
- After login you will find your home directory in the path "/home/loginname".
Windows users in the campus network can mount their home directory directly in the file explorer with their own drive letter.
- Selecting "Connect network drives..."
  - Selection of a free drive letter
  - folder: "\\vs-home.zih.tu-dresden.de\loginname"
  - tick "Connect with other credentials."
  - Click "Finish".
    - Authentication is done with the domain "dom", the ZIH login name and the corresponding password, similar to the following:
      - User information: dom\loginname
      - Password: zih-password
Mac users can mount it using the "Finder".
SFTP connection to the central SFTP server using a suitable application:
- Under Windows e.g. with FileZilla
- Linux users can use an SFTP connection (sshfs, sftp or scp)
- SFTP server is:
  - dgw.zih.tu-dresden.de (data gateway - SFTP server for all drives)
Connecting to the WWW area in the home directory
- In a web browser the following address is specified:
  - https://wwwpub.zih.tu-dresden.de/~loginname

Features and options for group drives

Group drives are designed to allow workgroups to store data centrally. Group drives can be requested in the Self-Service portal. Applicants and contact persons must be employees of the TU Dresden. Instructions on how to apply can be found in the FAQ .

All group drives are subject to an expiration time, which can be viewed in the Self-Service portal. An extension of the term to a maximum of 3 additional years can be made via the Self-Service portal.

Approximately 3 months after the runtime expires, the status of the group drive is set to "expired". After a further 3 months, it is locked. After another 6 months, the data of non-renewed group drives is packed and archived. Previously configured access rights are not preserved, so that afterwards, only the responsible manager can get the data back.

Sizes and limitations of group drives

As a consequence of the technologies used to provide the storage service, the group drives cannot be made available in unlimited quantities. In order to allow the data to be mirrored to a second location or that a backup can be provided, certain sizes may not be exceeded. Therefore, the group drives are divided into three different categories. When applying, a category must be selected, a change between categories is possible taking into account the resulting changes in snapshots and the backup.

Category backup/archive (restic, rsync or similar)

Used as destination for custom backup or to store data that is already stored at another location.

no mirror to the second location
no snapshot
no backup
up to a maximum of 2 TB data and 2.000.000 files

Category Institute, project and publication data

Used for institutes and projects for important data.

mirror to second location
snapshots available depending on settings, default is S2 (see below)
backup once a week
up to a maximum of 5 TB data and 5.000.000 files

Category raw research data

Used as storage for raw data from research projects.

mirror to second location
snapshots available depending on settings, default is S3 (see below)
no backup
up to a maximum of 50 TB of data and 10,000,000 files

Possible settings for snapshots

S0: no snapshots are created
S1: from 8 am - 4 pm on working days one snapshot every hour, these are kept for two days; one snapshot per day for two more days; 6 more snapshots for six more weeks
S2: one snapshot every 4 hours from 8 a.m. to 4 p.m. on work days, these are kept for two days; one snapshot per day for 12 more days; 13 snapshots each week; 6 more snapshots per month
S3: one snapshot per day for 2 days; 12 more 1x per week
S4: once per week for 12 weeks

Access to the group drives

CIFS/SMB, NFS3, NFS4 and SSH are offered as access protocols. Block-based devices (iSCSI) are not supported.

An SFTP program, e.g. FileZilla, is used to establish an SFTP connection to the data gateway:
- sftp dgw.zih.tu-dresden.de
- After authentication with login name+password and possibly necessary specification of a port (22), one is connected to the home directory and can navigate to all other directories.
  - The path of a group drive is usually "/glw/glwname/".
Mount as NFS drive on an institute server.
Connection via CIFS/Samba analogous to the home directories from Windows
- \\vs-grpNN.zih.tu-dresden.de\glwname
- Authentication is done with the domain "dom", the ZIH login name and the corresponding password similar to the following:
  - User information: dom\loginname
  - Password: ZIH password
Group drives are not available at the central login servers. However, one can initiate copy actions on the data gateway from there.

Anonymous FTP / data exchange without authentication is not supported.

Access rights are defined either according to Linux rules (UNIX) or Windows rules (NTFS).

The startup setting with Linux rules is "2770":

The owner login has full access
The owner group has full access
The owner group is usually inherited (setgid bit)
- Attention: permissions for newly created files and directories are set based on the umask of the respective user
The ZIH admins have root access only on the data gateway (full access)
All others have no access

Access rights with NTFS rules are defined directly on the files. A terminal server is available in ZIH for this purpose. One proceeds as follows

Establish a remote desktop connection to the TS "domts3.zih.tu-dresden.de"
Mount the group drive, select "Connect network drives...".
- Select drive letter
- enter folder: "\\vs-grpXX.zih.tu-dresden.de\glwname".
- check "Connect with other credentials."
- "click "Finish
  - Enter user information
    - dom\loginname
    - ZIH password

The initial settings with NTFS rules are:

Windows domain "dom" rules apply.
The domain admins (ZIH) have full access with inheritance
- For backup and possible maintenance
The admins specified in the application have full access with inheritance
- For administrative purposes
"Everyone" has read+execute access to the root directory without inheritance
- This can only be removed if accessibility via the SFTP server is not intended

Technologies used to secure the data

Securing data through erasure coding

All data is always distributed across multiple data stores in such a way that a failure of one or two data stores does not result in data loss.

Recovery options through snapshots

A snapshot is a user-readable older state of the stored files and folders. Usually these are accessible under Windows by accessing "previous versions" of files. Under Linux there is a subdirectory ".snapshot" which is hidden in every directory. Mac OS X you can find the snapshot directories under "~snapshot".

All snapshots are read-only and therefore offer protection against accidental or intentional (viruses, trojans) manipulation of the data. A disruptive program can destroy the current original of data, but has no write access to the snapshots.

Snapshots are available in varying quantities. Users must choose a use case for snapshots. The data stored in the snapshots counts towards the group drive quota.

Copy of the data at a second location

To safeguard against failure of the entire primary site (LZR), parts of the data are mirrored to a second site.

Backup for disaster recovery

In the event of a disaster, data is backed up weekly to a second storage technology. In the event of a failure, this allows the data to be restored to at least a state that is no more than one week old.

SVM or Storage Virtual Machine, virtual file servers for workgroups for self-administration.

Workgroups with their own user management, e.g. their own Windows subdomain, can apply for virtual file servers (SVM) with group drives in them. These are included in their own domain. They must be registered for the backup service by their administrators themselves.

Cooperation obligations & prerequisites

Access to the central drives via CIFS/Samba is done using a ZIH user login from the campus network. A VPN connection must be established from outside the campus. An encrypted connection, without a VPN gateway, is possible via the central data gateway. Here the services SSHFS, SCP and SFTP are supported.

Requirements:

ZIH user login, hardware (PC), internet access/campus network

Co-operation requirements:

Usage information for students

Usage information for other members of the TUD

Billing / costs

There are no costs for members of the TU Dresden.

Data security / data backup / encryption

The creation of snapshots is activated by default.
It is not possible to change the snapshot rules for individual home directories or group drives that are located in collective containers (older group drives).
All data mentioned here is included in the backup of the ZIH. Restore from tape requires the help of ZIH admins.
Professional backups of own systems to the home directories or group drives are not desired, as the resulting data cannot be effectively handled by the internal organization rules of the storage system.
There is a separate central backup service for this purpose. Backups usually involve constantly changing data, so for example, for every 10 GB of regular backup, up to 260 GB of data can accumulate in half a year.
- A backup copy of your own data without using a professional backup algorithm is usually acceptable.
Connections via SSH (scp, sftp) are encrypted.
CIFS connections are unencrypted unless you force the SMB3 protocol.
The data on the hard disks of the file servers are encrypted, because all of them are equipped with the option of self-encryption. However, when accessed, they are passed back to the transfer protocol in decrypted form.
Suitable programs must be used to encrypt directories or files. (e.g. BoxCryptor)