Login nodes
The two login nodes are the only accessible nodes from the AWI intranet.
Quantity | Name | Specification |
|---|---|---|
2x | albedo[0|1] |
|
Compute nodes
Quantity | Name | Partition | Specification | Notes |
|---|---|---|---|---|
240x | prod-[001-240] | smp, smpht |
| For our test phase, we have split the compute nodes in two sets, as announced by Malte in the HPC meeting: More information can be found on the slurm documentation. |
4x | fat-00[1-4] | fat, fatht |
| |
1x | gpu-001 | gpu |
| A comparison of the two different GPUs can be found here:
|
1x | gpu-002 | gpu |
|
Filesystem
Local user storage
- The local storage is a parallel GxFS Storage Appliance from NEC based on IBM Spectrum Scale: https://en.wikipedia.org/wiki/GPFS.
Tier 1: ~171 TiB NVMe as fast cache and/or burst buffer
Tier 2: ~5030 TiB NL-SAS HDD (NetApp EF300)
- All nodes are connected via a 100 Gb Mellanox/Inifiniband network.
| Personal directories | Project directories | |||||
|---|---|---|---|---|---|---|
| Mountpoint | /albedo/home/$USER | /albedo/work/user/$USER | /albedo/scratch/user/$USER | /albedo/work/projects/$PROJECT | /albedo/scratch/projects/$PROJECT | /albedo/burst |
| Comes with | HPC_user account: https://id.awi.de → Start a new request/Bestellung → IT Service → HPC → Add to chart/In den Einkaufswagen | Apply for Quota here: https://cloud.awi.de/#/projects | -- | |||
| Quota | 100 GB (fixed) | 3TB (fixed) | 50 TB (fixed) | 30 €/TB/yr (variable) | 10 €/TB/yr (variable) | |
| Delete | 90 days after user account expired | all data older than 90 days | 90 days after project expired | all data older then 90 days | after 10 days | |
| Security | Snapshots for 180 days | -- | Snapshots for 180 days | -- | -- | |
| Snapshots | /albedo/home/.snapshots/ | /albedo/work/user/.snapshots/ | -- | /albedo/work/projects/.snapshots/ | -- | |
| Owner:Group | $USER:hpc_user | $OWNER:$PROJECT | root:root | |||
| Permissions | 2700 → drwx--S--- | 2770 → rwxrws--- | 1777 → rwxwrxrwt | |||
| Focus | many small files | large files, large bandwidth | large files, large bandwidth | low latency, huge bandwidth | ||
Storage Pools
If you want to share data with other users use * Project directories*. For convenience project administrator may request a link from /albedo/pool/<something> → /albedo/work/projects/$PROJECT/<somewhere> via hpc@awi.de
Remote user storage (/isibhv)
- You can access your online space on the Isilon in Bremerhaven (see https://spaces.awi.de/x/a13-Eg for more information) via the nfs-mountpoints
/isibhv/projects
/isibhv/projects-noreplica
/isibhv/netscratch
/isibhv/platforms
/isibhv/home - Tape storage (HSM) is not mounted. However, you could archive your results with something like
rsync -Pauv /albedo/work/projects/$PROJECT/my_valuable_results/* hssrv1:/hs/projects/$PROJECT/my_valuable_results_from_albedo/
Network
Fast interconnect (beween albedo's nodes):
HDR Infiniband
Ethernet:
- albedo is connected to the AWI backbone (including the Isilon and the HSM) via four eth-100 Gb interfaces.
Each single albedo node has a 10 Gb interface.