Hardware
Login nodes
The two login nodes albedo[0|1] are the only accessible nodes from the AWI internal network. Their adress is
albedo[0|1].dmawi.de
Count | Name | Specification |
|---|---|---|
2x | albedo[0|1] |
|
Compute nodes
Count | Name | Partition | Specification |
|---|---|---|---|
240x | prod-[001-240] | smp, mpp |
|
4x | fat-00[1-4] | fat |
|
1x | gpu-001 | gpu |
|
1x | gpu-002 | gpu |
|
The FESOM2 Benchmark we used for the procurement on 240 Albedo nodes compares to 800 Ollie nodes.
Filesystem
Local user storage
- The local storage is a parallel GxFS Storage Appliance from NEC based on IBM Spectrum Scale: https://en.wikipedia.org/wiki/GPFS.
Tier 1: 220 TiB NVMe as fast cache and/or burst buffer
Tier 2: ~5.38 PiB NL-SAS HDD (NetApp EF300)
- All nodes are connected via a 100 Gb Mellanox/Inifiniband network.
| Personal directories | Project directories | |||||
|---|---|---|---|---|---|---|
| Mountpoint | /albedo/home/$USER | /albedo/work/user/$USER | /albedo/scratch/user/$USER | /albedo/work/projects/$PROJECT | /albedo/scratch/projects/$PROJECT | /albedo/burst |
| Comes with | HPC_user account: https://id.awi.de → Start a new request/Bestellung → IT Service → HPC → Add to chart/In den Einkaufswagen | Apply for Quota here: https://cloud.awi.de/#/projects | -- | |||
| Quota | 100 GB | 3TB | 50 TB | variable | variable | |
| Delete | 90 days after user account expired | all data older than 90 days | 90 days after project expired | all data older then 90 days | after 10 days | |
| Security | Snapshots for 180 days | -- | Snapshots for 180 days | -- | -- | |
| Snapshots | /albedo/home/.snapshots/ | /albedo/work/user/.snapshots/ | -- | /albedo_int/work/projects/.snapshots/ | -- | |
| Owner | $USER:hpc_user | $OWNER:$PROJECT | root:root | |||
| Permission | 2700 → drwx--S--- | 2770 → rwxrws--- | 1777 → rwxwrxrwt | |||
| Focus | many small files | large files, large bandwidth | large files, large bandwidth | low latency, huge bandwidth | ||
Remote user storage (/isibhv)
- You can access your online space on the Isilon in Bremerhaven (see https://spaces.awi.de/x/a13-Eg for more information) via the nfs-mountpoints
/isibhv/projects
/isibhv/projects-noreplica
/isibhv/netscratch
/isibhv/platforms
/isibhv/home
Network
Fast interconnect:
HDR Infiniband
Ethernet:
- albedo is connected to the AWI backbone (including the Isilon and the HSM) via four eth-100 Gb interfaces.
Each single albedo node has a 10 Gb interface.