...

The HSM (Hierarchical Storage Management) at AWI provides 

1. (nearly) unlimited storage space and
2.   two replicates on tape (in different buildings for security) and
3. a third copy on disk (for selected data, smaller files for faster access) → VTL (Virtual Tape Library, Silent Brick)
4. for a (comparable) low cost

However, there are two caveats:

• Your (larger) Data is archived on tape, so it will take a few minutes to get it back (unless it is online or has a disk-copy).
• You need a project to archive your data (apply for a project here: eResources).

HSM at AWI

Four domains for tape archiving at AWI

Purpose(mainly) PANGAEA and Registry on how Open to all usersOnly possible with sufficient metadata, see links Domain Purpose Path Disk Copy Description How to apply Usage A PANGAEA /hs/platforms Archive of sensor.awi.de Permanent archive for irrecoverable data. Metadata is needed for this domain. Please contact PANGAEA on how to submit your data. /hs/usero /hs/pangaea PANGAEA P Projects /hs/projects /hs/s3projects Project data Long term project data with predefined life time. A project can be created with eResources https://cloud.awi.de/#/projects  The setgid on directories ensures that new (sub-)directories belong to the project (POSIX) group automatically IB, IP Desaster recovery Daily (or weekly) automatic replication of online project data from the Isilon (when selected in eResources) in Bremerhaven and Potsdam, respectively D Internal IT stuff For internal IT use (used for additional backups and storing of expired user/project data) Domain Path Disk Copy Description How to apply A /hs/platforms Archive of sensor.awi.de . /hs/usero /hs/pangaea PANGAEA P /hs/projects /hs/s3projects Project data Please use eResources to create a project and request storage resources.  C Depricated /hs/usera /hs/userc /hs/userm Have vanished early 2020 IB, IP Desaster recovery /hs/isirep-... Project replica from the Isilons (Bhv, Pot) Disaster recovery only Daily (or weekly) automatic replication of online project data from the Isilon (when selected in eResources) in Bremerhaven and Potsdam, respectively. Used for disaster recovery only (=hopefully never ). D Internal IT stuffD /hs/backup samfs-dumps, ScoutFS-dumps, logfiles, Veeam IT internal use only /hs/store 10-year storing of expired user and project data /hs/s3gateway experimental storage A disk copyis available for specific (smaller) files for some file systems. This allows a fast significantly faster access of offline files. The availability of this disk archive depends on the actual resources/usage and might change.

...

• Three Dell R760xd server (hssrv2[a-c].dmawi.de)
• 4 Brocade (32Gb) 6610 SAN switches
• 270 TB SSD primary cache (Dell ME5084-SSD)
• 488 TB extended HDD-cache (Dell ME5084-HDD)
• Ab Ende Planed for end of 2024:
  1120 TB Virtual Tape Library (VTL) Silent Brick von Fast-LTA 
  • Controller G5200
  • SAS Switch
  • 7x SilentBrick Max

...