Table of Contents

HSM at AWI

Hardware

Tape Library

Introduction

A

...

...

HSM (Hierarchical Storage Management)

...

provides 

1. (nearly) unlimited storage space and
2. two replicates on tape (in different buildings for security)
3. a third copy on disk (for selected data, smaller files for faster access) → VTL (Virtual Tape Library, Silent Brick)

However, there are two caveats:

1. Your (larger) Data is archived on a tape, so it will take a while to get it back (unless it is online or has a disk-copy).
2. You need a project to archive your data (apply for a project here: eResources).

...

...

← Live View →
User: viewer PW: Viewer123!

...

Server and disks

• Three Dell R760xd server ( hssrv2[a-c] )
• 4 Brocade (32Gb) 6610 SAN switches
• 270 TB SSD primary cache (Dell ME5084)
• 488 TB Extended HDD-cache (Dell ME5084)
• Ab Ende 2024:
  1120 TB Virtual Tape Library (VTL) Silent Brick von Fast-LTA 
  • Controller G5200
  • SAS Switch
  • 7x SilentBrick Max

HSM at AWI

Concept

Principle Idea 

• HSM:  A (H)ierarchical (S)torage (M)anagement system consists of (at least) two storage systems: A cache speeds up access, and the hierarchy reduces cost. Based on a set of rules, data is stored on certain connected storage devices (tapes and optional disks).
• ScoutFS is used at AWI since 2024 and supersedes samfs (used since decades before)  www.versity.com/products/scoutfs/  www.scoutfs.org . 
• ScoutAM is used as (A)rchive (M)anager www.versity.com/products/scoutam/ and supersedes OHSM/VSM

...

• If the content of a file changes a new archive copy has to be produced. (You can not modify just the relevant bits on the tape.)
• The previous archive copy becomes useless (aside from having an additional backup of a previous version).
• If a file is deleted, the archive copy becomes useless, too.
• Both processes result in unused (invalid) sections on a tape. 
• Eventually only a small part of a tape contains relevant (up to date) information. The residual data can be archived on other tapes, the old tape is erased and can be used for future archive copies. This process is called recycling.

Hardware

Tape Library

• 2 TFinity Tape Libraries in two buildings
• each has 3100 licensed slots for LTO tapes
• 600 LTO-9 tapes in each library (600x ~20 TB → ~12 PB)

Image Added TFin-E	Image AddedTFin-D
Live View → Frame #2 or Frame #3 User: viewer PW: Viewer123!	Live View → Frame #2 or Frame #3 User: viewer PW: Viewer123!

Server, Switches and disks

• Three Dell R760xd server ( hssrv2[a-c] )
• 4 Brocade (32Gb) 6610 SAN switches
• 270 TB SSD primary cache (Dell ME5084)
• 488 TB extended HDD-cache (Dell ME5084)
• Ab Ende 2024:
  1120 TB Virtual Tape Library (VTL) Silent Brick von Fast-LTA 
  • Controller G5200
  • SAS Switch
  • 7x SilentBrick Max