Recommended storage locations
Overview
When you need more disk space you have four options:
- Option 1: Delete (unnecessary data)
- Option 2: /netscratch (temporary data)
- Option 3: Tape Archive (cold data)
- Option 4: Isilon (online data)
Option 1: Delete (unnecessary data) | |||||||||||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
What is not needed anymore can be deleted with the Windows Explorer or | |||||||||||||||||||||
| Option 2: /netscratch (Temporary data) | |||||||||||||||||||||
Features: 50 TB space available for up to 90 days at
Documentation: FileSharingvianetscratch | |||||||||||||||||||||
| Option 3: Tape Archive (HSM, cold data) | |||||||||||||||||||||
Features: Cheap (nearly) unlimited space available, two data copies in building D and E for security reasons.
Note 1: In case of a disaster (e.g., building E burns down) it might take some weeks/months to recover all data. | |||||||||||||||||||||
Option 4: Isilon (online data) | |||||||||||||||||||||
Features: Fast central network storage, daily snapshots (kept for six month, protection against user errors), erasure coding (protection against hardware failure), replication partially to Potsdam/Bremerhaven or HSM (protection against facility disaster)
Note 2: The personal data (4a) is replicated to the Isilon at the other location (Bremerhaven/Potsdam). Note 3: The project data (4b) can be replicated to the HSM, so data can be restored in case of a disaster. Note 4: Data on the Isilon is accessible from (parts of) ollie and could be processed using HPC. |
The exact path to your project data is shown in https://cloud.awi.de/#/projects this table is just for a quick overview:
| Online storage | (Linux-)Path | |||
|---|---|---|---|---|
| Location | Bremerhaven | /isibhv/projects | ||
| Potsdam | /isipd/projects | |||
| Access | Internal | |||
| External (only available for Bremerhaven) | -dmz | |||
| Replica | Tape | |||
| No replica | -noreplica | |||
| Tape storage/archive | ||||
| /hs/${FS}/ ${FS} depends on your section | ||||
Access:
The platform independent usage of the filesystems in Bremerhaven and Potsdam is possible by the protocols
- Network File System (NFS, Linux)
- Server-Message-Block-Protokoll (SMB , Windows & Mac)
Linux
The NFS filesystems are provided by an automounter and mounted as soon as they are required. The user just has to know the filesystems (e.g., /isibhv/projects/), which he wants to access.
Windows
On Windows & Mac, the user has to connect manually to the network drives from the File Explorer/Finder using the SMB-Server:
- Right click on This PC in the Windows File Explorer
- Select Map network drive.
- Enter \\smb-server\share (e.g.: \\smb.isibhv.dmawi.de\home)
- Users connecting from a external network using VPN, will need to use the option Connect using different credentials. Your AWI-Username and password are needed, for example:
Name: dmawi\wwichtig or wwichtig@dmawi.de
Passwort: xxxxxx
- External connections are connections via the Internet, connections from our internal wireless networks awi-guest and eduroam or our special network for private notebooks.
Mac-Users
For a manual connection from a Mac you will need to use the
- menu option go to => connect with server and enter the following:
- smb://server/share (e.g.: smb://smb.isibhv.dmawi.de/home)
After that, a login prompt will appear. Please note, the field "Workgroup/Domain" has to contain the domain name dmawi.
Snapshots (Backups)
Linux/Unix-Users
In every directory of the file server nfs.isibhv.dmawi.de, there is a hidden subdirectory called .snapshot. Subdirectories, with the creation date in their name, contain the snapshots in question.
Windows/Mac-Users
For every directory of the file server smb.isibhv.dmawi.de, you can right click on the directory and access the snapshots via the menu previous versions. The previous versions are sorted by the creation date of that snapshot.
File Sharing via netscratch
Windows, Linux and MacOS manage and interpret access permissions to files in different ways, sometimes not compatible with each other. For this reason two different /netscratch directories are provided:
/netscratch | /netscratch/public \\smb.isibhv.dmawi.de\netscratch | |
|---|---|---|
| Information | Everything you store here is, at first, only changeable by you. | Everything you store here is, at fist, readable, changable and deletable by everyone |
Windows | New files and directorties are only readable by you. You can add further users or groups via the Windows-Fileexplorer, in the tab "Properties → Security", but this will compromise or overwrite the previously defined Linux permissions. | You can change the permissions by using the Windows-Fileexplorer, in the tab - "Properties → Security" |
Linux | New files and directores are readable by everyone. You can change the access permissions for the owner and group by using chmod and/or chgrp. | You can change the permissions with chmod and chgrp, but this will compromise or overwrite the previously defined Windows permissions. |
MacOS | See Windows | See Windows |
Quota
You can check the occupied disk space with
Windows: File Explorer -> right click -> Properties
Linux
du-sh <path># shows physical disk usage (including data protection, accounted for quota)du-sh --apparent-size<path> # shows logical space- df -h <path> # shows quota and apparent size
- Projects: Project volume and quota can be checked with cloud.awi.de/#/projects