Recommended storage locations
Overview
When you need more disk space you have four options:
- Option 1: Delete (unnecessary data)
- Option 2: / netscratch (temporary data)
- Option 3: Tape Archive (cold data)
- Option 4: Isilon (online data)
...
Option 1: Delete (unnecessary data) |
---|
What is not needed anymore can be deleted with the Windows Explorer or rm -rf /<YOUR_DIRECTORY> |
Option 2: /netscratch (Temporary data) |
Features: 50 TB space available for up to 90 days (automatically deleted afterwards) at
- smb:\\smb.isibhv.dmawi.de\netscratch\public (Windows/MacOS/Linux) known as public in Windows
- /isibhv/netscratch (Linux)
Documentation: |
https://intranet.awi.de/infrastruktur/rechenzentrum/dienste/file-service.html FileSharingvianetscratch |
Option 3: Tape Archive (HSM, cold data)
|
Features: Cheap (nearly) unlimited space available, two data copies in building D and E for security reasons. Documentation: |
https://intranet.awi.de/infrastruktur/rechenzentrum/platforms/hsm-hierarchical-storage-management.htmlDeletedCost Personal data | hssrv2:/hs/userm/<USER> | To get write access to hssrv2 you need an HSM-account: https://eservices/account/extendThree months after user leaves AWI | 10 TB (fix) | free | 3b | Project data | hssrv2sys/<PROJEKTNAME>/10 TB50 (Default is free)3c | hssrv2Note 1: In case of a disaster (e.g., building E burns down) it might take some weeks/months to recover all data. |
Note 2: There will be no charge for relocated data (from unixfs/winfs) until further notice (right of continuance, "Bestandsschutz") |
Option 4: Isilon (online data) |
---|
Features: Fast central network storage, daily snapshots (kept for six month, protection against user errors), erasure coding (protection against hardware failure), replication partially to Potsdam/Bremerhaven or HSM (protection against facility disaster) |
Documentation: https://intranet.awi.de/infrastruktur/rechenzentrum/dienste/file-service.htmlOption | Utilisation | Path | Note |
---|
|
DeletedDefault Cost Budget (per TB and yr) |
---|
4a | Personal data | UNIX: /home/ UNIX:smb:\\smb.isibhv.dmawi.de/personal WIN:smb:\\smb.isibhv.dmawi.de/home WIN:smb:\\smb.isibhv.dmawi.de/personal | The separation /home vs. /personal is partly historic, based on differences beween Windows and Unix. |
|
Three months leaves left AWI | 100 GB (fix) | free | 4b | Project data | UNIX: /isibhv/ |
|
projects<PROJECT>projects[-noreplica]/project_name WIN:smb:\\smb.isibhv.dmawi.de\projects[-noreplica]/project_name | You can create a project and request |
|
HSM-online/Isilon resources here: |
|
Three months after project ends | 1 TB | 150 € (Default is free) | Note 1: This data is replicated to the Isilon in Potsdam, which ensures availability Note 1: "isibhv" indicates the Isilon in Bremerhaven. For Potsdam this would be "isipd".Note 2: The personal data (4a) is replicated to the Isilon at the other location (Bremerhaven/Potsdam). Note 3: The project data (4b) can be replicated to the HSM or not (your choice in https://cloud.awi.de/#/projects), so data can be restored in case of a disaster |
in Bremerhaven 2Option 3 (Tape Archive) could be used in addition to Option 3 for additional data protection in case of a disaster.Note 3: There will be no charge for relocated data (from unixfs/winfs) until further notice (right of continuance, "Bestandsschutz")
Note 4: Data on the Isilon is accassible from (parts of) ollie and could be processed using HPC....
Data on the Isilon is accessible from (parts of) ollie and could be processed using HPC. |
Option 5: Nextcloud ( Fileshareing with internal and external user )
|
Features: Filesharing and collaborative working on office files (collabora online). User can share files with users that are external users without an account. Every user has 1GB of nextcloud storage quota and up to 100GB on SMB share (Option 4a). For the SMB shares nextcloud is like a bridge to the data. You can also get access via network drive on your window file browser. EXTERNAL STORAGEShares | Mountable directories(Projects must be mouted by an administrator) |
---|
smb://smb.isibhv.dmawi.de | projects, personal/<username>, projects-dmz |
---|
smb://smb.isipd.dmawi.de |
---|
Backup / Restore dataThe backup is handled by the ISILON itself. To restore files you can use on Windows filesystems the option "previous versions" to get older backups. You can also get your files online in the nextcloud environment. To do so you need to modify your URL. Connect ProjectsWhen you create a project and get storage from cloud.awi.de, you can access the file on your storage via nextcloud. Internal Storage/var/www/nextcloud/data | Quota of 1GB per user |
---|
Backup / Restore dataThe data is backed up via VMWARE. A backup is created once a night. It is possible to restore individual files. This can take some time because the VMWARE administrator has to restore the files from the backup. |
The exact path to your project data is shown in https://cloud.awi.de/#/projects this table is just for a quick overview to understand the syntax:
Online storage |
| (Linux-)Path |
---|
Location | Bremerhaven | /isibhv/projects |
|
|
| Potsdam | /isipd/projects |
|
|
Access | Internal |
|
|
|
| External (only available for Bremerhaven) |
| -dmz |
|
Replica | Tape |
|
|
|
| No replica |
|
| -noreplica |
Tape storage/archive |
|
|
|
| /hs/projects/
|
|
|
Access:
The platform independent usage of the filesystems in Bremerhaven and Potsdam is possible by the protocols
- Network File System (NFS, Linux)
- Server-Message-Block-Protokoll (SMB , Windows & Mac)
- sftp (Linux, Windows, Mac, but tape storage only)
Linux
The NFS filesystems are provided by an automounter and mounted as soon as they are required. The user just has to know the filesystems (e.g., /isibhv/projects/), which he wants to access.
Windows
On Windows & Mac, the user has to connect manually to the network drives from the File Explorer/Finder using the SMB-Server:
- Right click on This PC in the Windows File Explorer
- Select Map network drive.
- Enter \\smb-server\share (e.g.: \\smb.isibhv.dmawi.de\home)
Info |
---|
- Users connecting from a external network using VPN, will need to use the option Connect using different credentials. Your AWI-Username and password are needed, for example:
Name: dmawi\wwichtig or wwichtig@dmawi.de Passwort: xxxxxx
- External connections are connections via the Internet, connections from our internal wireless networks awi-guest and eduroam or our special network for private notebooks.
|
Mac-Users
For a manual connection from a Mac you will need to use the
- menu option go to => connect with server and enter the following:
- smb://server/share (e.g.: smb://smb.isibhv.dmawi.de/home)
After that, a login prompt will appear. Please note, the field "Workgroup/Domain" has to contain the domain name dmawi.
Snapshots (Backups)
Linux/Unix-Users
In every directory of the file server nfs.isibhv.dmawi.de, there is a hidden subdirectory called .snapshot. Subdirectories, with the creation date in their name, contain the snapshots in question.
Windows-Users
For every directory of the file server smb.isibhv.dmawi.de, you can right click on the directory and access the snapshots via the menu previous versions. The previous versions are sorted by the creation date of that snapshot.
File Sharing via netscratch
Windows, Linux and MacOS manage and interpret access permissions to files in different ways, sometimes not compatible with each other. For this reason two different /netscratch directories are provided:
| /netscratch recommended for Linux
| /netscratch/public recommended for Windows \\smb.isibhv.dmawi.de\netscratch
|
---|
Information | Everything you store here is, at first, only changeable by you. | Everything you store here is, at fist, readable, changable and deletable by everyone |
Windows | New files and directorties are only readable by you. You can add further users or groups via the Windows-Fileexplorer, in the tab "Properties → Security", but this will compromise or overwrite the previously defined Linux permissions. | You can change the permissions by using the Windows-Fileexplorer, in the tab - "Properties → Security" |
Linux | New files and directores are readable by everyone. You can change the access permissions for the owner and group by using chmod and/or chgrp. | You can change the permissions with chmod and chgrp, but this will compromise or overwrite the previously defined Windows permissions. |
MacOS | See Windows | See Windows |
Quota
You can check the occupied disk space with
Windows: File Explorer -> right click -> Properties
Linux
du
-sh <path>
- df -h <path> # shows quota and apparent size
- Projects: Project volume and quota can be checked with cloud.awi.de/#/projects
Related articles
Content by Label |
---|
showLabels | false |
---|
max | 5 |
---|
showSpace | false |
---|
sort | modified |
---|
reverse | true |
---|
cql | label = "kb-how-to-article" and type = "page" and space = "HELP" |
---|
|
...