| Date | Topic | Slides / Comments / Material |
|---|
22nd Aug 2024 | - Announcements
- Conda licensing updates
- do not use defaults channel in conda
- instead, use conda-forge
- conda-forge is used by default in conda's modules in Albedo
- Disruption in albedo0 (login node) - due to maintenance
- for the time being use albedo1
- Q&A
- What should I do if I have existing conda environments to avoid using the defaults channel?
- run conda list and check if you have anything installed from the defaults channel. If you do you can:
conda config --remove channels defaults
conda config --add channels conda-forge
conda config --set channel_priority strict
- If you are still seeing the defaults channel (or no channel at all) after your conda list command, you'll have to rebuild your environment. If you have any problems by doing so write us an email to hpc@awi.de
| hybrid H-1049 |
27th Jun 2024 | - Announcements
- Q&A
- Presentation: Installing packages in Albedo with Spack - Seshadri Basava
| hybrid H-1049
Slides: https://notes.desy.de/p/Q2I-7FXT0#/ Slides as documentation: https://notes.desy.de/tPTGtbBMRB-tWOHcCtiq3Q?view Recording: https://nextcloud.awi.de/s/bxxadbF9AwsYssH |
23rd May 2024 | | hybrid
H-0049 Slides: https://nextcloud.awi.de/s/n2mng3F9Ey3Pcff |
25th Apr 2024 | - S3 object storage in HSM: prototype (Malte Thoma and Pavan Kumar Siligam )
- Q&A
- Presentation: AI/ML and climate analysis - it all starts with data - Christopher Kadow, DKRZ
- Christopher is the leader of the Data Analysis group at DKRZ. Among other tasks his group have developed the NextGEMS data portal, develop analysis tools, they are organizing Machine Learning workshops, working on the future DKRZ object storage, workflows with zarr data format...
- Abstract:
This talk gives an overview of the topics of the new department Data Analysis at DKRZ - using new technologies for climate research and data handling on the HPC Levante. We demonstrate the successful transition of an AI image processing method into a climate data reconstruction application. In addition to exciting results of reconstructed climate events in the distant past dating back to 1850, the importance of the data base in our field will be addressed. Understanding the content of the data collection is key to the proper application of the latest AI/ML and climate analysis. Linking different data sources for such target variables is part of the BMBF projects ClimXtreme, RegiKlim and WarmWorld - e.g. www.gems.dkrz.de . The results presented link observations with climate model simulations and, in combination with ML, provide an unprecedented insight into different climate applications such as data assimilation, downscaling and digital twins.
Kadow, C., Hall, D.M. & Ulbrich, U. Artificial intelligence reconstructs missing climate information. Nat. Geosci. 13, 408–413 (2020). https://doi.org/10.1038/s41561-020-0582-5 - You can request a consultation session with him on that day!
| hybrid
H-0049
Slides: https://nextcloud.awi.de/s/JHw5HXaxmRK53cM |
28th Mar 2024 → 11th Apr 2024
| | hybrid |
22nd Feb 2024 | - news about JupyterHub
- File system news
- Coming meetings and presentations
- Q&A
- Presentation: HPC and Data Process Service Catalogue - Miguel Andrés-Martínez
| hybrid
Slides:
https://nextcloud.awi.de/s/wPDd9KrrPsfFd9p |
25th Jan 2024 | - Advertisement of pool-tool workshop
- GPFS (file system) issues and work in progress
- New analysis-toolbox/01.2024
- Jupyter on Albedo
| Online |
23rd November 2023 | - Albedo support service update
- New Analysis-Toolbox in December
- JupyterHub work
- Demo by Sonal Rami: Machine learning overview: comparing CPU, GPU and Multi-GPU performance
| hybrid
Link to the recorded presentation: https://nextcloud.awi.de/s/oxmjEWg8AzHWHDR
Slides and Python script for the demo:
https://github.com/SonalRami/ML_demo-CPU-vs-GPU |
26th October 2023 | - Current support shortage
 - Last GPUs arrived
- User interviews and Fall plans for the group
- Updates on
- Spack 0.18.1 → 0.20.1 (Done!)
- gprofng (Done!)
- IDL-8.9.0/ENVI-5.7 (Done!)
- Documentation (Software: R, JupyterLab/Hub..., WIP)
- conda environments/python environments, spack documentation, guidelines (WIP)
- JupyterHub
- Invite users that use JupyterLab to request joinning the testing phase
- R in JupyterHub
- Consoles not working!
- From the previous meeting
- Albedo Queue information (WIP)
- statistics? (not urgent)
- idle time/processing time...
- Malte → sacct, sreport
- Directories permissions
- by default be able to read work directory?
- Malte → all employees would be able to read, not safe. Use eresources projects
- FESOM-1 status report Albedo
- CMORization status
- Pools for modelling updates
- Miguel → not our responsibility
- Jan → provide tools to mirror pools
- Malte → which one is the leading pool?
- Next Meeting in November
- ML demo with TensorFlow-Keras and GPUs, by Sonal Rami
| online |
28th September 2023 | - Current support shortage
- Updates on
- JupyterHub
- Invite users that use JupyterLab to request joinning the testing phase
- R in JupyterHub
| online |
25th August 2023 | - Albedo updates
- JupyterHub updates
- New team member
| hybrid, you can also join us in person: room H-1049 |
27th July 2023
| Meeting cancelled due to holidays of multiple team members and another simultaneous meeting |
|
22nd June 2023 | - Introduction round
- HPC Users Interviews announcement
- Albedo news
- Installation of new GPU node
- Ollie shut down
- JupyterHub progress
| hybrid, you can also join us in person: room H-1049 |
25th May →
Monday, May 22nd 2023 | - upcoming changes in the HPC team
- EOL Ollie approaching → last chance to save your data!
- QnA
| hybrid, you can also join us in person: room H-1049 |
| 27th April 2023 | ??? |
|
| 23 March 2023 | | only online |
| 23 February 2023 | - Albedo:
- Future of hyperthreading nodes on Albedo (smpht, mppht)
- Extension of GPU partition: 2 more nodes (Preferences on hardware?)
- Slurm Powersafe
- Parallel I/O
- Need for QOS "xlarge" (ollie) → 1wk (albedo)?
- QnA
| Hybrid meeting: Room Arktis G-301 & Webex |
26 January 2023
| no meeting |
|
| 22 December 2022 | QnA |
|
| 24 November | Current status of Albedo |
|
27 October | no meeting - holidays again (yes, in between, we do work!) | Albedo: slow progress, but impossible to give an exact date for power user access. |
22 September 2022 | no meeting - vacations & no updates on Albedo |
|
1 September 2022
(postponed by 1 week) | - Albedo timeline: delays... no user access before October
- Support team: Natalja will leave AWI end of October → reduced support capacity → strong focus on Albedo basics.
|
|
28 July | summer holidays - no meeting |
|
| 23 June 2022 | News on albedo... - Hardware has arrived at AWI server room
- The floor has to be reinforced: 4 July
- 5 July - ... installation, finally!
- Plans for 5PB file system: /albedo/work and /albedo/scratch, both with personal and project directories.
Space for projects via eResources (like /isibhv/projects/)
|
|
5 May 2022
(postponed by 1 week) | News on albedo: Delivery from 17th to 25th Mai 
Ollie: Accounting & Job Priorities |
Slurm on Ollie → Scheduling |
| 24March 2022 | Ollie: Bright+CentOS 7.5 → xCat + CentOS 7.9 | Partitions mpp_new, smp_new |
10 March 2022
(as an exception) | |
|
| 27 January 2022 | Albedo | Albedo_2022-01-27.pdf |
Stan End of Life 7 March 2022