HPC and NIRD toolkit course fall 2020
In the fall 2020, the Scientific Computing group at the IT department (UiB) is offering a basic hands-on course for current and future users of national HPC systems (Saga, Fram, Betzy) and the NIRD Toolkit. The course is aimed at those with little or no experience on these systems. For those who have little experience with UNIX-like operating systems such as Linux or who have never worked on the command line of a terminal, we will offer a hands-on pre-course about UNIX a week before the HPC & NIRD Toolkit lectures.
On each course day, we will start at 09:00 and conclude at 12:30. Each morning will include presentations and hands-on time on one of the actual HPC systems or the NIRD Toolkit. We recommend that participants use the afternoon on each day to repeat examples from the morning to get more experience on the systems.
Participants require access to a computer (not provided by the course organisers) with Internet connectivity and pre-installed programs to participate in the video meeting of the course (zoom, might work from a browser), to access the HPC systems (ssh or Putty) or the NIRD Toolkit (web browser).
Note, registrations will close a few days before the course dates, but we ask participants to register as soon as possible such that we can arrange access to one of the HPC systems or the NIRD Toolkit well in advance (arranging access during the course won't be very practical as we need to focus on the course itself).
Note 2, in order to use the NIRD Toolkit during the course you will need either a Feide account or a Feide OpenIDP guest account. You can create a Feide OpenIDP guest account at openidp.feide.no. If you're unsure about this, please contact us well in advance - minimum a week before the NIRD Toolkit sessions.
You can always contact us via hjelp.uib.no to get help or send an email to support@metacenter.no.
The courses are free of charge.
UNIX pre-course
Dates: October 19 and 20, 2020
Time: each day 09:00-12:30 with ~ 3 short breaks
Location: video meeting (information will be send to registered participants)
Participants may choose the lectures which they are interested in.
Registration
The course is free of charge and is organised by the Scientific Computing group at UiB IT department.
Registration https://skjemaker.app.uib.no/view.php?id=9042605
Registration closes on October 14th for those attending the pre-course (and the HPC & NIRD Toolkit part). We need a bit time to ensure that everyone has an account on the HPC systems and the NIRD Toolkit in case a participant does not already have one.
Agenda
Day 1 - Hello UNIX - 19.10.2020
Get introduced to UNIX and make your first steps on the command line.
- We will start by outlining the course goals and how we want to conduct the course via the video meeting.
- Then we plan to cover the following topics on day 1 (Note, exact details and order may still be changed.):
- logging into a remote machine using ssh or Putty
- exploring the $HOME folder (commands: ls, cd; brief excursion into program arguments)
- creating a directory (command: mkdir)
- creating a file (command: touch, echo; brief excursion into redirecting program output)
- copying a file (command: cp)
- showing the contents of a file (commands: cat, more, less; brief excursion into file/directory permissions)
- editing the contents of a file (commands: nano)
- Deleting files and directories (commands: rm, rmdir)
- Moving files and directories (command: mv; brief excursion into file names)
- Showing and using commands you have run before (command: history)
- A few exercises for the afternoon to practise the tools discussed on day 1.
Day 2 - Getting around - 20.10.2020
Learn & practise more commands to do your work more efficiently. Get to know what you should practise before the HPC and NIRD Toolkit course starts about a week later.
- We start with a quick recap of what was covered on day 1.
- Then we plan to cover the following topics on day 2 (Note, exact details and order may still be changed.):
- Showing and changing file/directory ownership and permissions (commands: chown, chmod)
- Finding data in files (command: grep; brief excursion into pipes)
- Inspecting parts of files (commands: head, tail)
- Un-/packing and compressing files and directories (commands: tar, gzip, zip)
- Writing shell scripts (understanding shebang, variables, comments, redirection)
- We conclude the pre-course with recommendations on how participants prepare themselves for the HPC and NIRD Toolkit course.
HPC and NIRD Toolkit course
Dates: October 27, 28 and 29, 2020
Time: each day 09:00-12:30 with ~ 3 short breaks
Location: video meeting (information will be send to registered participants)
Participants may choose the lectures which they are interested in.
Registration
The course is free of charge and is organised by the Scientific Computing group at UiB IT department.
Registration (same as for the UNIX pre-course, only needs to be filled once) https://skjemaker.app.uib.no/view.php?id=9042605
Registration closes on October 21st for those attending the HPC and NIRD Toolkit lectures only. We need a bit time to ensure that everyone has an account on the HPC systems and the NIRD Toolkit in case a participant does not already have one.
Agenda
Day 1 - Your first job on an HPC system - 27.10.2020
Get introduced to HPC and submit your job from the command line. (Note, exact details and order may still be changed.)
09:00-09:10 Welcome and overview of HPC resources and services available to students and researchers
09:10-09:20 Overview of the course and its goals, logistics, how to use zoom, chat, etc
09:20-09:30 What is HPC? Why may I need it?
09:30-09:40 Demo (login & first job)
09:40-09:50 break
09:50-10:20 Logging into an HPC system (presentation and hands-on; we use ssh (Linux & macOS) and Putty (Windows))
10:20-10:50 Using software modules (presentation and hands-on; we use the commands module avail, module load and module list)
10:50-11:00 break
11:00-11:20 Creating a directory for the job and copying some files (presentation and hands-on; we use the commands cd, mkdir and cp)
11:20-11:50 Submitting the first job (presentation and hands-on; we use the commands sbatch, squeue, ls and cat)
11:50-12:00 break
12:00-12:10 Recap and take aways from day 1; recommendations for practising during the afternoon
12:00-12:30 Q&A
Day 2 - How to organise your work on an HPC system - 28.10.2020
Learn & practise more commands to use an HPC system more efficiently. (Note, exact details and order may still be changed.)
09:00-09:40 Queues, Quality-of-Service and resource requirements (presentation and hands-on: we look at the documentation, use the commands squeue and time, look at job outputs and adjust the job script)
09:40-09:50 break
09:50-10:30 More about managing jobs (presentation and hands-on; we use commands scancel, srun/salloc and the slurmbrowser)
10:30-10:40 break
10:40-11:20 File systems, data transfers and disk usage (presentation and hands-on: we look at the documentation, use the command scp and the program winscp as well as dusage)
11:20-11:30 break
11:30-12:00 Best practices on HPC systems (presentation + discussion: topics may include login nodes, module environment, python environments)
12:00-12:10 How to start using an HPC system? (presentation: recap & take aways from day 1 and day 2)
12:10-12:30 Q&A
Day 3 - Hello NIRD Toolkit and How to get help - 29.10.2020
Get introduced to the NIRD Toolkit and learn about how to get help when you got stuck. (Note, exact details and order may still be changed.)
09:00-09:50 Introduction to the NIRD Toolkit (overview, configure services, resources, access, storage, adding software )
09:50-10:00 break
10:00-10:20 Different ways of plotting in Jupyter notebooks (presentation and hands-on)
10:20-10:30 Minio (transfer files)
10:30-10:40 Jupyterhub (collaboration)
10:40-10:50 Deep learning tools (small example with GPU)
10:50-11:00 break
11:00-11:50 How to get help if you get stuck (presentation and hands-on: where to find documentation, how does the support team work, how to write effective support requests)
11:50-12:00 break
12:00-12:10 Recap and take aways from day 3
12:10-12:20 Outlook to future courses
12:20-12:30 Q&A
Preparing your machine for the course
Linux / macOS
Any laptop or desktop running Linux or macOS should already have the necessary tools - a Terminal and ssh - installed.
You can verify this by launching a Terminal (a program like a browser) and typing a specific command into it (see step 3).
Launching a Terminal on, for example, Ubuntu version 20.04.1 LTS
If you have a different version of Ubuntu or of the desktop shown in screenshots below or even use a different Linux distribution, you need to figure out how to start a program, and in particular a Terminal. Google may help. If you get stuck, just contact us via email.
1. Click on the icon with the 9 dot matrix in the lower left corner in the screenshot below.
2. Type the word Terminal into the search field (in the upper center in the screenshot below) & click on the icon of the Terminal (shown under the search field).
Verify that ssh is installed
3. Type which ssh as shown in the screenshot below and hit the ENTER/RETURN key to execute the command. You should see a line such as /usr/bin/ssh (the exact line may be different on your machine). If you see a line starting with /usr/bin/which: no ssh in (...) then ssh may not be installed. Best is to contact us for help in this case. Note, the name of the program is case-sensitive - ssh is written in lowercase letters - any variant such as Ssh, SSH or similar will likely result in the message that there is no such program on your system.
Launching a Terminal on macOS
1. Start the Launchpad and type the word Terminal into the search field (upper center in the screenshot) and click the Terminal icon.
Verify that ssh is installed
2. Type which ssh as shown in the screenshot below and hit the ENTER/RETURN key to execute the command. You should see a line such as /usr/bin/ssh (the exact line may be different on your machine). If you see a line starting with /usr/bin/which: no ssh in (...) then ssh may not be installed. Best is to contact us for help in this case. Note, it seems macOS tolerates mixing uppercase and lowercase letters for the program name.
Windows
If your operating system is Windows, you may need to install the tool PuTTY on your system. We are using PuTTY to login into the HPC cluster Saga. If you are used to another program, for logging in into another machine, feel free to use that also for the course (we may not be able to help you in case of problems though -- if you are unsure, you can follow the steps below to install PuTTY).
1. We first check if PuTTY is already installed by searching for it. Click on the magnifier symbol in the lower left (indicated by the red arrow).
2. Type the word PuTTY into the search field and click on PuTTY under best match (if it is not found you likely have to install it first -- see below for instructions how to install it).
3. The PuTTY application should be shown as in the screenshot below. If so, you have the necessary tool installed.
Installing PuTTY on your Windows machine
We describe how you can install PuTTY even if you don't have administrator permissions on your system.
1. Open the web page Download PuTTY in a web browser.
2. Scroll to the box named Alternative binary files and download the 64-bit version for putty.exe (direct link: putty.exe).
3. Open the folder to which the file was downloaded. In the screenshot below you see how you can do this when using the browser Google Chrome (this is likely different when you are using another browser).
4. In the folder, move the pointer over the downloaded file and do a right-click to open a context menu. In the context menu, select first "Send to", then "Desktop (create shortcut)" and click. This should create a new icon on your desktop for the PuTTY application.
5. Double-click on the newly created icon on your desktop to open the PuTTY application.
