Skip to content

Getting an HPC Account#

Getting ready to request an account#

All users of AUGent can request an account on the HPC, which is part of the Flemish Supercomputing Centre (VSC).

See HPC policies for more information on who is entitled to an account.

The VSC, abbreviation of Flemish Supercomputer Centre, is a virtual supercomputer centre. It is a partnership between the five Flemish associations: the Association KU Leuven, Ghent University Association, Brussels University Association, Antwerp University Association and the University Colleges-Limburg. The VSC is funded by the Flemish Government.

There are two methods for connecting to HPC-UGent infrastructure:

The web portal offers a convenient way to upload files and gain shell access to the HPC-UGent infrastructure from a standard web browser (no software installation or configuration required).

If you would like use a terminal with SSH as this gives you more flexibility continue reading. However if you prefer to use the web portal, you can skip ahead to the following section: Applying for the account. Once you have successfully obtained an account, you can then delve into the details of utilizing the HPC-UGent web portal by reading Using the HPC-UGent web portal.

The HPC-UGent infrastructure clusters use public/private key pairs for user authentication (rather than passwords). Technically, the private key is stored on your local computer and always stays there; the public key is stored on the HPC. Access to the HPC is granted to anyone who can prove to have access to the corresponding private key on his local computer.

How do SSH keys work?#

  • an SSH public/private key pair can be seen as a lock and a key

  • the SSH public key is equivalent with a lock: you give it to the VSC and they put it on the door that gives access to your account.

  • the SSH private key is like a physical key: you don't hand it out to other people.

  • anyone who has the key (and the optional password) can unlock the door and log in to the account.

  • the door to your VSC account is special: it can have multiple locks (SSH public keys) attached to it, and you only need to open one lock with the corresponding key (SSH private key) to open the door (log in to the account).

Since all VSC clusters use Linux as their main operating system, you will need to get acquainted with using the command-line interface and using the terminal (see tutorial). To open a Terminal window in macOS, open the Finder and choose

>> Applications > Utilities > Terminal

Before requesting an account, you need to generate a pair of ssh keys. One popular way to do this on macOS is using the OpenSSH client included with macOS, which you can then also use to log on to the clusters.

Test OpenSSH#

Secure Shell (ssh) is a cryptographic network protocol for secure data communication, remote command-line login, remote command execution, and other secure network services between two networked computers. In short, ssh provides a secure connection between 2 computers via insecure channels (Network, Internet, telephone lines, ...).

"Secure" means that:

  1. the User is authenticated to the System; and

  2. the System is authenticated to the User; and

  3. all data is encrypted during transfer.

OpenSSH is a FREE implementation of the SSH connectivity protocol. macOS comes with its own implementation of OpenSSH, so you don't need to install any third-party software to use it. Just open a terminal window and jump in!

On all popular Linux distributions, the OpenSSH software is readily available, and most often installed by default. You can check whether the OpenSSH software is installed by opening a terminal and typing:

$ ssh -V
OpenSSH_7.4p1, OpenSSL 1.0.2k-fips 26 Jan 2017

To access the clusters and transfer your files, you will use the following commands:

  1. ssh-keygen: to generate the SSH key pair (public + private key);

  2. ssh: to open a shell on a remote machine;

  3. sftp: a secure equivalent of ftp;

  4. scp: a secure equivalent of the remote copy command rcp.

Generate a public/private key pair with OpenSSH#

A key pair might already be present in the default location inside your home directory. Therefore, we first check if a key is available with the "list short" ("ls") command:

ls ~/.ssh

If a key-pair is already available, you would normally get: 

authorized_keys     id_rsa      id_rsa.pub      known_hosts

Otherwise, the command will show:

ls: .ssh: No such file or directory

You can recognise a public/private key pair when a pair of files has the same name except for the extension ".pub" added to one of them. In this particular case, the private key is "id_rsa" and public key is "id_rsa.pub". You may have multiple keys (not necessarily in the directory "~/.ssh") if you or your operating system requires this. Be aware that your existing key pair might be too short, or not the right type.

You will need to generate a new key pair, when:

  1. you don't have a key pair yet

  2. you forgot the passphrase protecting your private key

  3. your private key was compromised

  4. your key pair is too short or not the right type

For extra security, the private key itself can be encrypted using a "passphrase", to prevent anyone from using your private key even when they manage to copy it. You have to "unlock" the private key by typing the passphrase. Be sure to never give away your private key, it is private and should stay private. You should not even copy it to one of your other machines, instead, you should create a new public/private key pair for each machine.

ssh-keygen -t rsa -b 4096

This will ask you for a file name to store the private and public key, and a passphrase to protect your private key. It needs to be emphasised that you really should choose the passphrase wisely! The system will ask you for it every time you want to use the private key that is every time you want to access the cluster or transfer your files.

Without your key pair, you won't be able to apply for a personal VSC account.

Using an SSH agent (optional)#

Most recent Unix derivatives include by default an SSH agent to keep and manage the user SSH keys. If you use one of these derivatives you must include the new keys into the SSH manager keyring to be able to connect to the HPC cluster. If not, SSH client will display an error message (see Connecting) similar to this:

Agent admitted failure to sign using the key. 
Permission denied (publickey,gssapi-keyex,gssapi-with-mic).

This could be fixed using the ssh-add command. You can include the new private keys' identities in your keyring with:

ssh-add

Tip

Without extra options ssh-add adds any key located at $HOME/.ssh directory, but you can specify the private key location path as argument, as example: ssh-add /path/to/my/id_rsa.

Check that your key is available from the keyring with:

ssh-add -l

After these changes the key agent will keep your SSH key to connect to the clusters as usual.

Tip

You should execute ssh-add command again if you generate a new SSH key.

Applying for the account#

Visit https://account.vscentrum.be/

You will be redirected to our WAYF (Where Are You From) service where you have to select your "Home Organisation".

image

Select "UGent" in the dropdown box and optionally select "Save my preference" and "permanently".

Click Confirm

You will now be taken to the authentication page of your institute.

You will now have to log in with CAS using your UGent account.

You either have a login name of maximum 8 characters, or a (non-UGent) email address if you are an external user. In case of problems with your UGent password, please visit: https://password.ugent.be/. After logging in, you may be requested to share your information. Click "Yes, continue".

image

After you log in using your UGent login and password, you will be asked to upload the file that contains your public key, i.e., the file "id_rsa.pub" which you have generated earlier. Make sure that your public key is actually accepted for upload, because if it is in a wrong format, wrong type or too short, then it will be refused.

This file has been stored in the directory "~/.ssh/".

Tip

As ".ssh" is an invisible directory, the Finder will not show it by default. The easiest way to access the folder, is by pressing Cmd+Shift+G (or Cmd+Shift+.), which will allow you to enter the name of a directory, which you would like to open in Finder. Here, type "~/.ssh" and press enter.

After you have uploaded your public key you will receive an e-mail with a link to confirm your e-mail address. After confirming your e-mail address the VSC staff will review and if applicable approve your account.

Welcome e-mail#

Within one day, you should receive a Welcome e-mail with your VSC account details.

Dear (Username), 
Your VSC-account has been approved by an administrator.
Your vsc-username is vsc40000

Your account should be fully active within one hour.

To check or update your account information please visit
https://account.vscentrum.be/

For further info please visit https://www.vscentrum.be/user-portal

Kind regards,
-- The VSC administrators

Now, you can start using the HPC. You can always look up your VSC id later by visiting https://account.vscentrum.be.

Adding multiple SSH public keys (optional)#

In case you are connecting from different computers to the login nodes, it is advised to use separate SSH public keys to do so. You should follow these steps.

  1. Create a new public/private SSH key pair from the new computer. Repeat the process described in section Generate a public/private key pair with OpenSSH.

  2. Go to https://account.vscentrum.be/django/account/edit

  3. Upload the new SSH public key using the Add public key section. Make sure that your public key is actually saved, because a public key will be refused if it is too short, wrong type, or in a wrong format.

  4. (optional) If you lost your key, you can delete the old key on the same page. You should keep at least one valid public SSH key in your account.

  5. Take into account that it will take some time before the new SSH public key is active in your account on the system; waiting for 15-30 minutes should be sufficient.

Computation Workflow on the HPC#

A typical Computation workflow will be:

  1. Connect to the HPC

  2. Transfer your files to the HPC

  3. Compile your code and test it

  4. Create a job script

  5. Submit your job

  6. Wait while

    1. your job gets into the queue

    2. your job gets executed

    3. your job finishes

  7. Move your results

We'll take you through the different tasks one by one in the following chapters.