Connecting to Your Cluster

When you submit a job to the Rescale platform, an HPC cluster is provisioned according to your job's hardware settings. You can interact with this cluster on Rescale in two main ways:

  • In-browser SSH - uses the browser interface of the Rescale platform.
  • SSH on local workstation - connect to your cluster using any terminal on your workstation.

This article describes the steps for both the methods.

The most straightforward method is to use Rescale's in-browser SSH. This method works without requiring any additional authentication or setup. This method is useful for users without easy access to a terminal for quickly checking cluster status, disk usage, or any other interactive function on the cluster.

Once your job has been submitted and the cluster has started, you will see a window appear below the "Live Tailing" section.

ssh.png

Click on the IP address of the compute node to launch the in-browser SSH terminal. You can now interact with the running cluster using this terminal, right from the web UI.

terminal.png

Due to browser restrictions, you will need to make use of our copy/paste tool in order to copy/paste text in or out of the in-browser SSH terminal. You can access this tool by clicking on the tab with 3 dots at the right hand side of the window.

copy-tool.png

This tool allows you to transfer clipboard data from your workstation to the cluster, and vice versa. Please follow the instructions on the pop up window.

If you would rather connect to your cluster using your own terminal on your machine, you must configure an SSH key in your Job settings prior to submitting your job. Instructions to for setting up your SSH key and connecting to your cluster can be found in the sections below.

Before connecting to the cluster, an existing public/private SSH keypair should exist or be newly created. The public key should then be added to your Rescale Account on the the Job Settings page. Please follow the steps given below depending on the type of workstation


Using PuTTY (Windows)

Generating you SSH keys

You can use PuTTYgen to generate your public/private keypair. PuTTY is a free open-source terminal emulator that functions much like the Terminal application in Mac OSX. You can download PuTTY from here. The PuTTYgen utility also comes with the PuTTY client installation.

To generate an SSH key with PuTTYgen, follow these steps:

  • Launch the PuTTYgen program
  • For Type of key to generate, select SSH-2 RSA under Key in the menu bar
  • Click the Generate button
  • Move your mouse in the area below the progress bar to provide randomness for the key generation algorithm. When the progress bar is full, PuTTYgen has generated your key pair

putty-generate-key.png

  • Click the Save private key button to save the private key. Please save it in an accessible location.

  • It is recommended that you use Pageant to save your SSH key for easy access and use within your command prompt



Copy the Public Key to the Clipboard

putty-copy-key.png

  • Right-click in the text field labeled Public key for pasting into OpenSSH authorized_keys file and choose Select All
  • Right-click again in the same text field and choose Copy


Configuring your Job Settings
  • On your Rescale platform account page, click on the <user>@rescale.com button at the top right corner
  • Select Account in the drop-down menu
  • Click on Job Settings option on the left side of the page

AccessJobSettings.png

  • Once you have copied the contents of your public key, paste the contents of the public key into the SSH Public Key field. Click Save and you are able to SSH into all your future clusters

  • The Access Restriction (CIDR range) field allows you to select which IP addresses can access your cluster. The options Everywhere or My Current IP Address can be set automatically from the pull-down menu. You can also specify your own custom Classless Inter-Domain Routing (CIDR) rule by editing the Access Restriction (CIDR) range text field directly



Connecting to Your Cluster
  • In Windows, you can use PuTTY as an SSH client to connect to your cluster on Rescale

  • When you provision a cluster, the Job Logs window on the Status tab will display a message showing the job's username on the system and the public IP address of the master node of the cluster. The message will look like this:

job-logs.png

  • In this example, the job username is uprod_qpWia and the public IP address is 107.23.132.81. You will need this information when using PuTTY. To log in to your machine, follow these instructions:

  • Launch the PuTTY program

  • Type the public IP address of your Rescale cluster in the Host Name (or IP address) field

  • In the Category panel on the left, expand the SSH section to expose the authentication (Auth) settings

putty-config-connect.png

  • Click Auth to display settings controlling SSH authentication.
  • Click Browse under Private key file for authentication and select the private key file you created previously

putty-config-auth.png

Based on the the type of connection to the cluster that you need for your workflow, you may need either a regular SSH session or an SSH tunnel connection:

For a regular SSH session

  • Click Open to start a PuTTY terminal session

For an SSH tunnel connection

  • Click Tunnels to display port forwarding settings.

  • Type the Source port and Destination in the fields provided; this is specific to the server program and is provided to you:

    • Source port: ####
    • Destination: localhost:####
  • Click Add to forward this port to your ssh tunnel configuration

  • Click Open to start a Putty terminal session

    ssh-tunnel.png



Terminal session and Logging in

When you connect to your cluster for the first time, you may see a PuTTY Security Alert warning you that the host's key is not cached in the the registry. Do not be alarmed; this is a normal warning. Click yes to continue.

Follow these steps to log in:

  • At the login as: prompt, type the username, for instance uprod_qpWia and press Enter
login as: uprod_qpWia
  Authenticating with public key "rsa-key-1234"
  Passphrase for key "rsa-key-1234":
  • You should now be logged into your machine


Connecting from a Terminal (Linux/MacOS)

Generating you SSH keys
On Mac OSX and Linux, SSH key generation utilities can be run from the command line of a terminal. To open the Mac OSX Terminal, open the Finder and choose Utilities from the Go menu. Find the Terminal application in the Utilities window and double-click it. The Terminal window opens with the command line prompt displaying the name of your machine and your username.

First check for existing keys on your computer. From the command line of the terminal type:

$ ls -al ~/.ssh

This lists the files in your .ssh directory, however, if you have a new Mac OSX or Linux installation, the .ssh directory may not yet exist.

The default public key file names are:

  • id_dsa.pub
  • id_ecdsa.pub
  • id_ed25519.pub
  • id_rsa.pub

If you wish to use an existing key file for your Rescale account, you can skip to the next step - Copy contents of your public key

Generate a new SSH key

By default, keys for all identities are added to the directory:

  • /Users/_yourname_/.ssh on Mac OSX
  • /home/_yourname_/.ssh on Linux

If you have an existing identity (public/private key pair) in this directory that you want to use on Rescale, skip this step. To create a new default identity:

  1. Open a terminal session on your local system
  2. Enter ssh-keygen at the command line
  3. The command prompts you for a file to save the key in. If the .ssh directory doesn't exist, the system creates one for you
  4. Accept the default location

The ssh-keygen command creates your default identity with its public and private keys. The whole interaction will look like this:

[mairi@centos ~]$ ssh-keygen
Generating public/private rsa key pair.
Enter file in which to save the key (/home/mairi/.ssh/id_rsa):
Enter passphrase (empty for no passphrase):
Enter same passphrase again:
Your identification has been saved in /home/mairi/.ssh/id_rsa.
Your public key has been saved in /home/mairi/.ssh/id_rsa.pub.
The key fingerprint is:
ed:88:95:91:38:e4:5e:ff:d6:73:70:f7:43:3e:f6:3b mairi@centos
The key's randomart image is:
+--[ RSA 2048]----+
|      .          |
|     o . .       |
|      + +        |
|     . o =       |
|      . S o   ..o|
|       o o . .ooo|
|      . . . o o=o|
|           .  .E+|
|               .+|
+-----------------+
  • You can, of course, save your new key pair to an alternative, i.e. non-default, location in your file system if you wish. By default, ssh-keygen generates 2048-bit RSA keys

  • ssh-keygen generates a public key and a private key. If not specified, the default public key will be saved as id_rsa.pub and the private key as id_rsa in your ~/ssh folder. Ensure that the ~/ssh is only accessible by you by setting the proper permissions to that folder:

    chmod 700 ~/.ssh


Copy the contents of your public key
  • On Linux, copy the contents of your chosen public key file by opening it in a text editor or by running the cat command (in this case the public key is stored in the id_rsa.pub file):
$ cat ~/.ssh/id_rsa.pub
  • On Mac OSX, you can use the command pbcopy in the terminal to copy the contents of your public key file to the clipboard as follows:
$ pbcopy < ~/.ssh/id_rsa.pub



Configuring your Job Settings
  • On your Rescale platform account page, click on the <user>@rescale.com button at the top right corner
  • Select Account in the drop-down menu
  • Click on Job Settings option on the left side of the page

AccessJobSettings.png

  • Once you have copied the contents of your public key, paste the contents of the public key into the SSH Public Key field. Click Save and you are able to SSH into all your future clusters

  • The Access Restriction (CIDR range) field allows you to select which IP addresses can access your cluster. The options Everywhere or My Current IP Address can be set automatically from the pull-down menu. You can also specify your own custom Classless Inter-Domain Routing (CIDR) range rule by editing the Access Restriction (CIDR range) text field directly




Connecting to your cluster

In Mac OSX and Linux, you can use the terminal to log into your machine.

When you provision a cluster, the Job Logs window on the Status tab will display a message showing the job's username on the system and the public IP address of the master node of the cluster. The message will look like this:

job-logs.png

In this example, the job username is uprod_qpWia and the public IP address is 107.23.132.81. You will need this information to log in to the cluster.

  • To log in to the cluster from a Terminal window, call ssh using a command of the form:
$ ssh -i ~/.ssh/<private key file> uprod_qpWia@107.23.132.81
  • Here, <private key file> corresponds to the previously genrated private key, id_rsa (Not id_rsa.pub)
  • If your privatekey file in an alternative directory, substitute the correct file path after the -i flag
  • The ssh command will log you in to your machine as the job user. If you defined a passphrase for your SSH key, the operating system will prompt you for it.

The terminal prompt should now show that you have logged in as the job user