hpc documentation

This page is read only. You can view the source, but not change it. Ask your administrator if you think this is wrong.
====== Connecting to Caviness from Mac/Linux ======

//This section uses the wiki's [[:#documentation-conventions|documentation conventions]].//

To initiate an SSH connection from a Mac/Linux system to the cluster's login (head) node, use
the generic command 

''ssh''<<//options//>> <<//HPC_username//>>@<<//HPC_hostname//>>

For example,
<code bash>
    ssh -Y traine@caviness.hpc.udel.edu
</code>

This **ssh** command is on your local system.  This example assumes you have a local command line interface, and are running X-Windows, for example, Mac/UNIX/Linux or cygwin/X (with the openssh client).


====== SSH Command line mode ======

The secure shell (SSH) is a network protocol to allow secure logins and file transfers to a remote host.  On a PC it is frequently bundled with a terminal emulator, as a place to type commands on the remote host.  Unix systems, include Mac, come with terminal emulator such as xterm, gnome-terminal or mac terminal.  IT recommends you start with the built-in terminal application of you local system, and use the command line SSH tools.   The three main tools are:

  - **SSH** - to start a secure shell connection, and typically login to the remote machine.  While connected to the remote machine, commands on the terminal will be remote commands.
  - **SFTP** - to open a session on the remote machine to browse the filesystem, and transfer files.  The commands in an SFTP session are very similar to ftp commands.
  - **SCP** - to copy files or directories directly, when you know the full path of the file name.  This one command is similar to the rcp command, and is particularly useful in script, since both the source a destination files need to be completely specified.

All of these commands use the hidden SSH directory to store identity keys, host keys and your configuration file.  There are a few SSH command utilities to help you manage your SSH directory.  The all begin with the ''ssh-'' prefix.
===== SSH directory =====
The SSH commands are Unix commands and they use a hidden directory in your home directory //~/.ssh// to store important files:
  * public/private key files (your identity)
  * List of known host keys (host identities you know)
  * List of authorized keys (identities you want to allow in)
  * Your personal configuration file
It is very important to have this directory and all the files permitted properly.  Others should not be able to see your secret information (such as the private key).  We suggest you let the SSH commands created the files as needed.  If the file is not permitted properly, either it will be ignored or you will get an error message.

To connect to a remote host, you must first exchange identity keys.  The remote host has a system level SSH directory ///etc/ssh// as well as the SSH directory in your home directory on the remote host.  You must append the appropriate id keys to the end of the key lists - known hosts and authorized keys.

^^   Key exchanges   ^^^^^^
^^ On Local Host ^^  ^^ On Remote Host^^
|| ~/.ssh/known_hosts || <<---|| /etc/ssh public key ||
|| ~/.ssh public key file || --->> || ~/.ssh/authorized_keys ||

There are SSH tools to help with the key exchange.  When these tools work, you will not need to know the details of file names and directory locations.
===== Initializing SSH directory=====
We will map out a basic method for getting files in your hidden SSH directory.  These SSH commands will create new directories and files, as needed, and tell you when new files are created.  Commands to create a SSH directory with public/private keys, adding the keys to the SSH agent, and finally sending the public key to a remote SSH directory.

  -''ssh-keygen'' -- to create public/private key pair
  -''ssh-add''   -- to add public key to SSH agent
  -''ssh-copy-id'' //account//@//remote_host// -- to copy public key to remote host
  -''modify your .ssh/config file'' -- optional step to create an SSH service

Repeat the last two steps command for each remote host where you have an account with the home directory setup for ssh.


==== Adding keys to SSH agent ====

Second, you want to add your keys to your SSH agent.

    ssh-add

Here is an example session output where you enter the same passphrase you entered above (twice).
<code>
[trainf@login00 ~]$ ssh-add
Enter passphrase for /home/1200/.ssh/id_ecdsa: 
Identity added: /home/1200/.ssh/id_ecdsa (/home/1200/.ssh/id_ecdsa)
</code>

You can list the keys in your SSH agent with:
<code>
[trainf@login00 ~]$ ssh-add -l
2048 ba:b5:b6:0e:d9:71:f2:f9:9e:e7:17:39:57:6b:6b:d6 /home/1200/.ssh/id_ecdsa (RSA)
</code>

==== Adding keys to known host file ====

To connect to a remote you must know the key and have it stored in your known host file.  The
system will do this exchange automatically, the first time you try to connect.

<code>
[trainf@centos .ssh]$ ssh notme@caviness.hpc.udel.edu
The authenticity of host 'caviness.hpc.udel.edu (128.175.98.19)' can't be established.
RSA key fingerprint is 19:28:fd:da:66:48:f2:8b:e3:ca:80:7e:b1:ec:b0:2c.
Are you sure you want to continue connecting (yes/no)? yes
Warning: Permanently added 'caviness.hpc.udel.edu,128.175.98.19' (RSA) to the list of known hosts.
</code>
==== Copy ID keys to remote host ====
Third, you want to copy the key in your SSH agent to a remote host.  You need an account and password.  

    ssh-copy-id account@remote_host


<note tip>**Not always available** --  The ''ssh-copy-id'' is not installed with all SSH tool sets.  You will get
a "command not found" message with the above command.  For example, it is not included on the Mac distribution.  
For a one time copy of your id, after a local ssh-add, use the command:

  ssh -A account@remote_host 'ssh-add -L >> .ssh/authorized_keys'

to append the keys in your agent to the end of your remote authorized keys file.

Or you can copy the relatively simple shell executable (and man page) from a host that does have it installed.  Both Caviness and Farber have the script in **''/usr/bin/ssh-copy-id''** and the man page in **''/usr/share/man/man1/ssh-copy-id.1.gx''**.

</note>
Here is an example session using the ssh-copy-id command.
Answer
"yes", after you verify the host name.  Then you will be asked for your password on the remote host.
<code>
[trainf@login00 ~]$ ssh-copy-id jdoe@centos.css.udel.edu
The authenticity of host 'centos.css.udel.edu (128.175.68.83)' can't be established.
RSA key fingerprint is d2:f9:58:79:08:d6:18:22:3f:23:ac:4b:f4:a7:b9:3b.
Are you sure you want to continue connecting (yes/no)? yes
Warning: Permanently added 'centos.css.udel.edu,128.175.68.83' (RSA) to the list of known hosts.
jdoe@centos.css.udel.edu's password: 
Now try logging into the machine, with "ssh 'jdoe@centos.css.udel.edu'", and check in:

  .ssh/authorized_keys

to make sure we haven't added extra keys that you weren't expecting.
</code>

In this case, you have "permanently added" the host name to your list of known hosts, and you have added
keys to your remote ''.ssh/authorized_keys'' file (one key per line).
Check the file permissions and dates with commands:

<code bash>
ls -dl .ssh .ssh/*
ssh jdoe@centos.css.udel.edu 'ls -dl .ssh .ssh/*'
</code>


==== Create SSH services ====

An SSH service is a short cut that is saved you ssh configuration file.  Collect this information for each service
you want to add:

^^ Short name ^^ Account name ^^ Full server name ^^ Options you want to add^^
|| caviness || trainf || caviness.hpc.udel.edu || Trusted X11 tunnelling & arcfour ciphers ||
|| vm || jdoe || centos.css.udel.edu || no password authentication ||

For these two services add these lines to your ''.ssh/config'' file

<code>
Host caviness
  User trainf
  Hostname caviness.hpc.udel.edu
  ForwardX11 yes
  ForwardX11Trusted yes
  Ciphers arcfour256,arcfour128
Host vm
  User jdoe
  Hostname centos.css.udel.edu
  PasswordAuthentication no
</code>

With these lines in your ''.ssh/config'' file, you can logon with the command:

<code>
ssh caviness 
</code>
    
and you can copy files from you current local directory to your remote home directory with the command:

<code>
scp localfile.txt caviness:remotefile.txt
</code>