epoc

Setup required to run an EPOC ensemble member

In order to run an EPOC workflow, you must have the following accounts:

• Archer2 and Puma2
• MOSRS
• Jasmin

You will then need to setup the following:

• Archer2 and puma2 environments to run the UM
• Globus transfer from Archer2 to Jasmin
• Epoc GWS access on Jasmin:
  • Request epoc GWS access via the Jasmin accounts portal.
  • Then contact Jon to be made a GWS deputy
  • Then contact the Jasmin helpdesk for tape access.
• Cylc submission to Jasmin (see below)
• Jasmin transfer cache (see below)
• Data migration to Jasmin tape archive (see below)

Cylc submission to Jasmin

In order to submit jobs to Jasmin from puma2 (for jdma and monitor tasks), you need to configure your ssh settings to connect to the Jasmin sci servers.

1. Add the following to your .ssh/config file on PUMA2.

Host login-0?.jasmin.ac.uk
    User <JAMSIN-USERNAME>
    IdentityFile ~/.ssh/<JASMIN-SSH-KEY>
    ForwardAgent yes

Host sci-vm-0? sci-ph-0?
    User <JASMIN-USERNAME>
    IdentityFile ~/.ssh/<JASMIN-SSH-KEY>
    ForwardAgent yes
    ProxyJump login-03.jasmin.ac.uk

• Replace <JASMIN-USERNAME> with your Jasmin username, e.g. aosprey.
• Replace <JASMIN-SSH-KEY> with the name of your Jasmin key, e.g. id_rsa_jasmin.

2. Add your Jasmin ssh-key to your ssh-agent:

• Run ssh-add ~/.ssh/<JASMIN-SSH-KEY>
• You will be prompted for your Jasmin ssh key passphrase.

3. Test connection to Jasmin:

• Try for example ssh sci-vm-02
• You should be logged into the Jasmin sci node without prompt for your passphrase.

4. Add path to Rose/Cylc to your ~/.bash_profile on JASMIN:

if [[ $(hostname) = sci*.jasmin.ac.uk || $(hostname) = cylc*.jasmin.ac.uk || $(hostname) = host*.jc.rl.ac.uk ]]; then
  # Rose/cylc on jasmin-sci & Lotus nodes
  export PATH=/apps/jasmin/metomi/bin:$PATH
fi

Jasmin transfer cache

The Jasmin transfer cache (XFC) is used in the EPOC workflows to stage data before migration to elastic tape, avoiding group workspace.

To setup an XFC space:

1. First install the XFC client
2. Then follow the initialisation steps

Data migration to Jasmin tape archive

To setup your environment for migration of data to Jasmin Elastic Tape as part of the UM workflow:

1. Login to one of the Jasmin sci servers
2. Load the jdma client:

   . ~umshared/venvs/jdma_venv_py3/bin/activate
   

3. Follow the setup steps in the JDMA documentation