HPC Cluster

A detailed look at the job script

In order to understand job scripts let's look again at an extended example, this time with line numbers:

1 #!/bin/bash --login
2 #$ -cwd
3 #$ -N hello_world
4 #$ -j y
5 #$ -o hello_world.out
6 #$ -l h_rt=120000
7 #$ -l mem_free=4G
8 #$ -m e
9 #$ -M meine.mail@adresse

10 echo "Hello world!"

The scripts consists of three parts: the hashbang (line 1), the header (lines 2-9) and the actual script (from line 10 on).

The hashbang determines (like in other Unix shell scripts) which interpreter the script should be executed with. In this example it's the "bash"-Shell.
One could use other scripting languages like csh, perl or python by specifying their path on line 1.
For scripts executed with 'bash' it's advisable to add the '--login' option due to the way the bash-shell reads initialization files.

Lines beginning with '#$' are interpreted by the batch system and are scanned for embedded options. The option '-cwd' on line 2, for instance, specifies that the jobs is to be started in the directory from which the job was submitted.

The name of the job is specified by the '-N' option (line 3). It appears for instance in the output of the 'qstat' command.

The '-j y' option (line 4) requests that output to both stdout and stderr should be written to the same output file, the name of which is given by the '-o' option (line 5).

The option '-l'  (lines 6 and 7) describe resource requests and can appear multiple times. For a job to start, all such requests must be met. A job may wait indefinitely if not.
The resource 'h_rt'  on line 6 gives an upper limit for the run time of the job (in seconds) after which the job will be terminated by the batch system. 'h_rt' should always be specified for any jobs other than really short test jobs as the default is currenty 10 minutes. Currently, the maximum run time is 792000 seconds (= 220 hours). Jobs with shorter runtime are likely to start sooner.

mem_free declares the estimated upper limit of the required memory.

Further options can be found on the man page of the 'qsub' command.