User Serial

Running serial jobs on SciNet

You should not submit purely serial jobs to the queue (on either GPC or TCS), since it would mean wasting the computational power of 7 (or 31, on the TCS) cpus while the jobs is running. While we encourage you to try and parallelize your code, sometimes it is beneficial to run several serial codes at the same time. Note that because TCS is machine specialized for parallel computing, you should only use the GPC for concurrent serial runs.

Serial jobs of similar duration

In the FAQ it is explained how to run bunches of eight jobs at the same time, so that all cpus are kept busy. This is a good solution if:

• the jobs have the same or similar duration
• all eight jobs fit in memory simultaneously
• there are no memory contention issues
• there are not too many jobs: if you have many jobs to bunch, eventually some of these will be unbalanced.

Serial jobs of varying duration

If you have a lot (50+) of relatively short serial jobs to do, and you know that eight jobs fit in memory without memory issues, the following strategy in your submission script maximizes the cpu utilization: <source lang="bash">

1. !/bin/bash
2. MOAB/Torque submission script for multiple, dynamically-run
3. serial jobs on SciNet GPC
5. PBS -l nodes=1:ppn=8,walltime=1:00:00
6. PBS -N serialdynamic

1. DIRECTORY TO RUN - $PBS_O_WORKDIR is directory job was submitted from

cd $PBS_O_WORKDIR

1. COMMANDS ARE ASSUMED TO BE SCRIPTS CALLED job* in the directory that the job was submitted from

listofjobs = `ls job*`

1. THESE SCRIPTS ARE ASSUMED TO CALL THE SAME MAIN EXECUTABLE 'myrun', WHOSE NAME WILL POP UP IN 'ps'

psname = 'myrun'

1. (SEVERAL EXECUTABLES COULD BE SPECIFIED, SEPARATED BY COMMAS)

1. EXECUTE COMMANDS

for serialjob in $listofjobs do

   sleep 5
   njobs=`ps -C $psname|wc -l`
   while [ $njobs -gt 8 ]
   do
       sleep 5
       njobs=`ps -C $psname|wc -l`
   done
   $serialjob &

done wait </source> Notes:

• This is the simplest case of dynamically run serial jobs.
• Doing many serial jobs often entails doing many disk reads and writes, which can be detrimental to the performance. In that case, running off the ramdisk may be an option.
• When using a ramdisk, make sure you copy your results from the Ramdisk back to the scratch after the runs, or when the job is killed because time has run out.