Gamess

GAMESS(US)

The GAMESS version January 12, 2009 R3 was built using the Intel v11.1 compilers and v3.2.2 MPI library, according to the instructions in http://software.intel.com/en-us/articles/building-gamess-with-intel-compilers-intel-mkl-and-intel-mpi-on-linux/

The required build scripts - comp, compall, lked - and run script - rungms - were modified to account for our own installation. In order to build GAMESS one first must ensure that the intel and intelmpi modules are loaded ("module load intel intelmpi"). This applies to running GAMESS as well. The module "gamess" must also be loaded in order to run GAMESS ("module load gamess").

The modified scripts are in the file /scinet/gpc/src/gamess-on-scinet.tar.gz

Running GAMESS

- Make sure the directory /scratch/$USER/gamess-scratch exists (the $SCINET_RUNGMS script will create it if it does not exist)

- Make sure the modules: intel, intelmpi, gamess are loaded (in your .bashrc: "module load intel intelmpi gamess").

- Create a torque script to run GAMESS. Here is an example:

- The GAMESS executable is in $SCINET_GAMESS_HOME/gamess.00.x - The rungms script is in $SCINET_GAMESS_HOME/rungms (actually it is $SCINET_RUNGMS)

- For multinode runs, use the $SCINET_RUNGMS_IB script to run on the InfiniBand portion of the GPC

- The rungms script takes 4 arguments: input file, executable number, number of compute processes, processors per node

For example, in order to run with the input file /scratch/$USER/gamesstest01, on 8 cpus, and the default version (00) of the executable on a machine with 8 cores:

  # load the gamess and other required modules in .bashrc
  module load gcc intel intelmpi gamess  

  # run the program
  $SCINET_RUNGMS /scratch/$USER/gamesstest01 00 8 8

Here is a sample torque script for running a GAMESS calculation, on a single 8-core node. Note the comments regarding the use of the ramdisk, a technique that may speed up calculations significantly. It is restricted to runs on a single node, and there is book-keeping required before the job ends, namely copying any useful output from the ramdisk (/dev/shm) to the user's directory on the /scratch filesystem.

<source lang="bash">

1. !/bin/bash
2. PBS -l nodes=1:ppn=8,walltime=48:00:00,os=centos53computeA
3. PBS -N gamessjob

1. 1. To submit type: qsub gms.sh

1. If not an interactive job (i.e. -I), then cd into the directory where
2. I typed qsub.

if [ "$PBS_ENVIRONMENT" != "PBS_INTERACTIVE" ]; then

  if [ -n "$PBS_O_WORKDIR" ]; then
    cd $PBS_O_WORKDIR
  fi

fi

1. the input file is typically named something like "gamesjob.inp"
2. so the script will be run like "$SCINET_RUNGMS gamessjob 00 8 8"

1. load the gamess module if not in .bashrc already
2. module load gcc intel intelmpi gamess

1. If - and only if - the run is on a single node, and does not require too much memory,
2. then the runtime scratch directory can be put on ramdisk on the
3. compute node. This helps speed up the code, and avoids contention for
4. disk resources. Note that the maximum size of the ramdisk is 8 GB,
5. and the maximum total memory available to jobs is about 14 GB per node
6. (including program memory and ramdisk).
8. In order to use the ramdisk uncomment the following definition:
9. export SCINET_GAMESS_SCRATCH=/dev/shm

1. run the program

$SCINET_RUNGMS gamessjob 00 8 8

1. copy any files of interest from the $SCINET_GAMESS_SCRATCH directory
2. to a "real" directory on disk, e.g. /scratch/$USER/gamessjob
3. cp $SCINET_GAMESS_SCRATCH/* /scratch/$USER/gamessjob

</source>

Here is a similar script, but this one uses 2 InfiniBand-connected nodes, and runs the appropriate $SCINET_RUNGMS_IB script to actually run the job:

<source lang="bash">

1. !/bin/bash
2. PBS -l nodes=2:ib:ppn=8,walltime=48:00:00,os=centos53computeA
3. PBS -N gamessjob

1. 1. To submit type: qsub gmsib.sh

1. If not an interactive job (i.e. -I), then cd into the directory where
2. I typed qsub.

if [ "$PBS_ENVIRONMENT" != "PBS_INTERACTIVE" ]; then

  if [ -n "$PBS_O_WORKDIR" ]; then
    cd $PBS_O_WORKDIR
  fi

fi

1. the input file is typically named something like "gamesjob.inp"
2. so the script will be run like "$SCINET_RUNGMS gamessjob 00 8 8"

1. load the gamess module if not in .bashrc already
2. actually, it MUST be in .bashrc
3. module load gamess

1. This script requests InfiniBand-connected nodes (:ib above)
2. so it must run with the IB version of the rungms script,
3. $SCINET_RUNGMS_IB

$SCINET_RUNGMS_IB gamessjob 00 16 8 </source>

-- dgruner 5 October 2009 -- dgruner Updated on 14 September 2010