oldwiki.scinet.utoronto.ca - User contributions [en-gb]

https://oldwiki.scinet.utoronto.ca/api.php?action=feedcontributions&feedformat=atom&user=Jchu oldwiki.scinet.utoronto.ca - User contributions [en-gb] 2026-04-08T06:54:07Z User contributions MediaWiki 1.35.12 https://oldwiki.scinet.utoronto.ca/index.php?title=User_Codes&diff=884 User Codes 2010-02-02T19:13:37Z

<p>Jchu: /* LAMMPS */</p> <hr /> <div>__FORCETOC__<br /> <br /> ==Astrophysics==<br /> <br /> ===Athena (explicit, uniform grid MHD code)===<br /> <br /> [[Image:StrongScalingAthenaGPC.png|thumb|right|320px|Athena scaling on GPC with OpenMPI and MVAPICH2 on GigE, and OpenMPI on InfiniBand]]<br /> <br /> [http://www.astro.princeton.edu/~jstone/athena.html Athena] is a straightforward C code which doesn't use a lot of libraries so it is pretty straightforward to build and compile on new machines. <br /> <br /> It encapsulates its compiler flags, etc in an <tt>Makeoptions.in</tt> file which is then processed by <tt>configure</tt>. I've used the following additions to <tt>Makeoptions.in</tt> on TCS and GPC:<br /> <br /> <source lang="sh"><br /> ifeq ($(MACHINE),scinettcs)<br /> CC = mpcc_r<br /> LDR = mpcc_r<br /> OPT = -O5 -q64 -qarch=pwr6 -qtune=pwr6 -qcache=auto -qlargepage -qstrict<br /> MPIINC =<br /> MPILIB =<br /> CFLAGS = $(OPT)<br /> LIB = -ldl -lm<br /> else<br /> ifeq ($(MACHINE),scinetgpc)<br /> CC = mpicc<br /> LDR = mpicc<br /> OPT = -O3<br /> MPIINC =<br /> MPILIB =<br /> CFLAGS = $(OPT)<br /> LIB = -lm<br /> else<br /> ...<br /> endif<br /> endif<br /> </source><br /> It performs quite well on the GPC, scaling extremely well even on a strong scaling test out to about 256 cores (32 nodes) on Gigabit ethernet, and performing beautifully on InfiniBand out to 512 cores (64 nodes). <br /> <br /> <br /> -- [[User:Ljdursi|ljdursi]] 19:20, 13 August 2009 (UTC)<br /> <br /> ===FLASH3 (Adaptive Mesh reactive hydrodynamics; explict hydro/MHD)===<br /> <br /> [[Image:weak-scaling-example.png|thumb|right|320px|Weak scaling test of the 2d sod problem on both the GPC and TCS. The results are actually somewhat faster on the GPC; in both cases (weak) scaling is very good out at least to 256 cores]]<br /> <br /> [http://flash.uchicago.edu FLASH] encapsulates its machine-dependant information in the <tt>FLASH3/sites</tt> directory. For the GPC, you'll have to<br /> <pre><br /> module load intel<br /> module load openmpi<br /> module load hdf5/183-v16-openmpi<br /> </pre><br /> <br /> and with that, the following file (<tt>sites/scinetgpc/Makefile.h</tt>) works for me:<br /> <source lang="sh"><br /> ## Must do module load hdf5/183-v16-openmpi<br /> HDF5_PATH = ${SCINET_HDF5_BASE}<br /> ZLIB_PATH = /usr/local<br /> <br /> #----------------------------------------------------------------------------<br /> # Compiler and linker commands<br /> #<br /> # We use the f90 compiler as the linker, so some C libraries may explicitly<br /> # need to be added into the link line.<br /> #----------------------------------------------------------------------------<br /> <br /> ## modules will put the right mpi in our path<br /> FCOMP = mpif77<br /> CCOMP = mpicc<br /> CPPCOMP = mpiCC<br /> LINK = mpif77<br /> <br /> #----------------------------------------------------------------------------<br /> # Compilation flags<br /> #<br /> # Three sets of compilation/linking flags are defined: one for optimized<br /> # code, one for testing, and one for debugging. The default is to use the <br /> # _OPT version. Specifying -debug to setup will pick the _DEBUG version,<br /> # these should enable bounds checking. Specifying -test is used for <br /> # flash_test, and is set for quick code generation, and (sometimes) <br /> # profiling. The Makefile generated by setup will assign the generic token <br /> # (ex. FFLAGS) to the proper set of flags (ex. FFLAGS_OPT).<br /> #----------------------------------------------------------------------------<br /> <br /> FFLAGS_OPT = -c -r8 -i4 -O3 -xSSE4.2<br /> FFLAGS_DEBUG = -c -g -r8 -i4 -O0<br /> FFLAGS_TEST = -c -r8 -i4<br /> <br /> <br /> # if we are using HDF5, we need to specify the path to the include files<br /> CFLAGS_HDF5 = -I${HDF5_PATH}/include<br /> <br /> CFLAGS_OPT = -c -O3 -xSSE4.2<br /> CFLAGS_TEST = -c -O2 <br /> CFLAGS_DEBUG = -c -g <br /> <br /> MDEFS = <br /> <br /> .SUFFIXES: .o .c .f .F .h .fh .F90 .f90<br /> <br /> #----------------------------------------------------------------------------<br /> # Linker flags<br /> #<br /> # There is a seperate version of the linker flags for each of the _OPT, <br /> # _DEBUG, and _TEST cases.<br /> #----------------------------------------------------------------------------<br /> <br /> LFLAGS_OPT = -o<br /> LFLAGS_TEST = -o<br /> LFLAGS_DEBUG = -g -o<br /> <br /> MACHOBJ = <br /> <br /> <br /> MV = mv -f<br /> AR = ar -r<br /> RM = rm -f<br /> CD = cd<br /> RL = ranlib<br /> ECHO = echo<br /> </source><br /> <br /> <br /> -- [[User:Ljdursi|ljdursi]] 22:11, 13 August 2009 (UTC)<br /> <br /> ==Aeronautics==<br /> <br /> ==Chemistry==<br /> <br /> ===GAMESS (US)===<br /> The GAMESS version January 12, 2009 R3 was built using the Intel v11.1 compilers and v3.2.2 MPI library, according to the instructions in http://software.intel.com/en-us/articles/building-gamess-with-intel-compilers-intel-mkl-and-intel-mpi-on-linux/<br /> <br /> The required build scripts - comp, compall, lked - and run script - rungms - were modified to account for our own installation. In order to build GAMESS one first must ensure that the intel and intelmpi modules are loaded ("module load intel intelmpi"). This applies to running GAMESS as well.<br /> The module "gamess" must also be loaded in order to run GAMESS ("module load gamess").<br /> <br /> The modified scripts are in the file /scinet/gpc/src/gamess-on-scinet.tar.gz<br /> <br /> ====Running GAMESS====<br /> - Make sure the directory /scratch/$USER/gamess-scratch exists (the $SCINET_RUNGMS script will create it if it does not exist)<br /> <br /> - Make sure the modules: intel, intelmpi, gamess are loaded (in your .bashrc: "module load intel intelmpi gamess").<br /> <br /> - Create a torque script to run GAMESS. Here is an example:<br /> <br /> - The GAMESS executable is in $SCINET_GAMESS_HOME/gamess.00.x<br /> - The rungms script is in $SCINET_GAMESS_HOME/rungms (actually it is $SCINET_RUNGMS)<br /> <br /> - For IB multinode runs, use the $SCINET_RUNGMS_IB script<br /> <br /> - The rungms script takes 4 arguments: input file, executable number, number of compute processes, processors per node<br /> <br /> For example, in order to run with the input file /scratch/$USER/gamesstest01, on 8 cpus, and the default version (00)<br /> of the executable on a machine with 8 cores:<br /> <br /> # load the gamess module in .bashrc<br /> module load gamess <br /> <br /> # run the program<br /> $SCINET_RUNGMS /scratch/$USER/gamesstest01 00 8 8<br /> <br /> Here is a sample torque script for running a GAMESS calculation, on a single 8-core node:<br /> <br /> <source lang="bash"><br /> #!/bin/bash<br /> #PBS -l nodes=1:ppn=8,walltime=48:00:00,os=centos53computeA<br /> #PBS -N gamessjob<br /> <br /> ## To submit type: qsub gms.sh<br /> <br /> # If not an interactive job (i.e. -I), then cd into the directory where<br /> # I typed qsub.<br /> if [ "$PBS_ENVIRONMENT" != "PBS_INTERACTIVE" ]; then<br /> if [ -n "$PBS_O_WORKDIR" ]; then<br /> cd $PBS_O_WORKDIR<br /> fi<br /> fi<br /> <br /> # the input file is typically named something like "gamesjob.inp"<br /> # so the script will be run like "$SCINET_RUNGMS gamessjob 00 8 8"<br /> <br /> # load the gamess module if not in .bashrc already<br /> # actually, it MUST be in .bashrc<br /> # module load gamess<br /> <br /> # run the program<br /> <br /> $SCINET_RUNGMS gamessjob 00 8 8<br /> </source><br /> <br /> Here is a similar script, but this one uses 2 InfiniBand-connected nodes,<br /> and runs the appropriate $SCINET_RUNGMS_IB script to actually run the job:<br /> <br /> <source lang="bash"><br /> #!/bin/bash<br /> #PBS -l nodes=2:ib:ppn=8,walltime=48:00:00,os=centos53computeA<br /> #PBS -N gamessjob<br /> <br /> ## To submit type: qsub gmsib.sh<br /> <br /> # If not an interactive job (i.e. -I), then cd into the directory where<br /> # I typed qsub.<br /> if [ "$PBS_ENVIRONMENT" != "PBS_INTERACTIVE" ]; then<br /> if [ -n "$PBS_O_WORKDIR" ]; then<br /> cd $PBS_O_WORKDIR<br /> fi<br /> fi<br /> <br /> # the input file is typically named something like "gamesjob.inp"<br /> # so the script will be run like "$SCINET_RUNGMS gamessjob 00 8 8"<br /> <br /> # load the gamess module if not in .bashrc already<br /> # actually, it MUST be in .bashrc<br /> # module load gamess<br /> <br /> # This script requests InfiniBand-connected nodes (:ib above)<br /> # so it must run with the IB version of the rungms script,<br /> # $SCINET_RUNGMS_IB<br /> <br /> $SCINET_RUNGMS_IB gamessjob 00 16 8<br /> </source><br /> <br /> <br /> -- [[User:Dgruner|dgruner]] 5 October 2009<br /> <br /> ===Tips from the Fekl Lab===<br /> <br /> Through trial and error, we have found a few useful things that we would like to share:<br /> <br /> 1. Two very useful, open-source programs for visualization of output files from GAMESS(US) and for generation of input files are [http://www.scl.ameslab.gov/MacMolPlt/ MacMolPlt]and [http://avogadro.openmolecules.net/wiki/Main_Page Avogadro]. The are available for UNIX/LINUX, Windows and Mac based machines, HOWEVER: any input files that we have generated with these programs on a Windows-based machine do not run on Mac based machines. We don't know why.<br /> <br /> 2. [http://winscp.net/eng/index.php WinSCP] is a very useful tool that has a graphical user interface for moving files from a local machine to SCINET and vice versa. It also has text editing capabilities.<br /> <br /> 3. The [https://bse.pnl.gov/bse/portal ESML Basis Set Exchange] is an excellent source for custom basis set or effective core potential parameters. Make sure that you specify "Gamess-US" in the format drop-down box.<br /> <br /> 4. The commercial program [http://www.chemcraftprog.com/ ChemCraft] is a highly useful visualization program that has the ability to edit molecules in a very similar fashion to GaussView. It can also be customized to build GAMESS(US) input files.<br /> <br /> ====Anatomy of a GAMESS(US) Input File with Basis Set Info in an External File====<br /> <br /> $CONTRL SCFTYP=RHF RUNTYP=OPTIMIZE DFTTYP=M06-L MAXIT=199 MULT=1 NOSYM=1<br /> ECP=READ $END<br /> $SYSTEM TIMLIM=525600 MWORDS=1750 PARALL=.TRUE. $END<br /> $BASIS GBASIS=CUSTOMNI EXTFIL=.t. $END<br /> $SCF DIRSCF=.TRUE. FDIFF=.f. $END<br /> $STATPT OPTTOL=0.0001 NSTEP=500 HSSEND=.t. $END<br /> $DATA<br /> Mo_BDT3<br /> C1<br /> MOLYBDENUM 42.0 5.7556500000 4.4039600000 16.5808400000<br /> SULFUR 16.0 7.4169700000 3.1956300000 15.2089300000<br /> SULFUR 16.0 4.0966800000 3.2258300000 15.1761100000<br /> SULFUR 16.0 3.9677300000 4.4940500000 18.3266100000<br /> SULFUR 16.0 7.1776900000 3.5815000000 18.4485200000<br /> SULFUR 16.0 4.3776600000 6.2447400000 15.6786900000<br /> SULFUR 16.0 7.5478700000 6.0679800000 16.2223700000<br /> CARBON 6.0 6.4716900000 2.1004800000 14.1902300000<br /> CARBON 6.0 5.0690300000 2.1781400000 14.1080700000<br /> CARBON 6.0 4.8421800000 4.2701300000 19.8855500000<br /> CARBON 6.0 6.1969000000 3.9249600000 19.9397400000<br /> CARBON 6.0 6.8280600000 3.7834200000 21.1913200000<br /> CARBON 6.0 5.7697600000 7.6933500000 17.4241800000<br /> CARBON 6.0 7.2043100000 7.9413600000 17.8281100000<br /> CARBON 6.0 5.5051400000 7.0409700000 14.5903800000<br /> CARBON 6.0 6.8905200000 6.9194700000 14.7626200000<br /> CARBON 6.0 7.7396400000 7.5379800000 13.8285700000<br /> HYDROGEN 1.0 8.8190700000 7.4520600000 13.9252200000<br /> CARBON 6.0 7.2169400000 8.2960300000 12.7704100000<br /> HYDROGEN 1.0 7.8667000000 8.7825100000 12.0575600000<br /> CARBON 6.0 5.8260300000 8.4502300000 12.6467800000<br /> HYDROGEN 1.0 5.4143000000 9.0544300000 11.8493100000<br /> CARBON 6.0 4.9881500000 7.8192300000 13.5528400000<br /> HYDROGEN 1.0 3.9090500000 7.9420000000 13.4583700000<br /> CARBON 6.0 7.1538500000 1.1569600000 13.4143900000<br /> CARBON 6.0 4.4018100000 1.3603900000 13.1919900000<br /> CARBON 6.0 6.4791600000 0.3185500000 12.5353300000<br /> CARBON 6.0 5.0837400000 0.4369500000 12.4084900000<br /> HYDROGEN 1.0 7.0116000000 -0.4099400000 11.9434600000<br /> HYDROGEN 1.0 8.2399000000 1.0702400000 13.4937600000<br /> HYDROGEN 1.0 3.3185600000 1.4368700000 13.0953100000<br /> HYDROGEN 1.0 4.5549800000 -0.1997300000 11.7165200000<br /> CARBON 6.0 6.1105700000 3.9639000000 22.3866100000<br /> CARBON 6.0 4.1216300000 4.4424400000 21.1020100000<br /> HYDROGEN 1.0 7.8732900000 3.5217100000 21.2520500000<br /> CARBON 6.0 4.7606000000 4.2868500000 22.3363800000<br /> HYDROGEN 1.0 6.6064200000 3.8406000000 23.3428500000<br /> HYDROGEN 1.0 4.2065000000 4.4170700000 23.2667100000<br /> HYDROGEN 1.0 3.0674000000 4.6893500000 21.0889000000<br /> HYDROGEN 1.0 7.4249200000 7.7545300000 18.8583200000<br /> HYDROGEN 1.0 7.6651700000 8.9049700000 17.7652100000<br /> HYDROGEN 1.0 5.3324000000 8.6487800000 17.2222700000<br /> HYDROGEN 1.0 5.5015000000 7.1039000000 18.2759400000<br /> $END<br /> $ECP<br /> MO-ECP GEN 28 3<br /> 5 ----- f potential -----<br /> -0.0469492 0 537.9667807 <br /> -20.2080084 1 147.8982938 <br /> -106.2116302 2 45.7358898 <br /> -41.8107368 2 13.2911467 <br /> -4.2054103 2 4.7059961 <br /> 3 ----- s-f potential -----<br /> 2.8063717 0 110.2991760 <br /> 44.5162012 1 23.2014645 <br /> 82.7785227 2 5.3530131 <br /> 4 ----- p-f potential -----<br /> 4.9420876 0 63.2901397 <br /> 25.8604976 1 23.3315302 <br /> 132.4708742 2 24.6759423 <br /> 57.3149794 2 4.6493040 <br /> 5 ----- d-f potential -----<br /> 3.0054591 0 104.4839977 <br /> 26.3637851 1 66.2307245 <br /> 183.3849199 2 39.1283176 <br /> 98.4453068 2 13.1164437 <br /> 22.4901377 2 3.6280263 <br /> S NONE<br /> S NONE<br /> S NONE<br /> S NONE<br /> S NONE<br /> S NONE<br /> C NONE<br /> C NONE<br /> C NONE<br /> C NONE<br /> C NONE<br /> C NONE<br /> C NONE<br /> C NONE<br /> C NONE<br /> C NONE<br /> H NONE<br /> C NONE<br /> H NONE<br /> C NONE<br /> H NONE<br /> C NONE<br /> H NONE<br /> C NONE<br /> C NONE<br /> C NONE<br /> C NONE<br /> H NONE<br /> H NONE<br /> H NONE<br /> H NONE<br /> C NONE<br /> C NONE<br /> H NONE<br /> C NONE<br /> H NONE<br /> H NONE<br /> H NONE<br /> H NONE<br /> H NONE<br /> H NONE<br /> H NONE<br /> $END<br /> <br /> =====The Input Deck=====<br /> <br /> Below is the input deck. It is where you tell GAMESS(US) what job type to execute and where all you individual parameters are entered for your specific job type. The example input deck below is for a geometry optimization and frequency calculation. This input deck is equivalent to the Gaussian job with "opt" and "freq" in the route section.<br /> <br /> $CONTRL SCFTYP=RHF RUNTYP=OPTIMIZE DFTTYP=M06-L MAXIT=199 MULT=1 NOSYM=1<br /> ECP=READ $END<br /> $SYSTEM TIMLIM=2850 MWORDS=1750 MEMDDI=20 PARALL=.TRUE. $END<br /> $BASIS GBASIS=CUSTOMNI EXTFIL=.t. $END<br /> $SCF DIRSCF=.TRUE. FDIFF=.f. $END<br /> $STATPT OPTTOL=0.0001 NSTEP=500 HSSEND=.t. $END<br /> $DATA<br /> <br /> An important thing to note is the spacing. In the input deck, there must be 1 space at the beginning of each line of the input deck. If not, the job will fail. Most builders will insert this space anyway, but it helps to double check.<br /> <br /> The end of the input deck is marked by the "$DATA" line.<br /> <br /> =====Job Title Line=====<br /> <br /> The next line of the file is the job title. It can be anthing you wish, however, we have found that to be on the safe side, we avoide using symbols or spaces<br /> <br /> Mo_BDT3<br /> <br /> =====Symmetry Point Group=====<br /> <br /> The next line of the file is the symmetry point group of your molecule. Note that there is no leading space before the point group.<br /> <br /> C1<br /> <br /> =====Coordinates=====<br /> <br /> The next block of text is set aside for the coordinates of the molecule. This can be in internal (or z-matrix) format or cartesian coordinates. Note that there is no leading space before the coordinates. One may use the chemical symbol or the full name of each atom in the molecule. Note that the end of the coordinates is signified by an "$END", which MUST have one space preceding it. The coordinates below do NOT have any basis set information inserted. It is possible to insert basis set information directly into the input file. This is accomplished by obtaining the desired basis set parameters from the EMSL and then inserting them below each relevant atom. An example input file with inserted basis set information will be shown later.<br /> <br /> MOLYBDENUM 42.0 5.7556500000 4.4039600000 16.5808400000<br /> SULFUR 16.0 7.4169700000 3.1956300000 15.2089300000<br /> SULFUR 16.0 4.0966800000 3.2258300000 15.1761100000<br /> SULFUR 16.0 3.9677300000 4.4940500000 18.3266100000<br /> SULFUR 16.0 7.1776900000 3.5815000000 18.4485200000<br /> SULFUR 16.0 4.3776600000 6.2447400000 15.6786900000<br /> SULFUR 16.0 7.5478700000 6.0679800000 16.2223700000<br /> CARBON 6.0 6.4716900000 2.1004800000 14.1902300000<br /> CARBON 6.0 5.0690300000 2.1781400000 14.1080700000<br /> CARBON 6.0 4.8421800000 4.2701300000 19.8855500000<br /> CARBON 6.0 6.1969000000 3.9249600000 19.9397400000<br /> CARBON 6.0 6.8280600000 3.7834200000 21.1913200000<br /> CARBON 6.0 5.7697600000 7.6933500000 17.4241800000<br /> CARBON 6.0 7.2043100000 7.9413600000 17.8281100000<br /> CARBON 6.0 5.5051400000 7.0409700000 14.5903800000<br /> CARBON 6.0 6.8905200000 6.9194700000 14.7626200000<br /> CARBON 6.0 7.7396400000 7.5379800000 13.8285700000<br /> HYDROGEN 1.0 8.8190700000 7.4520600000 13.9252200000<br /> CARBON 6.0 7.2169400000 8.2960300000 12.7704100000<br /> HYDROGEN 1.0 7.8667000000 8.7825100000 12.0575600000<br /> CARBON 6.0 5.8260300000 8.4502300000 12.6467800000<br /> HYDROGEN 1.0 5.4143000000 9.0544300000 11.8493100000<br /> CARBON 6.0 4.9881500000 7.8192300000 13.5528400000<br /> HYDROGEN 1.0 3.9090500000 7.9420000000 13.4583700000<br /> CARBON 6.0 7.1538500000 1.1569600000 13.4143900000<br /> CARBON 6.0 4.4018100000 1.3603900000 13.1919900000<br /> CARBON 6.0 6.4791600000 0.3185500000 12.5353300000<br /> CARBON 6.0 5.0837400000 0.4369500000 12.4084900000<br /> HYDROGEN 1.0 7.0116000000 -0.4099400000 11.9434600000<br /> HYDROGEN 1.0 8.2399000000 1.0702400000 13.4937600000<br /> HYDROGEN 1.0 3.3185600000 1.4368700000 13.0953100000<br /> HYDROGEN 1.0 4.5549800000 -0.1997300000 11.7165200000<br /> CARBON 6.0 6.1105700000 3.9639000000 22.3866100000<br /> CARBON 6.0 4.1216300000 4.4424400000 21.1020100000<br /> HYDROGEN 1.0 7.8732900000 3.5217100000 21.2520500000<br /> CARBON 6.0 4.7606000000 4.2868500000 22.3363800000<br /> HYDROGEN 1.0 6.6064200000 3.8406000000 23.3428500000<br /> HYDROGEN 1.0 4.2065000000 4.4170700000 23.2667100000<br /> HYDROGEN 1.0 3.0674000000 4.6893500000 21.0889000000<br /> HYDROGEN 1.0 7.4249200000 7.7545300000 18.8583200000<br /> HYDROGEN 1.0 7.6651700000 8.9049700000 17.7652100000<br /> HYDROGEN 1.0 5.3324000000 8.6487800000 17.2222700000<br /> HYDROGEN 1.0 5.5015000000 7.1039000000 18.2759400000<br /> $END<br /> <br /> =====Effective Core Potential Data=====<br /> <br /> The effective core potential (ECP) data is entered after the coordinates. It starts with "$ECP", which must be preceded with a space. The atoms of the molecule are listed in the same order as in the coordinates section and the parameters for the ECP are listed after each atom. Note that for any atom that does NOT have an ECP, one must enter "ECP-NONE" or "NONE" after each atom without an ECP.<br /> <br /> $ECP<br /> MO-ECP GEN 28 3<br /> 5 ----- f potential -----<br /> -0.0469492 0 537.9667807 <br /> -20.2080084 1 147.8982938 <br /> -106.2116302 2 45.7358898 <br /> -41.8107368 2 13.2911467 <br /> -4.2054103 2 4.7059961 <br /> 3 ----- s-f potential -----<br /> 2.8063717 0 110.2991760 <br /> 44.5162012 1 23.2014645 <br /> 82.7785227 2 5.3530131 <br /> 4 ----- p-f potential -----<br /> 4.9420876 0 63.2901397 <br /> 25.8604976 1 23.3315302 <br /> 132.4708742 2 24.6759423 <br /> 57.3149794 2 4.6493040 <br /> 5 ----- d-f potential -----<br /> 3.0054591 0 104.4839977 <br /> 26.3637851 1 66.2307245 <br /> 183.3849199 2 39.1283176 <br /> 98.4453068 2 13.1164437 <br /> 22.4901377 2 3.6280263 <br /> S NONE<br /> S NONE<br /> S NONE<br /> S NONE<br /> S NONE<br /> S NONE<br /> C NONE<br /> C NONE<br /> C NONE<br /> C NONE<br /> C NONE<br /> C NONE<br /> C NONE<br /> C NONE<br /> C NONE<br /> C NONE<br /> H NONE<br /> C NONE<br /> H NONE<br /> C NONE<br /> H NONE<br /> C NONE<br /> H NONE<br /> C NONE<br /> C NONE<br /> C NONE<br /> C NONE<br /> H NONE<br /> H NONE<br /> H NONE<br /> H NONE<br /> C NONE<br /> C NONE<br /> H NONE<br /> C NONE<br /> H NONE<br /> H NONE<br /> H NONE<br /> H NONE<br /> H NONE<br /> H NONE<br /> H NONE<br /> $END<br /> <br /> -- [[User:M.Zimmer-De Iuliis|mzd]] 16 November 2009<br /> <br /> ====Using Effective Core Potentials in GAMESS(US)====<br /> <br /> For many metal containing compounds, it is very convenient and time saving to use an effective core potential (ECP) for the core metal electrons, as they are usually not important to the reactivity of the complex or the geometry around the metal. Since GAMESS(US) has a limited number of built-in ECPs, one may want to make GAMESS(US) read an external file that contains the ECP data using the "EXTFIL" keyword in the $GBASIS command line of the input file. In addition, to make GAMESS(US) use this external file, one must copy the "rungms" file and modify it accordingly. The following is a list of instructions with commands that will work from a terminal. One could also use WinSCP to do all of this with a GUI rather than a TUI. <br /> <br /> =====Modifiying rungms to Use Custom Basis Set File=====<br /> 1. Copy "rungms" from /scinet/gpc/Applications/gamess to one's own /scratch/$USER/ directory:<br /> cp /scinet/gpc/Applications/gamess/rungms /scratch/$USER/<br /> <br /> 2. Change to the scratch directory and check to see if "rungms" has copied successfully.<br /> cd /scratch/$USER<br /> ls<br /> <br /> 3. Edit line 147 of the vi. <br /> vi rungms<br /> Move the cursor down to line 147 using the arrow keys. It should say "setenv EXTBAS /dev/null". Using the arrow keys, move the cursor to the first "/" and then hit "i" to insert text. Put the path to your external basis file here. For example, /scratch/$USER/basisset. Then hit "escape". To save the changes and exit vi, type ":" and you should see a colon appear at the bottom of the window. Type "wq" (which should appear at the bottom of the window next to the colon) and then hit enter. Now you are done with vi.<br /> <br /> =====Creating a Custom Basis Set File=====<br /> 1. To create a custom basis set file, you need create a new text document. Our group's common practice is to comment out the first line of this file by inserting an exclamation mark (!) followed by noting the specific basis sets and ECPs that are going to be used for each of the atoms. Let us the molecule Mo(CO)6, Molybdenum hexacarbonyl, as an example. Below is the first line of the the external file, which we will call "CUSTOMMO" (NOTE: you can use any name for the external file that suits you, as long as it has no spaces and is 8 characters or less).<br /> <br /> ! 6-31G on C and O and LANL2D2 ECP on Mo<br /> <br /> 2. The next step is to visit the [https://bse.pnl.gov/bse/portal EMSL Basis Set exchange] and select C and O from the periodic table. Then, on the left of the page, select "6-31G" as the basis set. Finally, make sure the output is in GAMESS(US) format using the drop-down menu and then click "get basis set".<br /> <br /> [[File:C_O_6_31G_basisset.JPG|centre]]<br /> <br /> 3. A new window should appear with text in it. For our example case, the text looks like this:<br /> <br /> ! 6-31G EMSL Basis Set Exchange Library 10/13/09 11:12 AM<br /> ! Elements References<br /> ! -------- ----------<br /> ! H - He: W.J. Hehre, R. Ditchfield and J.A. Pople, J. Chem. Phys. 56,<br /> ! Li - Ne: 2257 (1972). Note: Li and B come from J.D. Dill and J.A.<br /> ! Pople, J. Chem. Phys. 62, 2921 (1975).<br /> ! Na - Ar: M.M. Francl, W.J. Petro, W.J. Hehre, J.S. Binkley, M.S. Gordon,<br /> ! D.J. DeFrees and J.A. Pople, J. Chem. Phys. 77, 3654 (1982)<br /> ! K - Zn: V. Rassolov, J.A. Pople, M. Ratner and T.L. Windus, J. Chem. Phys.<br /> ! 109, 1223 (1998)<br /> ! Note: He and Ne are unpublished basis sets taken from the Gaussian<br /> ! program<br /> ! <br /> $DATA<br /><br /> CARBON<br /> S 6<br /> 1 3047.5249000 0.0018347 <br /> 2 457.3695100 0.0140373 <br /> 3 103.9486900 0.0688426 <br /> 4 29.2101550 0.2321844 <br /> 5 9.2866630 0.4679413 <br /> 6 3.1639270 0.3623120 <br /> L 3<br /> 1 7.8682724 -0.1193324 0.0689991 <br /> 2 1.8812885 -0.1608542 0.3164240 <br /> 3 0.5442493 1.1434564 0.7443083 <br /> L 1<br /> 1 0.1687144 1.0000000 1.0000000<br /> <br /> OXYGEN<br /> S 6<br /> 1 5484.6717000 0.0018311 <br /> 2 825.2349500 0.0139501 <br /> 3 188.0469600 0.0684451 <br /> 4 52.9645000 0.2327143 <br /> 5 16.8975700 0.4701930 <br /> 6 5.7996353 0.3585209 <br /> L 3<br /> 1 15.5396160 -0.1107775 0.0708743 <br /> 2 3.5999336 -0.1480263 0.3397528 <br /> 3 1.0137618 1.1307670 0.7271586 <br /> L 1<br /> 1 0.2700058 1.0000000 1.0000000 <br /> $END<br /> <br /> 3. Now, copy and paste the text between the $DATA and $END headings onto our external text file, CUSTOMMO. We also need to change the change the name of each element to it symbol in the periodic table. Finally, we need to add the name of the external file next to the element symbol, separated by one space. Note that there should be a blank line separating the basis set information and the first, commented-out line (The line starting with the '!'). The CUSTOMMO should look like this:<br /> <br /> ! 6-31G on C and O and LANL2D2 ECP on Mo<br /><br /> C CUSTOMMO<br /> S 6<br /> 1 3047.5249000 0.0018347 <br /> 2 457.3695100 0.0140373 <br /> 3 103.9486900 0.0688426 <br /> 4 29.2101550 0.2321844 <br /> 5 9.2866630 0.4679413 <br /> 6 3.1639270 0.3623120 <br /> L 3<br /> 1 7.8682724 -0.1193324 0.0689991 <br /> 2 1.8812885 -0.1608542 0.3164240 <br /> 3 0.5442493 1.1434564 0.7443083 <br /> L 1<br /> 1 0.1687144 1.0000000 1.0000000<br /> <br /> O CUSTOMMO<br /> S 6<br /> 1 5484.6717000 0.0018311 <br /> 2 825.2349500 0.0139501 <br /> 3 188.0469600 0.0684451 <br /> 4 52.9645000 0.2327143 <br /> 5 16.8975700 0.4701930 <br /> 6 5.7996353 0.3585209 <br /> L 3<br /> 1 15.5396160 -0.1107775 0.0708743 <br /> 2 3.5999336 -0.1480263 0.3397528 <br /> 3 1.0137618 1.1307670 0.7271586 <br /> L 1<br /> 1 0.2700058 1.0000000 1.0000000 <br /> <br /> 4. Repeat Step 3 above but choose Mo and select the LANL2DZ ECP instead. A new window will pop up with the basis set information as well as the ECP data we need, since we specified the LANL2DZ '''ECP'''. The ECP data is not inserted into the external file, rather it is placed into the input file itself (More on this later). <br /> <br /> [[File:Mo_LANL2DZ_basisset.JPG|centre]]<br /> <br /> 5. After copying the molybdenum basis set information, your fiished external basis set file should look like this:<br /> ! 6-31G on C and O and LANL2D2 ECP on Mo<br /><br /> C CUSTOMMO<br /> S 6<br /> 1 3047.5249000 0.0018347 <br /> 2 457.3695100 0.0140373 <br /> 3 103.9486900 0.0688426 <br /> 4 29.2101550 0.2321844 <br /> 5 9.2866630 0.4679413 <br /> 6 3.1639270 0.3623120 <br /> L 3<br /> 1 7.8682724 -0.1193324 0.0689991 <br /> 2 1.8812885 -0.1608542 0.3164240 <br /> 3 0.5442493 1.1434564 0.7443083 <br /> L 1<br /> 1 0.1687144 1.0000000 1.0000000<br /> <br /> O CUSTOMMO<br /> S 6<br /> 1 5484.6717000 0.0018311 <br /> 2 825.2349500 0.0139501 <br /> 3 188.0469600 0.0684451 <br /> 4 52.9645000 0.2327143 <br /> 5 16.8975700 0.4701930 <br /> 6 5.7996353 0.3585209 <br /> L 3<br /> 1 15.5396160 -0.1107775 0.0708743 <br /> 2 3.5999336 -0.1480263 0.3397528 <br /> 3 1.0137618 1.1307670 0.7271586 <br /> L 1<br /> 1 0.2700058 1.0000000 1.0000000<br /> <br /> Mo CUSTOMO<br /> S 3<br /> 1 2.3610000 -0.9121760 <br /> 2 1.3090000 1.1477453 <br /> 3 0.4500000 0.6097109 <br /> S 4<br /> 1 2.3610000 0.8139259 <br /> 2 1.3090000 -1.1360084 <br /> 3 0.4500000 -1.1611592 <br /> 4 0.1681000 1.0064786 <br /> S 1<br /> 1 0.0423000 1.0000000 <br /> P 3<br /> 1 4.8950000 -0.0908258 <br /> 2 1.0440000 0.7042899 <br /> 3 0.3877000 0.3973179 <br /> P 2<br /> 1 0.4995000 -0.1081945 <br /> 2 0.0780000 1.0368093 <br /> P 1<br /> 1 0.0247000 1.0000000 <br /> D 3<br /> 1 2.9930000 0.0527063 <br /> 2 1.0630000 0.5003907 <br /> 3 0.3721000 0.5794024 <br /> D 1<br /> 1 0.1178000 1.0000000<br /> <br /> -- [[User:M.Zimmer-De Iuliis|mzd]] 13 October 2009<br /> <br /> ==Climate Modelling==<br /> <br /> ==Medicine/Bio==<br /> <br /> ==High Energy Physics==<br /> <br /> ==Structural Biology==<br /> Molecular simulation of proteins, lipids, carbohydrates, and other biologically relevant molecules.<br /> ===Molecular Dynamics (MD) simulation===<br /> ====GROMACS====<br /> Please refer to the [[gromacs|GROMACS]] page<br /> ====AMBER====<br /> Please refer to the [[amber|AMBER]] page<br /> ====NAMD====<br /> NAMD is one of the better scaling MD packages out there. With sufficiently large systems, it is able to scale to hundreds or thousands of cores on Scinet. Below are details for compiling and running NAMD on Scinet.<br /> <br /> More information regarding performance and different compile options coming soon...<br /> <br /> =====Compiling NAMD for GPC=====<br /> Ensure the proper compiler/mpi modules are loaded.<br /> <source lang="sh"><br /> module load intel<br /> module load openmpi/1.3.3-intel-v11.0-ofed<br /> </source><br /> <br /> '''Compile Charm++ and NAMD'''<br /> <source lang="sh"><br /> #Unpack source files and get required support libraries<br /> tar -xzf NAMD_2.7b1_Source.tar.gz<br /> cd NAMD_2.7b1_Source<br /> tar -xf charm-6.1.tar<br /> wget http://www.ks.uiuc.edu/Research/namd/libraries/fftw-linux-x86_64.tar.gz<br /> wget http://www.ks.uiuc.edu/Research/namd/libraries/tcl-linux-x86_64.tar.gz<br /> tar -xzf fftw-linux-x86_64.tar.gz; mv linux-x86_64 fftw<br /> tar -xzf tcl-linux-x86_64.tar.gz; mv linux-x86_64 tcl<br /> #Compile Charm++<br /> cd charm-6.1<br /> ./build charm++ mpi-linux-x86_64 icc --basedir /scinet/gpc/mpi/openmpi/1.3.3-intel-v11.0-ofed/ --no-shared -O -DCMK_OPTIMIZE=1<br /> cd ..<br /> #Compile NAMD. <br /> #Edit arch/Linux-x86_64-icc.arch and add "-lmpi" to the end of the CXXOPTS and COPTS line.<br /> #Make a builds directory if you want different versions of NAMD compiled at the same time.<br /> mkdir builds<br /> ./config builds/Linux-x86_64-icc --charm-arch mpi-linux-x86_64-icc<br /> cd builds/Linux-x86_64-icc/<br /> make -j4 namd2 # Adjust value of j as desired to specify number of simultaneous make targets. <br /> </source><br /> --[[User:Cmadill|Cmadill]] 16:18, 27 August 2009 (UTC)<br /> <br /> =====Running Fortran=====<br /> On the development nodes, there is an old gcc. The associated libraries are not on the compute nodes. Ensure the line:<br /> <br /> module load gcc<br /> <br /> is in your .bashrc file.<br /> <br /> ====LAMMPS====<br /> [[Image:StrongScalingLAMMPS.png|thumb|320px|right|Strong scaling test on GPC with OpenMPI and IntelMPI on Ethernet and InfiniBand]]<br /> [[Image:WeakScalingLAMMPS.png|thumb|320px|right|Weak scaling test on GPC with OpenMPI and IntelMPI on Ethernet and InfiniBand]]<br /> LAMMPS is a parallel MD code that can be found [http://lammps.sandia.gov/ here].<br /> <br /> '''Scaling Tests on GPC'''<br /> <br /> Results from strong scaling tests for LAMMPS using EAM potentials on GPC are shown in the graph on the right. Test simulation ran 500 timesteps for 4,000,000 atoms.<br /> <br /> Results from weak scaling tests for LAMMPS using EAM potentials on GPC are shown in the graph on the right. Test simulation ran 500 timesteps for 32,000 atoms per processor.<br /> <br /> OpenMPI version used: openmpi/1.4.1-intel-v11.0-ofed<br /> <br /> IntelMPI version used: intelmpi/impi-4.0.0.013<br /> <br /> LAMMPS version used: 15 Jan 2010<br /> <br /> '''Summary of Scaling Tests'''<br /> <br /> Results show good scaling for both OpenMPI and IntelMPI on Ethernet up to 16 processors, after which performance begins to suffer. On Infiniband, excellent scaling is maintained to 512 processors.<br /> <br /> IntelMPI shows slightly better performance compared to OpenMPI when running with Infiniband.<br /> <br /> --[[User:jchu|jchu]] 14:08 Feb 2, 2010<br /> <br /> ===Monte Carlo (MC) simulation===</div>

Jchu https://oldwiki.scinet.utoronto.ca/index.php?title=User_Codes&diff=883 User Codes 2010-02-02T19:08:53Z

<p>Jchu: /* LAMMPS */</p> <hr /> <div>__FORCETOC__<br /> <br /> ==Astrophysics==<br /> <br /> ===Athena (explicit, uniform grid MHD code)===<br /> <br /> [[Image:StrongScalingAthenaGPC.png|thumb|right|320px|Athena scaling on GPC with OpenMPI and MVAPICH2 on GigE, and OpenMPI on InfiniBand]]<br /> <br /> [http://www.astro.princeton.edu/~jstone/athena.html Athena] is a straightforward C code which doesn't use a lot of libraries so it is pretty straightforward to build and compile on new machines. <br /> <br /> It encapsulates its compiler flags, etc in an <tt>Makeoptions.in</tt> file which is then processed by <tt>configure</tt>. I've used the following additions to <tt>Makeoptions.in</tt> on TCS and GPC:<br /> <br /> <source lang="sh"><br /> ifeq ($(MACHINE),scinettcs)<br /> CC = mpcc_r<br /> LDR = mpcc_r<br /> OPT = -O5 -q64 -qarch=pwr6 -qtune=pwr6 -qcache=auto -qlargepage -qstrict<br /> MPIINC =<br /> MPILIB =<br /> CFLAGS = $(OPT)<br /> LIB = -ldl -lm<br /> else<br /> ifeq ($(MACHINE),scinetgpc)<br /> CC = mpicc<br /> LDR = mpicc<br /> OPT = -O3<br /> MPIINC =<br /> MPILIB =<br /> CFLAGS = $(OPT)<br /> LIB = -lm<br /> else<br /> ...<br /> endif<br /> endif<br /> </source><br /> It performs quite well on the GPC, scaling extremely well even on a strong scaling test out to about 256 cores (32 nodes) on Gigabit ethernet, and performing beautifully on InfiniBand out to 512 cores (64 nodes). <br /> <br /> <br /> -- [[User:Ljdursi|ljdursi]] 19:20, 13 August 2009 (UTC)<br /> <br /> ===FLASH3 (Adaptive Mesh reactive hydrodynamics; explict hydro/MHD)===<br /> <br /> [[Image:weak-scaling-example.png|thumb|right|320px|Weak scaling test of the 2d sod problem on both the GPC and TCS. The results are actually somewhat faster on the GPC; in both cases (weak) scaling is very good out at least to 256 cores]]<br /> <br /> [http://flash.uchicago.edu FLASH] encapsulates its machine-dependant information in the <tt>FLASH3/sites</tt> directory. For the GPC, you'll have to<br /> <pre><br /> module load intel<br /> module load openmpi<br /> module load hdf5/183-v16-openmpi<br /> </pre><br /> <br /> and with that, the following file (<tt>sites/scinetgpc/Makefile.h</tt>) works for me:<br /> <source lang="sh"><br /> ## Must do module load hdf5/183-v16-openmpi<br /> HDF5_PATH = ${SCINET_HDF5_BASE}<br /> ZLIB_PATH = /usr/local<br /> <br /> #----------------------------------------------------------------------------<br /> # Compiler and linker commands<br /> #<br /> # We use the f90 compiler as the linker, so some C libraries may explicitly<br /> # need to be added into the link line.<br /> #----------------------------------------------------------------------------<br /> <br /> ## modules will put the right mpi in our path<br /> FCOMP = mpif77<br /> CCOMP = mpicc<br /> CPPCOMP = mpiCC<br /> LINK = mpif77<br /> <br /> #----------------------------------------------------------------------------<br /> # Compilation flags<br /> #<br /> # Three sets of compilation/linking flags are defined: one for optimized<br /> # code, one for testing, and one for debugging. The default is to use the <br /> # _OPT version. Specifying -debug to setup will pick the _DEBUG version,<br /> # these should enable bounds checking. Specifying -test is used for <br /> # flash_test, and is set for quick code generation, and (sometimes) <br /> # profiling. The Makefile generated by setup will assign the generic token <br /> # (ex. FFLAGS) to the proper set of flags (ex. FFLAGS_OPT).<br /> #----------------------------------------------------------------------------<br /> <br /> FFLAGS_OPT = -c -r8 -i4 -O3 -xSSE4.2<br /> FFLAGS_DEBUG = -c -g -r8 -i4 -O0<br /> FFLAGS_TEST = -c -r8 -i4<br /> <br /> <br /> # if we are using HDF5, we need to specify the path to the include files<br /> CFLAGS_HDF5 = -I${HDF5_PATH}/include<br /> <br /> CFLAGS_OPT = -c -O3 -xSSE4.2<br /> CFLAGS_TEST = -c -O2 <br /> CFLAGS_DEBUG = -c -g <br /> <br /> MDEFS = <br /> <br /> .SUFFIXES: .o .c .f .F .h .fh .F90 .f90<br /> <br /> #----------------------------------------------------------------------------<br /> # Linker flags<br /> #<br /> # There is a seperate version of the linker flags for each of the _OPT, <br /> # _DEBUG, and _TEST cases.<br /> #----------------------------------------------------------------------------<br /> <br /> LFLAGS_OPT = -o<br /> LFLAGS_TEST = -o<br /> LFLAGS_DEBUG = -g -o<br /> <br /> MACHOBJ = <br /> <br /> <br /> MV = mv -f<br /> AR = ar -r<br /> RM = rm -f<br /> CD = cd<br /> RL = ranlib<br /> ECHO = echo<br /> </source><br /> <br /> <br /> -- [[User:Ljdursi|ljdursi]] 22:11, 13 August 2009 (UTC)<br /> <br /> ==Aeronautics==<br /> <br /> ==Chemistry==<br /> <br /> ===GAMESS (US)===<br /> The GAMESS version January 12, 2009 R3 was built using the Intel v11.1 compilers and v3.2.2 MPI library, according to the instructions in http://software.intel.com/en-us/articles/building-gamess-with-intel-compilers-intel-mkl-and-intel-mpi-on-linux/<br /> <br /> The required build scripts - comp, compall, lked - and run script - rungms - were modified to account for our own installation. In order to build GAMESS one first must ensure that the intel and intelmpi modules are loaded ("module load intel intelmpi"). This applies to running GAMESS as well.<br /> The module "gamess" must also be loaded in order to run GAMESS ("module load gamess").<br /> <br /> The modified scripts are in the file /scinet/gpc/src/gamess-on-scinet.tar.gz<br /> <br /> ====Running GAMESS====<br /> - Make sure the directory /scratch/$USER/gamess-scratch exists (the $SCINET_RUNGMS script will create it if it does not exist)<br /> <br /> - Make sure the modules: intel, intelmpi, gamess are loaded (in your .bashrc: "module load intel intelmpi gamess").<br /> <br /> - Create a torque script to run GAMESS. Here is an example:<br /> <br /> - The GAMESS executable is in $SCINET_GAMESS_HOME/gamess.00.x<br /> - The rungms script is in $SCINET_GAMESS_HOME/rungms (actually it is $SCINET_RUNGMS)<br /> <br /> - For IB multinode runs, use the $SCINET_RUNGMS_IB script<br /> <br /> - The rungms script takes 4 arguments: input file, executable number, number of compute processes, processors per node<br /> <br /> For example, in order to run with the input file /scratch/$USER/gamesstest01, on 8 cpus, and the default version (00)<br /> of the executable on a machine with 8 cores:<br /> <br /> # load the gamess module in .bashrc<br /> module load gamess <br /> <br /> # run the program<br /> $SCINET_RUNGMS /scratch/$USER/gamesstest01 00 8 8<br /> <br /> Here is a sample torque script for running a GAMESS calculation, on a single 8-core node:<br /> <br /> <source lang="bash"><br /> #!/bin/bash<br /> #PBS -l nodes=1:ppn=8,walltime=48:00:00,os=centos53computeA<br /> #PBS -N gamessjob<br /> <br /> ## To submit type: qsub gms.sh<br /> <br /> # If not an interactive job (i.e. -I), then cd into the directory where<br /> # I typed qsub.<br /> if [ "$PBS_ENVIRONMENT" != "PBS_INTERACTIVE" ]; then<br /> if [ -n "$PBS_O_WORKDIR" ]; then<br /> cd $PBS_O_WORKDIR<br /> fi<br /> fi<br /> <br /> # the input file is typically named something like "gamesjob.inp"<br /> # so the script will be run like "$SCINET_RUNGMS gamessjob 00 8 8"<br /> <br /> # load the gamess module if not in .bashrc already<br /> # actually, it MUST be in .bashrc<br /> # module load gamess<br /> <br /> # run the program<br /> <br /> $SCINET_RUNGMS gamessjob 00 8 8<br /> </source><br /> <br /> Here is a similar script, but this one uses 2 InfiniBand-connected nodes,<br /> and runs the appropriate $SCINET_RUNGMS_IB script to actually run the job:<br /> <br /> <source lang="bash"><br /> #!/bin/bash<br /> #PBS -l nodes=2:ib:ppn=8,walltime=48:00:00,os=centos53computeA<br /> #PBS -N gamessjob<br /> <br /> ## To submit type: qsub gmsib.sh<br /> <br /> # If not an interactive job (i.e. -I), then cd into the directory where<br /> # I typed qsub.<br /> if [ "$PBS_ENVIRONMENT" != "PBS_INTERACTIVE" ]; then<br /> if [ -n "$PBS_O_WORKDIR" ]; then<br /> cd $PBS_O_WORKDIR<br /> fi<br /> fi<br /> <br /> # the input file is typically named something like "gamesjob.inp"<br /> # so the script will be run like "$SCINET_RUNGMS gamessjob 00 8 8"<br /> <br /> # load the gamess module if not in .bashrc already<br /> # actually, it MUST be in .bashrc<br /> # module load gamess<br /> <br /> # This script requests InfiniBand-connected nodes (:ib above)<br /> # so it must run with the IB version of the rungms script,<br /> # $SCINET_RUNGMS_IB<br /> <br /> $SCINET_RUNGMS_IB gamessjob 00 16 8<br /> </source><br /> <br /> <br /> -- [[User:Dgruner|dgruner]] 5 October 2009<br /> <br /> ===Tips from the Fekl Lab===<br /> <br /> Through trial and error, we have found a few useful things that we would like to share:<br /> <br /> 1. Two very useful, open-source programs for visualization of output files from GAMESS(US) and for generation of input files are [http://www.scl.ameslab.gov/MacMolPlt/ MacMolPlt]and [http://avogadro.openmolecules.net/wiki/Main_Page Avogadro]. The are available for UNIX/LINUX, Windows and Mac based machines, HOWEVER: any input files that we have generated with these programs on a Windows-based machine do not run on Mac based machines. We don't know why.<br /> <br /> 2. [http://winscp.net/eng/index.php WinSCP] is a very useful tool that has a graphical user interface for moving files from a local machine to SCINET and vice versa. It also has text editing capabilities.<br /> <br /> 3. The [https://bse.pnl.gov/bse/portal ESML Basis Set Exchange] is an excellent source for custom basis set or effective core potential parameters. Make sure that you specify "Gamess-US" in the format drop-down box.<br /> <br /> 4. The commercial program [http://www.chemcraftprog.com/ ChemCraft] is a highly useful visualization program that has the ability to edit molecules in a very similar fashion to GaussView. It can also be customized to build GAMESS(US) input files.<br /> <br /> ====Anatomy of a GAMESS(US) Input File with Basis Set Info in an External File====<br /> <br /> $CONTRL SCFTYP=RHF RUNTYP=OPTIMIZE DFTTYP=M06-L MAXIT=199 MULT=1 NOSYM=1<br /> ECP=READ $END<br /> $SYSTEM TIMLIM=525600 MWORDS=1750 PARALL=.TRUE. $END<br /> $BASIS GBASIS=CUSTOMNI EXTFIL=.t. $END<br /> $SCF DIRSCF=.TRUE. FDIFF=.f. $END<br /> $STATPT OPTTOL=0.0001 NSTEP=500 HSSEND=.t. $END<br /> $DATA<br /> Mo_BDT3<br /> C1<br /> MOLYBDENUM 42.0 5.7556500000 4.4039600000 16.5808400000<br /> SULFUR 16.0 7.4169700000 3.1956300000 15.2089300000<br /> SULFUR 16.0 4.0966800000 3.2258300000 15.1761100000<br /> SULFUR 16.0 3.9677300000 4.4940500000 18.3266100000<br /> SULFUR 16.0 7.1776900000 3.5815000000 18.4485200000<br /> SULFUR 16.0 4.3776600000 6.2447400000 15.6786900000<br /> SULFUR 16.0 7.5478700000 6.0679800000 16.2223700000<br /> CARBON 6.0 6.4716900000 2.1004800000 14.1902300000<br /> CARBON 6.0 5.0690300000 2.1781400000 14.1080700000<br /> CARBON 6.0 4.8421800000 4.2701300000 19.8855500000<br /> CARBON 6.0 6.1969000000 3.9249600000 19.9397400000<br /> CARBON 6.0 6.8280600000 3.7834200000 21.1913200000<br /> CARBON 6.0 5.7697600000 7.6933500000 17.4241800000<br /> CARBON 6.0 7.2043100000 7.9413600000 17.8281100000<br /> CARBON 6.0 5.5051400000 7.0409700000 14.5903800000<br /> CARBON 6.0 6.8905200000 6.9194700000 14.7626200000<br /> CARBON 6.0 7.7396400000 7.5379800000 13.8285700000<br /> HYDROGEN 1.0 8.8190700000 7.4520600000 13.9252200000<br /> CARBON 6.0 7.2169400000 8.2960300000 12.7704100000<br /> HYDROGEN 1.0 7.8667000000 8.7825100000 12.0575600000<br /> CARBON 6.0 5.8260300000 8.4502300000 12.6467800000<br /> HYDROGEN 1.0 5.4143000000 9.0544300000 11.8493100000<br /> CARBON 6.0 4.9881500000 7.8192300000 13.5528400000<br /> HYDROGEN 1.0 3.9090500000 7.9420000000 13.4583700000<br /> CARBON 6.0 7.1538500000 1.1569600000 13.4143900000<br /> CARBON 6.0 4.4018100000 1.3603900000 13.1919900000<br /> CARBON 6.0 6.4791600000 0.3185500000 12.5353300000<br /> CARBON 6.0 5.0837400000 0.4369500000 12.4084900000<br /> HYDROGEN 1.0 7.0116000000 -0.4099400000 11.9434600000<br /> HYDROGEN 1.0 8.2399000000 1.0702400000 13.4937600000<br /> HYDROGEN 1.0 3.3185600000 1.4368700000 13.0953100000<br /> HYDROGEN 1.0 4.5549800000 -0.1997300000 11.7165200000<br /> CARBON 6.0 6.1105700000 3.9639000000 22.3866100000<br /> CARBON 6.0 4.1216300000 4.4424400000 21.1020100000<br /> HYDROGEN 1.0 7.8732900000 3.5217100000 21.2520500000<br /> CARBON 6.0 4.7606000000 4.2868500000 22.3363800000<br /> HYDROGEN 1.0 6.6064200000 3.8406000000 23.3428500000<br /> HYDROGEN 1.0 4.2065000000 4.4170700000 23.2667100000<br /> HYDROGEN 1.0 3.0674000000 4.6893500000 21.0889000000<br /> HYDROGEN 1.0 7.4249200000 7.7545300000 18.8583200000<br /> HYDROGEN 1.0 7.6651700000 8.9049700000 17.7652100000<br /> HYDROGEN 1.0 5.3324000000 8.6487800000 17.2222700000<br /> HYDROGEN 1.0 5.5015000000 7.1039000000 18.2759400000<br /> $END<br /> $ECP<br /> MO-ECP GEN 28 3<br /> 5 ----- f potential -----<br /> -0.0469492 0 537.9667807 <br /> -20.2080084 1 147.8982938 <br /> -106.2116302 2 45.7358898 <br /> -41.8107368 2 13.2911467 <br /> -4.2054103 2 4.7059961 <br /> 3 ----- s-f potential -----<br /> 2.8063717 0 110.2991760 <br /> 44.5162012 1 23.2014645 <br /> 82.7785227 2 5.3530131 <br /> 4 ----- p-f potential -----<br /> 4.9420876 0 63.2901397 <br /> 25.8604976 1 23.3315302 <br /> 132.4708742 2 24.6759423 <br /> 57.3149794 2 4.6493040 <br /> 5 ----- d-f potential -----<br /> 3.0054591 0 104.4839977 <br /> 26.3637851 1 66.2307245 <br /> 183.3849199 2 39.1283176 <br /> 98.4453068 2 13.1164437 <br /> 22.4901377 2 3.6280263 <br /> S NONE<br /> S NONE<br /> S NONE<br /> S NONE<br /> S NONE<br /> S NONE<br /> C NONE<br /> C NONE<br /> C NONE<br /> C NONE<br /> C NONE<br /> C NONE<br /> C NONE<br /> C NONE<br /> C NONE<br /> C NONE<br /> H NONE<br /> C NONE<br /> H NONE<br /> C NONE<br /> H NONE<br /> C NONE<br /> H NONE<br /> C NONE<br /> C NONE<br /> C NONE<br /> C NONE<br /> H NONE<br /> H NONE<br /> H NONE<br /> H NONE<br /> C NONE<br /> C NONE<br /> H NONE<br /> C NONE<br /> H NONE<br /> H NONE<br /> H NONE<br /> H NONE<br /> H NONE<br /> H NONE<br /> H NONE<br /> $END<br /> <br /> =====The Input Deck=====<br /> <br /> Below is the input deck. It is where you tell GAMESS(US) what job type to execute and where all you individual parameters are entered for your specific job type. The example input deck below is for a geometry optimization and frequency calculation. This input deck is equivalent to the Gaussian job with "opt" and "freq" in the route section.<br /> <br /> $CONTRL SCFTYP=RHF RUNTYP=OPTIMIZE DFTTYP=M06-L MAXIT=199 MULT=1 NOSYM=1<br /> ECP=READ $END<br /> $SYSTEM TIMLIM=2850 MWORDS=1750 MEMDDI=20 PARALL=.TRUE. $END<br /> $BASIS GBASIS=CUSTOMNI EXTFIL=.t. $END<br /> $SCF DIRSCF=.TRUE. FDIFF=.f. $END<br /> $STATPT OPTTOL=0.0001 NSTEP=500 HSSEND=.t. $END<br /> $DATA<br /> <br /> An important thing to note is the spacing. In the input deck, there must be 1 space at the beginning of each line of the input deck. If not, the job will fail. Most builders will insert this space anyway, but it helps to double check.<br /> <br /> The end of the input deck is marked by the "$DATA" line.<br /> <br /> =====Job Title Line=====<br /> <br /> The next line of the file is the job title. It can be anthing you wish, however, we have found that to be on the safe side, we avoide using symbols or spaces<br /> <br /> Mo_BDT3<br /> <br /> =====Symmetry Point Group=====<br /> <br /> The next line of the file is the symmetry point group of your molecule. Note that there is no leading space before the point group.<br /> <br /> C1<br /> <br /> =====Coordinates=====<br /> <br /> The next block of text is set aside for the coordinates of the molecule. This can be in internal (or z-matrix) format or cartesian coordinates. Note that there is no leading space before the coordinates. One may use the chemical symbol or the full name of each atom in the molecule. Note that the end of the coordinates is signified by an "$END", which MUST have one space preceding it. The coordinates below do NOT have any basis set information inserted. It is possible to insert basis set information directly into the input file. This is accomplished by obtaining the desired basis set parameters from the EMSL and then inserting them below each relevant atom. An example input file with inserted basis set information will be shown later.<br /> <br /> MOLYBDENUM 42.0 5.7556500000 4.4039600000 16.5808400000<br /> SULFUR 16.0 7.4169700000 3.1956300000 15.2089300000<br /> SULFUR 16.0 4.0966800000 3.2258300000 15.1761100000<br /> SULFUR 16.0 3.9677300000 4.4940500000 18.3266100000<br /> SULFUR 16.0 7.1776900000 3.5815000000 18.4485200000<br /> SULFUR 16.0 4.3776600000 6.2447400000 15.6786900000<br /> SULFUR 16.0 7.5478700000 6.0679800000 16.2223700000<br /> CARBON 6.0 6.4716900000 2.1004800000 14.1902300000<br /> CARBON 6.0 5.0690300000 2.1781400000 14.1080700000<br /> CARBON 6.0 4.8421800000 4.2701300000 19.8855500000<br /> CARBON 6.0 6.1969000000 3.9249600000 19.9397400000<br /> CARBON 6.0 6.8280600000 3.7834200000 21.1913200000<br /> CARBON 6.0 5.7697600000 7.6933500000 17.4241800000<br /> CARBON 6.0 7.2043100000 7.9413600000 17.8281100000<br /> CARBON 6.0 5.5051400000 7.0409700000 14.5903800000<br /> CARBON 6.0 6.8905200000 6.9194700000 14.7626200000<br /> CARBON 6.0 7.7396400000 7.5379800000 13.8285700000<br /> HYDROGEN 1.0 8.8190700000 7.4520600000 13.9252200000<br /> CARBON 6.0 7.2169400000 8.2960300000 12.7704100000<br /> HYDROGEN 1.0 7.8667000000 8.7825100000 12.0575600000<br /> CARBON 6.0 5.8260300000 8.4502300000 12.6467800000<br /> HYDROGEN 1.0 5.4143000000 9.0544300000 11.8493100000<br /> CARBON 6.0 4.9881500000 7.8192300000 13.5528400000<br /> HYDROGEN 1.0 3.9090500000 7.9420000000 13.4583700000<br /> CARBON 6.0 7.1538500000 1.1569600000 13.4143900000<br /> CARBON 6.0 4.4018100000 1.3603900000 13.1919900000<br /> CARBON 6.0 6.4791600000 0.3185500000 12.5353300000<br /> CARBON 6.0 5.0837400000 0.4369500000 12.4084900000<br /> HYDROGEN 1.0 7.0116000000 -0.4099400000 11.9434600000<br /> HYDROGEN 1.0 8.2399000000 1.0702400000 13.4937600000<br /> HYDROGEN 1.0 3.3185600000 1.4368700000 13.0953100000<br /> HYDROGEN 1.0 4.5549800000 -0.1997300000 11.7165200000<br /> CARBON 6.0 6.1105700000 3.9639000000 22.3866100000<br /> CARBON 6.0 4.1216300000 4.4424400000 21.1020100000<br /> HYDROGEN 1.0 7.8732900000 3.5217100000 21.2520500000<br /> CARBON 6.0 4.7606000000 4.2868500000 22.3363800000<br /> HYDROGEN 1.0 6.6064200000 3.8406000000 23.3428500000<br /> HYDROGEN 1.0 4.2065000000 4.4170700000 23.2667100000<br /> HYDROGEN 1.0 3.0674000000 4.6893500000 21.0889000000<br /> HYDROGEN 1.0 7.4249200000 7.7545300000 18.8583200000<br /> HYDROGEN 1.0 7.6651700000 8.9049700000 17.7652100000<br /> HYDROGEN 1.0 5.3324000000 8.6487800000 17.2222700000<br /> HYDROGEN 1.0 5.5015000000 7.1039000000 18.2759400000<br /> $END<br /> <br /> =====Effective Core Potential Data=====<br /> <br /> The effective core potential (ECP) data is entered after the coordinates. It starts with "$ECP", which must be preceded with a space. The atoms of the molecule are listed in the same order as in the coordinates section and the parameters for the ECP are listed after each atom. Note that for any atom that does NOT have an ECP, one must enter "ECP-NONE" or "NONE" after each atom without an ECP.<br /> <br /> $ECP<br /> MO-ECP GEN 28 3<br /> 5 ----- f potential -----<br /> -0.0469492 0 537.9667807 <br /> -20.2080084 1 147.8982938 <br /> -106.2116302 2 45.7358898 <br /> -41.8107368 2 13.2911467 <br /> -4.2054103 2 4.7059961 <br /> 3 ----- s-f potential -----<br /> 2.8063717 0 110.2991760 <br /> 44.5162012 1 23.2014645 <br /> 82.7785227 2 5.3530131 <br /> 4 ----- p-f potential -----<br /> 4.9420876 0 63.2901397 <br /> 25.8604976 1 23.3315302 <br /> 132.4708742 2 24.6759423 <br /> 57.3149794 2 4.6493040 <br /> 5 ----- d-f potential -----<br /> 3.0054591 0 104.4839977 <br /> 26.3637851 1 66.2307245 <br /> 183.3849199 2 39.1283176 <br /> 98.4453068 2 13.1164437 <br /> 22.4901377 2 3.6280263 <br /> S NONE<br /> S NONE<br /> S NONE<br /> S NONE<br /> S NONE<br /> S NONE<br /> C NONE<br /> C NONE<br /> C NONE<br /> C NONE<br /> C NONE<br /> C NONE<br /> C NONE<br /> C NONE<br /> C NONE<br /> C NONE<br /> H NONE<br /> C NONE<br /> H NONE<br /> C NONE<br /> H NONE<br /> C NONE<br /> H NONE<br /> C NONE<br /> C NONE<br /> C NONE<br /> C NONE<br /> H NONE<br /> H NONE<br /> H NONE<br /> H NONE<br /> C NONE<br /> C NONE<br /> H NONE<br /> C NONE<br /> H NONE<br /> H NONE<br /> H NONE<br /> H NONE<br /> H NONE<br /> H NONE<br /> H NONE<br /> $END<br /> <br /> -- [[User:M.Zimmer-De Iuliis|mzd]] 16 November 2009<br /> <br /> ====Using Effective Core Potentials in GAMESS(US)====<br /> <br /> For many metal containing compounds, it is very convenient and time saving to use an effective core potential (ECP) for the core metal electrons, as they are usually not important to the reactivity of the complex or the geometry around the metal. Since GAMESS(US) has a limited number of built-in ECPs, one may want to make GAMESS(US) read an external file that contains the ECP data using the "EXTFIL" keyword in the $GBASIS command line of the input file. In addition, to make GAMESS(US) use this external file, one must copy the "rungms" file and modify it accordingly. The following is a list of instructions with commands that will work from a terminal. One could also use WinSCP to do all of this with a GUI rather than a TUI. <br /> <br /> =====Modifiying rungms to Use Custom Basis Set File=====<br /> 1. Copy "rungms" from /scinet/gpc/Applications/gamess to one's own /scratch/$USER/ directory:<br /> cp /scinet/gpc/Applications/gamess/rungms /scratch/$USER/<br /> <br /> 2. Change to the scratch directory and check to see if "rungms" has copied successfully.<br /> cd /scratch/$USER<br /> ls<br /> <br /> 3. Edit line 147 of the vi. <br /> vi rungms<br /> Move the cursor down to line 147 using the arrow keys. It should say "setenv EXTBAS /dev/null". Using the arrow keys, move the cursor to the first "/" and then hit "i" to insert text. Put the path to your external basis file here. For example, /scratch/$USER/basisset. Then hit "escape". To save the changes and exit vi, type ":" and you should see a colon appear at the bottom of the window. Type "wq" (which should appear at the bottom of the window next to the colon) and then hit enter. Now you are done with vi.<br /> <br /> =====Creating a Custom Basis Set File=====<br /> 1. To create a custom basis set file, you need create a new text document. Our group's common practice is to comment out the first line of this file by inserting an exclamation mark (!) followed by noting the specific basis sets and ECPs that are going to be used for each of the atoms. Let us the molecule Mo(CO)6, Molybdenum hexacarbonyl, as an example. Below is the first line of the the external file, which we will call "CUSTOMMO" (NOTE: you can use any name for the external file that suits you, as long as it has no spaces and is 8 characters or less).<br /> <br /> ! 6-31G on C and O and LANL2D2 ECP on Mo<br /> <br /> 2. The next step is to visit the [https://bse.pnl.gov/bse/portal EMSL Basis Set exchange] and select C and O from the periodic table. Then, on the left of the page, select "6-31G" as the basis set. Finally, make sure the output is in GAMESS(US) format using the drop-down menu and then click "get basis set".<br /> <br /> [[File:C_O_6_31G_basisset.JPG|centre]]<br /> <br /> 3. A new window should appear with text in it. For our example case, the text looks like this:<br /> <br /> ! 6-31G EMSL Basis Set Exchange Library 10/13/09 11:12 AM<br /> ! Elements References<br /> ! -------- ----------<br /> ! H - He: W.J. Hehre, R. Ditchfield and J.A. Pople, J. Chem. Phys. 56,<br /> ! Li - Ne: 2257 (1972). Note: Li and B come from J.D. Dill and J.A.<br /> ! Pople, J. Chem. Phys. 62, 2921 (1975).<br /> ! Na - Ar: M.M. Francl, W.J. Petro, W.J. Hehre, J.S. Binkley, M.S. Gordon,<br /> ! D.J. DeFrees and J.A. Pople, J. Chem. Phys. 77, 3654 (1982)<br /> ! K - Zn: V. Rassolov, J.A. Pople, M. Ratner and T.L. Windus, J. Chem. Phys.<br /> ! 109, 1223 (1998)<br /> ! Note: He and Ne are unpublished basis sets taken from the Gaussian<br /> ! program<br /> ! <br /> $DATA<br /><br /> CARBON<br /> S 6<br /> 1 3047.5249000 0.0018347 <br /> 2 457.3695100 0.0140373 <br /> 3 103.9486900 0.0688426 <br /> 4 29.2101550 0.2321844 <br /> 5 9.2866630 0.4679413 <br /> 6 3.1639270 0.3623120 <br /> L 3<br /> 1 7.8682724 -0.1193324 0.0689991 <br /> 2 1.8812885 -0.1608542 0.3164240 <br /> 3 0.5442493 1.1434564 0.7443083 <br /> L 1<br /> 1 0.1687144 1.0000000 1.0000000<br /> <br /> OXYGEN<br /> S 6<br /> 1 5484.6717000 0.0018311 <br /> 2 825.2349500 0.0139501 <br /> 3 188.0469600 0.0684451 <br /> 4 52.9645000 0.2327143 <br /> 5 16.8975700 0.4701930 <br /> 6 5.7996353 0.3585209 <br /> L 3<br /> 1 15.5396160 -0.1107775 0.0708743 <br /> 2 3.5999336 -0.1480263 0.3397528 <br /> 3 1.0137618 1.1307670 0.7271586 <br /> L 1<br /> 1 0.2700058 1.0000000 1.0000000 <br /> $END<br /> <br /> 3. Now, copy and paste the text between the $DATA and $END headings onto our external text file, CUSTOMMO. We also need to change the change the name of each element to it symbol in the periodic table. Finally, we need to add the name of the external file next to the element symbol, separated by one space. Note that there should be a blank line separating the basis set information and the first, commented-out line (The line starting with the '!'). The CUSTOMMO should look like this:<br /> <br /> ! 6-31G on C and O and LANL2D2 ECP on Mo<br /><br /> C CUSTOMMO<br /> S 6<br /> 1 3047.5249000 0.0018347 <br /> 2 457.3695100 0.0140373 <br /> 3 103.9486900 0.0688426 <br /> 4 29.2101550 0.2321844 <br /> 5 9.2866630 0.4679413 <br /> 6 3.1639270 0.3623120 <br /> L 3<br /> 1 7.8682724 -0.1193324 0.0689991 <br /> 2 1.8812885 -0.1608542 0.3164240 <br /> 3 0.5442493 1.1434564 0.7443083 <br /> L 1<br /> 1 0.1687144 1.0000000 1.0000000<br /> <br /> O CUSTOMMO<br /> S 6<br /> 1 5484.6717000 0.0018311 <br /> 2 825.2349500 0.0139501 <br /> 3 188.0469600 0.0684451 <br /> 4 52.9645000 0.2327143 <br /> 5 16.8975700 0.4701930 <br /> 6 5.7996353 0.3585209 <br /> L 3<br /> 1 15.5396160 -0.1107775 0.0708743 <br /> 2 3.5999336 -0.1480263 0.3397528 <br /> 3 1.0137618 1.1307670 0.7271586 <br /> L 1<br /> 1 0.2700058 1.0000000 1.0000000 <br /> <br /> 4. Repeat Step 3 above but choose Mo and select the LANL2DZ ECP instead. A new window will pop up with the basis set information as well as the ECP data we need, since we specified the LANL2DZ '''ECP'''. The ECP data is not inserted into the external file, rather it is placed into the input file itself (More on this later). <br /> <br /> [[File:Mo_LANL2DZ_basisset.JPG|centre]]<br /> <br /> 5. After copying the molybdenum basis set information, your fiished external basis set file should look like this:<br /> ! 6-31G on C and O and LANL2D2 ECP on Mo<br /><br /> C CUSTOMMO<br /> S 6<br /> 1 3047.5249000 0.0018347 <br /> 2 457.3695100 0.0140373 <br /> 3 103.9486900 0.0688426 <br /> 4 29.2101550 0.2321844 <br /> 5 9.2866630 0.4679413 <br /> 6 3.1639270 0.3623120 <br /> L 3<br /> 1 7.8682724 -0.1193324 0.0689991 <br /> 2 1.8812885 -0.1608542 0.3164240 <br /> 3 0.5442493 1.1434564 0.7443083 <br /> L 1<br /> 1 0.1687144 1.0000000 1.0000000<br /> <br /> O CUSTOMMO<br /> S 6<br /> 1 5484.6717000 0.0018311 <br /> 2 825.2349500 0.0139501 <br /> 3 188.0469600 0.0684451 <br /> 4 52.9645000 0.2327143 <br /> 5 16.8975700 0.4701930 <br /> 6 5.7996353 0.3585209 <br /> L 3<br /> 1 15.5396160 -0.1107775 0.0708743 <br /> 2 3.5999336 -0.1480263 0.3397528 <br /> 3 1.0137618 1.1307670 0.7271586 <br /> L 1<br /> 1 0.2700058 1.0000000 1.0000000<br /> <br /> Mo CUSTOMO<br /> S 3<br /> 1 2.3610000 -0.9121760 <br /> 2 1.3090000 1.1477453 <br /> 3 0.4500000 0.6097109 <br /> S 4<br /> 1 2.3610000 0.8139259 <br /> 2 1.3090000 -1.1360084 <br /> 3 0.4500000 -1.1611592 <br /> 4 0.1681000 1.0064786 <br /> S 1<br /> 1 0.0423000 1.0000000 <br /> P 3<br /> 1 4.8950000 -0.0908258 <br /> 2 1.0440000 0.7042899 <br /> 3 0.3877000 0.3973179 <br /> P 2<br /> 1 0.4995000 -0.1081945 <br /> 2 0.0780000 1.0368093 <br /> P 1<br /> 1 0.0247000 1.0000000 <br /> D 3<br /> 1 2.9930000 0.0527063 <br /> 2 1.0630000 0.5003907 <br /> 3 0.3721000 0.5794024 <br /> D 1<br /> 1 0.1178000 1.0000000<br /> <br /> -- [[User:M.Zimmer-De Iuliis|mzd]] 13 October 2009<br /> <br /> ==Climate Modelling==<br /> <br /> ==Medicine/Bio==<br /> <br /> ==High Energy Physics==<br /> <br /> ==Structural Biology==<br /> Molecular simulation of proteins, lipids, carbohydrates, and other biologically relevant molecules.<br /> ===Molecular Dynamics (MD) simulation===<br /> ====GROMACS====<br /> Please refer to the [[gromacs|GROMACS]] page<br /> ====AMBER====<br /> Please refer to the [[amber|AMBER]] page<br /> ====NAMD====<br /> NAMD is one of the better scaling MD packages out there. With sufficiently large systems, it is able to scale to hundreds or thousands of cores on Scinet. Below are details for compiling and running NAMD on Scinet.<br /> <br /> More information regarding performance and different compile options coming soon...<br /> <br /> =====Compiling NAMD for GPC=====<br /> Ensure the proper compiler/mpi modules are loaded.<br /> <source lang="sh"><br /> module load intel<br /> module load openmpi/1.3.3-intel-v11.0-ofed<br /> </source><br /> <br /> '''Compile Charm++ and NAMD'''<br /> <source lang="sh"><br /> #Unpack source files and get required support libraries<br /> tar -xzf NAMD_2.7b1_Source.tar.gz<br /> cd NAMD_2.7b1_Source<br /> tar -xf charm-6.1.tar<br /> wget http://www.ks.uiuc.edu/Research/namd/libraries/fftw-linux-x86_64.tar.gz<br /> wget http://www.ks.uiuc.edu/Research/namd/libraries/tcl-linux-x86_64.tar.gz<br /> tar -xzf fftw-linux-x86_64.tar.gz; mv linux-x86_64 fftw<br /> tar -xzf tcl-linux-x86_64.tar.gz; mv linux-x86_64 tcl<br /> #Compile Charm++<br /> cd charm-6.1<br /> ./build charm++ mpi-linux-x86_64 icc --basedir /scinet/gpc/mpi/openmpi/1.3.3-intel-v11.0-ofed/ --no-shared -O -DCMK_OPTIMIZE=1<br /> cd ..<br /> #Compile NAMD. <br /> #Edit arch/Linux-x86_64-icc.arch and add "-lmpi" to the end of the CXXOPTS and COPTS line.<br /> #Make a builds directory if you want different versions of NAMD compiled at the same time.<br /> mkdir builds<br /> ./config builds/Linux-x86_64-icc --charm-arch mpi-linux-x86_64-icc<br /> cd builds/Linux-x86_64-icc/<br /> make -j4 namd2 # Adjust value of j as desired to specify number of simultaneous make targets. <br /> </source><br /> --[[User:Cmadill|Cmadill]] 16:18, 27 August 2009 (UTC)<br /> <br /> =====Running Fortran=====<br /> On the development nodes, there is an old gcc. The associated libraries are not on the compute nodes. Ensure the line:<br /> <br /> module load gcc<br /> <br /> is in your .bashrc file.<br /> <br /> ====LAMMPS====<br /> [[Image:StrongScalingLAMMPS.png|thumb|320px|right|Strong scaling test on GPC with OpenMPI and IntelMPI on Ethernet and InfiniBand]]<br /> [[Image:WeakScalingLAMMPS.png|thumb|320px|right|Weak scaling test on GPC with OpenMPI and IntelMPI on Ethernet and InfiniBand]]<br /> LAMMPS is a parallel MD code that can be found [http://lammps.sandia.gov/ here].<br /> <br /> '''Scaling Tests on GPC'''<br /> <br /> Results from strong scaling tests for LAMMPS using EAM potentials on GPC are shown in the graph on the right. Test simulation ran 500 timesteps for 4,000,000 atoms.<br /> <br /> Results from weak scaling tests for LAMMPS using EAM potentials on GPC are shown in the graph on the right. Test simulation ran 500 timesteps for 32,000 atoms per processor.<br /> <br /> OpenMPI version used: openmpi/1.4.1-intel-v11.0-ofed<br /> <br /> IntelMPI version used: intelmpi/impi-4.0.0.013<br /> <br /> '''Summary of Scaling Tests'''<br /> <br /> Results show good scaling for both OpenMPI and IntelMPI on Ethernet up to 16 processors, after which performance begins to suffer. On Infiniband, excellent scaling is maintained to 512 processors.<br /> <br /> IntelMPI shows slightly better performance compared to OpenMPI when running with Infiniband.<br /> <br /> --[[User:jchu|jchu]] 14:08 Feb 2, 2010<br /> <br /> ===Monte Carlo (MC) simulation===</div>

Jchu https://oldwiki.scinet.utoronto.ca/index.php?title=User_Codes&diff=882 User Codes 2010-02-02T19:07:24Z

<p>Jchu: /* LAMMPS */</p> <hr /> <div>__FORCETOC__<br /> <br /> ==Astrophysics==<br /> <br /> ===Athena (explicit, uniform grid MHD code)===<br /> <br /> [[Image:StrongScalingAthenaGPC.png|thumb|right|320px|Athena scaling on GPC with OpenMPI and MVAPICH2 on GigE, and OpenMPI on InfiniBand]]<br /> <br /> [http://www.astro.princeton.edu/~jstone/athena.html Athena] is a straightforward C code which doesn't use a lot of libraries so it is pretty straightforward to build and compile on new machines. <br /> <br /> It encapsulates its compiler flags, etc in an <tt>Makeoptions.in</tt> file which is then processed by <tt>configure</tt>. I've used the following additions to <tt>Makeoptions.in</tt> on TCS and GPC:<br /> <br /> <source lang="sh"><br /> ifeq ($(MACHINE),scinettcs)<br /> CC = mpcc_r<br /> LDR = mpcc_r<br /> OPT = -O5 -q64 -qarch=pwr6 -qtune=pwr6 -qcache=auto -qlargepage -qstrict<br /> MPIINC =<br /> MPILIB =<br /> CFLAGS = $(OPT)<br /> LIB = -ldl -lm<br /> else<br /> ifeq ($(MACHINE),scinetgpc)<br /> CC = mpicc<br /> LDR = mpicc<br /> OPT = -O3<br /> MPIINC =<br /> MPILIB =<br /> CFLAGS = $(OPT)<br /> LIB = -lm<br /> else<br /> ...<br /> endif<br /> endif<br /> </source><br /> It performs quite well on the GPC, scaling extremely well even on a strong scaling test out to about 256 cores (32 nodes) on Gigabit ethernet, and performing beautifully on InfiniBand out to 512 cores (64 nodes). <br /> <br /> <br /> -- [[User:Ljdursi|ljdursi]] 19:20, 13 August 2009 (UTC)<br /> <br /> ===FLASH3 (Adaptive Mesh reactive hydrodynamics; explict hydro/MHD)===<br /> <br /> [[Image:weak-scaling-example.png|thumb|right|320px|Weak scaling test of the 2d sod problem on both the GPC and TCS. The results are actually somewhat faster on the GPC; in both cases (weak) scaling is very good out at least to 256 cores]]<br /> <br /> [http://flash.uchicago.edu FLASH] encapsulates its machine-dependant information in the <tt>FLASH3/sites</tt> directory. For the GPC, you'll have to<br /> <pre><br /> module load intel<br /> module load openmpi<br /> module load hdf5/183-v16-openmpi<br /> </pre><br /> <br /> and with that, the following file (<tt>sites/scinetgpc/Makefile.h</tt>) works for me:<br /> <source lang="sh"><br /> ## Must do module load hdf5/183-v16-openmpi<br /> HDF5_PATH = ${SCINET_HDF5_BASE}<br /> ZLIB_PATH = /usr/local<br /> <br /> #----------------------------------------------------------------------------<br /> # Compiler and linker commands<br /> #<br /> # We use the f90 compiler as the linker, so some C libraries may explicitly<br /> # need to be added into the link line.<br /> #----------------------------------------------------------------------------<br /> <br /> ## modules will put the right mpi in our path<br /> FCOMP = mpif77<br /> CCOMP = mpicc<br /> CPPCOMP = mpiCC<br /> LINK = mpif77<br /> <br /> #----------------------------------------------------------------------------<br /> # Compilation flags<br /> #<br /> # Three sets of compilation/linking flags are defined: one for optimized<br /> # code, one for testing, and one for debugging. The default is to use the <br /> # _OPT version. Specifying -debug to setup will pick the _DEBUG version,<br /> # these should enable bounds checking. Specifying -test is used for <br /> # flash_test, and is set for quick code generation, and (sometimes) <br /> # profiling. The Makefile generated by setup will assign the generic token <br /> # (ex. FFLAGS) to the proper set of flags (ex. FFLAGS_OPT).<br /> #----------------------------------------------------------------------------<br /> <br /> FFLAGS_OPT = -c -r8 -i4 -O3 -xSSE4.2<br /> FFLAGS_DEBUG = -c -g -r8 -i4 -O0<br /> FFLAGS_TEST = -c -r8 -i4<br /> <br /> <br /> # if we are using HDF5, we need to specify the path to the include files<br /> CFLAGS_HDF5 = -I${HDF5_PATH}/include<br /> <br /> CFLAGS_OPT = -c -O3 -xSSE4.2<br /> CFLAGS_TEST = -c -O2 <br /> CFLAGS_DEBUG = -c -g <br /> <br /> MDEFS = <br /> <br /> .SUFFIXES: .o .c .f .F .h .fh .F90 .f90<br /> <br /> #----------------------------------------------------------------------------<br /> # Linker flags<br /> #<br /> # There is a seperate version of the linker flags for each of the _OPT, <br /> # _DEBUG, and _TEST cases.<br /> #----------------------------------------------------------------------------<br /> <br /> LFLAGS_OPT = -o<br /> LFLAGS_TEST = -o<br /> LFLAGS_DEBUG = -g -o<br /> <br /> MACHOBJ = <br /> <br /> <br /> MV = mv -f<br /> AR = ar -r<br /> RM = rm -f<br /> CD = cd<br /> RL = ranlib<br /> ECHO = echo<br /> </source><br /> <br /> <br /> -- [[User:Ljdursi|ljdursi]] 22:11, 13 August 2009 (UTC)<br /> <br /> ==Aeronautics==<br /> <br /> ==Chemistry==<br /> <br /> ===GAMESS (US)===<br /> The GAMESS version January 12, 2009 R3 was built using the Intel v11.1 compilers and v3.2.2 MPI library, according to the instructions in http://software.intel.com/en-us/articles/building-gamess-with-intel-compilers-intel-mkl-and-intel-mpi-on-linux/<br /> <br /> The required build scripts - comp, compall, lked - and run script - rungms - were modified to account for our own installation. In order to build GAMESS one first must ensure that the intel and intelmpi modules are loaded ("module load intel intelmpi"). This applies to running GAMESS as well.<br /> The module "gamess" must also be loaded in order to run GAMESS ("module load gamess").<br /> <br /> The modified scripts are in the file /scinet/gpc/src/gamess-on-scinet.tar.gz<br /> <br /> ====Running GAMESS====<br /> - Make sure the directory /scratch/$USER/gamess-scratch exists (the $SCINET_RUNGMS script will create it if it does not exist)<br /> <br /> - Make sure the modules: intel, intelmpi, gamess are loaded (in your .bashrc: "module load intel intelmpi gamess").<br /> <br /> - Create a torque script to run GAMESS. Here is an example:<br /> <br /> - The GAMESS executable is in $SCINET_GAMESS_HOME/gamess.00.x<br /> - The rungms script is in $SCINET_GAMESS_HOME/rungms (actually it is $SCINET_RUNGMS)<br /> <br /> - For IB multinode runs, use the $SCINET_RUNGMS_IB script<br /> <br /> - The rungms script takes 4 arguments: input file, executable number, number of compute processes, processors per node<br /> <br /> For example, in order to run with the input file /scratch/$USER/gamesstest01, on 8 cpus, and the default version (00)<br /> of the executable on a machine with 8 cores:<br /> <br /> # load the gamess module in .bashrc<br /> module load gamess <br /> <br /> # run the program<br /> $SCINET_RUNGMS /scratch/$USER/gamesstest01 00 8 8<br /> <br /> Here is a sample torque script for running a GAMESS calculation, on a single 8-core node:<br /> <br /> <source lang="bash"><br /> #!/bin/bash<br /> #PBS -l nodes=1:ppn=8,walltime=48:00:00,os=centos53computeA<br /> #PBS -N gamessjob<br /> <br /> ## To submit type: qsub gms.sh<br /> <br /> # If not an interactive job (i.e. -I), then cd into the directory where<br /> # I typed qsub.<br /> if [ "$PBS_ENVIRONMENT" != "PBS_INTERACTIVE" ]; then<br /> if [ -n "$PBS_O_WORKDIR" ]; then<br /> cd $PBS_O_WORKDIR<br /> fi<br /> fi<br /> <br /> # the input file is typically named something like "gamesjob.inp"<br /> # so the script will be run like "$SCINET_RUNGMS gamessjob 00 8 8"<br /> <br /> # load the gamess module if not in .bashrc already<br /> # actually, it MUST be in .bashrc<br /> # module load gamess<br /> <br /> # run the program<br /> <br /> $SCINET_RUNGMS gamessjob 00 8 8<br /> </source><br /> <br /> Here is a similar script, but this one uses 2 InfiniBand-connected nodes,<br /> and runs the appropriate $SCINET_RUNGMS_IB script to actually run the job:<br /> <br /> <source lang="bash"><br /> #!/bin/bash<br /> #PBS -l nodes=2:ib:ppn=8,walltime=48:00:00,os=centos53computeA<br /> #PBS -N gamessjob<br /> <br /> ## To submit type: qsub gmsib.sh<br /> <br /> # If not an interactive job (i.e. -I), then cd into the directory where<br /> # I typed qsub.<br /> if [ "$PBS_ENVIRONMENT" != "PBS_INTERACTIVE" ]; then<br /> if [ -n "$PBS_O_WORKDIR" ]; then<br /> cd $PBS_O_WORKDIR<br /> fi<br /> fi<br /> <br /> # the input file is typically named something like "gamesjob.inp"<br /> # so the script will be run like "$SCINET_RUNGMS gamessjob 00 8 8"<br /> <br /> # load the gamess module if not in .bashrc already<br /> # actually, it MUST be in .bashrc<br /> # module load gamess<br /> <br /> # This script requests InfiniBand-connected nodes (:ib above)<br /> # so it must run with the IB version of the rungms script,<br /> # $SCINET_RUNGMS_IB<br /> <br /> $SCINET_RUNGMS_IB gamessjob 00 16 8<br /> </source><br /> <br /> <br /> -- [[User:Dgruner|dgruner]] 5 October 2009<br /> <br /> ===Tips from the Fekl Lab===<br /> <br /> Through trial and error, we have found a few useful things that we would like to share:<br /> <br /> 1. Two very useful, open-source programs for visualization of output files from GAMESS(US) and for generation of input files are [http://www.scl.ameslab.gov/MacMolPlt/ MacMolPlt]and [http://avogadro.openmolecules.net/wiki/Main_Page Avogadro]. The are available for UNIX/LINUX, Windows and Mac based machines, HOWEVER: any input files that we have generated with these programs on a Windows-based machine do not run on Mac based machines. We don't know why.<br /> <br /> 2. [http://winscp.net/eng/index.php WinSCP] is a very useful tool that has a graphical user interface for moving files from a local machine to SCINET and vice versa. It also has text editing capabilities.<br /> <br /> 3. The [https://bse.pnl.gov/bse/portal ESML Basis Set Exchange] is an excellent source for custom basis set or effective core potential parameters. Make sure that you specify "Gamess-US" in the format drop-down box.<br /> <br /> 4. The commercial program [http://www.chemcraftprog.com/ ChemCraft] is a highly useful visualization program that has the ability to edit molecules in a very similar fashion to GaussView. It can also be customized to build GAMESS(US) input files.<br /> <br /> ====Anatomy of a GAMESS(US) Input File with Basis Set Info in an External File====<br /> <br /> $CONTRL SCFTYP=RHF RUNTYP=OPTIMIZE DFTTYP=M06-L MAXIT=199 MULT=1 NOSYM=1<br /> ECP=READ $END<br /> $SYSTEM TIMLIM=525600 MWORDS=1750 PARALL=.TRUE. $END<br /> $BASIS GBASIS=CUSTOMNI EXTFIL=.t. $END<br /> $SCF DIRSCF=.TRUE. FDIFF=.f. $END<br /> $STATPT OPTTOL=0.0001 NSTEP=500 HSSEND=.t. $END<br /> $DATA<br /> Mo_BDT3<br /> C1<br /> MOLYBDENUM 42.0 5.7556500000 4.4039600000 16.5808400000<br /> SULFUR 16.0 7.4169700000 3.1956300000 15.2089300000<br /> SULFUR 16.0 4.0966800000 3.2258300000 15.1761100000<br /> SULFUR 16.0 3.9677300000 4.4940500000 18.3266100000<br /> SULFUR 16.0 7.1776900000 3.5815000000 18.4485200000<br /> SULFUR 16.0 4.3776600000 6.2447400000 15.6786900000<br /> SULFUR 16.0 7.5478700000 6.0679800000 16.2223700000<br /> CARBON 6.0 6.4716900000 2.1004800000 14.1902300000<br /> CARBON 6.0 5.0690300000 2.1781400000 14.1080700000<br /> CARBON 6.0 4.8421800000 4.2701300000 19.8855500000<br /> CARBON 6.0 6.1969000000 3.9249600000 19.9397400000<br /> CARBON 6.0 6.8280600000 3.7834200000 21.1913200000<br /> CARBON 6.0 5.7697600000 7.6933500000 17.4241800000<br /> CARBON 6.0 7.2043100000 7.9413600000 17.8281100000<br /> CARBON 6.0 5.5051400000 7.0409700000 14.5903800000<br /> CARBON 6.0 6.8905200000 6.9194700000 14.7626200000<br /> CARBON 6.0 7.7396400000 7.5379800000 13.8285700000<br /> HYDROGEN 1.0 8.8190700000 7.4520600000 13.9252200000<br /> CARBON 6.0 7.2169400000 8.2960300000 12.7704100000<br /> HYDROGEN 1.0 7.8667000000 8.7825100000 12.0575600000<br /> CARBON 6.0 5.8260300000 8.4502300000 12.6467800000<br /> HYDROGEN 1.0 5.4143000000 9.0544300000 11.8493100000<br /> CARBON 6.0 4.9881500000 7.8192300000 13.5528400000<br /> HYDROGEN 1.0 3.9090500000 7.9420000000 13.4583700000<br /> CARBON 6.0 7.1538500000 1.1569600000 13.4143900000<br /> CARBON 6.0 4.4018100000 1.3603900000 13.1919900000<br /> CARBON 6.0 6.4791600000 0.3185500000 12.5353300000<br /> CARBON 6.0 5.0837400000 0.4369500000 12.4084900000<br /> HYDROGEN 1.0 7.0116000000 -0.4099400000 11.9434600000<br /> HYDROGEN 1.0 8.2399000000 1.0702400000 13.4937600000<br /> HYDROGEN 1.0 3.3185600000 1.4368700000 13.0953100000<br /> HYDROGEN 1.0 4.5549800000 -0.1997300000 11.7165200000<br /> CARBON 6.0 6.1105700000 3.9639000000 22.3866100000<br /> CARBON 6.0 4.1216300000 4.4424400000 21.1020100000<br /> HYDROGEN 1.0 7.8732900000 3.5217100000 21.2520500000<br /> CARBON 6.0 4.7606000000 4.2868500000 22.3363800000<br /> HYDROGEN 1.0 6.6064200000 3.8406000000 23.3428500000<br /> HYDROGEN 1.0 4.2065000000 4.4170700000 23.2667100000<br /> HYDROGEN 1.0 3.0674000000 4.6893500000 21.0889000000<br /> HYDROGEN 1.0 7.4249200000 7.7545300000 18.8583200000<br /> HYDROGEN 1.0 7.6651700000 8.9049700000 17.7652100000<br /> HYDROGEN 1.0 5.3324000000 8.6487800000 17.2222700000<br /> HYDROGEN 1.0 5.5015000000 7.1039000000 18.2759400000<br /> $END<br /> $ECP<br /> MO-ECP GEN 28 3<br /> 5 ----- f potential -----<br /> -0.0469492 0 537.9667807 <br /> -20.2080084 1 147.8982938 <br /> -106.2116302 2 45.7358898 <br /> -41.8107368 2 13.2911467 <br /> -4.2054103 2 4.7059961 <br /> 3 ----- s-f potential -----<br /> 2.8063717 0 110.2991760 <br /> 44.5162012 1 23.2014645 <br /> 82.7785227 2 5.3530131 <br /> 4 ----- p-f potential -----<br /> 4.9420876 0 63.2901397 <br /> 25.8604976 1 23.3315302 <br /> 132.4708742 2 24.6759423 <br /> 57.3149794 2 4.6493040 <br /> 5 ----- d-f potential -----<br /> 3.0054591 0 104.4839977 <br /> 26.3637851 1 66.2307245 <br /> 183.3849199 2 39.1283176 <br /> 98.4453068 2 13.1164437 <br /> 22.4901377 2 3.6280263 <br /> S NONE<br /> S NONE<br /> S NONE<br /> S NONE<br /> S NONE<br /> S NONE<br /> C NONE<br /> C NONE<br /> C NONE<br /> C NONE<br /> C NONE<br /> C NONE<br /> C NONE<br /> C NONE<br /> C NONE<br /> C NONE<br /> H NONE<br /> C NONE<br /> H NONE<br /> C NONE<br /> H NONE<br /> C NONE<br /> H NONE<br /> C NONE<br /> C NONE<br /> C NONE<br /> C NONE<br /> H NONE<br /> H NONE<br /> H NONE<br /> H NONE<br /> C NONE<br /> C NONE<br /> H NONE<br /> C NONE<br /> H NONE<br /> H NONE<br /> H NONE<br /> H NONE<br /> H NONE<br /> H NONE<br /> H NONE<br /> $END<br /> <br /> =====The Input Deck=====<br /> <br /> Below is the input deck. It is where you tell GAMESS(US) what job type to execute and where all you individual parameters are entered for your specific job type. The example input deck below is for a geometry optimization and frequency calculation. This input deck is equivalent to the Gaussian job with "opt" and "freq" in the route section.<br /> <br /> $CONTRL SCFTYP=RHF RUNTYP=OPTIMIZE DFTTYP=M06-L MAXIT=199 MULT=1 NOSYM=1<br /> ECP=READ $END<br /> $SYSTEM TIMLIM=2850 MWORDS=1750 MEMDDI=20 PARALL=.TRUE. $END<br /> $BASIS GBASIS=CUSTOMNI EXTFIL=.t. $END<br /> $SCF DIRSCF=.TRUE. FDIFF=.f. $END<br /> $STATPT OPTTOL=0.0001 NSTEP=500 HSSEND=.t. $END<br /> $DATA<br /> <br /> An important thing to note is the spacing. In the input deck, there must be 1 space at the beginning of each line of the input deck. If not, the job will fail. Most builders will insert this space anyway, but it helps to double check.<br /> <br /> The end of the input deck is marked by the "$DATA" line.<br /> <br /> =====Job Title Line=====<br /> <br /> The next line of the file is the job title. It can be anthing you wish, however, we have found that to be on the safe side, we avoide using symbols or spaces<br /> <br /> Mo_BDT3<br /> <br /> =====Symmetry Point Group=====<br /> <br /> The next line of the file is the symmetry point group of your molecule. Note that there is no leading space before the point group.<br /> <br /> C1<br /> <br /> =====Coordinates=====<br /> <br /> The next block of text is set aside for the coordinates of the molecule. This can be in internal (or z-matrix) format or cartesian coordinates. Note that there is no leading space before the coordinates. One may use the chemical symbol or the full name of each atom in the molecule. Note that the end of the coordinates is signified by an "$END", which MUST have one space preceding it. The coordinates below do NOT have any basis set information inserted. It is possible to insert basis set information directly into the input file. This is accomplished by obtaining the desired basis set parameters from the EMSL and then inserting them below each relevant atom. An example input file with inserted basis set information will be shown later.<br /> <br /> MOLYBDENUM 42.0 5.7556500000 4.4039600000 16.5808400000<br /> SULFUR 16.0 7.4169700000 3.1956300000 15.2089300000<br /> SULFUR 16.0 4.0966800000 3.2258300000 15.1761100000<br /> SULFUR 16.0 3.9677300000 4.4940500000 18.3266100000<br /> SULFUR 16.0 7.1776900000 3.5815000000 18.4485200000<br /> SULFUR 16.0 4.3776600000 6.2447400000 15.6786900000<br /> SULFUR 16.0 7.5478700000 6.0679800000 16.2223700000<br /> CARBON 6.0 6.4716900000 2.1004800000 14.1902300000<br /> CARBON 6.0 5.0690300000 2.1781400000 14.1080700000<br /> CARBON 6.0 4.8421800000 4.2701300000 19.8855500000<br /> CARBON 6.0 6.1969000000 3.9249600000 19.9397400000<br /> CARBON 6.0 6.8280600000 3.7834200000 21.1913200000<br /> CARBON 6.0 5.7697600000 7.6933500000 17.4241800000<br /> CARBON 6.0 7.2043100000 7.9413600000 17.8281100000<br /> CARBON 6.0 5.5051400000 7.0409700000 14.5903800000<br /> CARBON 6.0 6.8905200000 6.9194700000 14.7626200000<br /> CARBON 6.0 7.7396400000 7.5379800000 13.8285700000<br /> HYDROGEN 1.0 8.8190700000 7.4520600000 13.9252200000<br /> CARBON 6.0 7.2169400000 8.2960300000 12.7704100000<br /> HYDROGEN 1.0 7.8667000000 8.7825100000 12.0575600000<br /> CARBON 6.0 5.8260300000 8.4502300000 12.6467800000<br /> HYDROGEN 1.0 5.4143000000 9.0544300000 11.8493100000<br /> CARBON 6.0 4.9881500000 7.8192300000 13.5528400000<br /> HYDROGEN 1.0 3.9090500000 7.9420000000 13.4583700000<br /> CARBON 6.0 7.1538500000 1.1569600000 13.4143900000<br /> CARBON 6.0 4.4018100000 1.3603900000 13.1919900000<br /> CARBON 6.0 6.4791600000 0.3185500000 12.5353300000<br /> CARBON 6.0 5.0837400000 0.4369500000 12.4084900000<br /> HYDROGEN 1.0 7.0116000000 -0.4099400000 11.9434600000<br /> HYDROGEN 1.0 8.2399000000 1.0702400000 13.4937600000<br /> HYDROGEN 1.0 3.3185600000 1.4368700000 13.0953100000<br /> HYDROGEN 1.0 4.5549800000 -0.1997300000 11.7165200000<br /> CARBON 6.0 6.1105700000 3.9639000000 22.3866100000<br /> CARBON 6.0 4.1216300000 4.4424400000 21.1020100000<br /> HYDROGEN 1.0 7.8732900000 3.5217100000 21.2520500000<br /> CARBON 6.0 4.7606000000 4.2868500000 22.3363800000<br /> HYDROGEN 1.0 6.6064200000 3.8406000000 23.3428500000<br /> HYDROGEN 1.0 4.2065000000 4.4170700000 23.2667100000<br /> HYDROGEN 1.0 3.0674000000 4.6893500000 21.0889000000<br /> HYDROGEN 1.0 7.4249200000 7.7545300000 18.8583200000<br /> HYDROGEN 1.0 7.6651700000 8.9049700000 17.7652100000<br /> HYDROGEN 1.0 5.3324000000 8.6487800000 17.2222700000<br /> HYDROGEN 1.0 5.5015000000 7.1039000000 18.2759400000<br /> $END<br /> <br /> =====Effective Core Potential Data=====<br /> <br /> The effective core potential (ECP) data is entered after the coordinates. It starts with "$ECP", which must be preceded with a space. The atoms of the molecule are listed in the same order as in the coordinates section and the parameters for the ECP are listed after each atom. Note that for any atom that does NOT have an ECP, one must enter "ECP-NONE" or "NONE" after each atom without an ECP.<br /> <br /> $ECP<br /> MO-ECP GEN 28 3<br /> 5 ----- f potential -----<br /> -0.0469492 0 537.9667807 <br /> -20.2080084 1 147.8982938 <br /> -106.2116302 2 45.7358898 <br /> -41.8107368 2 13.2911467 <br /> -4.2054103 2 4.7059961 <br /> 3 ----- s-f potential -----<br /> 2.8063717 0 110.2991760 <br /> 44.5162012 1 23.2014645 <br /> 82.7785227 2 5.3530131 <br /> 4 ----- p-f potential -----<br /> 4.9420876 0 63.2901397 <br /> 25.8604976 1 23.3315302 <br /> 132.4708742 2 24.6759423 <br /> 57.3149794 2 4.6493040 <br /> 5 ----- d-f potential -----<br /> 3.0054591 0 104.4839977 <br /> 26.3637851 1 66.2307245 <br /> 183.3849199 2 39.1283176 <br /> 98.4453068 2 13.1164437 <br /> 22.4901377 2 3.6280263 <br /> S NONE<br /> S NONE<br /> S NONE<br /> S NONE<br /> S NONE<br /> S NONE<br /> C NONE<br /> C NONE<br /> C NONE<br /> C NONE<br /> C NONE<br /> C NONE<br /> C NONE<br /> C NONE<br /> C NONE<br /> C NONE<br /> H NONE<br /> C NONE<br /> H NONE<br /> C NONE<br /> H NONE<br /> C NONE<br /> H NONE<br /> C NONE<br /> C NONE<br /> C NONE<br /> C NONE<br /> H NONE<br /> H NONE<br /> H NONE<br /> H NONE<br /> C NONE<br /> C NONE<br /> H NONE<br /> C NONE<br /> H NONE<br /> H NONE<br /> H NONE<br /> H NONE<br /> H NONE<br /> H NONE<br /> H NONE<br /> $END<br /> <br /> -- [[User:M.Zimmer-De Iuliis|mzd]] 16 November 2009<br /> <br /> ====Using Effective Core Potentials in GAMESS(US)====<br /> <br /> For many metal containing compounds, it is very convenient and time saving to use an effective core potential (ECP) for the core metal electrons, as they are usually not important to the reactivity of the complex or the geometry around the metal. Since GAMESS(US) has a limited number of built-in ECPs, one may want to make GAMESS(US) read an external file that contains the ECP data using the "EXTFIL" keyword in the $GBASIS command line of the input file. In addition, to make GAMESS(US) use this external file, one must copy the "rungms" file and modify it accordingly. The following is a list of instructions with commands that will work from a terminal. One could also use WinSCP to do all of this with a GUI rather than a TUI. <br /> <br /> =====Modifiying rungms to Use Custom Basis Set File=====<br /> 1. Copy "rungms" from /scinet/gpc/Applications/gamess to one's own /scratch/$USER/ directory:<br /> cp /scinet/gpc/Applications/gamess/rungms /scratch/$USER/<br /> <br /> 2. Change to the scratch directory and check to see if "rungms" has copied successfully.<br /> cd /scratch/$USER<br /> ls<br /> <br /> 3. Edit line 147 of the vi. <br /> vi rungms<br /> Move the cursor down to line 147 using the arrow keys. It should say "setenv EXTBAS /dev/null". Using the arrow keys, move the cursor to the first "/" and then hit "i" to insert text. Put the path to your external basis file here. For example, /scratch/$USER/basisset. Then hit "escape". To save the changes and exit vi, type ":" and you should see a colon appear at the bottom of the window. Type "wq" (which should appear at the bottom of the window next to the colon) and then hit enter. Now you are done with vi.<br /> <br /> =====Creating a Custom Basis Set File=====<br /> 1. To create a custom basis set file, you need create a new text document. Our group's common practice is to comment out the first line of this file by inserting an exclamation mark (!) followed by noting the specific basis sets and ECPs that are going to be used for each of the atoms. Let us the molecule Mo(CO)6, Molybdenum hexacarbonyl, as an example. Below is the first line of the the external file, which we will call "CUSTOMMO" (NOTE: you can use any name for the external file that suits you, as long as it has no spaces and is 8 characters or less).<br /> <br /> ! 6-31G on C and O and LANL2D2 ECP on Mo<br /> <br /> 2. The next step is to visit the [https://bse.pnl.gov/bse/portal EMSL Basis Set exchange] and select C and O from the periodic table. Then, on the left of the page, select "6-31G" as the basis set. Finally, make sure the output is in GAMESS(US) format using the drop-down menu and then click "get basis set".<br /> <br /> [[File:C_O_6_31G_basisset.JPG|centre]]<br /> <br /> 3. A new window should appear with text in it. For our example case, the text looks like this:<br /> <br /> ! 6-31G EMSL Basis Set Exchange Library 10/13/09 11:12 AM<br /> ! Elements References<br /> ! -------- ----------<br /> ! H - He: W.J. Hehre, R. Ditchfield and J.A. Pople, J. Chem. Phys. 56,<br /> ! Li - Ne: 2257 (1972). Note: Li and B come from J.D. Dill and J.A.<br /> ! Pople, J. Chem. Phys. 62, 2921 (1975).<br /> ! Na - Ar: M.M. Francl, W.J. Petro, W.J. Hehre, J.S. Binkley, M.S. Gordon,<br /> ! D.J. DeFrees and J.A. Pople, J. Chem. Phys. 77, 3654 (1982)<br /> ! K - Zn: V. Rassolov, J.A. Pople, M. Ratner and T.L. Windus, J. Chem. Phys.<br /> ! 109, 1223 (1998)<br /> ! Note: He and Ne are unpublished basis sets taken from the Gaussian<br /> ! program<br /> ! <br /> $DATA<br /><br /> CARBON<br /> S 6<br /> 1 3047.5249000 0.0018347 <br /> 2 457.3695100 0.0140373 <br /> 3 103.9486900 0.0688426 <br /> 4 29.2101550 0.2321844 <br /> 5 9.2866630 0.4679413 <br /> 6 3.1639270 0.3623120 <br /> L 3<br /> 1 7.8682724 -0.1193324 0.0689991 <br /> 2 1.8812885 -0.1608542 0.3164240 <br /> 3 0.5442493 1.1434564 0.7443083 <br /> L 1<br /> 1 0.1687144 1.0000000 1.0000000<br /> <br /> OXYGEN<br /> S 6<br /> 1 5484.6717000 0.0018311 <br /> 2 825.2349500 0.0139501 <br /> 3 188.0469600 0.0684451 <br /> 4 52.9645000 0.2327143 <br /> 5 16.8975700 0.4701930 <br /> 6 5.7996353 0.3585209 <br /> L 3<br /> 1 15.5396160 -0.1107775 0.0708743 <br /> 2 3.5999336 -0.1480263 0.3397528 <br /> 3 1.0137618 1.1307670 0.7271586 <br /> L 1<br /> 1 0.2700058 1.0000000 1.0000000 <br /> $END<br /> <br /> 3. Now, copy and paste the text between the $DATA and $END headings onto our external text file, CUSTOMMO. We also need to change the change the name of each element to it symbol in the periodic table. Finally, we need to add the name of the external file next to the element symbol, separated by one space. Note that there should be a blank line separating the basis set information and the first, commented-out line (The line starting with the '!'). The CUSTOMMO should look like this:<br /> <br /> ! 6-31G on C and O and LANL2D2 ECP on Mo<br /><br /> C CUSTOMMO<br /> S 6<br /> 1 3047.5249000 0.0018347 <br /> 2 457.3695100 0.0140373 <br /> 3 103.9486900 0.0688426 <br /> 4 29.2101550 0.2321844 <br /> 5 9.2866630 0.4679413 <br /> 6 3.1639270 0.3623120 <br /> L 3<br /> 1 7.8682724 -0.1193324 0.0689991 <br /> 2 1.8812885 -0.1608542 0.3164240 <br /> 3 0.5442493 1.1434564 0.7443083 <br /> L 1<br /> 1 0.1687144 1.0000000 1.0000000<br /> <br /> O CUSTOMMO<br /> S 6<br /> 1 5484.6717000 0.0018311 <br /> 2 825.2349500 0.0139501 <br /> 3 188.0469600 0.0684451 <br /> 4 52.9645000 0.2327143 <br /> 5 16.8975700 0.4701930 <br /> 6 5.7996353 0.3585209 <br /> L 3<br /> 1 15.5396160 -0.1107775 0.0708743 <br /> 2 3.5999336 -0.1480263 0.3397528 <br /> 3 1.0137618 1.1307670 0.7271586 <br /> L 1<br /> 1 0.2700058 1.0000000 1.0000000 <br /> <br /> 4. Repeat Step 3 above but choose Mo and select the LANL2DZ ECP instead. A new window will pop up with the basis set information as well as the ECP data we need, since we specified the LANL2DZ '''ECP'''. The ECP data is not inserted into the external file, rather it is placed into the input file itself (More on this later). <br /> <br /> [[File:Mo_LANL2DZ_basisset.JPG|centre]]<br /> <br /> 5. After copying the molybdenum basis set information, your fiished external basis set file should look like this:<br /> ! 6-31G on C and O and LANL2D2 ECP on Mo<br /><br /> C CUSTOMMO<br /> S 6<br /> 1 3047.5249000 0.0018347 <br /> 2 457.3695100 0.0140373 <br /> 3 103.9486900 0.0688426 <br /> 4 29.2101550 0.2321844 <br /> 5 9.2866630 0.4679413 <br /> 6 3.1639270 0.3623120 <br /> L 3<br /> 1 7.8682724 -0.1193324 0.0689991 <br /> 2 1.8812885 -0.1608542 0.3164240 <br /> 3 0.5442493 1.1434564 0.7443083 <br /> L 1<br /> 1 0.1687144 1.0000000 1.0000000<br /> <br /> O CUSTOMMO<br /> S 6<br /> 1 5484.6717000 0.0018311 <br /> 2 825.2349500 0.0139501 <br /> 3 188.0469600 0.0684451 <br /> 4 52.9645000 0.2327143 <br /> 5 16.8975700 0.4701930 <br /> 6 5.7996353 0.3585209 <br /> L 3<br /> 1 15.5396160 -0.1107775 0.0708743 <br /> 2 3.5999336 -0.1480263 0.3397528 <br /> 3 1.0137618 1.1307670 0.7271586 <br /> L 1<br /> 1 0.2700058 1.0000000 1.0000000<br /> <br /> Mo CUSTOMO<br /> S 3<br /> 1 2.3610000 -0.9121760 <br /> 2 1.3090000 1.1477453 <br /> 3 0.4500000 0.6097109 <br /> S 4<br /> 1 2.3610000 0.8139259 <br /> 2 1.3090000 -1.1360084 <br /> 3 0.4500000 -1.1611592 <br /> 4 0.1681000 1.0064786 <br /> S 1<br /> 1 0.0423000 1.0000000 <br /> P 3<br /> 1 4.8950000 -0.0908258 <br /> 2 1.0440000 0.7042899 <br /> 3 0.3877000 0.3973179 <br /> P 2<br /> 1 0.4995000 -0.1081945 <br /> 2 0.0780000 1.0368093 <br /> P 1<br /> 1 0.0247000 1.0000000 <br /> D 3<br /> 1 2.9930000 0.0527063 <br /> 2 1.0630000 0.5003907 <br /> 3 0.3721000 0.5794024 <br /> D 1<br /> 1 0.1178000 1.0000000<br /> <br /> -- [[User:M.Zimmer-De Iuliis|mzd]] 13 October 2009<br /> <br /> ==Climate Modelling==<br /> <br /> ==Medicine/Bio==<br /> <br /> ==High Energy Physics==<br /> <br /> ==Structural Biology==<br /> Molecular simulation of proteins, lipids, carbohydrates, and other biologically relevant molecules.<br /> ===Molecular Dynamics (MD) simulation===<br /> ====GROMACS====<br /> Please refer to the [[gromacs|GROMACS]] page<br /> ====AMBER====<br /> Please refer to the [[amber|AMBER]] page<br /> ====NAMD====<br /> NAMD is one of the better scaling MD packages out there. With sufficiently large systems, it is able to scale to hundreds or thousands of cores on Scinet. Below are details for compiling and running NAMD on Scinet.<br /> <br /> More information regarding performance and different compile options coming soon...<br /> <br /> =====Compiling NAMD for GPC=====<br /> Ensure the proper compiler/mpi modules are loaded.<br /> <source lang="sh"><br /> module load intel<br /> module load openmpi/1.3.3-intel-v11.0-ofed<br /> </source><br /> <br /> '''Compile Charm++ and NAMD'''<br /> <source lang="sh"><br /> #Unpack source files and get required support libraries<br /> tar -xzf NAMD_2.7b1_Source.tar.gz<br /> cd NAMD_2.7b1_Source<br /> tar -xf charm-6.1.tar<br /> wget http://www.ks.uiuc.edu/Research/namd/libraries/fftw-linux-x86_64.tar.gz<br /> wget http://www.ks.uiuc.edu/Research/namd/libraries/tcl-linux-x86_64.tar.gz<br /> tar -xzf fftw-linux-x86_64.tar.gz; mv linux-x86_64 fftw<br /> tar -xzf tcl-linux-x86_64.tar.gz; mv linux-x86_64 tcl<br /> #Compile Charm++<br /> cd charm-6.1<br /> ./build charm++ mpi-linux-x86_64 icc --basedir /scinet/gpc/mpi/openmpi/1.3.3-intel-v11.0-ofed/ --no-shared -O -DCMK_OPTIMIZE=1<br /> cd ..<br /> #Compile NAMD. <br /> #Edit arch/Linux-x86_64-icc.arch and add "-lmpi" to the end of the CXXOPTS and COPTS line.<br /> #Make a builds directory if you want different versions of NAMD compiled at the same time.<br /> mkdir builds<br /> ./config builds/Linux-x86_64-icc --charm-arch mpi-linux-x86_64-icc<br /> cd builds/Linux-x86_64-icc/<br /> make -j4 namd2 # Adjust value of j as desired to specify number of simultaneous make targets. <br /> </source><br /> --[[User:Cmadill|Cmadill]] 16:18, 27 August 2009 (UTC)<br /> <br /> =====Running Fortran=====<br /> On the development nodes, there is an old gcc. The associated libraries are not on the compute nodes. Ensure the line:<br /> <br /> module load gcc<br /> <br /> is in your .bashrc file.<br /> <br /> ====LAMMPS====<br /> [[Image:StrongScalingLAMMPS.png|thumb|320px|right|Strong scaling test on GPC with OpenMPI and IntelMPI on Ethernet and InfiniBand]]<br /> [[Image:WeakScalingLAMMPS.png|thumb|320px|right|Weak scaling test on GPC with OpenMPI and IntelMPI on Ethernet and InfiniBand]]<br /> LAMMPS is a parallel MD code that can be found [http://lammps.sandia.gov/ here].<br /> <br /> '''Scaling Tests on GPC'''<br /> <br /> Results from strong scaling tests for LAMMPS using EAM potentials on GPC are shown in the graph on the right. Test simulation ran 500 timesteps for 4,000,000 atoms.<br /> <br /> Results from weak scaling tests for LAMMPS using EAM potentials on GPC are shown in the graph on the right. Test simulation ran 500 timesteps for 32,000 atoms per processor.<br /> <br /> OpenMPI version used: openmpi/1.4.1-intel-v11.0-ofed<br /> <br /> IntelMPI version used: intelmpi/impi-4.0.0.013<br /> <br /> '''Summary of Scaling Tests'''<br /> <br /> Results show good scaling for both OpenMPI and IntelMPI up to 16 processors, after which performance begins to suffer. On Infiniband, excellent scaling is maintained to 512 processors.<br /> <br /> IntelMPI shows slightly better performance compared to OpenMPI when running with Infiniband.<br /> <br /> --[[User:jchu|jchu]] Feb 2, 2010<br /> <br /> ===Monte Carlo (MC) simulation===</div>

Jchu https://oldwiki.scinet.utoronto.ca/index.php?title=User_Codes&diff=881 User Codes 2010-02-02T19:06:05Z

<p>Jchu: /* LAMMPS */</p> <hr /> <div>__FORCETOC__<br /> <br /> ==Astrophysics==<br /> <br /> ===Athena (explicit, uniform grid MHD code)===<br /> <br /> [[Image:StrongScalingAthenaGPC.png|thumb|right|320px|Athena scaling on GPC with OpenMPI and MVAPICH2 on GigE, and OpenMPI on InfiniBand]]<br /> <br /> [http://www.astro.princeton.edu/~jstone/athena.html Athena] is a straightforward C code which doesn't use a lot of libraries so it is pretty straightforward to build and compile on new machines. <br /> <br /> It encapsulates its compiler flags, etc in an <tt>Makeoptions.in</tt> file which is then processed by <tt>configure</tt>. I've used the following additions to <tt>Makeoptions.in</tt> on TCS and GPC:<br /> <br /> <source lang="sh"><br /> ifeq ($(MACHINE),scinettcs)<br /> CC = mpcc_r<br /> LDR = mpcc_r<br /> OPT = -O5 -q64 -qarch=pwr6 -qtune=pwr6 -qcache=auto -qlargepage -qstrict<br /> MPIINC =<br /> MPILIB =<br /> CFLAGS = $(OPT)<br /> LIB = -ldl -lm<br /> else<br /> ifeq ($(MACHINE),scinetgpc)<br /> CC = mpicc<br /> LDR = mpicc<br /> OPT = -O3<br /> MPIINC =<br /> MPILIB =<br /> CFLAGS = $(OPT)<br /> LIB = -lm<br /> else<br /> ...<br /> endif<br /> endif<br /> </source><br /> It performs quite well on the GPC, scaling extremely well even on a strong scaling test out to about 256 cores (32 nodes) on Gigabit ethernet, and performing beautifully on InfiniBand out to 512 cores (64 nodes). <br /> <br /> <br /> -- [[User:Ljdursi|ljdursi]] 19:20, 13 August 2009 (UTC)<br /> <br /> ===FLASH3 (Adaptive Mesh reactive hydrodynamics; explict hydro/MHD)===<br /> <br /> [[Image:weak-scaling-example.png|thumb|right|320px|Weak scaling test of the 2d sod problem on both the GPC and TCS. The results are actually somewhat faster on the GPC; in both cases (weak) scaling is very good out at least to 256 cores]]<br /> <br /> [http://flash.uchicago.edu FLASH] encapsulates its machine-dependant information in the <tt>FLASH3/sites</tt> directory. For the GPC, you'll have to<br /> <pre><br /> module load intel<br /> module load openmpi<br /> module load hdf5/183-v16-openmpi<br /> </pre><br /> <br /> and with that, the following file (<tt>sites/scinetgpc/Makefile.h</tt>) works for me:<br /> <source lang="sh"><br /> ## Must do module load hdf5/183-v16-openmpi<br /> HDF5_PATH = ${SCINET_HDF5_BASE}<br /> ZLIB_PATH = /usr/local<br /> <br /> #----------------------------------------------------------------------------<br /> # Compiler and linker commands<br /> #<br /> # We use the f90 compiler as the linker, so some C libraries may explicitly<br /> # need to be added into the link line.<br /> #----------------------------------------------------------------------------<br /> <br /> ## modules will put the right mpi in our path<br /> FCOMP = mpif77<br /> CCOMP = mpicc<br /> CPPCOMP = mpiCC<br /> LINK = mpif77<br /> <br /> #----------------------------------------------------------------------------<br /> # Compilation flags<br /> #<br /> # Three sets of compilation/linking flags are defined: one for optimized<br /> # code, one for testing, and one for debugging. The default is to use the <br /> # _OPT version. Specifying -debug to setup will pick the _DEBUG version,<br /> # these should enable bounds checking. Specifying -test is used for <br /> # flash_test, and is set for quick code generation, and (sometimes) <br /> # profiling. The Makefile generated by setup will assign the generic token <br /> # (ex. FFLAGS) to the proper set of flags (ex. FFLAGS_OPT).<br /> #----------------------------------------------------------------------------<br /> <br /> FFLAGS_OPT = -c -r8 -i4 -O3 -xSSE4.2<br /> FFLAGS_DEBUG = -c -g -r8 -i4 -O0<br /> FFLAGS_TEST = -c -r8 -i4<br /> <br /> <br /> # if we are using HDF5, we need to specify the path to the include files<br /> CFLAGS_HDF5 = -I${HDF5_PATH}/include<br /> <br /> CFLAGS_OPT = -c -O3 -xSSE4.2<br /> CFLAGS_TEST = -c -O2 <br /> CFLAGS_DEBUG = -c -g <br /> <br /> MDEFS = <br /> <br /> .SUFFIXES: .o .c .f .F .h .fh .F90 .f90<br /> <br /> #----------------------------------------------------------------------------<br /> # Linker flags<br /> #<br /> # There is a seperate version of the linker flags for each of the _OPT, <br /> # _DEBUG, and _TEST cases.<br /> #----------------------------------------------------------------------------<br /> <br /> LFLAGS_OPT = -o<br /> LFLAGS_TEST = -o<br /> LFLAGS_DEBUG = -g -o<br /> <br /> MACHOBJ = <br /> <br /> <br /> MV = mv -f<br /> AR = ar -r<br /> RM = rm -f<br /> CD = cd<br /> RL = ranlib<br /> ECHO = echo<br /> </source><br /> <br /> <br /> -- [[User:Ljdursi|ljdursi]] 22:11, 13 August 2009 (UTC)<br /> <br /> ==Aeronautics==<br /> <br /> ==Chemistry==<br /> <br /> ===GAMESS (US)===<br /> The GAMESS version January 12, 2009 R3 was built using the Intel v11.1 compilers and v3.2.2 MPI library, according to the instructions in http://software.intel.com/en-us/articles/building-gamess-with-intel-compilers-intel-mkl-and-intel-mpi-on-linux/<br /> <br /> The required build scripts - comp, compall, lked - and run script - rungms - were modified to account for our own installation. In order to build GAMESS one first must ensure that the intel and intelmpi modules are loaded ("module load intel intelmpi"). This applies to running GAMESS as well.<br /> The module "gamess" must also be loaded in order to run GAMESS ("module load gamess").<br /> <br /> The modified scripts are in the file /scinet/gpc/src/gamess-on-scinet.tar.gz<br /> <br /> ====Running GAMESS====<br /> - Make sure the directory /scratch/$USER/gamess-scratch exists (the $SCINET_RUNGMS script will create it if it does not exist)<br /> <br /> - Make sure the modules: intel, intelmpi, gamess are loaded (in your .bashrc: "module load intel intelmpi gamess").<br /> <br /> - Create a torque script to run GAMESS. Here is an example:<br /> <br /> - The GAMESS executable is in $SCINET_GAMESS_HOME/gamess.00.x<br /> - The rungms script is in $SCINET_GAMESS_HOME/rungms (actually it is $SCINET_RUNGMS)<br /> <br /> - For IB multinode runs, use the $SCINET_RUNGMS_IB script<br /> <br /> - The rungms script takes 4 arguments: input file, executable number, number of compute processes, processors per node<br /> <br /> For example, in order to run with the input file /scratch/$USER/gamesstest01, on 8 cpus, and the default version (00)<br /> of the executable on a machine with 8 cores:<br /> <br /> # load the gamess module in .bashrc<br /> module load gamess <br /> <br /> # run the program<br /> $SCINET_RUNGMS /scratch/$USER/gamesstest01 00 8 8<br /> <br /> Here is a sample torque script for running a GAMESS calculation, on a single 8-core node:<br /> <br /> <source lang="bash"><br /> #!/bin/bash<br /> #PBS -l nodes=1:ppn=8,walltime=48:00:00,os=centos53computeA<br /> #PBS -N gamessjob<br /> <br /> ## To submit type: qsub gms.sh<br /> <br /> # If not an interactive job (i.e. -I), then cd into the directory where<br /> # I typed qsub.<br /> if [ "$PBS_ENVIRONMENT" != "PBS_INTERACTIVE" ]; then<br /> if [ -n "$PBS_O_WORKDIR" ]; then<br /> cd $PBS_O_WORKDIR<br /> fi<br /> fi<br /> <br /> # the input file is typically named something like "gamesjob.inp"<br /> # so the script will be run like "$SCINET_RUNGMS gamessjob 00 8 8"<br /> <br /> # load the gamess module if not in .bashrc already<br /> # actually, it MUST be in .bashrc<br /> # module load gamess<br /> <br /> # run the program<br /> <br /> $SCINET_RUNGMS gamessjob 00 8 8<br /> </source><br /> <br /> Here is a similar script, but this one uses 2 InfiniBand-connected nodes,<br /> and runs the appropriate $SCINET_RUNGMS_IB script to actually run the job:<br /> <br /> <source lang="bash"><br /> #!/bin/bash<br /> #PBS -l nodes=2:ib:ppn=8,walltime=48:00:00,os=centos53computeA<br /> #PBS -N gamessjob<br /> <br /> ## To submit type: qsub gmsib.sh<br /> <br /> # If not an interactive job (i.e. -I), then cd into the directory where<br /> # I typed qsub.<br /> if [ "$PBS_ENVIRONMENT" != "PBS_INTERACTIVE" ]; then<br /> if [ -n "$PBS_O_WORKDIR" ]; then<br /> cd $PBS_O_WORKDIR<br /> fi<br /> fi<br /> <br /> # the input file is typically named something like "gamesjob.inp"<br /> # so the script will be run like "$SCINET_RUNGMS gamessjob 00 8 8"<br /> <br /> # load the gamess module if not in .bashrc already<br /> # actually, it MUST be in .bashrc<br /> # module load gamess<br /> <br /> # This script requests InfiniBand-connected nodes (:ib above)<br /> # so it must run with the IB version of the rungms script,<br /> # $SCINET_RUNGMS_IB<br /> <br /> $SCINET_RUNGMS_IB gamessjob 00 16 8<br /> </source><br /> <br /> <br /> -- [[User:Dgruner|dgruner]] 5 October 2009<br /> <br /> ===Tips from the Fekl Lab===<br /> <br /> Through trial and error, we have found a few useful things that we would like to share:<br /> <br /> 1. Two very useful, open-source programs for visualization of output files from GAMESS(US) and for generation of input files are [http://www.scl.ameslab.gov/MacMolPlt/ MacMolPlt]and [http://avogadro.openmolecules.net/wiki/Main_Page Avogadro]. The are available for UNIX/LINUX, Windows and Mac based machines, HOWEVER: any input files that we have generated with these programs on a Windows-based machine do not run on Mac based machines. We don't know why.<br /> <br /> 2. [http://winscp.net/eng/index.php WinSCP] is a very useful tool that has a graphical user interface for moving files from a local machine to SCINET and vice versa. It also has text editing capabilities.<br /> <br /> 3. The [https://bse.pnl.gov/bse/portal ESML Basis Set Exchange] is an excellent source for custom basis set or effective core potential parameters. Make sure that you specify "Gamess-US" in the format drop-down box.<br /> <br /> 4. The commercial program [http://www.chemcraftprog.com/ ChemCraft] is a highly useful visualization program that has the ability to edit molecules in a very similar fashion to GaussView. It can also be customized to build GAMESS(US) input files.<br /> <br /> ====Anatomy of a GAMESS(US) Input File with Basis Set Info in an External File====<br /> <br /> $CONTRL SCFTYP=RHF RUNTYP=OPTIMIZE DFTTYP=M06-L MAXIT=199 MULT=1 NOSYM=1<br /> ECP=READ $END<br /> $SYSTEM TIMLIM=525600 MWORDS=1750 PARALL=.TRUE. $END<br /> $BASIS GBASIS=CUSTOMNI EXTFIL=.t. $END<br /> $SCF DIRSCF=.TRUE. FDIFF=.f. $END<br /> $STATPT OPTTOL=0.0001 NSTEP=500 HSSEND=.t. $END<br /> $DATA<br /> Mo_BDT3<br /> C1<br /> MOLYBDENUM 42.0 5.7556500000 4.4039600000 16.5808400000<br /> SULFUR 16.0 7.4169700000 3.1956300000 15.2089300000<br /> SULFUR 16.0 4.0966800000 3.2258300000 15.1761100000<br /> SULFUR 16.0 3.9677300000 4.4940500000 18.3266100000<br /> SULFUR 16.0 7.1776900000 3.5815000000 18.4485200000<br /> SULFUR 16.0 4.3776600000 6.2447400000 15.6786900000<br /> SULFUR 16.0 7.5478700000 6.0679800000 16.2223700000<br /> CARBON 6.0 6.4716900000 2.1004800000 14.1902300000<br /> CARBON 6.0 5.0690300000 2.1781400000 14.1080700000<br /> CARBON 6.0 4.8421800000 4.2701300000 19.8855500000<br /> CARBON 6.0 6.1969000000 3.9249600000 19.9397400000<br /> CARBON 6.0 6.8280600000 3.7834200000 21.1913200000<br /> CARBON 6.0 5.7697600000 7.6933500000 17.4241800000<br /> CARBON 6.0 7.2043100000 7.9413600000 17.8281100000<br /> CARBON 6.0 5.5051400000 7.0409700000 14.5903800000<br /> CARBON 6.0 6.8905200000 6.9194700000 14.7626200000<br /> CARBON 6.0 7.7396400000 7.5379800000 13.8285700000<br /> HYDROGEN 1.0 8.8190700000 7.4520600000 13.9252200000<br /> CARBON 6.0 7.2169400000 8.2960300000 12.7704100000<br /> HYDROGEN 1.0 7.8667000000 8.7825100000 12.0575600000<br /> CARBON 6.0 5.8260300000 8.4502300000 12.6467800000<br /> HYDROGEN 1.0 5.4143000000 9.0544300000 11.8493100000<br /> CARBON 6.0 4.9881500000 7.8192300000 13.5528400000<br /> HYDROGEN 1.0 3.9090500000 7.9420000000 13.4583700000<br /> CARBON 6.0 7.1538500000 1.1569600000 13.4143900000<br /> CARBON 6.0 4.4018100000 1.3603900000 13.1919900000<br /> CARBON 6.0 6.4791600000 0.3185500000 12.5353300000<br /> CARBON 6.0 5.0837400000 0.4369500000 12.4084900000<br /> HYDROGEN 1.0 7.0116000000 -0.4099400000 11.9434600000<br /> HYDROGEN 1.0 8.2399000000 1.0702400000 13.4937600000<br /> HYDROGEN 1.0 3.3185600000 1.4368700000 13.0953100000<br /> HYDROGEN 1.0 4.5549800000 -0.1997300000 11.7165200000<br /> CARBON 6.0 6.1105700000 3.9639000000 22.3866100000<br /> CARBON 6.0 4.1216300000 4.4424400000 21.1020100000<br /> HYDROGEN 1.0 7.8732900000 3.5217100000 21.2520500000<br /> CARBON 6.0 4.7606000000 4.2868500000 22.3363800000<br /> HYDROGEN 1.0 6.6064200000 3.8406000000 23.3428500000<br /> HYDROGEN 1.0 4.2065000000 4.4170700000 23.2667100000<br /> HYDROGEN 1.0 3.0674000000 4.6893500000 21.0889000000<br /> HYDROGEN 1.0 7.4249200000 7.7545300000 18.8583200000<br /> HYDROGEN 1.0 7.6651700000 8.9049700000 17.7652100000<br /> HYDROGEN 1.0 5.3324000000 8.6487800000 17.2222700000<br /> HYDROGEN 1.0 5.5015000000 7.1039000000 18.2759400000<br /> $END<br /> $ECP<br /> MO-ECP GEN 28 3<br /> 5 ----- f potential -----<br /> -0.0469492 0 537.9667807 <br /> -20.2080084 1 147.8982938 <br /> -106.2116302 2 45.7358898 <br /> -41.8107368 2 13.2911467 <br /> -4.2054103 2 4.7059961 <br /> 3 ----- s-f potential -----<br /> 2.8063717 0 110.2991760 <br /> 44.5162012 1 23.2014645 <br /> 82.7785227 2 5.3530131 <br /> 4 ----- p-f potential -----<br /> 4.9420876 0 63.2901397 <br /> 25.8604976 1 23.3315302 <br /> 132.4708742 2 24.6759423 <br /> 57.3149794 2 4.6493040 <br /> 5 ----- d-f potential -----<br /> 3.0054591 0 104.4839977 <br /> 26.3637851 1 66.2307245 <br /> 183.3849199 2 39.1283176 <br /> 98.4453068 2 13.1164437 <br /> 22.4901377 2 3.6280263 <br /> S NONE<br /> S NONE<br /> S NONE<br /> S NONE<br /> S NONE<br /> S NONE<br /> C NONE<br /> C NONE<br /> C NONE<br /> C NONE<br /> C NONE<br /> C NONE<br /> C NONE<br /> C NONE<br /> C NONE<br /> C NONE<br /> H NONE<br /> C NONE<br /> H NONE<br /> C NONE<br /> H NONE<br /> C NONE<br /> H NONE<br /> C NONE<br /> C NONE<br /> C NONE<br /> C NONE<br /> H NONE<br /> H NONE<br /> H NONE<br /> H NONE<br /> C NONE<br /> C NONE<br /> H NONE<br /> C NONE<br /> H NONE<br /> H NONE<br /> H NONE<br /> H NONE<br /> H NONE<br /> H NONE<br /> H NONE<br /> $END<br /> <br /> =====The Input Deck=====<br /> <br /> Below is the input deck. It is where you tell GAMESS(US) what job type to execute and where all you individual parameters are entered for your specific job type. The example input deck below is for a geometry optimization and frequency calculation. This input deck is equivalent to the Gaussian job with "opt" and "freq" in the route section.<br /> <br /> $CONTRL SCFTYP=RHF RUNTYP=OPTIMIZE DFTTYP=M06-L MAXIT=199 MULT=1 NOSYM=1<br /> ECP=READ $END<br /> $SYSTEM TIMLIM=2850 MWORDS=1750 MEMDDI=20 PARALL=.TRUE. $END<br /> $BASIS GBASIS=CUSTOMNI EXTFIL=.t. $END<br /> $SCF DIRSCF=.TRUE. FDIFF=.f. $END<br /> $STATPT OPTTOL=0.0001 NSTEP=500 HSSEND=.t. $END<br /> $DATA<br /> <br /> An important thing to note is the spacing. In the input deck, there must be 1 space at the beginning of each line of the input deck. If not, the job will fail. Most builders will insert this space anyway, but it helps to double check.<br /> <br /> The end of the input deck is marked by the "$DATA" line.<br /> <br /> =====Job Title Line=====<br /> <br /> The next line of the file is the job title. It can be anthing you wish, however, we have found that to be on the safe side, we avoide using symbols or spaces<br /> <br /> Mo_BDT3<br /> <br /> =====Symmetry Point Group=====<br /> <br /> The next line of the file is the symmetry point group of your molecule. Note that there is no leading space before the point group.<br /> <br /> C1<br /> <br /> =====Coordinates=====<br /> <br /> The next block of text is set aside for the coordinates of the molecule. This can be in internal (or z-matrix) format or cartesian coordinates. Note that there is no leading space before the coordinates. One may use the chemical symbol or the full name of each atom in the molecule. Note that the end of the coordinates is signified by an "$END", which MUST have one space preceding it. The coordinates below do NOT have any basis set information inserted. It is possible to insert basis set information directly into the input file. This is accomplished by obtaining the desired basis set parameters from the EMSL and then inserting them below each relevant atom. An example input file with inserted basis set information will be shown later.<br /> <br /> MOLYBDENUM 42.0 5.7556500000 4.4039600000 16.5808400000<br /> SULFUR 16.0 7.4169700000 3.1956300000 15.2089300000<br /> SULFUR 16.0 4.0966800000 3.2258300000 15.1761100000<br /> SULFUR 16.0 3.9677300000 4.4940500000 18.3266100000<br /> SULFUR 16.0 7.1776900000 3.5815000000 18.4485200000<br /> SULFUR 16.0 4.3776600000 6.2447400000 15.6786900000<br /> SULFUR 16.0 7.5478700000 6.0679800000 16.2223700000<br /> CARBON 6.0 6.4716900000 2.1004800000 14.1902300000<br /> CARBON 6.0 5.0690300000 2.1781400000 14.1080700000<br /> CARBON 6.0 4.8421800000 4.2701300000 19.8855500000<br /> CARBON 6.0 6.1969000000 3.9249600000 19.9397400000<br /> CARBON 6.0 6.8280600000 3.7834200000 21.1913200000<br /> CARBON 6.0 5.7697600000 7.6933500000 17.4241800000<br /> CARBON 6.0 7.2043100000 7.9413600000 17.8281100000<br /> CARBON 6.0 5.5051400000 7.0409700000 14.5903800000<br /> CARBON 6.0 6.8905200000 6.9194700000 14.7626200000<br /> CARBON 6.0 7.7396400000 7.5379800000 13.8285700000<br /> HYDROGEN 1.0 8.8190700000 7.4520600000 13.9252200000<br /> CARBON 6.0 7.2169400000 8.2960300000 12.7704100000<br /> HYDROGEN 1.0 7.8667000000 8.7825100000 12.0575600000<br /> CARBON 6.0 5.8260300000 8.4502300000 12.6467800000<br /> HYDROGEN 1.0 5.4143000000 9.0544300000 11.8493100000<br /> CARBON 6.0 4.9881500000 7.8192300000 13.5528400000<br /> HYDROGEN 1.0 3.9090500000 7.9420000000 13.4583700000<br /> CARBON 6.0 7.1538500000 1.1569600000 13.4143900000<br /> CARBON 6.0 4.4018100000 1.3603900000 13.1919900000<br /> CARBON 6.0 6.4791600000 0.3185500000 12.5353300000<br /> CARBON 6.0 5.0837400000 0.4369500000 12.4084900000<br /> HYDROGEN 1.0 7.0116000000 -0.4099400000 11.9434600000<br /> HYDROGEN 1.0 8.2399000000 1.0702400000 13.4937600000<br /> HYDROGEN 1.0 3.3185600000 1.4368700000 13.0953100000<br /> HYDROGEN 1.0 4.5549800000 -0.1997300000 11.7165200000<br /> CARBON 6.0 6.1105700000 3.9639000000 22.3866100000<br /> CARBON 6.0 4.1216300000 4.4424400000 21.1020100000<br /> HYDROGEN 1.0 7.8732900000 3.5217100000 21.2520500000<br /> CARBON 6.0 4.7606000000 4.2868500000 22.3363800000<br /> HYDROGEN 1.0 6.6064200000 3.8406000000 23.3428500000<br /> HYDROGEN 1.0 4.2065000000 4.4170700000 23.2667100000<br /> HYDROGEN 1.0 3.0674000000 4.6893500000 21.0889000000<br /> HYDROGEN 1.0 7.4249200000 7.7545300000 18.8583200000<br /> HYDROGEN 1.0 7.6651700000 8.9049700000 17.7652100000<br /> HYDROGEN 1.0 5.3324000000 8.6487800000 17.2222700000<br /> HYDROGEN 1.0 5.5015000000 7.1039000000 18.2759400000<br /> $END<br /> <br /> =====Effective Core Potential Data=====<br /> <br /> The effective core potential (ECP) data is entered after the coordinates. It starts with "$ECP", which must be preceded with a space. The atoms of the molecule are listed in the same order as in the coordinates section and the parameters for the ECP are listed after each atom. Note that for any atom that does NOT have an ECP, one must enter "ECP-NONE" or "NONE" after each atom without an ECP.<br /> <br /> $ECP<br /> MO-ECP GEN 28 3<br /> 5 ----- f potential -----<br /> -0.0469492 0 537.9667807 <br /> -20.2080084 1 147.8982938 <br /> -106.2116302 2 45.7358898 <br /> -41.8107368 2 13.2911467 <br /> -4.2054103 2 4.7059961 <br /> 3 ----- s-f potential -----<br /> 2.8063717 0 110.2991760 <br /> 44.5162012 1 23.2014645 <br /> 82.7785227 2 5.3530131 <br /> 4 ----- p-f potential -----<br /> 4.9420876 0 63.2901397 <br /> 25.8604976 1 23.3315302 <br /> 132.4708742 2 24.6759423 <br /> 57.3149794 2 4.6493040 <br /> 5 ----- d-f potential -----<br /> 3.0054591 0 104.4839977 <br /> 26.3637851 1 66.2307245 <br /> 183.3849199 2 39.1283176 <br /> 98.4453068 2 13.1164437 <br /> 22.4901377 2 3.6280263 <br /> S NONE<br /> S NONE<br /> S NONE<br /> S NONE<br /> S NONE<br /> S NONE<br /> C NONE<br /> C NONE<br /> C NONE<br /> C NONE<br /> C NONE<br /> C NONE<br /> C NONE<br /> C NONE<br /> C NONE<br /> C NONE<br /> H NONE<br /> C NONE<br /> H NONE<br /> C NONE<br /> H NONE<br /> C NONE<br /> H NONE<br /> C NONE<br /> C NONE<br /> C NONE<br /> C NONE<br /> H NONE<br /> H NONE<br /> H NONE<br /> H NONE<br /> C NONE<br /> C NONE<br /> H NONE<br /> C NONE<br /> H NONE<br /> H NONE<br /> H NONE<br /> H NONE<br /> H NONE<br /> H NONE<br /> H NONE<br /> $END<br /> <br /> -- [[User:M.Zimmer-De Iuliis|mzd]] 16 November 2009<br /> <br /> ====Using Effective Core Potentials in GAMESS(US)====<br /> <br /> For many metal containing compounds, it is very convenient and time saving to use an effective core potential (ECP) for the core metal electrons, as they are usually not important to the reactivity of the complex or the geometry around the metal. Since GAMESS(US) has a limited number of built-in ECPs, one may want to make GAMESS(US) read an external file that contains the ECP data using the "EXTFIL" keyword in the $GBASIS command line of the input file. In addition, to make GAMESS(US) use this external file, one must copy the "rungms" file and modify it accordingly. The following is a list of instructions with commands that will work from a terminal. One could also use WinSCP to do all of this with a GUI rather than a TUI. <br /> <br /> =====Modifiying rungms to Use Custom Basis Set File=====<br /> 1. Copy "rungms" from /scinet/gpc/Applications/gamess to one's own /scratch/$USER/ directory:<br /> cp /scinet/gpc/Applications/gamess/rungms /scratch/$USER/<br /> <br /> 2. Change to the scratch directory and check to see if "rungms" has copied successfully.<br /> cd /scratch/$USER<br /> ls<br /> <br /> 3. Edit line 147 of the vi. <br /> vi rungms<br /> Move the cursor down to line 147 using the arrow keys. It should say "setenv EXTBAS /dev/null". Using the arrow keys, move the cursor to the first "/" and then hit "i" to insert text. Put the path to your external basis file here. For example, /scratch/$USER/basisset. Then hit "escape". To save the changes and exit vi, type ":" and you should see a colon appear at the bottom of the window. Type "wq" (which should appear at the bottom of the window next to the colon) and then hit enter. Now you are done with vi.<br /> <br /> =====Creating a Custom Basis Set File=====<br /> 1. To create a custom basis set file, you need create a new text document. Our group's common practice is to comment out the first line of this file by inserting an exclamation mark (!) followed by noting the specific basis sets and ECPs that are going to be used for each of the atoms. Let us the molecule Mo(CO)6, Molybdenum hexacarbonyl, as an example. Below is the first line of the the external file, which we will call "CUSTOMMO" (NOTE: you can use any name for the external file that suits you, as long as it has no spaces and is 8 characters or less).<br /> <br /> ! 6-31G on C and O and LANL2D2 ECP on Mo<br /> <br /> 2. The next step is to visit the [https://bse.pnl.gov/bse/portal EMSL Basis Set exchange] and select C and O from the periodic table. Then, on the left of the page, select "6-31G" as the basis set. Finally, make sure the output is in GAMESS(US) format using the drop-down menu and then click "get basis set".<br /> <br /> [[File:C_O_6_31G_basisset.JPG|centre]]<br /> <br /> 3. A new window should appear with text in it. For our example case, the text looks like this:<br /> <br /> ! 6-31G EMSL Basis Set Exchange Library 10/13/09 11:12 AM<br /> ! Elements References<br /> ! -------- ----------<br /> ! H - He: W.J. Hehre, R. Ditchfield and J.A. Pople, J. Chem. Phys. 56,<br /> ! Li - Ne: 2257 (1972). Note: Li and B come from J.D. Dill and J.A.<br /> ! Pople, J. Chem. Phys. 62, 2921 (1975).<br /> ! Na - Ar: M.M. Francl, W.J. Petro, W.J. Hehre, J.S. Binkley, M.S. Gordon,<br /> ! D.J. DeFrees and J.A. Pople, J. Chem. Phys. 77, 3654 (1982)<br /> ! K - Zn: V. Rassolov, J.A. Pople, M. Ratner and T.L. Windus, J. Chem. Phys.<br /> ! 109, 1223 (1998)<br /> ! Note: He and Ne are unpublished basis sets taken from the Gaussian<br /> ! program<br /> ! <br /> $DATA<br /><br /> CARBON<br /> S 6<br /> 1 3047.5249000 0.0018347 <br /> 2 457.3695100 0.0140373 <br /> 3 103.9486900 0.0688426 <br /> 4 29.2101550 0.2321844 <br /> 5 9.2866630 0.4679413 <br /> 6 3.1639270 0.3623120 <br /> L 3<br /> 1 7.8682724 -0.1193324 0.0689991 <br /> 2 1.8812885 -0.1608542 0.3164240 <br /> 3 0.5442493 1.1434564 0.7443083 <br /> L 1<br /> 1 0.1687144 1.0000000 1.0000000<br /> <br /> OXYGEN<br /> S 6<br /> 1 5484.6717000 0.0018311 <br /> 2 825.2349500 0.0139501 <br /> 3 188.0469600 0.0684451 <br /> 4 52.9645000 0.2327143 <br /> 5 16.8975700 0.4701930 <br /> 6 5.7996353 0.3585209 <br /> L 3<br /> 1 15.5396160 -0.1107775 0.0708743 <br /> 2 3.5999336 -0.1480263 0.3397528 <br /> 3 1.0137618 1.1307670 0.7271586 <br /> L 1<br /> 1 0.2700058 1.0000000 1.0000000 <br /> $END<br /> <br /> 3. Now, copy and paste the text between the $DATA and $END headings onto our external text file, CUSTOMMO. We also need to change the change the name of each element to it symbol in the periodic table. Finally, we need to add the name of the external file next to the element symbol, separated by one space. Note that there should be a blank line separating the basis set information and the first, commented-out line (The line starting with the '!'). The CUSTOMMO should look like this:<br /> <br /> ! 6-31G on C and O and LANL2D2 ECP on Mo<br /><br /> C CUSTOMMO<br /> S 6<br /> 1 3047.5249000 0.0018347 <br /> 2 457.3695100 0.0140373 <br /> 3 103.9486900 0.0688426 <br /> 4 29.2101550 0.2321844 <br /> 5 9.2866630 0.4679413 <br /> 6 3.1639270 0.3623120 <br /> L 3<br /> 1 7.8682724 -0.1193324 0.0689991 <br /> 2 1.8812885 -0.1608542 0.3164240 <br /> 3 0.5442493 1.1434564 0.7443083 <br /> L 1<br /> 1 0.1687144 1.0000000 1.0000000<br /> <br /> O CUSTOMMO<br /> S 6<br /> 1 5484.6717000 0.0018311 <br /> 2 825.2349500 0.0139501 <br /> 3 188.0469600 0.0684451 <br /> 4 52.9645000 0.2327143 <br /> 5 16.8975700 0.4701930 <br /> 6 5.7996353 0.3585209 <br /> L 3<br /> 1 15.5396160 -0.1107775 0.0708743 <br /> 2 3.5999336 -0.1480263 0.3397528 <br /> 3 1.0137618 1.1307670 0.7271586 <br /> L 1<br /> 1 0.2700058 1.0000000 1.0000000 <br /> <br /> 4. Repeat Step 3 above but choose Mo and select the LANL2DZ ECP instead. A new window will pop up with the basis set information as well as the ECP data we need, since we specified the LANL2DZ '''ECP'''. The ECP data is not inserted into the external file, rather it is placed into the input file itself (More on this later). <br /> <br /> [[File:Mo_LANL2DZ_basisset.JPG|centre]]<br /> <br /> 5. After copying the molybdenum basis set information, your fiished external basis set file should look like this:<br /> ! 6-31G on C and O and LANL2D2 ECP on Mo<br /><br /> C CUSTOMMO<br /> S 6<br /> 1 3047.5249000 0.0018347 <br /> 2 457.3695100 0.0140373 <br /> 3 103.9486900 0.0688426 <br /> 4 29.2101550 0.2321844 <br /> 5 9.2866630 0.4679413 <br /> 6 3.1639270 0.3623120 <br /> L 3<br /> 1 7.8682724 -0.1193324 0.0689991 <br /> 2 1.8812885 -0.1608542 0.3164240 <br /> 3 0.5442493 1.1434564 0.7443083 <br /> L 1<br /> 1 0.1687144 1.0000000 1.0000000<br /> <br /> O CUSTOMMO<br /> S 6<br /> 1 5484.6717000 0.0018311 <br /> 2 825.2349500 0.0139501 <br /> 3 188.0469600 0.0684451 <br /> 4 52.9645000 0.2327143 <br /> 5 16.8975700 0.4701930 <br /> 6 5.7996353 0.3585209 <br /> L 3<br /> 1 15.5396160 -0.1107775 0.0708743 <br /> 2 3.5999336 -0.1480263 0.3397528 <br /> 3 1.0137618 1.1307670 0.7271586 <br /> L 1<br /> 1 0.2700058 1.0000000 1.0000000<br /> <br /> Mo CUSTOMO<br /> S 3<br /> 1 2.3610000 -0.9121760 <br /> 2 1.3090000 1.1477453 <br /> 3 0.4500000 0.6097109 <br /> S 4<br /> 1 2.3610000 0.8139259 <br /> 2 1.3090000 -1.1360084 <br /> 3 0.4500000 -1.1611592 <br /> 4 0.1681000 1.0064786 <br /> S 1<br /> 1 0.0423000 1.0000000 <br /> P 3<br /> 1 4.8950000 -0.0908258 <br /> 2 1.0440000 0.7042899 <br /> 3 0.3877000 0.3973179 <br /> P 2<br /> 1 0.4995000 -0.1081945 <br /> 2 0.0780000 1.0368093 <br /> P 1<br /> 1 0.0247000 1.0000000 <br /> D 3<br /> 1 2.9930000 0.0527063 <br /> 2 1.0630000 0.5003907 <br /> 3 0.3721000 0.5794024 <br /> D 1<br /> 1 0.1178000 1.0000000<br /> <br /> -- [[User:M.Zimmer-De Iuliis|mzd]] 13 October 2009<br /> <br /> ==Climate Modelling==<br /> <br /> ==Medicine/Bio==<br /> <br /> ==High Energy Physics==<br /> <br /> ==Structural Biology==<br /> Molecular simulation of proteins, lipids, carbohydrates, and other biologically relevant molecules.<br /> ===Molecular Dynamics (MD) simulation===<br /> ====GROMACS====<br /> Please refer to the [[gromacs|GROMACS]] page<br /> ====AMBER====<br /> Please refer to the [[amber|AMBER]] page<br /> ====NAMD====<br /> NAMD is one of the better scaling MD packages out there. With sufficiently large systems, it is able to scale to hundreds or thousands of cores on Scinet. Below are details for compiling and running NAMD on Scinet.<br /> <br /> More information regarding performance and different compile options coming soon...<br /> <br /> =====Compiling NAMD for GPC=====<br /> Ensure the proper compiler/mpi modules are loaded.<br /> <source lang="sh"><br /> module load intel<br /> module load openmpi/1.3.3-intel-v11.0-ofed<br /> </source><br /> <br /> '''Compile Charm++ and NAMD'''<br /> <source lang="sh"><br /> #Unpack source files and get required support libraries<br /> tar -xzf NAMD_2.7b1_Source.tar.gz<br /> cd NAMD_2.7b1_Source<br /> tar -xf charm-6.1.tar<br /> wget http://www.ks.uiuc.edu/Research/namd/libraries/fftw-linux-x86_64.tar.gz<br /> wget http://www.ks.uiuc.edu/Research/namd/libraries/tcl-linux-x86_64.tar.gz<br /> tar -xzf fftw-linux-x86_64.tar.gz; mv linux-x86_64 fftw<br /> tar -xzf tcl-linux-x86_64.tar.gz; mv linux-x86_64 tcl<br /> #Compile Charm++<br /> cd charm-6.1<br /> ./build charm++ mpi-linux-x86_64 icc --basedir /scinet/gpc/mpi/openmpi/1.3.3-intel-v11.0-ofed/ --no-shared -O -DCMK_OPTIMIZE=1<br /> cd ..<br /> #Compile NAMD. <br /> #Edit arch/Linux-x86_64-icc.arch and add "-lmpi" to the end of the CXXOPTS and COPTS line.<br /> #Make a builds directory if you want different versions of NAMD compiled at the same time.<br /> mkdir builds<br /> ./config builds/Linux-x86_64-icc --charm-arch mpi-linux-x86_64-icc<br /> cd builds/Linux-x86_64-icc/<br /> make -j4 namd2 # Adjust value of j as desired to specify number of simultaneous make targets. <br /> </source><br /> --[[User:Cmadill|Cmadill]] 16:18, 27 August 2009 (UTC)<br /> <br /> =====Running Fortran=====<br /> On the development nodes, there is an old gcc. The associated libraries are not on the compute nodes. Ensure the line:<br /> <br /> module load gcc<br /> <br /> is in your .bashrc file.<br /> <br /> ====LAMMPS====<br /> [[Image:StrongScalingLAMMPS.png|thumb|320px|right|Strong scaling test on GPC with OpenMPI and IntelMPI on Ethernet and InfiniBand]]<br /> [[Image:WeakScalingLAMMPS.png|thumb|320px|right|Weak scaling test on GPC with OpenMPI and IntelMPI on Ethernet and InfiniBand]]<br /> LAMMPS is a parallel MD code that can be found [http://lammps.sandia.gov/ here].<br /> <br /> '''Scaling Tests'''<br /> <br /> Results from strong scaling tests for LAMMPS using EAM potentials on GPC are shown in the graph on the right. Test simulation ran 500 timesteps for 4,000,000 atoms.<br /> <br /> Results from weak scaling tests for LAMMPS using EAM potentials on GPC are shown in the graph on the right. Test simulation ran 500 timesteps for 32,000 atoms per processor.<br /> <br /> OpenMPI version used: openmpi/1.4.1-intel-v11.0-ofed<br /> <br /> IntelMPI version used: intelmpi/impi-4.0.0.013<br /> <br /> '''Summary of Scaling Tests'''<br /> <br /> Results show good scaling for both OpenMPI and IntelMPI up to 16 processors, after which performance begins to suffer. On Infiniband, excellent scaling is maintained to 512 processors.<br /> <br /> IntelMPI shows slightly better performance compared to OpenMPI when running with Infiniband.<br /> <br /> --[[User:jchu|jchu]] Feb 2, 2010<br /> <br /> ===Monte Carlo (MC) simulation===</div>

Jchu https://oldwiki.scinet.utoronto.ca/index.php?title=User_Codes&diff=880 User Codes 2010-02-02T19:05:23Z

<p>Jchu: /* LAMMPS */</p> <hr /> <div>__FORCETOC__<br /> <br /> ==Astrophysics==<br /> <br /> ===Athena (explicit, uniform grid MHD code)===<br /> <br /> [[Image:StrongScalingAthenaGPC.png|thumb|right|320px|Athena scaling on GPC with OpenMPI and MVAPICH2 on GigE, and OpenMPI on InfiniBand]]<br /> <br /> [http://www.astro.princeton.edu/~jstone/athena.html Athena] is a straightforward C code which doesn't use a lot of libraries so it is pretty straightforward to build and compile on new machines. <br /> <br /> It encapsulates its compiler flags, etc in an <tt>Makeoptions.in</tt> file which is then processed by <tt>configure</tt>. I've used the following additions to <tt>Makeoptions.in</tt> on TCS and GPC:<br /> <br /> <source lang="sh"><br /> ifeq ($(MACHINE),scinettcs)<br /> CC = mpcc_r<br /> LDR = mpcc_r<br /> OPT = -O5 -q64 -qarch=pwr6 -qtune=pwr6 -qcache=auto -qlargepage -qstrict<br /> MPIINC =<br /> MPILIB =<br /> CFLAGS = $(OPT)<br /> LIB = -ldl -lm<br /> else<br /> ifeq ($(MACHINE),scinetgpc)<br /> CC = mpicc<br /> LDR = mpicc<br /> OPT = -O3<br /> MPIINC =<br /> MPILIB =<br /> CFLAGS = $(OPT)<br /> LIB = -lm<br /> else<br /> ...<br /> endif<br /> endif<br /> </source><br /> It performs quite well on the GPC, scaling extremely well even on a strong scaling test out to about 256 cores (32 nodes) on Gigabit ethernet, and performing beautifully on InfiniBand out to 512 cores (64 nodes). <br /> <br /> <br /> -- [[User:Ljdursi|ljdursi]] 19:20, 13 August 2009 (UTC)<br /> <br /> ===FLASH3 (Adaptive Mesh reactive hydrodynamics; explict hydro/MHD)===<br /> <br /> [[Image:weak-scaling-example.png|thumb|right|320px|Weak scaling test of the 2d sod problem on both the GPC and TCS. The results are actually somewhat faster on the GPC; in both cases (weak) scaling is very good out at least to 256 cores]]<br /> <br /> [http://flash.uchicago.edu FLASH] encapsulates its machine-dependant information in the <tt>FLASH3/sites</tt> directory. For the GPC, you'll have to<br /> <pre><br /> module load intel<br /> module load openmpi<br /> module load hdf5/183-v16-openmpi<br /> </pre><br /> <br /> and with that, the following file (<tt>sites/scinetgpc/Makefile.h</tt>) works for me:<br /> <source lang="sh"><br /> ## Must do module load hdf5/183-v16-openmpi<br /> HDF5_PATH = ${SCINET_HDF5_BASE}<br /> ZLIB_PATH = /usr/local<br /> <br /> #----------------------------------------------------------------------------<br /> # Compiler and linker commands<br /> #<br /> # We use the f90 compiler as the linker, so some C libraries may explicitly<br /> # need to be added into the link line.<br /> #----------------------------------------------------------------------------<br /> <br /> ## modules will put the right mpi in our path<br /> FCOMP = mpif77<br /> CCOMP = mpicc<br /> CPPCOMP = mpiCC<br /> LINK = mpif77<br /> <br /> #----------------------------------------------------------------------------<br /> # Compilation flags<br /> #<br /> # Three sets of compilation/linking flags are defined: one for optimized<br /> # code, one for testing, and one for debugging. The default is to use the <br /> # _OPT version. Specifying -debug to setup will pick the _DEBUG version,<br /> # these should enable bounds checking. Specifying -test is used for <br /> # flash_test, and is set for quick code generation, and (sometimes) <br /> # profiling. The Makefile generated by setup will assign the generic token <br /> # (ex. FFLAGS) to the proper set of flags (ex. FFLAGS_OPT).<br /> #----------------------------------------------------------------------------<br /> <br /> FFLAGS_OPT = -c -r8 -i4 -O3 -xSSE4.2<br /> FFLAGS_DEBUG = -c -g -r8 -i4 -O0<br /> FFLAGS_TEST = -c -r8 -i4<br /> <br /> <br /> # if we are using HDF5, we need to specify the path to the include files<br /> CFLAGS_HDF5 = -I${HDF5_PATH}/include<br /> <br /> CFLAGS_OPT = -c -O3 -xSSE4.2<br /> CFLAGS_TEST = -c -O2 <br /> CFLAGS_DEBUG = -c -g <br /> <br /> MDEFS = <br /> <br /> .SUFFIXES: .o .c .f .F .h .fh .F90 .f90<br /> <br /> #----------------------------------------------------------------------------<br /> # Linker flags<br /> #<br /> # There is a seperate version of the linker flags for each of the _OPT, <br /> # _DEBUG, and _TEST cases.<br /> #----------------------------------------------------------------------------<br /> <br /> LFLAGS_OPT = -o<br /> LFLAGS_TEST = -o<br /> LFLAGS_DEBUG = -g -o<br /> <br /> MACHOBJ = <br /> <br /> <br /> MV = mv -f<br /> AR = ar -r<br /> RM = rm -f<br /> CD = cd<br /> RL = ranlib<br /> ECHO = echo<br /> </source><br /> <br /> <br /> -- [[User:Ljdursi|ljdursi]] 22:11, 13 August 2009 (UTC)<br /> <br /> ==Aeronautics==<br /> <br /> ==Chemistry==<br /> <br /> ===GAMESS (US)===<br /> The GAMESS version January 12, 2009 R3 was built using the Intel v11.1 compilers and v3.2.2 MPI library, according to the instructions in http://software.intel.com/en-us/articles/building-gamess-with-intel-compilers-intel-mkl-and-intel-mpi-on-linux/<br /> <br /> The required build scripts - comp, compall, lked - and run script - rungms - were modified to account for our own installation. In order to build GAMESS one first must ensure that the intel and intelmpi modules are loaded ("module load intel intelmpi"). This applies to running GAMESS as well.<br /> The module "gamess" must also be loaded in order to run GAMESS ("module load gamess").<br /> <br /> The modified scripts are in the file /scinet/gpc/src/gamess-on-scinet.tar.gz<br /> <br /> ====Running GAMESS====<br /> - Make sure the directory /scratch/$USER/gamess-scratch exists (the $SCINET_RUNGMS script will create it if it does not exist)<br /> <br /> - Make sure the modules: intel, intelmpi, gamess are loaded (in your .bashrc: "module load intel intelmpi gamess").<br /> <br /> - Create a torque script to run GAMESS. Here is an example:<br /> <br /> - The GAMESS executable is in $SCINET_GAMESS_HOME/gamess.00.x<br /> - The rungms script is in $SCINET_GAMESS_HOME/rungms (actually it is $SCINET_RUNGMS)<br /> <br /> - For IB multinode runs, use the $SCINET_RUNGMS_IB script<br /> <br /> - The rungms script takes 4 arguments: input file, executable number, number of compute processes, processors per node<br /> <br /> For example, in order to run with the input file /scratch/$USER/gamesstest01, on 8 cpus, and the default version (00)<br /> of the executable on a machine with 8 cores:<br /> <br /> # load the gamess module in .bashrc<br /> module load gamess <br /> <br /> # run the program<br /> $SCINET_RUNGMS /scratch/$USER/gamesstest01 00 8 8<br /> <br /> Here is a sample torque script for running a GAMESS calculation, on a single 8-core node:<br /> <br /> <source lang="bash"><br /> #!/bin/bash<br /> #PBS -l nodes=1:ppn=8,walltime=48:00:00,os=centos53computeA<br /> #PBS -N gamessjob<br /> <br /> ## To submit type: qsub gms.sh<br /> <br /> # If not an interactive job (i.e. -I), then cd into the directory where<br /> # I typed qsub.<br /> if [ "$PBS_ENVIRONMENT" != "PBS_INTERACTIVE" ]; then<br /> if [ -n "$PBS_O_WORKDIR" ]; then<br /> cd $PBS_O_WORKDIR<br /> fi<br /> fi<br /> <br /> # the input file is typically named something like "gamesjob.inp"<br /> # so the script will be run like "$SCINET_RUNGMS gamessjob 00 8 8"<br /> <br /> # load the gamess module if not in .bashrc already<br /> # actually, it MUST be in .bashrc<br /> # module load gamess<br /> <br /> # run the program<br /> <br /> $SCINET_RUNGMS gamessjob 00 8 8<br /> </source><br /> <br /> Here is a similar script, but this one uses 2 InfiniBand-connected nodes,<br /> and runs the appropriate $SCINET_RUNGMS_IB script to actually run the job:<br /> <br /> <source lang="bash"><br /> #!/bin/bash<br /> #PBS -l nodes=2:ib:ppn=8,walltime=48:00:00,os=centos53computeA<br /> #PBS -N gamessjob<br /> <br /> ## To submit type: qsub gmsib.sh<br /> <br /> # If not an interactive job (i.e. -I), then cd into the directory where<br /> # I typed qsub.<br /> if [ "$PBS_ENVIRONMENT" != "PBS_INTERACTIVE" ]; then<br /> if [ -n "$PBS_O_WORKDIR" ]; then<br /> cd $PBS_O_WORKDIR<br /> fi<br /> fi<br /> <br /> # the input file is typically named something like "gamesjob.inp"<br /> # so the script will be run like "$SCINET_RUNGMS gamessjob 00 8 8"<br /> <br /> # load the gamess module if not in .bashrc already<br /> # actually, it MUST be in .bashrc<br /> # module load gamess<br /> <br /> # This script requests InfiniBand-connected nodes (:ib above)<br /> # so it must run with the IB version of the rungms script,<br /> # $SCINET_RUNGMS_IB<br /> <br /> $SCINET_RUNGMS_IB gamessjob 00 16 8<br /> </source><br /> <br /> <br /> -- [[User:Dgruner|dgruner]] 5 October 2009<br /> <br /> ===Tips from the Fekl Lab===<br /> <br /> Through trial and error, we have found a few useful things that we would like to share:<br /> <br /> 1. Two very useful, open-source programs for visualization of output files from GAMESS(US) and for generation of input files are [http://www.scl.ameslab.gov/MacMolPlt/ MacMolPlt]and [http://avogadro.openmolecules.net/wiki/Main_Page Avogadro]. The are available for UNIX/LINUX, Windows and Mac based machines, HOWEVER: any input files that we have generated with these programs on a Windows-based machine do not run on Mac based machines. We don't know why.<br /> <br /> 2. [http://winscp.net/eng/index.php WinSCP] is a very useful tool that has a graphical user interface for moving files from a local machine to SCINET and vice versa. It also has text editing capabilities.<br /> <br /> 3. The [https://bse.pnl.gov/bse/portal ESML Basis Set Exchange] is an excellent source for custom basis set or effective core potential parameters. Make sure that you specify "Gamess-US" in the format drop-down box.<br /> <br /> 4. The commercial program [http://www.chemcraftprog.com/ ChemCraft] is a highly useful visualization program that has the ability to edit molecules in a very similar fashion to GaussView. It can also be customized to build GAMESS(US) input files.<br /> <br /> ====Anatomy of a GAMESS(US) Input File with Basis Set Info in an External File====<br /> <br /> $CONTRL SCFTYP=RHF RUNTYP=OPTIMIZE DFTTYP=M06-L MAXIT=199 MULT=1 NOSYM=1<br /> ECP=READ $END<br /> $SYSTEM TIMLIM=525600 MWORDS=1750 PARALL=.TRUE. $END<br /> $BASIS GBASIS=CUSTOMNI EXTFIL=.t. $END<br /> $SCF DIRSCF=.TRUE. FDIFF=.f. $END<br /> $STATPT OPTTOL=0.0001 NSTEP=500 HSSEND=.t. $END<br /> $DATA<br /> Mo_BDT3<br /> C1<br /> MOLYBDENUM 42.0 5.7556500000 4.4039600000 16.5808400000<br /> SULFUR 16.0 7.4169700000 3.1956300000 15.2089300000<br /> SULFUR 16.0 4.0966800000 3.2258300000 15.1761100000<br /> SULFUR 16.0 3.9677300000 4.4940500000 18.3266100000<br /> SULFUR 16.0 7.1776900000 3.5815000000 18.4485200000<br /> SULFUR 16.0 4.3776600000 6.2447400000 15.6786900000<br /> SULFUR 16.0 7.5478700000 6.0679800000 16.2223700000<br /> CARBON 6.0 6.4716900000 2.1004800000 14.1902300000<br /> CARBON 6.0 5.0690300000 2.1781400000 14.1080700000<br /> CARBON 6.0 4.8421800000 4.2701300000 19.8855500000<br /> CARBON 6.0 6.1969000000 3.9249600000 19.9397400000<br /> CARBON 6.0 6.8280600000 3.7834200000 21.1913200000<br /> CARBON 6.0 5.7697600000 7.6933500000 17.4241800000<br /> CARBON 6.0 7.2043100000 7.9413600000 17.8281100000<br /> CARBON 6.0 5.5051400000 7.0409700000 14.5903800000<br /> CARBON 6.0 6.8905200000 6.9194700000 14.7626200000<br /> CARBON 6.0 7.7396400000 7.5379800000 13.8285700000<br /> HYDROGEN 1.0 8.8190700000 7.4520600000 13.9252200000<br /> CARBON 6.0 7.2169400000 8.2960300000 12.7704100000<br /> HYDROGEN 1.0 7.8667000000 8.7825100000 12.0575600000<br /> CARBON 6.0 5.8260300000 8.4502300000 12.6467800000<br /> HYDROGEN 1.0 5.4143000000 9.0544300000 11.8493100000<br /> CARBON 6.0 4.9881500000 7.8192300000 13.5528400000<br /> HYDROGEN 1.0 3.9090500000 7.9420000000 13.4583700000<br /> CARBON 6.0 7.1538500000 1.1569600000 13.4143900000<br /> CARBON 6.0 4.4018100000 1.3603900000 13.1919900000<br /> CARBON 6.0 6.4791600000 0.3185500000 12.5353300000<br /> CARBON 6.0 5.0837400000 0.4369500000 12.4084900000<br /> HYDROGEN 1.0 7.0116000000 -0.4099400000 11.9434600000<br /> HYDROGEN 1.0 8.2399000000 1.0702400000 13.4937600000<br /> HYDROGEN 1.0 3.3185600000 1.4368700000 13.0953100000<br /> HYDROGEN 1.0 4.5549800000 -0.1997300000 11.7165200000<br /> CARBON 6.0 6.1105700000 3.9639000000 22.3866100000<br /> CARBON 6.0 4.1216300000 4.4424400000 21.1020100000<br /> HYDROGEN 1.0 7.8732900000 3.5217100000 21.2520500000<br /> CARBON 6.0 4.7606000000 4.2868500000 22.3363800000<br /> HYDROGEN 1.0 6.6064200000 3.8406000000 23.3428500000<br /> HYDROGEN 1.0 4.2065000000 4.4170700000 23.2667100000<br /> HYDROGEN 1.0 3.0674000000 4.6893500000 21.0889000000<br /> HYDROGEN 1.0 7.4249200000 7.7545300000 18.8583200000<br /> HYDROGEN 1.0 7.6651700000 8.9049700000 17.7652100000<br /> HYDROGEN 1.0 5.3324000000 8.6487800000 17.2222700000<br /> HYDROGEN 1.0 5.5015000000 7.1039000000 18.2759400000<br /> $END<br /> $ECP<br /> MO-ECP GEN 28 3<br /> 5 ----- f potential -----<br /> -0.0469492 0 537.9667807 <br /> -20.2080084 1 147.8982938 <br /> -106.2116302 2 45.7358898 <br /> -41.8107368 2 13.2911467 <br /> -4.2054103 2 4.7059961 <br /> 3 ----- s-f potential -----<br /> 2.8063717 0 110.2991760 <br /> 44.5162012 1 23.2014645 <br /> 82.7785227 2 5.3530131 <br /> 4 ----- p-f potential -----<br /> 4.9420876 0 63.2901397 <br /> 25.8604976 1 23.3315302 <br /> 132.4708742 2 24.6759423 <br /> 57.3149794 2 4.6493040 <br /> 5 ----- d-f potential -----<br /> 3.0054591 0 104.4839977 <br /> 26.3637851 1 66.2307245 <br /> 183.3849199 2 39.1283176 <br /> 98.4453068 2 13.1164437 <br /> 22.4901377 2 3.6280263 <br /> S NONE<br /> S NONE<br /> S NONE<br /> S NONE<br /> S NONE<br /> S NONE<br /> C NONE<br /> C NONE<br /> C NONE<br /> C NONE<br /> C NONE<br /> C NONE<br /> C NONE<br /> C NONE<br /> C NONE<br /> C NONE<br /> H NONE<br /> C NONE<br /> H NONE<br /> C NONE<br /> H NONE<br /> C NONE<br /> H NONE<br /> C NONE<br /> C NONE<br /> C NONE<br /> C NONE<br /> H NONE<br /> H NONE<br /> H NONE<br /> H NONE<br /> C NONE<br /> C NONE<br /> H NONE<br /> C NONE<br /> H NONE<br /> H NONE<br /> H NONE<br /> H NONE<br /> H NONE<br /> H NONE<br /> H NONE<br /> $END<br /> <br /> =====The Input Deck=====<br /> <br /> Below is the input deck. It is where you tell GAMESS(US) what job type to execute and where all you individual parameters are entered for your specific job type. The example input deck below is for a geometry optimization and frequency calculation. This input deck is equivalent to the Gaussian job with "opt" and "freq" in the route section.<br /> <br /> $CONTRL SCFTYP=RHF RUNTYP=OPTIMIZE DFTTYP=M06-L MAXIT=199 MULT=1 NOSYM=1<br /> ECP=READ $END<br /> $SYSTEM TIMLIM=2850 MWORDS=1750 MEMDDI=20 PARALL=.TRUE. $END<br /> $BASIS GBASIS=CUSTOMNI EXTFIL=.t. $END<br /> $SCF DIRSCF=.TRUE. FDIFF=.f. $END<br /> $STATPT OPTTOL=0.0001 NSTEP=500 HSSEND=.t. $END<br /> $DATA<br /> <br /> An important thing to note is the spacing. In the input deck, there must be 1 space at the beginning of each line of the input deck. If not, the job will fail. Most builders will insert this space anyway, but it helps to double check.<br /> <br /> The end of the input deck is marked by the "$DATA" line.<br /> <br /> =====Job Title Line=====<br /> <br /> The next line of the file is the job title. It can be anthing you wish, however, we have found that to be on the safe side, we avoide using symbols or spaces<br /> <br /> Mo_BDT3<br /> <br /> =====Symmetry Point Group=====<br /> <br /> The next line of the file is the symmetry point group of your molecule. Note that there is no leading space before the point group.<br /> <br /> C1<br /> <br /> =====Coordinates=====<br /> <br /> The next block of text is set aside for the coordinates of the molecule. This can be in internal (or z-matrix) format or cartesian coordinates. Note that there is no leading space before the coordinates. One may use the chemical symbol or the full name of each atom in the molecule. Note that the end of the coordinates is signified by an "$END", which MUST have one space preceding it. The coordinates below do NOT have any basis set information inserted. It is possible to insert basis set information directly into the input file. This is accomplished by obtaining the desired basis set parameters from the EMSL and then inserting them below each relevant atom. An example input file with inserted basis set information will be shown later.<br /> <br /> MOLYBDENUM 42.0 5.7556500000 4.4039600000 16.5808400000<br /> SULFUR 16.0 7.4169700000 3.1956300000 15.2089300000<br /> SULFUR 16.0 4.0966800000 3.2258300000 15.1761100000<br /> SULFUR 16.0 3.9677300000 4.4940500000 18.3266100000<br /> SULFUR 16.0 7.1776900000 3.5815000000 18.4485200000<br /> SULFUR 16.0 4.3776600000 6.2447400000 15.6786900000<br /> SULFUR 16.0 7.5478700000 6.0679800000 16.2223700000<br /> CARBON 6.0 6.4716900000 2.1004800000 14.1902300000<br /> CARBON 6.0 5.0690300000 2.1781400000 14.1080700000<br /> CARBON 6.0 4.8421800000 4.2701300000 19.8855500000<br /> CARBON 6.0 6.1969000000 3.9249600000 19.9397400000<br /> CARBON 6.0 6.8280600000 3.7834200000 21.1913200000<br /> CARBON 6.0 5.7697600000 7.6933500000 17.4241800000<br /> CARBON 6.0 7.2043100000 7.9413600000 17.8281100000<br /> CARBON 6.0 5.5051400000 7.0409700000 14.5903800000<br /> CARBON 6.0 6.8905200000 6.9194700000 14.7626200000<br /> CARBON 6.0 7.7396400000 7.5379800000 13.8285700000<br /> HYDROGEN 1.0 8.8190700000 7.4520600000 13.9252200000<br /> CARBON 6.0 7.2169400000 8.2960300000 12.7704100000<br /> HYDROGEN 1.0 7.8667000000 8.7825100000 12.0575600000<br /> CARBON 6.0 5.8260300000 8.4502300000 12.6467800000<br /> HYDROGEN 1.0 5.4143000000 9.0544300000 11.8493100000<br /> CARBON 6.0 4.9881500000 7.8192300000 13.5528400000<br /> HYDROGEN 1.0 3.9090500000 7.9420000000 13.4583700000<br /> CARBON 6.0 7.1538500000 1.1569600000 13.4143900000<br /> CARBON 6.0 4.4018100000 1.3603900000 13.1919900000<br /> CARBON 6.0 6.4791600000 0.3185500000 12.5353300000<br /> CARBON 6.0 5.0837400000 0.4369500000 12.4084900000<br /> HYDROGEN 1.0 7.0116000000 -0.4099400000 11.9434600000<br /> HYDROGEN 1.0 8.2399000000 1.0702400000 13.4937600000<br /> HYDROGEN 1.0 3.3185600000 1.4368700000 13.0953100000<br /> HYDROGEN 1.0 4.5549800000 -0.1997300000 11.7165200000<br /> CARBON 6.0 6.1105700000 3.9639000000 22.3866100000<br /> CARBON 6.0 4.1216300000 4.4424400000 21.1020100000<br /> HYDROGEN 1.0 7.8732900000 3.5217100000 21.2520500000<br /> CARBON 6.0 4.7606000000 4.2868500000 22.3363800000<br /> HYDROGEN 1.0 6.6064200000 3.8406000000 23.3428500000<br /> HYDROGEN 1.0 4.2065000000 4.4170700000 23.2667100000<br /> HYDROGEN 1.0 3.0674000000 4.6893500000 21.0889000000<br /> HYDROGEN 1.0 7.4249200000 7.7545300000 18.8583200000<br /> HYDROGEN 1.0 7.6651700000 8.9049700000 17.7652100000<br /> HYDROGEN 1.0 5.3324000000 8.6487800000 17.2222700000<br /> HYDROGEN 1.0 5.5015000000 7.1039000000 18.2759400000<br /> $END<br /> <br /> =====Effective Core Potential Data=====<br /> <br /> The effective core potential (ECP) data is entered after the coordinates. It starts with "$ECP", which must be preceded with a space. The atoms of the molecule are listed in the same order as in the coordinates section and the parameters for the ECP are listed after each atom. Note that for any atom that does NOT have an ECP, one must enter "ECP-NONE" or "NONE" after each atom without an ECP.<br /> <br /> $ECP<br /> MO-ECP GEN 28 3<br /> 5 ----- f potential -----<br /> -0.0469492 0 537.9667807 <br /> -20.2080084 1 147.8982938 <br /> -106.2116302 2 45.7358898 <br /> -41.8107368 2 13.2911467 <br /> -4.2054103 2 4.7059961 <br /> 3 ----- s-f potential -----<br /> 2.8063717 0 110.2991760 <br /> 44.5162012 1 23.2014645 <br /> 82.7785227 2 5.3530131 <br /> 4 ----- p-f potential -----<br /> 4.9420876 0 63.2901397 <br /> 25.8604976 1 23.3315302 <br /> 132.4708742 2 24.6759423 <br /> 57.3149794 2 4.6493040 <br /> 5 ----- d-f potential -----<br /> 3.0054591 0 104.4839977 <br /> 26.3637851 1 66.2307245 <br /> 183.3849199 2 39.1283176 <br /> 98.4453068 2 13.1164437 <br /> 22.4901377 2 3.6280263 <br /> S NONE<br /> S NONE<br /> S NONE<br /> S NONE<br /> S NONE<br /> S NONE<br /> C NONE<br /> C NONE<br /> C NONE<br /> C NONE<br /> C NONE<br /> C NONE<br /> C NONE<br /> C NONE<br /> C NONE<br /> C NONE<br /> H NONE<br /> C NONE<br /> H NONE<br /> C NONE<br /> H NONE<br /> C NONE<br /> H NONE<br /> C NONE<br /> C NONE<br /> C NONE<br /> C NONE<br /> H NONE<br /> H NONE<br /> H NONE<br /> H NONE<br /> C NONE<br /> C NONE<br /> H NONE<br /> C NONE<br /> H NONE<br /> H NONE<br /> H NONE<br /> H NONE<br /> H NONE<br /> H NONE<br /> H NONE<br /> $END<br /> <br /> -- [[User:M.Zimmer-De Iuliis|mzd]] 16 November 2009<br /> <br /> ====Using Effective Core Potentials in GAMESS(US)====<br /> <br /> For many metal containing compounds, it is very convenient and time saving to use an effective core potential (ECP) for the core metal electrons, as they are usually not important to the reactivity of the complex or the geometry around the metal. Since GAMESS(US) has a limited number of built-in ECPs, one may want to make GAMESS(US) read an external file that contains the ECP data using the "EXTFIL" keyword in the $GBASIS command line of the input file. In addition, to make GAMESS(US) use this external file, one must copy the "rungms" file and modify it accordingly. The following is a list of instructions with commands that will work from a terminal. One could also use WinSCP to do all of this with a GUI rather than a TUI. <br /> <br /> =====Modifiying rungms to Use Custom Basis Set File=====<br /> 1. Copy "rungms" from /scinet/gpc/Applications/gamess to one's own /scratch/$USER/ directory:<br /> cp /scinet/gpc/Applications/gamess/rungms /scratch/$USER/<br /> <br /> 2. Change to the scratch directory and check to see if "rungms" has copied successfully.<br /> cd /scratch/$USER<br /> ls<br /> <br /> 3. Edit line 147 of the vi. <br /> vi rungms<br /> Move the cursor down to line 147 using the arrow keys. It should say "setenv EXTBAS /dev/null". Using the arrow keys, move the cursor to the first "/" and then hit "i" to insert text. Put the path to your external basis file here. For example, /scratch/$USER/basisset. Then hit "escape". To save the changes and exit vi, type ":" and you should see a colon appear at the bottom of the window. Type "wq" (which should appear at the bottom of the window next to the colon) and then hit enter. Now you are done with vi.<br /> <br /> =====Creating a Custom Basis Set File=====<br /> 1. To create a custom basis set file, you need create a new text document. Our group's common practice is to comment out the first line of this file by inserting an exclamation mark (!) followed by noting the specific basis sets and ECPs that are going to be used for each of the atoms. Let us the molecule Mo(CO)6, Molybdenum hexacarbonyl, as an example. Below is the first line of the the external file, which we will call "CUSTOMMO" (NOTE: you can use any name for the external file that suits you, as long as it has no spaces and is 8 characters or less).<br /> <br /> ! 6-31G on C and O and LANL2D2 ECP on Mo<br /> <br /> 2. The next step is to visit the [https://bse.pnl.gov/bse/portal EMSL Basis Set exchange] and select C and O from the periodic table. Then, on the left of the page, select "6-31G" as the basis set. Finally, make sure the output is in GAMESS(US) format using the drop-down menu and then click "get basis set".<br /> <br /> [[File:C_O_6_31G_basisset.JPG|centre]]<br /> <br /> 3. A new window should appear with text in it. For our example case, the text looks like this:<br /> <br /> ! 6-31G EMSL Basis Set Exchange Library 10/13/09 11:12 AM<br /> ! Elements References<br /> ! -------- ----------<br /> ! H - He: W.J. Hehre, R. Ditchfield and J.A. Pople, J. Chem. Phys. 56,<br /> ! Li - Ne: 2257 (1972). Note: Li and B come from J.D. Dill and J.A.<br /> ! Pople, J. Chem. Phys. 62, 2921 (1975).<br /> ! Na - Ar: M.M. Francl, W.J. Petro, W.J. Hehre, J.S. Binkley, M.S. Gordon,<br /> ! D.J. DeFrees and J.A. Pople, J. Chem. Phys. 77, 3654 (1982)<br /> ! K - Zn: V. Rassolov, J.A. Pople, M. Ratner and T.L. Windus, J. Chem. Phys.<br /> ! 109, 1223 (1998)<br /> ! Note: He and Ne are unpublished basis sets taken from the Gaussian<br /> ! program<br /> ! <br /> $DATA<br /><br /> CARBON<br /> S 6<br /> 1 3047.5249000 0.0018347 <br /> 2 457.3695100 0.0140373 <br /> 3 103.9486900 0.0688426 <br /> 4 29.2101550 0.2321844 <br /> 5 9.2866630 0.4679413 <br /> 6 3.1639270 0.3623120 <br /> L 3<br /> 1 7.8682724 -0.1193324 0.0689991 <br /> 2 1.8812885 -0.1608542 0.3164240 <br /> 3 0.5442493 1.1434564 0.7443083 <br /> L 1<br /> 1 0.1687144 1.0000000 1.0000000<br /> <br /> OXYGEN<br /> S 6<br /> 1 5484.6717000 0.0018311 <br /> 2 825.2349500 0.0139501 <br /> 3 188.0469600 0.0684451 <br /> 4 52.9645000 0.2327143 <br /> 5 16.8975700 0.4701930 <br /> 6 5.7996353 0.3585209 <br /> L 3<br /> 1 15.5396160 -0.1107775 0.0708743 <br /> 2 3.5999336 -0.1480263 0.3397528 <br /> 3 1.0137618 1.1307670 0.7271586 <br /> L 1<br /> 1 0.2700058 1.0000000 1.0000000 <br /> $END<br /> <br /> 3. Now, copy and paste the text between the $DATA and $END headings onto our external text file, CUSTOMMO. We also need to change the change the name of each element to it symbol in the periodic table. Finally, we need to add the name of the external file next to the element symbol, separated by one space. Note that there should be a blank line separating the basis set information and the first, commented-out line (The line starting with the '!'). The CUSTOMMO should look like this:<br /> <br /> ! 6-31G on C and O and LANL2D2 ECP on Mo<br /><br /> C CUSTOMMO<br /> S 6<br /> 1 3047.5249000 0.0018347 <br /> 2 457.3695100 0.0140373 <br /> 3 103.9486900 0.0688426 <br /> 4 29.2101550 0.2321844 <br /> 5 9.2866630 0.4679413 <br /> 6 3.1639270 0.3623120 <br /> L 3<br /> 1 7.8682724 -0.1193324 0.0689991 <br /> 2 1.8812885 -0.1608542 0.3164240 <br /> 3 0.5442493 1.1434564 0.7443083 <br /> L 1<br /> 1 0.1687144 1.0000000 1.0000000<br /> <br /> O CUSTOMMO<br /> S 6<br /> 1 5484.6717000 0.0018311 <br /> 2 825.2349500 0.0139501 <br /> 3 188.0469600 0.0684451 <br /> 4 52.9645000 0.2327143 <br /> 5 16.8975700 0.4701930 <br /> 6 5.7996353 0.3585209 <br /> L 3<br /> 1 15.5396160 -0.1107775 0.0708743 <br /> 2 3.5999336 -0.1480263 0.3397528 <br /> 3 1.0137618 1.1307670 0.7271586 <br /> L 1<br /> 1 0.2700058 1.0000000 1.0000000 <br /> <br /> 4. Repeat Step 3 above but choose Mo and select the LANL2DZ ECP instead. A new window will pop up with the basis set information as well as the ECP data we need, since we specified the LANL2DZ '''ECP'''. The ECP data is not inserted into the external file, rather it is placed into the input file itself (More on this later). <br /> <br /> [[File:Mo_LANL2DZ_basisset.JPG|centre]]<br /> <br /> 5. After copying the molybdenum basis set information, your fiished external basis set file should look like this:<br /> ! 6-31G on C and O and LANL2D2 ECP on Mo<br /><br /> C CUSTOMMO<br /> S 6<br /> 1 3047.5249000 0.0018347 <br /> 2 457.3695100 0.0140373 <br /> 3 103.9486900 0.0688426 <br /> 4 29.2101550 0.2321844 <br /> 5 9.2866630 0.4679413 <br /> 6 3.1639270 0.3623120 <br /> L 3<br /> 1 7.8682724 -0.1193324 0.0689991 <br /> 2 1.8812885 -0.1608542 0.3164240 <br /> 3 0.5442493 1.1434564 0.7443083 <br /> L 1<br /> 1 0.1687144 1.0000000 1.0000000<br /> <br /> O CUSTOMMO<br /> S 6<br /> 1 5484.6717000 0.0018311 <br /> 2 825.2349500 0.0139501 <br /> 3 188.0469600 0.0684451 <br /> 4 52.9645000 0.2327143 <br /> 5 16.8975700 0.4701930 <br /> 6 5.7996353 0.3585209 <br /> L 3<br /> 1 15.5396160 -0.1107775 0.0708743 <br /> 2 3.5999336 -0.1480263 0.3397528 <br /> 3 1.0137618 1.1307670 0.7271586 <br /> L 1<br /> 1 0.2700058 1.0000000 1.0000000<br /> <br /> Mo CUSTOMO<br /> S 3<br /> 1 2.3610000 -0.9121760 <br /> 2 1.3090000 1.1477453 <br /> 3 0.4500000 0.6097109 <br /> S 4<br /> 1 2.3610000 0.8139259 <br /> 2 1.3090000 -1.1360084 <br /> 3 0.4500000 -1.1611592 <br /> 4 0.1681000 1.0064786 <br /> S 1<br /> 1 0.0423000 1.0000000 <br /> P 3<br /> 1 4.8950000 -0.0908258 <br /> 2 1.0440000 0.7042899 <br /> 3 0.3877000 0.3973179 <br /> P 2<br /> 1 0.4995000 -0.1081945 <br /> 2 0.0780000 1.0368093 <br /> P 1<br /> 1 0.0247000 1.0000000 <br /> D 3<br /> 1 2.9930000 0.0527063 <br /> 2 1.0630000 0.5003907 <br /> 3 0.3721000 0.5794024 <br /> D 1<br /> 1 0.1178000 1.0000000<br /> <br /> -- [[User:M.Zimmer-De Iuliis|mzd]] 13 October 2009<br /> <br /> ==Climate Modelling==<br /> <br /> ==Medicine/Bio==<br /> <br /> ==High Energy Physics==<br /> <br /> ==Structural Biology==<br /> Molecular simulation of proteins, lipids, carbohydrates, and other biologically relevant molecules.<br /> ===Molecular Dynamics (MD) simulation===<br /> ====GROMACS====<br /> Please refer to the [[gromacs|GROMACS]] page<br /> ====AMBER====<br /> Please refer to the [[amber|AMBER]] page<br /> ====NAMD====<br /> NAMD is one of the better scaling MD packages out there. With sufficiently large systems, it is able to scale to hundreds or thousands of cores on Scinet. Below are details for compiling and running NAMD on Scinet.<br /> <br /> More information regarding performance and different compile options coming soon...<br /> <br /> =====Compiling NAMD for GPC=====<br /> Ensure the proper compiler/mpi modules are loaded.<br /> <source lang="sh"><br /> module load intel<br /> module load openmpi/1.3.3-intel-v11.0-ofed<br /> </source><br /> <br /> '''Compile Charm++ and NAMD'''<br /> <source lang="sh"><br /> #Unpack source files and get required support libraries<br /> tar -xzf NAMD_2.7b1_Source.tar.gz<br /> cd NAMD_2.7b1_Source<br /> tar -xf charm-6.1.tar<br /> wget http://www.ks.uiuc.edu/Research/namd/libraries/fftw-linux-x86_64.tar.gz<br /> wget http://www.ks.uiuc.edu/Research/namd/libraries/tcl-linux-x86_64.tar.gz<br /> tar -xzf fftw-linux-x86_64.tar.gz; mv linux-x86_64 fftw<br /> tar -xzf tcl-linux-x86_64.tar.gz; mv linux-x86_64 tcl<br /> #Compile Charm++<br /> cd charm-6.1<br /> ./build charm++ mpi-linux-x86_64 icc --basedir /scinet/gpc/mpi/openmpi/1.3.3-intel-v11.0-ofed/ --no-shared -O -DCMK_OPTIMIZE=1<br /> cd ..<br /> #Compile NAMD. <br /> #Edit arch/Linux-x86_64-icc.arch and add "-lmpi" to the end of the CXXOPTS and COPTS line.<br /> #Make a builds directory if you want different versions of NAMD compiled at the same time.<br /> mkdir builds<br /> ./config builds/Linux-x86_64-icc --charm-arch mpi-linux-x86_64-icc<br /> cd builds/Linux-x86_64-icc/<br /> make -j4 namd2 # Adjust value of j as desired to specify number of simultaneous make targets. <br /> </source><br /> --[[User:Cmadill|Cmadill]] 16:18, 27 August 2009 (UTC)<br /> <br /> =====Running Fortran=====<br /> On the development nodes, there is an old gcc. The associated libraries are not on the compute nodes. Ensure the line:<br /> <br /> module load gcc<br /> <br /> is in your .bashrc file.<br /> <br /> ====LAMMPS====<br /> LAMMPS is a parallel MD code that can be found [here http://lammps.sandia.gov/].<br /> <br /> [[Image:StrongScalingLAMMPS.png|thumb|320px|right|Strong scaling test on GPC with OpenMPI and IntelMPI on Ethernet and InfiniBand]]<br /> [[Image:WeakScalingLAMMPS.png|thumb|320px|right|Weak scaling test on GPC with OpenMPI and IntelMPI on Ethernet and InfiniBand]]<br /> <br /> '''Scaling Tests'''<br /> <br /> Results from strong scaling tests for LAMMPS using EAM potentials on GPC are shown in the graph on the right. Test simulation ran 500 timesteps for 4,000,000 atoms.<br /> <br /> Results from weak scaling tests for LAMMPS using EAM potentials on GPC are shown in the graph on the right. Test simulation ran 500 timesteps for 32,000 atoms per processor.<br /> <br /> OpenMPI version used: openmpi/1.4.1-intel-v11.0-ofed<br /> <br /> IntelMPI version used: intelmpi/impi-4.0.0.013<br /> <br /> '''Summary of Scaling Tests'''<br /> <br /> Results show good scaling for both OpenMPI and IntelMPI up to 16 processors, after which performance begins to suffer. On Infiniband, excellent scaling is maintained to 512 processors.<br /> <br /> IntelMPI shows slightly better performance compared to OpenMPI when running with Infiniband.<br /> <br /> --[[User:jchu|jchu]] Feb 2, 2010<br /> <br /> ===Monte Carlo (MC) simulation===</div>

Jchu https://oldwiki.scinet.utoronto.ca/index.php?title=User_Codes&diff=879 User Codes 2010-02-02T19:02:42Z

<p>Jchu: /* LAMMPS */</p> <hr /> <div>__FORCETOC__<br /> <br /> ==Astrophysics==<br /> <br /> ===Athena (explicit, uniform grid MHD code)===<br /> <br /> [[Image:StrongScalingAthenaGPC.png|thumb|right|320px|Athena scaling on GPC with OpenMPI and MVAPICH2 on GigE, and OpenMPI on InfiniBand]]<br /> <br /> [http://www.astro.princeton.edu/~jstone/athena.html Athena] is a straightforward C code which doesn't use a lot of libraries so it is pretty straightforward to build and compile on new machines. <br /> <br /> It encapsulates its compiler flags, etc in an <tt>Makeoptions.in</tt> file which is then processed by <tt>configure</tt>. I've used the following additions to <tt>Makeoptions.in</tt> on TCS and GPC:<br /> <br /> <source lang="sh"><br /> ifeq ($(MACHINE),scinettcs)<br /> CC = mpcc_r<br /> LDR = mpcc_r<br /> OPT = -O5 -q64 -qarch=pwr6 -qtune=pwr6 -qcache=auto -qlargepage -qstrict<br /> MPIINC =<br /> MPILIB =<br /> CFLAGS = $(OPT)<br /> LIB = -ldl -lm<br /> else<br /> ifeq ($(MACHINE),scinetgpc)<br /> CC = mpicc<br /> LDR = mpicc<br /> OPT = -O3<br /> MPIINC =<br /> MPILIB =<br /> CFLAGS = $(OPT)<br /> LIB = -lm<br /> else<br /> ...<br /> endif<br /> endif<br /> </source><br /> It performs quite well on the GPC, scaling extremely well even on a strong scaling test out to about 256 cores (32 nodes) on Gigabit ethernet, and performing beautifully on InfiniBand out to 512 cores (64 nodes). <br /> <br /> <br /> -- [[User:Ljdursi|ljdursi]] 19:20, 13 August 2009 (UTC)<br /> <br /> ===FLASH3 (Adaptive Mesh reactive hydrodynamics; explict hydro/MHD)===<br /> <br /> [[Image:weak-scaling-example.png|thumb|right|320px|Weak scaling test of the 2d sod problem on both the GPC and TCS. The results are actually somewhat faster on the GPC; in both cases (weak) scaling is very good out at least to 256 cores]]<br /> <br /> [http://flash.uchicago.edu FLASH] encapsulates its machine-dependant information in the <tt>FLASH3/sites</tt> directory. For the GPC, you'll have to<br /> <pre><br /> module load intel<br /> module load openmpi<br /> module load hdf5/183-v16-openmpi<br /> </pre><br /> <br /> and with that, the following file (<tt>sites/scinetgpc/Makefile.h</tt>) works for me:<br /> <source lang="sh"><br /> ## Must do module load hdf5/183-v16-openmpi<br /> HDF5_PATH = ${SCINET_HDF5_BASE}<br /> ZLIB_PATH = /usr/local<br /> <br /> #----------------------------------------------------------------------------<br /> # Compiler and linker commands<br /> #<br /> # We use the f90 compiler as the linker, so some C libraries may explicitly<br /> # need to be added into the link line.<br /> #----------------------------------------------------------------------------<br /> <br /> ## modules will put the right mpi in our path<br /> FCOMP = mpif77<br /> CCOMP = mpicc<br /> CPPCOMP = mpiCC<br /> LINK = mpif77<br /> <br /> #----------------------------------------------------------------------------<br /> # Compilation flags<br /> #<br /> # Three sets of compilation/linking flags are defined: one for optimized<br /> # code, one for testing, and one for debugging. The default is to use the <br /> # _OPT version. Specifying -debug to setup will pick the _DEBUG version,<br /> # these should enable bounds checking. Specifying -test is used for <br /> # flash_test, and is set for quick code generation, and (sometimes) <br /> # profiling. The Makefile generated by setup will assign the generic token <br /> # (ex. FFLAGS) to the proper set of flags (ex. FFLAGS_OPT).<br /> #----------------------------------------------------------------------------<br /> <br /> FFLAGS_OPT = -c -r8 -i4 -O3 -xSSE4.2<br /> FFLAGS_DEBUG = -c -g -r8 -i4 -O0<br /> FFLAGS_TEST = -c -r8 -i4<br /> <br /> <br /> # if we are using HDF5, we need to specify the path to the include files<br /> CFLAGS_HDF5 = -I${HDF5_PATH}/include<br /> <br /> CFLAGS_OPT = -c -O3 -xSSE4.2<br /> CFLAGS_TEST = -c -O2 <br /> CFLAGS_DEBUG = -c -g <br /> <br /> MDEFS = <br /> <br /> .SUFFIXES: .o .c .f .F .h .fh .F90 .f90<br /> <br /> #----------------------------------------------------------------------------<br /> # Linker flags<br /> #<br /> # There is a seperate version of the linker flags for each of the _OPT, <br /> # _DEBUG, and _TEST cases.<br /> #----------------------------------------------------------------------------<br /> <br /> LFLAGS_OPT = -o<br /> LFLAGS_TEST = -o<br /> LFLAGS_DEBUG = -g -o<br /> <br /> MACHOBJ = <br /> <br /> <br /> MV = mv -f<br /> AR = ar -r<br /> RM = rm -f<br /> CD = cd<br /> RL = ranlib<br /> ECHO = echo<br /> </source><br /> <br /> <br /> -- [[User:Ljdursi|ljdursi]] 22:11, 13 August 2009 (UTC)<br /> <br /> ==Aeronautics==<br /> <br /> ==Chemistry==<br /> <br /> ===GAMESS (US)===<br /> The GAMESS version January 12, 2009 R3 was built using the Intel v11.1 compilers and v3.2.2 MPI library, according to the instructions in http://software.intel.com/en-us/articles/building-gamess-with-intel-compilers-intel-mkl-and-intel-mpi-on-linux/<br /> <br /> The required build scripts - comp, compall, lked - and run script - rungms - were modified to account for our own installation. In order to build GAMESS one first must ensure that the intel and intelmpi modules are loaded ("module load intel intelmpi"). This applies to running GAMESS as well.<br /> The module "gamess" must also be loaded in order to run GAMESS ("module load gamess").<br /> <br /> The modified scripts are in the file /scinet/gpc/src/gamess-on-scinet.tar.gz<br /> <br /> ====Running GAMESS====<br /> - Make sure the directory /scratch/$USER/gamess-scratch exists (the $SCINET_RUNGMS script will create it if it does not exist)<br /> <br /> - Make sure the modules: intel, intelmpi, gamess are loaded (in your .bashrc: "module load intel intelmpi gamess").<br /> <br /> - Create a torque script to run GAMESS. Here is an example:<br /> <br /> - The GAMESS executable is in $SCINET_GAMESS_HOME/gamess.00.x<br /> - The rungms script is in $SCINET_GAMESS_HOME/rungms (actually it is $SCINET_RUNGMS)<br /> <br /> - For IB multinode runs, use the $SCINET_RUNGMS_IB script<br /> <br /> - The rungms script takes 4 arguments: input file, executable number, number of compute processes, processors per node<br /> <br /> For example, in order to run with the input file /scratch/$USER/gamesstest01, on 8 cpus, and the default version (00)<br /> of the executable on a machine with 8 cores:<br /> <br /> # load the gamess module in .bashrc<br /> module load gamess <br /> <br /> # run the program<br /> $SCINET_RUNGMS /scratch/$USER/gamesstest01 00 8 8<br /> <br /> Here is a sample torque script for running a GAMESS calculation, on a single 8-core node:<br /> <br /> <source lang="bash"><br /> #!/bin/bash<br /> #PBS -l nodes=1:ppn=8,walltime=48:00:00,os=centos53computeA<br /> #PBS -N gamessjob<br /> <br /> ## To submit type: qsub gms.sh<br /> <br /> # If not an interactive job (i.e. -I), then cd into the directory where<br /> # I typed qsub.<br /> if [ "$PBS_ENVIRONMENT" != "PBS_INTERACTIVE" ]; then<br /> if [ -n "$PBS_O_WORKDIR" ]; then<br /> cd $PBS_O_WORKDIR<br /> fi<br /> fi<br /> <br /> # the input file is typically named something like "gamesjob.inp"<br /> # so the script will be run like "$SCINET_RUNGMS gamessjob 00 8 8"<br /> <br /> # load the gamess module if not in .bashrc already<br /> # actually, it MUST be in .bashrc<br /> # module load gamess<br /> <br /> # run the program<br /> <br /> $SCINET_RUNGMS gamessjob 00 8 8<br /> </source><br /> <br /> Here is a similar script, but this one uses 2 InfiniBand-connected nodes,<br /> and runs the appropriate $SCINET_RUNGMS_IB script to actually run the job:<br /> <br /> <source lang="bash"><br /> #!/bin/bash<br /> #PBS -l nodes=2:ib:ppn=8,walltime=48:00:00,os=centos53computeA<br /> #PBS -N gamessjob<br /> <br /> ## To submit type: qsub gmsib.sh<br /> <br /> # If not an interactive job (i.e. -I), then cd into the directory where<br /> # I typed qsub.<br /> if [ "$PBS_ENVIRONMENT" != "PBS_INTERACTIVE" ]; then<br /> if [ -n "$PBS_O_WORKDIR" ]; then<br /> cd $PBS_O_WORKDIR<br /> fi<br /> fi<br /> <br /> # the input file is typically named something like "gamesjob.inp"<br /> # so the script will be run like "$SCINET_RUNGMS gamessjob 00 8 8"<br /> <br /> # load the gamess module if not in .bashrc already<br /> # actually, it MUST be in .bashrc<br /> # module load gamess<br /> <br /> # This script requests InfiniBand-connected nodes (:ib above)<br /> # so it must run with the IB version of the rungms script,<br /> # $SCINET_RUNGMS_IB<br /> <br /> $SCINET_RUNGMS_IB gamessjob 00 16 8<br /> </source><br /> <br /> <br /> -- [[User:Dgruner|dgruner]] 5 October 2009<br /> <br /> ===Tips from the Fekl Lab===<br /> <br /> Through trial and error, we have found a few useful things that we would like to share:<br /> <br /> 1. Two very useful, open-source programs for visualization of output files from GAMESS(US) and for generation of input files are [http://www.scl.ameslab.gov/MacMolPlt/ MacMolPlt]and [http://avogadro.openmolecules.net/wiki/Main_Page Avogadro]. The are available for UNIX/LINUX, Windows and Mac based machines, HOWEVER: any input files that we have generated with these programs on a Windows-based machine do not run on Mac based machines. We don't know why.<br /> <br /> 2. [http://winscp.net/eng/index.php WinSCP] is a very useful tool that has a graphical user interface for moving files from a local machine to SCINET and vice versa. It also has text editing capabilities.<br /> <br /> 3. The [https://bse.pnl.gov/bse/portal ESML Basis Set Exchange] is an excellent source for custom basis set or effective core potential parameters. Make sure that you specify "Gamess-US" in the format drop-down box.<br /> <br /> 4. The commercial program [http://www.chemcraftprog.com/ ChemCraft] is a highly useful visualization program that has the ability to edit molecules in a very similar fashion to GaussView. It can also be customized to build GAMESS(US) input files.<br /> <br /> ====Anatomy of a GAMESS(US) Input File with Basis Set Info in an External File====<br /> <br /> $CONTRL SCFTYP=RHF RUNTYP=OPTIMIZE DFTTYP=M06-L MAXIT=199 MULT=1 NOSYM=1<br /> ECP=READ $END<br /> $SYSTEM TIMLIM=525600 MWORDS=1750 PARALL=.TRUE. $END<br /> $BASIS GBASIS=CUSTOMNI EXTFIL=.t. $END<br /> $SCF DIRSCF=.TRUE. FDIFF=.f. $END<br /> $STATPT OPTTOL=0.0001 NSTEP=500 HSSEND=.t. $END<br /> $DATA<br /> Mo_BDT3<br /> C1<br /> MOLYBDENUM 42.0 5.7556500000 4.4039600000 16.5808400000<br /> SULFUR 16.0 7.4169700000 3.1956300000 15.2089300000<br /> SULFUR 16.0 4.0966800000 3.2258300000 15.1761100000<br /> SULFUR 16.0 3.9677300000 4.4940500000 18.3266100000<br /> SULFUR 16.0 7.1776900000 3.5815000000 18.4485200000<br /> SULFUR 16.0 4.3776600000 6.2447400000 15.6786900000<br /> SULFUR 16.0 7.5478700000 6.0679800000 16.2223700000<br /> CARBON 6.0 6.4716900000 2.1004800000 14.1902300000<br /> CARBON 6.0 5.0690300000 2.1781400000 14.1080700000<br /> CARBON 6.0 4.8421800000 4.2701300000 19.8855500000<br /> CARBON 6.0 6.1969000000 3.9249600000 19.9397400000<br /> CARBON 6.0 6.8280600000 3.7834200000 21.1913200000<br /> CARBON 6.0 5.7697600000 7.6933500000 17.4241800000<br /> CARBON 6.0 7.2043100000 7.9413600000 17.8281100000<br /> CARBON 6.0 5.5051400000 7.0409700000 14.5903800000<br /> CARBON 6.0 6.8905200000 6.9194700000 14.7626200000<br /> CARBON 6.0 7.7396400000 7.5379800000 13.8285700000<br /> HYDROGEN 1.0 8.8190700000 7.4520600000 13.9252200000<br /> CARBON 6.0 7.2169400000 8.2960300000 12.7704100000<br /> HYDROGEN 1.0 7.8667000000 8.7825100000 12.0575600000<br /> CARBON 6.0 5.8260300000 8.4502300000 12.6467800000<br /> HYDROGEN 1.0 5.4143000000 9.0544300000 11.8493100000<br /> CARBON 6.0 4.9881500000 7.8192300000 13.5528400000<br /> HYDROGEN 1.0 3.9090500000 7.9420000000 13.4583700000<br /> CARBON 6.0 7.1538500000 1.1569600000 13.4143900000<br /> CARBON 6.0 4.4018100000 1.3603900000 13.1919900000<br /> CARBON 6.0 6.4791600000 0.3185500000 12.5353300000<br /> CARBON 6.0 5.0837400000 0.4369500000 12.4084900000<br /> HYDROGEN 1.0 7.0116000000 -0.4099400000 11.9434600000<br /> HYDROGEN 1.0 8.2399000000 1.0702400000 13.4937600000<br /> HYDROGEN 1.0 3.3185600000 1.4368700000 13.0953100000<br /> HYDROGEN 1.0 4.5549800000 -0.1997300000 11.7165200000<br /> CARBON 6.0 6.1105700000 3.9639000000 22.3866100000<br /> CARBON 6.0 4.1216300000 4.4424400000 21.1020100000<br /> HYDROGEN 1.0 7.8732900000 3.5217100000 21.2520500000<br /> CARBON 6.0 4.7606000000 4.2868500000 22.3363800000<br /> HYDROGEN 1.0 6.6064200000 3.8406000000 23.3428500000<br /> HYDROGEN 1.0 4.2065000000 4.4170700000 23.2667100000<br /> HYDROGEN 1.0 3.0674000000 4.6893500000 21.0889000000<br /> HYDROGEN 1.0 7.4249200000 7.7545300000 18.8583200000<br /> HYDROGEN 1.0 7.6651700000 8.9049700000 17.7652100000<br /> HYDROGEN 1.0 5.3324000000 8.6487800000 17.2222700000<br /> HYDROGEN 1.0 5.5015000000 7.1039000000 18.2759400000<br /> $END<br /> $ECP<br /> MO-ECP GEN 28 3<br /> 5 ----- f potential -----<br /> -0.0469492 0 537.9667807 <br /> -20.2080084 1 147.8982938 <br /> -106.2116302 2 45.7358898 <br /> -41.8107368 2 13.2911467 <br /> -4.2054103 2 4.7059961 <br /> 3 ----- s-f potential -----<br /> 2.8063717 0 110.2991760 <br /> 44.5162012 1 23.2014645 <br /> 82.7785227 2 5.3530131 <br /> 4 ----- p-f potential -----<br /> 4.9420876 0 63.2901397 <br /> 25.8604976 1 23.3315302 <br /> 132.4708742 2 24.6759423 <br /> 57.3149794 2 4.6493040 <br /> 5 ----- d-f potential -----<br /> 3.0054591 0 104.4839977 <br /> 26.3637851 1 66.2307245 <br /> 183.3849199 2 39.1283176 <br /> 98.4453068 2 13.1164437 <br /> 22.4901377 2 3.6280263 <br /> S NONE<br /> S NONE<br /> S NONE<br /> S NONE<br /> S NONE<br /> S NONE<br /> C NONE<br /> C NONE<br /> C NONE<br /> C NONE<br /> C NONE<br /> C NONE<br /> C NONE<br /> C NONE<br /> C NONE<br /> C NONE<br /> H NONE<br /> C NONE<br /> H NONE<br /> C NONE<br /> H NONE<br /> C NONE<br /> H NONE<br /> C NONE<br /> C NONE<br /> C NONE<br /> C NONE<br /> H NONE<br /> H NONE<br /> H NONE<br /> H NONE<br /> C NONE<br /> C NONE<br /> H NONE<br /> C NONE<br /> H NONE<br /> H NONE<br /> H NONE<br /> H NONE<br /> H NONE<br /> H NONE<br /> H NONE<br /> $END<br /> <br /> =====The Input Deck=====<br /> <br /> Below is the input deck. It is where you tell GAMESS(US) what job type to execute and where all you individual parameters are entered for your specific job type. The example input deck below is for a geometry optimization and frequency calculation. This input deck is equivalent to the Gaussian job with "opt" and "freq" in the route section.<br /> <br /> $CONTRL SCFTYP=RHF RUNTYP=OPTIMIZE DFTTYP=M06-L MAXIT=199 MULT=1 NOSYM=1<br /> ECP=READ $END<br /> $SYSTEM TIMLIM=2850 MWORDS=1750 MEMDDI=20 PARALL=.TRUE. $END<br /> $BASIS GBASIS=CUSTOMNI EXTFIL=.t. $END<br /> $SCF DIRSCF=.TRUE. FDIFF=.f. $END<br /> $STATPT OPTTOL=0.0001 NSTEP=500 HSSEND=.t. $END<br /> $DATA<br /> <br /> An important thing to note is the spacing. In the input deck, there must be 1 space at the beginning of each line of the input deck. If not, the job will fail. Most builders will insert this space anyway, but it helps to double check.<br /> <br /> The end of the input deck is marked by the "$DATA" line.<br /> <br /> =====Job Title Line=====<br /> <br /> The next line of the file is the job title. It can be anthing you wish, however, we have found that to be on the safe side, we avoide using symbols or spaces<br /> <br /> Mo_BDT3<br /> <br /> =====Symmetry Point Group=====<br /> <br /> The next line of the file is the symmetry point group of your molecule. Note that there is no leading space before the point group.<br /> <br /> C1<br /> <br /> =====Coordinates=====<br /> <br /> The next block of text is set aside for the coordinates of the molecule. This can be in internal (or z-matrix) format or cartesian coordinates. Note that there is no leading space before the coordinates. One may use the chemical symbol or the full name of each atom in the molecule. Note that the end of the coordinates is signified by an "$END", which MUST have one space preceding it. The coordinates below do NOT have any basis set information inserted. It is possible to insert basis set information directly into the input file. This is accomplished by obtaining the desired basis set parameters from the EMSL and then inserting them below each relevant atom. An example input file with inserted basis set information will be shown later.<br /> <br /> MOLYBDENUM 42.0 5.7556500000 4.4039600000 16.5808400000<br /> SULFUR 16.0 7.4169700000 3.1956300000 15.2089300000<br /> SULFUR 16.0 4.0966800000 3.2258300000 15.1761100000<br /> SULFUR 16.0 3.9677300000 4.4940500000 18.3266100000<br /> SULFUR 16.0 7.1776900000 3.5815000000 18.4485200000<br /> SULFUR 16.0 4.3776600000 6.2447400000 15.6786900000<br /> SULFUR 16.0 7.5478700000 6.0679800000 16.2223700000<br /> CARBON 6.0 6.4716900000 2.1004800000 14.1902300000<br /> CARBON 6.0 5.0690300000 2.1781400000 14.1080700000<br /> CARBON 6.0 4.8421800000 4.2701300000 19.8855500000<br /> CARBON 6.0 6.1969000000 3.9249600000 19.9397400000<br /> CARBON 6.0 6.8280600000 3.7834200000 21.1913200000<br /> CARBON 6.0 5.7697600000 7.6933500000 17.4241800000<br /> CARBON 6.0 7.2043100000 7.9413600000 17.8281100000<br /> CARBON 6.0 5.5051400000 7.0409700000 14.5903800000<br /> CARBON 6.0 6.8905200000 6.9194700000 14.7626200000<br /> CARBON 6.0 7.7396400000 7.5379800000 13.8285700000<br /> HYDROGEN 1.0 8.8190700000 7.4520600000 13.9252200000<br /> CARBON 6.0 7.2169400000 8.2960300000 12.7704100000<br /> HYDROGEN 1.0 7.8667000000 8.7825100000 12.0575600000<br /> CARBON 6.0 5.8260300000 8.4502300000 12.6467800000<br /> HYDROGEN 1.0 5.4143000000 9.0544300000 11.8493100000<br /> CARBON 6.0 4.9881500000 7.8192300000 13.5528400000<br /> HYDROGEN 1.0 3.9090500000 7.9420000000 13.4583700000<br /> CARBON 6.0 7.1538500000 1.1569600000 13.4143900000<br /> CARBON 6.0 4.4018100000 1.3603900000 13.1919900000<br /> CARBON 6.0 6.4791600000 0.3185500000 12.5353300000<br /> CARBON 6.0 5.0837400000 0.4369500000 12.4084900000<br /> HYDROGEN 1.0 7.0116000000 -0.4099400000 11.9434600000<br /> HYDROGEN 1.0 8.2399000000 1.0702400000 13.4937600000<br /> HYDROGEN 1.0 3.3185600000 1.4368700000 13.0953100000<br /> HYDROGEN 1.0 4.5549800000 -0.1997300000 11.7165200000<br /> CARBON 6.0 6.1105700000 3.9639000000 22.3866100000<br /> CARBON 6.0 4.1216300000 4.4424400000 21.1020100000<br /> HYDROGEN 1.0 7.8732900000 3.5217100000 21.2520500000<br /> CARBON 6.0 4.7606000000 4.2868500000 22.3363800000<br /> HYDROGEN 1.0 6.6064200000 3.8406000000 23.3428500000<br /> HYDROGEN 1.0 4.2065000000 4.4170700000 23.2667100000<br /> HYDROGEN 1.0 3.0674000000 4.6893500000 21.0889000000<br /> HYDROGEN 1.0 7.4249200000 7.7545300000 18.8583200000<br /> HYDROGEN 1.0 7.6651700000 8.9049700000 17.7652100000<br /> HYDROGEN 1.0 5.3324000000 8.6487800000 17.2222700000<br /> HYDROGEN 1.0 5.5015000000 7.1039000000 18.2759400000<br /> $END<br /> <br /> =====Effective Core Potential Data=====<br /> <br /> The effective core potential (ECP) data is entered after the coordinates. It starts with "$ECP", which must be preceded with a space. The atoms of the molecule are listed in the same order as in the coordinates section and the parameters for the ECP are listed after each atom. Note that for any atom that does NOT have an ECP, one must enter "ECP-NONE" or "NONE" after each atom without an ECP.<br /> <br /> $ECP<br /> MO-ECP GEN 28 3<br /> 5 ----- f potential -----<br /> -0.0469492 0 537.9667807 <br /> -20.2080084 1 147.8982938 <br /> -106.2116302 2 45.7358898 <br /> -41.8107368 2 13.2911467 <br /> -4.2054103 2 4.7059961 <br /> 3 ----- s-f potential -----<br /> 2.8063717 0 110.2991760 <br /> 44.5162012 1 23.2014645 <br /> 82.7785227 2 5.3530131 <br /> 4 ----- p-f potential -----<br /> 4.9420876 0 63.2901397 <br /> 25.8604976 1 23.3315302 <br /> 132.4708742 2 24.6759423 <br /> 57.3149794 2 4.6493040 <br /> 5 ----- d-f potential -----<br /> 3.0054591 0 104.4839977 <br /> 26.3637851 1 66.2307245 <br /> 183.3849199 2 39.1283176 <br /> 98.4453068 2 13.1164437 <br /> 22.4901377 2 3.6280263 <br /> S NONE<br /> S NONE<br /> S NONE<br /> S NONE<br /> S NONE<br /> S NONE<br /> C NONE<br /> C NONE<br /> C NONE<br /> C NONE<br /> C NONE<br /> C NONE<br /> C NONE<br /> C NONE<br /> C NONE<br /> C NONE<br /> H NONE<br /> C NONE<br /> H NONE<br /> C NONE<br /> H NONE<br /> C NONE<br /> H NONE<br /> C NONE<br /> C NONE<br /> C NONE<br /> C NONE<br /> H NONE<br /> H NONE<br /> H NONE<br /> H NONE<br /> C NONE<br /> C NONE<br /> H NONE<br /> C NONE<br /> H NONE<br /> H NONE<br /> H NONE<br /> H NONE<br /> H NONE<br /> H NONE<br /> H NONE<br /> $END<br /> <br /> -- [[User:M.Zimmer-De Iuliis|mzd]] 16 November 2009<br /> <br /> ====Using Effective Core Potentials in GAMESS(US)====<br /> <br /> For many metal containing compounds, it is very convenient and time saving to use an effective core potential (ECP) for the core metal electrons, as they are usually not important to the reactivity of the complex or the geometry around the metal. Since GAMESS(US) has a limited number of built-in ECPs, one may want to make GAMESS(US) read an external file that contains the ECP data using the "EXTFIL" keyword in the $GBASIS command line of the input file. In addition, to make GAMESS(US) use this external file, one must copy the "rungms" file and modify it accordingly. The following is a list of instructions with commands that will work from a terminal. One could also use WinSCP to do all of this with a GUI rather than a TUI. <br /> <br /> =====Modifiying rungms to Use Custom Basis Set File=====<br /> 1. Copy "rungms" from /scinet/gpc/Applications/gamess to one's own /scratch/$USER/ directory:<br /> cp /scinet/gpc/Applications/gamess/rungms /scratch/$USER/<br /> <br /> 2. Change to the scratch directory and check to see if "rungms" has copied successfully.<br /> cd /scratch/$USER<br /> ls<br /> <br /> 3. Edit line 147 of the vi. <br /> vi rungms<br /> Move the cursor down to line 147 using the arrow keys. It should say "setenv EXTBAS /dev/null". Using the arrow keys, move the cursor to the first "/" and then hit "i" to insert text. Put the path to your external basis file here. For example, /scratch/$USER/basisset. Then hit "escape". To save the changes and exit vi, type ":" and you should see a colon appear at the bottom of the window. Type "wq" (which should appear at the bottom of the window next to the colon) and then hit enter. Now you are done with vi.<br /> <br /> =====Creating a Custom Basis Set File=====<br /> 1. To create a custom basis set file, you need create a new text document. Our group's common practice is to comment out the first line of this file by inserting an exclamation mark (!) followed by noting the specific basis sets and ECPs that are going to be used for each of the atoms. Let us the molecule Mo(CO)6, Molybdenum hexacarbonyl, as an example. Below is the first line of the the external file, which we will call "CUSTOMMO" (NOTE: you can use any name for the external file that suits you, as long as it has no spaces and is 8 characters or less).<br /> <br /> ! 6-31G on C and O and LANL2D2 ECP on Mo<br /> <br /> 2. The next step is to visit the [https://bse.pnl.gov/bse/portal EMSL Basis Set exchange] and select C and O from the periodic table. Then, on the left of the page, select "6-31G" as the basis set. Finally, make sure the output is in GAMESS(US) format using the drop-down menu and then click "get basis set".<br /> <br /> [[File:C_O_6_31G_basisset.JPG|centre]]<br /> <br /> 3. A new window should appear with text in it. For our example case, the text looks like this:<br /> <br /> ! 6-31G EMSL Basis Set Exchange Library 10/13/09 11:12 AM<br /> ! Elements References<br /> ! -------- ----------<br /> ! H - He: W.J. Hehre, R. Ditchfield and J.A. Pople, J. Chem. Phys. 56,<br /> ! Li - Ne: 2257 (1972). Note: Li and B come from J.D. Dill and J.A.<br /> ! Pople, J. Chem. Phys. 62, 2921 (1975).<br /> ! Na - Ar: M.M. Francl, W.J. Petro, W.J. Hehre, J.S. Binkley, M.S. Gordon,<br /> ! D.J. DeFrees and J.A. Pople, J. Chem. Phys. 77, 3654 (1982)<br /> ! K - Zn: V. Rassolov, J.A. Pople, M. Ratner and T.L. Windus, J. Chem. Phys.<br /> ! 109, 1223 (1998)<br /> ! Note: He and Ne are unpublished basis sets taken from the Gaussian<br /> ! program<br /> ! <br /> $DATA<br /><br /> CARBON<br /> S 6<br /> 1 3047.5249000 0.0018347 <br /> 2 457.3695100 0.0140373 <br /> 3 103.9486900 0.0688426 <br /> 4 29.2101550 0.2321844 <br /> 5 9.2866630 0.4679413 <br /> 6 3.1639270 0.3623120 <br /> L 3<br /> 1 7.8682724 -0.1193324 0.0689991 <br /> 2 1.8812885 -0.1608542 0.3164240 <br /> 3 0.5442493 1.1434564 0.7443083 <br /> L 1<br /> 1 0.1687144 1.0000000 1.0000000<br /> <br /> OXYGEN<br /> S 6<br /> 1 5484.6717000 0.0018311 <br /> 2 825.2349500 0.0139501 <br /> 3 188.0469600 0.0684451 <br /> 4 52.9645000 0.2327143 <br /> 5 16.8975700 0.4701930 <br /> 6 5.7996353 0.3585209 <br /> L 3<br /> 1 15.5396160 -0.1107775 0.0708743 <br /> 2 3.5999336 -0.1480263 0.3397528 <br /> 3 1.0137618 1.1307670 0.7271586 <br /> L 1<br /> 1 0.2700058 1.0000000 1.0000000 <br /> $END<br /> <br /> 3. Now, copy and paste the text between the $DATA and $END headings onto our external text file, CUSTOMMO. We also need to change the change the name of each element to it symbol in the periodic table. Finally, we need to add the name of the external file next to the element symbol, separated by one space. Note that there should be a blank line separating the basis set information and the first, commented-out line (The line starting with the '!'). The CUSTOMMO should look like this:<br /> <br /> ! 6-31G on C and O and LANL2D2 ECP on Mo<br /><br /> C CUSTOMMO<br /> S 6<br /> 1 3047.5249000 0.0018347 <br /> 2 457.3695100 0.0140373 <br /> 3 103.9486900 0.0688426 <br /> 4 29.2101550 0.2321844 <br /> 5 9.2866630 0.4679413 <br /> 6 3.1639270 0.3623120 <br /> L 3<br /> 1 7.8682724 -0.1193324 0.0689991 <br /> 2 1.8812885 -0.1608542 0.3164240 <br /> 3 0.5442493 1.1434564 0.7443083 <br /> L 1<br /> 1 0.1687144 1.0000000 1.0000000<br /> <br /> O CUSTOMMO<br /> S 6<br /> 1 5484.6717000 0.0018311 <br /> 2 825.2349500 0.0139501 <br /> 3 188.0469600 0.0684451 <br /> 4 52.9645000 0.2327143 <br /> 5 16.8975700 0.4701930 <br /> 6 5.7996353 0.3585209 <br /> L 3<br /> 1 15.5396160 -0.1107775 0.0708743 <br /> 2 3.5999336 -0.1480263 0.3397528 <br /> 3 1.0137618 1.1307670 0.7271586 <br /> L 1<br /> 1 0.2700058 1.0000000 1.0000000 <br /> <br /> 4. Repeat Step 3 above but choose Mo and select the LANL2DZ ECP instead. A new window will pop up with the basis set information as well as the ECP data we need, since we specified the LANL2DZ '''ECP'''. The ECP data is not inserted into the external file, rather it is placed into the input file itself (More on this later). <br /> <br /> [[File:Mo_LANL2DZ_basisset.JPG|centre]]<br /> <br /> 5. After copying the molybdenum basis set information, your fiished external basis set file should look like this:<br /> ! 6-31G on C and O and LANL2D2 ECP on Mo<br /><br /> C CUSTOMMO<br /> S 6<br /> 1 3047.5249000 0.0018347 <br /> 2 457.3695100 0.0140373 <br /> 3 103.9486900 0.0688426 <br /> 4 29.2101550 0.2321844 <br /> 5 9.2866630 0.4679413 <br /> 6 3.1639270 0.3623120 <br /> L 3<br /> 1 7.8682724 -0.1193324 0.0689991 <br /> 2 1.8812885 -0.1608542 0.3164240 <br /> 3 0.5442493 1.1434564 0.7443083 <br /> L 1<br /> 1 0.1687144 1.0000000 1.0000000<br /> <br /> O CUSTOMMO<br /> S 6<br /> 1 5484.6717000 0.0018311 <br /> 2 825.2349500 0.0139501 <br /> 3 188.0469600 0.0684451 <br /> 4 52.9645000 0.2327143 <br /> 5 16.8975700 0.4701930 <br /> 6 5.7996353 0.3585209 <br /> L 3<br /> 1 15.5396160 -0.1107775 0.0708743 <br /> 2 3.5999336 -0.1480263 0.3397528 <br /> 3 1.0137618 1.1307670 0.7271586 <br /> L 1<br /> 1 0.2700058 1.0000000 1.0000000<br /> <br /> Mo CUSTOMO<br /> S 3<br /> 1 2.3610000 -0.9121760 <br /> 2 1.3090000 1.1477453 <br /> 3 0.4500000 0.6097109 <br /> S 4<br /> 1 2.3610000 0.8139259 <br /> 2 1.3090000 -1.1360084 <br /> 3 0.4500000 -1.1611592 <br /> 4 0.1681000 1.0064786 <br /> S 1<br /> 1 0.0423000 1.0000000 <br /> P 3<br /> 1 4.8950000 -0.0908258 <br /> 2 1.0440000 0.7042899 <br /> 3 0.3877000 0.3973179 <br /> P 2<br /> 1 0.4995000 -0.1081945 <br /> 2 0.0780000 1.0368093 <br /> P 1<br /> 1 0.0247000 1.0000000 <br /> D 3<br /> 1 2.9930000 0.0527063 <br /> 2 1.0630000 0.5003907 <br /> 3 0.3721000 0.5794024 <br /> D 1<br /> 1 0.1178000 1.0000000<br /> <br /> -- [[User:M.Zimmer-De Iuliis|mzd]] 13 October 2009<br /> <br /> ==Climate Modelling==<br /> <br /> ==Medicine/Bio==<br /> <br /> ==High Energy Physics==<br /> <br /> ==Structural Biology==<br /> Molecular simulation of proteins, lipids, carbohydrates, and other biologically relevant molecules.<br /> ===Molecular Dynamics (MD) simulation===<br /> ====GROMACS====<br /> Please refer to the [[gromacs|GROMACS]] page<br /> ====AMBER====<br /> Please refer to the [[amber|AMBER]] page<br /> ====NAMD====<br /> NAMD is one of the better scaling MD packages out there. With sufficiently large systems, it is able to scale to hundreds or thousands of cores on Scinet. Below are details for compiling and running NAMD on Scinet.<br /> <br /> More information regarding performance and different compile options coming soon...<br /> <br /> =====Compiling NAMD for GPC=====<br /> Ensure the proper compiler/mpi modules are loaded.<br /> <source lang="sh"><br /> module load intel<br /> module load openmpi/1.3.3-intel-v11.0-ofed<br /> </source><br /> <br /> '''Compile Charm++ and NAMD'''<br /> <source lang="sh"><br /> #Unpack source files and get required support libraries<br /> tar -xzf NAMD_2.7b1_Source.tar.gz<br /> cd NAMD_2.7b1_Source<br /> tar -xf charm-6.1.tar<br /> wget http://www.ks.uiuc.edu/Research/namd/libraries/fftw-linux-x86_64.tar.gz<br /> wget http://www.ks.uiuc.edu/Research/namd/libraries/tcl-linux-x86_64.tar.gz<br /> tar -xzf fftw-linux-x86_64.tar.gz; mv linux-x86_64 fftw<br /> tar -xzf tcl-linux-x86_64.tar.gz; mv linux-x86_64 tcl<br /> #Compile Charm++<br /> cd charm-6.1<br /> ./build charm++ mpi-linux-x86_64 icc --basedir /scinet/gpc/mpi/openmpi/1.3.3-intel-v11.0-ofed/ --no-shared -O -DCMK_OPTIMIZE=1<br /> cd ..<br /> #Compile NAMD. <br /> #Edit arch/Linux-x86_64-icc.arch and add "-lmpi" to the end of the CXXOPTS and COPTS line.<br /> #Make a builds directory if you want different versions of NAMD compiled at the same time.<br /> mkdir builds<br /> ./config builds/Linux-x86_64-icc --charm-arch mpi-linux-x86_64-icc<br /> cd builds/Linux-x86_64-icc/<br /> make -j4 namd2 # Adjust value of j as desired to specify number of simultaneous make targets. <br /> </source><br /> --[[User:Cmadill|Cmadill]] 16:18, 27 August 2009 (UTC)<br /> <br /> =====Running Fortran=====<br /> On the development nodes, there is an old gcc. The associated libraries are not on the compute nodes. Ensure the line:<br /> <br /> module load gcc<br /> <br /> is in your .bashrc file.<br /> <br /> ====LAMMPS====<br /> [[Image:StrongScalingLAMMPS.png|thumb|320px|right|Strong scaling test on GPC with OpenMPI and IntelMPI on Ethernet and InfiniBand]]<br /> [[Image:WeakScalingLAMMPS.png|thumb|320px|right|Weak scaling test on GPC with OpenMPI and IntelMPI on Ethernet and InfiniBand]]<br /> Results from strong scaling tests for LAMMPS using EAM potentials on GPC are shown in the graph on the right. The test simulation ran 500 timesteps for 4,000,000 atoms.<br /> <br /> Results from weak scaling tests for LAMMPS using EAM potentials on GPC are shown in the graph on the right. The test simulation ran 500 timesteps for 32,000 atoms per processor.<br /> <br /> OpenMPI version used: openmpi/1.4.1-intel-v11.0-ofed<br /> <br /> IntelMPI version used: intelmpi/impi-4.0.0.013<br /> <br /> Results show good scaling for both OpenMPI and IntelMPI up to 16 processors, after which performance begins to suffer. On Infiniband, excellent scaling is maintained to 512 processors.<br /> <br /> IntelMPI shows slightly better performance compared to OpenMPI when running with Infiniband.<br /> <br /> --[[User:jchu|jchu]] Feb 2, 2010<br /> <br /> ===Monte Carlo (MC) simulation===</div>

Jchu https://oldwiki.scinet.utoronto.ca/index.php?title=User_Codes&diff=878 User Codes 2010-02-02T19:00:08Z

<p>Jchu: /* LAMMPS */</p> <hr /> <div>__FORCETOC__<br /> <br /> ==Astrophysics==<br /> <br /> ===Athena (explicit, uniform grid MHD code)===<br /> <br /> [[Image:StrongScalingAthenaGPC.png|thumb|right|320px|Athena scaling on GPC with OpenMPI and MVAPICH2 on GigE, and OpenMPI on InfiniBand]]<br /> <br /> [http://www.astro.princeton.edu/~jstone/athena.html Athena] is a straightforward C code which doesn't use a lot of libraries so it is pretty straightforward to build and compile on new machines. <br /> <br /> It encapsulates its compiler flags, etc in an <tt>Makeoptions.in</tt> file which is then processed by <tt>configure</tt>. I've used the following additions to <tt>Makeoptions.in</tt> on TCS and GPC:<br /> <br /> <source lang="sh"><br /> ifeq ($(MACHINE),scinettcs)<br /> CC = mpcc_r<br /> LDR = mpcc_r<br /> OPT = -O5 -q64 -qarch=pwr6 -qtune=pwr6 -qcache=auto -qlargepage -qstrict<br /> MPIINC =<br /> MPILIB =<br /> CFLAGS = $(OPT)<br /> LIB = -ldl -lm<br /> else<br /> ifeq ($(MACHINE),scinetgpc)<br /> CC = mpicc<br /> LDR = mpicc<br /> OPT = -O3<br /> MPIINC =<br /> MPILIB =<br /> CFLAGS = $(OPT)<br /> LIB = -lm<br /> else<br /> ...<br /> endif<br /> endif<br /> </source><br /> It performs quite well on the GPC, scaling extremely well even on a strong scaling test out to about 256 cores (32 nodes) on Gigabit ethernet, and performing beautifully on InfiniBand out to 512 cores (64 nodes). <br /> <br /> <br /> -- [[User:Ljdursi|ljdursi]] 19:20, 13 August 2009 (UTC)<br /> <br /> ===FLASH3 (Adaptive Mesh reactive hydrodynamics; explict hydro/MHD)===<br /> <br /> [[Image:weak-scaling-example.png|thumb|right|320px|Weak scaling test of the 2d sod problem on both the GPC and TCS. The results are actually somewhat faster on the GPC; in both cases (weak) scaling is very good out at least to 256 cores]]<br /> <br /> [http://flash.uchicago.edu FLASH] encapsulates its machine-dependant information in the <tt>FLASH3/sites</tt> directory. For the GPC, you'll have to<br /> <pre><br /> module load intel<br /> module load openmpi<br /> module load hdf5/183-v16-openmpi<br /> </pre><br /> <br /> and with that, the following file (<tt>sites/scinetgpc/Makefile.h</tt>) works for me:<br /> <source lang="sh"><br /> ## Must do module load hdf5/183-v16-openmpi<br /> HDF5_PATH = ${SCINET_HDF5_BASE}<br /> ZLIB_PATH = /usr/local<br /> <br /> #----------------------------------------------------------------------------<br /> # Compiler and linker commands<br /> #<br /> # We use the f90 compiler as the linker, so some C libraries may explicitly<br /> # need to be added into the link line.<br /> #----------------------------------------------------------------------------<br /> <br /> ## modules will put the right mpi in our path<br /> FCOMP = mpif77<br /> CCOMP = mpicc<br /> CPPCOMP = mpiCC<br /> LINK = mpif77<br /> <br /> #----------------------------------------------------------------------------<br /> # Compilation flags<br /> #<br /> # Three sets of compilation/linking flags are defined: one for optimized<br /> # code, one for testing, and one for debugging. The default is to use the <br /> # _OPT version. Specifying -debug to setup will pick the _DEBUG version,<br /> # these should enable bounds checking. Specifying -test is used for <br /> # flash_test, and is set for quick code generation, and (sometimes) <br /> # profiling. The Makefile generated by setup will assign the generic token <br /> # (ex. FFLAGS) to the proper set of flags (ex. FFLAGS_OPT).<br /> #----------------------------------------------------------------------------<br /> <br /> FFLAGS_OPT = -c -r8 -i4 -O3 -xSSE4.2<br /> FFLAGS_DEBUG = -c -g -r8 -i4 -O0<br /> FFLAGS_TEST = -c -r8 -i4<br /> <br /> <br /> # if we are using HDF5, we need to specify the path to the include files<br /> CFLAGS_HDF5 = -I${HDF5_PATH}/include<br /> <br /> CFLAGS_OPT = -c -O3 -xSSE4.2<br /> CFLAGS_TEST = -c -O2 <br /> CFLAGS_DEBUG = -c -g <br /> <br /> MDEFS = <br /> <br /> .SUFFIXES: .o .c .f .F .h .fh .F90 .f90<br /> <br /> #----------------------------------------------------------------------------<br /> # Linker flags<br /> #<br /> # There is a seperate version of the linker flags for each of the _OPT, <br /> # _DEBUG, and _TEST cases.<br /> #----------------------------------------------------------------------------<br /> <br /> LFLAGS_OPT = -o<br /> LFLAGS_TEST = -o<br /> LFLAGS_DEBUG = -g -o<br /> <br /> MACHOBJ = <br /> <br /> <br /> MV = mv -f<br /> AR = ar -r<br /> RM = rm -f<br /> CD = cd<br /> RL = ranlib<br /> ECHO = echo<br /> </source><br /> <br /> <br /> -- [[User:Ljdursi|ljdursi]] 22:11, 13 August 2009 (UTC)<br /> <br /> ==Aeronautics==<br /> <br /> ==Chemistry==<br /> <br /> ===GAMESS (US)===<br /> The GAMESS version January 12, 2009 R3 was built using the Intel v11.1 compilers and v3.2.2 MPI library, according to the instructions in http://software.intel.com/en-us/articles/building-gamess-with-intel-compilers-intel-mkl-and-intel-mpi-on-linux/<br /> <br /> The required build scripts - comp, compall, lked - and run script - rungms - were modified to account for our own installation. In order to build GAMESS one first must ensure that the intel and intelmpi modules are loaded ("module load intel intelmpi"). This applies to running GAMESS as well.<br /> The module "gamess" must also be loaded in order to run GAMESS ("module load gamess").<br /> <br /> The modified scripts are in the file /scinet/gpc/src/gamess-on-scinet.tar.gz<br /> <br /> ====Running GAMESS====<br /> - Make sure the directory /scratch/$USER/gamess-scratch exists (the $SCINET_RUNGMS script will create it if it does not exist)<br /> <br /> - Make sure the modules: intel, intelmpi, gamess are loaded (in your .bashrc: "module load intel intelmpi gamess").<br /> <br /> - Create a torque script to run GAMESS. Here is an example:<br /> <br /> - The GAMESS executable is in $SCINET_GAMESS_HOME/gamess.00.x<br /> - The rungms script is in $SCINET_GAMESS_HOME/rungms (actually it is $SCINET_RUNGMS)<br /> <br /> - For IB multinode runs, use the $SCINET_RUNGMS_IB script<br /> <br /> - The rungms script takes 4 arguments: input file, executable number, number of compute processes, processors per node<br /> <br /> For example, in order to run with the input file /scratch/$USER/gamesstest01, on 8 cpus, and the default version (00)<br /> of the executable on a machine with 8 cores:<br /> <br /> # load the gamess module in .bashrc<br /> module load gamess <br /> <br /> # run the program<br /> $SCINET_RUNGMS /scratch/$USER/gamesstest01 00 8 8<br /> <br /> Here is a sample torque script for running a GAMESS calculation, on a single 8-core node:<br /> <br /> <source lang="bash"><br /> #!/bin/bash<br /> #PBS -l nodes=1:ppn=8,walltime=48:00:00,os=centos53computeA<br /> #PBS -N gamessjob<br /> <br /> ## To submit type: qsub gms.sh<br /> <br /> # If not an interactive job (i.e. -I), then cd into the directory where<br /> # I typed qsub.<br /> if [ "$PBS_ENVIRONMENT" != "PBS_INTERACTIVE" ]; then<br /> if [ -n "$PBS_O_WORKDIR" ]; then<br /> cd $PBS_O_WORKDIR<br /> fi<br /> fi<br /> <br /> # the input file is typically named something like "gamesjob.inp"<br /> # so the script will be run like "$SCINET_RUNGMS gamessjob 00 8 8"<br /> <br /> # load the gamess module if not in .bashrc already<br /> # actually, it MUST be in .bashrc<br /> # module load gamess<br /> <br /> # run the program<br /> <br /> $SCINET_RUNGMS gamessjob 00 8 8<br /> </source><br /> <br /> Here is a similar script, but this one uses 2 InfiniBand-connected nodes,<br /> and runs the appropriate $SCINET_RUNGMS_IB script to actually run the job:<br /> <br /> <source lang="bash"><br /> #!/bin/bash<br /> #PBS -l nodes=2:ib:ppn=8,walltime=48:00:00,os=centos53computeA<br /> #PBS -N gamessjob<br /> <br /> ## To submit type: qsub gmsib.sh<br /> <br /> # If not an interactive job (i.e. -I), then cd into the directory where<br /> # I typed qsub.<br /> if [ "$PBS_ENVIRONMENT" != "PBS_INTERACTIVE" ]; then<br /> if [ -n "$PBS_O_WORKDIR" ]; then<br /> cd $PBS_O_WORKDIR<br /> fi<br /> fi<br /> <br /> # the input file is typically named something like "gamesjob.inp"<br /> # so the script will be run like "$SCINET_RUNGMS gamessjob 00 8 8"<br /> <br /> # load the gamess module if not in .bashrc already<br /> # actually, it MUST be in .bashrc<br /> # module load gamess<br /> <br /> # This script requests InfiniBand-connected nodes (:ib above)<br /> # so it must run with the IB version of the rungms script,<br /> # $SCINET_RUNGMS_IB<br /> <br /> $SCINET_RUNGMS_IB gamessjob 00 16 8<br /> </source><br /> <br /> <br /> -- [[User:Dgruner|dgruner]] 5 October 2009<br /> <br /> ===Tips from the Fekl Lab===<br /> <br /> Through trial and error, we have found a few useful things that we would like to share:<br /> <br /> 1. Two very useful, open-source programs for visualization of output files from GAMESS(US) and for generation of input files are [http://www.scl.ameslab.gov/MacMolPlt/ MacMolPlt]and [http://avogadro.openmolecules.net/wiki/Main_Page Avogadro]. The are available for UNIX/LINUX, Windows and Mac based machines, HOWEVER: any input files that we have generated with these programs on a Windows-based machine do not run on Mac based machines. We don't know why.<br /> <br /> 2. [http://winscp.net/eng/index.php WinSCP] is a very useful tool that has a graphical user interface for moving files from a local machine to SCINET and vice versa. It also has text editing capabilities.<br /> <br /> 3. The [https://bse.pnl.gov/bse/portal ESML Basis Set Exchange] is an excellent source for custom basis set or effective core potential parameters. Make sure that you specify "Gamess-US" in the format drop-down box.<br /> <br /> 4. The commercial program [http://www.chemcraftprog.com/ ChemCraft] is a highly useful visualization program that has the ability to edit molecules in a very similar fashion to GaussView. It can also be customized to build GAMESS(US) input files.<br /> <br /> ====Anatomy of a GAMESS(US) Input File with Basis Set Info in an External File====<br /> <br /> $CONTRL SCFTYP=RHF RUNTYP=OPTIMIZE DFTTYP=M06-L MAXIT=199 MULT=1 NOSYM=1<br /> ECP=READ $END<br /> $SYSTEM TIMLIM=525600 MWORDS=1750 PARALL=.TRUE. $END<br /> $BASIS GBASIS=CUSTOMNI EXTFIL=.t. $END<br /> $SCF DIRSCF=.TRUE. FDIFF=.f. $END<br /> $STATPT OPTTOL=0.0001 NSTEP=500 HSSEND=.t. $END<br /> $DATA<br /> Mo_BDT3<br /> C1<br /> MOLYBDENUM 42.0 5.7556500000 4.4039600000 16.5808400000<br /> SULFUR 16.0 7.4169700000 3.1956300000 15.2089300000<br /> SULFUR 16.0 4.0966800000 3.2258300000 15.1761100000<br /> SULFUR 16.0 3.9677300000 4.4940500000 18.3266100000<br /> SULFUR 16.0 7.1776900000 3.5815000000 18.4485200000<br /> SULFUR 16.0 4.3776600000 6.2447400000 15.6786900000<br /> SULFUR 16.0 7.5478700000 6.0679800000 16.2223700000<br /> CARBON 6.0 6.4716900000 2.1004800000 14.1902300000<br /> CARBON 6.0 5.0690300000 2.1781400000 14.1080700000<br /> CARBON 6.0 4.8421800000 4.2701300000 19.8855500000<br /> CARBON 6.0 6.1969000000 3.9249600000 19.9397400000<br /> CARBON 6.0 6.8280600000 3.7834200000 21.1913200000<br /> CARBON 6.0 5.7697600000 7.6933500000 17.4241800000<br /> CARBON 6.0 7.2043100000 7.9413600000 17.8281100000<br /> CARBON 6.0 5.5051400000 7.0409700000 14.5903800000<br /> CARBON 6.0 6.8905200000 6.9194700000 14.7626200000<br /> CARBON 6.0 7.7396400000 7.5379800000 13.8285700000<br /> HYDROGEN 1.0 8.8190700000 7.4520600000 13.9252200000<br /> CARBON 6.0 7.2169400000 8.2960300000 12.7704100000<br /> HYDROGEN 1.0 7.8667000000 8.7825100000 12.0575600000<br /> CARBON 6.0 5.8260300000 8.4502300000 12.6467800000<br /> HYDROGEN 1.0 5.4143000000 9.0544300000 11.8493100000<br /> CARBON 6.0 4.9881500000 7.8192300000 13.5528400000<br /> HYDROGEN 1.0 3.9090500000 7.9420000000 13.4583700000<br /> CARBON 6.0 7.1538500000 1.1569600000 13.4143900000<br /> CARBON 6.0 4.4018100000 1.3603900000 13.1919900000<br /> CARBON 6.0 6.4791600000 0.3185500000 12.5353300000<br /> CARBON 6.0 5.0837400000 0.4369500000 12.4084900000<br /> HYDROGEN 1.0 7.0116000000 -0.4099400000 11.9434600000<br /> HYDROGEN 1.0 8.2399000000 1.0702400000 13.4937600000<br /> HYDROGEN 1.0 3.3185600000 1.4368700000 13.0953100000<br /> HYDROGEN 1.0 4.5549800000 -0.1997300000 11.7165200000<br /> CARBON 6.0 6.1105700000 3.9639000000 22.3866100000<br /> CARBON 6.0 4.1216300000 4.4424400000 21.1020100000<br /> HYDROGEN 1.0 7.8732900000 3.5217100000 21.2520500000<br /> CARBON 6.0 4.7606000000 4.2868500000 22.3363800000<br /> HYDROGEN 1.0 6.6064200000 3.8406000000 23.3428500000<br /> HYDROGEN 1.0 4.2065000000 4.4170700000 23.2667100000<br /> HYDROGEN 1.0 3.0674000000 4.6893500000 21.0889000000<br /> HYDROGEN 1.0 7.4249200000 7.7545300000 18.8583200000<br /> HYDROGEN 1.0 7.6651700000 8.9049700000 17.7652100000<br /> HYDROGEN 1.0 5.3324000000 8.6487800000 17.2222700000<br /> HYDROGEN 1.0 5.5015000000 7.1039000000 18.2759400000<br /> $END<br /> $ECP<br /> MO-ECP GEN 28 3<br /> 5 ----- f potential -----<br /> -0.0469492 0 537.9667807 <br /> -20.2080084 1 147.8982938 <br /> -106.2116302 2 45.7358898 <br /> -41.8107368 2 13.2911467 <br /> -4.2054103 2 4.7059961 <br /> 3 ----- s-f potential -----<br /> 2.8063717 0 110.2991760 <br /> 44.5162012 1 23.2014645 <br /> 82.7785227 2 5.3530131 <br /> 4 ----- p-f potential -----<br /> 4.9420876 0 63.2901397 <br /> 25.8604976 1 23.3315302 <br /> 132.4708742 2 24.6759423 <br /> 57.3149794 2 4.6493040 <br /> 5 ----- d-f potential -----<br /> 3.0054591 0 104.4839977 <br /> 26.3637851 1 66.2307245 <br /> 183.3849199 2 39.1283176 <br /> 98.4453068 2 13.1164437 <br /> 22.4901377 2 3.6280263 <br /> S NONE<br /> S NONE<br /> S NONE<br /> S NONE<br /> S NONE<br /> S NONE<br /> C NONE<br /> C NONE<br /> C NONE<br /> C NONE<br /> C NONE<br /> C NONE<br /> C NONE<br /> C NONE<br /> C NONE<br /> C NONE<br /> H NONE<br /> C NONE<br /> H NONE<br /> C NONE<br /> H NONE<br /> C NONE<br /> H NONE<br /> C NONE<br /> C NONE<br /> C NONE<br /> C NONE<br /> H NONE<br /> H NONE<br /> H NONE<br /> H NONE<br /> C NONE<br /> C NONE<br /> H NONE<br /> C NONE<br /> H NONE<br /> H NONE<br /> H NONE<br /> H NONE<br /> H NONE<br /> H NONE<br /> H NONE<br /> $END<br /> <br /> =====The Input Deck=====<br /> <br /> Below is the input deck. It is where you tell GAMESS(US) what job type to execute and where all you individual parameters are entered for your specific job type. The example input deck below is for a geometry optimization and frequency calculation. This input deck is equivalent to the Gaussian job with "opt" and "freq" in the route section.<br /> <br /> $CONTRL SCFTYP=RHF RUNTYP=OPTIMIZE DFTTYP=M06-L MAXIT=199 MULT=1 NOSYM=1<br /> ECP=READ $END<br /> $SYSTEM TIMLIM=2850 MWORDS=1750 MEMDDI=20 PARALL=.TRUE. $END<br /> $BASIS GBASIS=CUSTOMNI EXTFIL=.t. $END<br /> $SCF DIRSCF=.TRUE. FDIFF=.f. $END<br /> $STATPT OPTTOL=0.0001 NSTEP=500 HSSEND=.t. $END<br /> $DATA<br /> <br /> An important thing to note is the spacing. In the input deck, there must be 1 space at the beginning of each line of the input deck. If not, the job will fail. Most builders will insert this space anyway, but it helps to double check.<br /> <br /> The end of the input deck is marked by the "$DATA" line.<br /> <br /> =====Job Title Line=====<br /> <br /> The next line of the file is the job title. It can be anthing you wish, however, we have found that to be on the safe side, we avoide using symbols or spaces<br /> <br /> Mo_BDT3<br /> <br /> =====Symmetry Point Group=====<br /> <br /> The next line of the file is the symmetry point group of your molecule. Note that there is no leading space before the point group.<br /> <br /> C1<br /> <br /> =====Coordinates=====<br /> <br /> The next block of text is set aside for the coordinates of the molecule. This can be in internal (or z-matrix) format or cartesian coordinates. Note that there is no leading space before the coordinates. One may use the chemical symbol or the full name of each atom in the molecule. Note that the end of the coordinates is signified by an "$END", which MUST have one space preceding it. The coordinates below do NOT have any basis set information inserted. It is possible to insert basis set information directly into the input file. This is accomplished by obtaining the desired basis set parameters from the EMSL and then inserting them below each relevant atom. An example input file with inserted basis set information will be shown later.<br /> <br /> MOLYBDENUM 42.0 5.7556500000 4.4039600000 16.5808400000<br /> SULFUR 16.0 7.4169700000 3.1956300000 15.2089300000<br /> SULFUR 16.0 4.0966800000 3.2258300000 15.1761100000<br /> SULFUR 16.0 3.9677300000 4.4940500000 18.3266100000<br /> SULFUR 16.0 7.1776900000 3.5815000000 18.4485200000<br /> SULFUR 16.0 4.3776600000 6.2447400000 15.6786900000<br /> SULFUR 16.0 7.5478700000 6.0679800000 16.2223700000<br /> CARBON 6.0 6.4716900000 2.1004800000 14.1902300000<br /> CARBON 6.0 5.0690300000 2.1781400000 14.1080700000<br /> CARBON 6.0 4.8421800000 4.2701300000 19.8855500000<br /> CARBON 6.0 6.1969000000 3.9249600000 19.9397400000<br /> CARBON 6.0 6.8280600000 3.7834200000 21.1913200000<br /> CARBON 6.0 5.7697600000 7.6933500000 17.4241800000<br /> CARBON 6.0 7.2043100000 7.9413600000 17.8281100000<br /> CARBON 6.0 5.5051400000 7.0409700000 14.5903800000<br /> CARBON 6.0 6.8905200000 6.9194700000 14.7626200000<br /> CARBON 6.0 7.7396400000 7.5379800000 13.8285700000<br /> HYDROGEN 1.0 8.8190700000 7.4520600000 13.9252200000<br /> CARBON 6.0 7.2169400000 8.2960300000 12.7704100000<br /> HYDROGEN 1.0 7.8667000000 8.7825100000 12.0575600000<br /> CARBON 6.0 5.8260300000 8.4502300000 12.6467800000<br /> HYDROGEN 1.0 5.4143000000 9.0544300000 11.8493100000<br /> CARBON 6.0 4.9881500000 7.8192300000 13.5528400000<br /> HYDROGEN 1.0 3.9090500000 7.9420000000 13.4583700000<br /> CARBON 6.0 7.1538500000 1.1569600000 13.4143900000<br /> CARBON 6.0 4.4018100000 1.3603900000 13.1919900000<br /> CARBON 6.0 6.4791600000 0.3185500000 12.5353300000<br /> CARBON 6.0 5.0837400000 0.4369500000 12.4084900000<br /> HYDROGEN 1.0 7.0116000000 -0.4099400000 11.9434600000<br /> HYDROGEN 1.0 8.2399000000 1.0702400000 13.4937600000<br /> HYDROGEN 1.0 3.3185600000 1.4368700000 13.0953100000<br /> HYDROGEN 1.0 4.5549800000 -0.1997300000 11.7165200000<br /> CARBON 6.0 6.1105700000 3.9639000000 22.3866100000<br /> CARBON 6.0 4.1216300000 4.4424400000 21.1020100000<br /> HYDROGEN 1.0 7.8732900000 3.5217100000 21.2520500000<br /> CARBON 6.0 4.7606000000 4.2868500000 22.3363800000<br /> HYDROGEN 1.0 6.6064200000 3.8406000000 23.3428500000<br /> HYDROGEN 1.0 4.2065000000 4.4170700000 23.2667100000<br /> HYDROGEN 1.0 3.0674000000 4.6893500000 21.0889000000<br /> HYDROGEN 1.0 7.4249200000 7.7545300000 18.8583200000<br /> HYDROGEN 1.0 7.6651700000 8.9049700000 17.7652100000<br /> HYDROGEN 1.0 5.3324000000 8.6487800000 17.2222700000<br /> HYDROGEN 1.0 5.5015000000 7.1039000000 18.2759400000<br /> $END<br /> <br /> =====Effective Core Potential Data=====<br /> <br /> The effective core potential (ECP) data is entered after the coordinates. It starts with "$ECP", which must be preceded with a space. The atoms of the molecule are listed in the same order as in the coordinates section and the parameters for the ECP are listed after each atom. Note that for any atom that does NOT have an ECP, one must enter "ECP-NONE" or "NONE" after each atom without an ECP.<br /> <br /> $ECP<br /> MO-ECP GEN 28 3<br /> 5 ----- f potential -----<br /> -0.0469492 0 537.9667807 <br /> -20.2080084 1 147.8982938 <br /> -106.2116302 2 45.7358898 <br /> -41.8107368 2 13.2911467 <br /> -4.2054103 2 4.7059961 <br /> 3 ----- s-f potential -----<br /> 2.8063717 0 110.2991760 <br /> 44.5162012 1 23.2014645 <br /> 82.7785227 2 5.3530131 <br /> 4 ----- p-f potential -----<br /> 4.9420876 0 63.2901397 <br /> 25.8604976 1 23.3315302 <br /> 132.4708742 2 24.6759423 <br /> 57.3149794 2 4.6493040 <br /> 5 ----- d-f potential -----<br /> 3.0054591 0 104.4839977 <br /> 26.3637851 1 66.2307245 <br /> 183.3849199 2 39.1283176 <br /> 98.4453068 2 13.1164437 <br /> 22.4901377 2 3.6280263 <br /> S NONE<br /> S NONE<br /> S NONE<br /> S NONE<br /> S NONE<br /> S NONE<br /> C NONE<br /> C NONE<br /> C NONE<br /> C NONE<br /> C NONE<br /> C NONE<br /> C NONE<br /> C NONE<br /> C NONE<br /> C NONE<br /> H NONE<br /> C NONE<br /> H NONE<br /> C NONE<br /> H NONE<br /> C NONE<br /> H NONE<br /> C NONE<br /> C NONE<br /> C NONE<br /> C NONE<br /> H NONE<br /> H NONE<br /> H NONE<br /> H NONE<br /> C NONE<br /> C NONE<br /> H NONE<br /> C NONE<br /> H NONE<br /> H NONE<br /> H NONE<br /> H NONE<br /> H NONE<br /> H NONE<br /> H NONE<br /> $END<br /> <br /> -- [[User:M.Zimmer-De Iuliis|mzd]] 16 November 2009<br /> <br /> ====Using Effective Core Potentials in GAMESS(US)====<br /> <br /> For many metal containing compounds, it is very convenient and time saving to use an effective core potential (ECP) for the core metal electrons, as they are usually not important to the reactivity of the complex or the geometry around the metal. Since GAMESS(US) has a limited number of built-in ECPs, one may want to make GAMESS(US) read an external file that contains the ECP data using the "EXTFIL" keyword in the $GBASIS command line of the input file. In addition, to make GAMESS(US) use this external file, one must copy the "rungms" file and modify it accordingly. The following is a list of instructions with commands that will work from a terminal. One could also use WinSCP to do all of this with a GUI rather than a TUI. <br /> <br /> =====Modifiying rungms to Use Custom Basis Set File=====<br /> 1. Copy "rungms" from /scinet/gpc/Applications/gamess to one's own /scratch/$USER/ directory:<br /> cp /scinet/gpc/Applications/gamess/rungms /scratch/$USER/<br /> <br /> 2. Change to the scratch directory and check to see if "rungms" has copied successfully.<br /> cd /scratch/$USER<br /> ls<br /> <br /> 3. Edit line 147 of the vi. <br /> vi rungms<br /> Move the cursor down to line 147 using the arrow keys. It should say "setenv EXTBAS /dev/null". Using the arrow keys, move the cursor to the first "/" and then hit "i" to insert text. Put the path to your external basis file here. For example, /scratch/$USER/basisset. Then hit "escape". To save the changes and exit vi, type ":" and you should see a colon appear at the bottom of the window. Type "wq" (which should appear at the bottom of the window next to the colon) and then hit enter. Now you are done with vi.<br /> <br /> =====Creating a Custom Basis Set File=====<br /> 1. To create a custom basis set file, you need create a new text document. Our group's common practice is to comment out the first line of this file by inserting an exclamation mark (!) followed by noting the specific basis sets and ECPs that are going to be used for each of the atoms. Let us the molecule Mo(CO)6, Molybdenum hexacarbonyl, as an example. Below is the first line of the the external file, which we will call "CUSTOMMO" (NOTE: you can use any name for the external file that suits you, as long as it has no spaces and is 8 characters or less).<br /> <br /> ! 6-31G on C and O and LANL2D2 ECP on Mo<br /> <br /> 2. The next step is to visit the [https://bse.pnl.gov/bse/portal EMSL Basis Set exchange] and select C and O from the periodic table. Then, on the left of the page, select "6-31G" as the basis set. Finally, make sure the output is in GAMESS(US) format using the drop-down menu and then click "get basis set".<br /> <br /> [[File:C_O_6_31G_basisset.JPG|centre]]<br /> <br /> 3. A new window should appear with text in it. For our example case, the text looks like this:<br /> <br /> ! 6-31G EMSL Basis Set Exchange Library 10/13/09 11:12 AM<br /> ! Elements References<br /> ! -------- ----------<br /> ! H - He: W.J. Hehre, R. Ditchfield and J.A. Pople, J. Chem. Phys. 56,<br /> ! Li - Ne: 2257 (1972). Note: Li and B come from J.D. Dill and J.A.<br /> ! Pople, J. Chem. Phys. 62, 2921 (1975).<br /> ! Na - Ar: M.M. Francl, W.J. Petro, W.J. Hehre, J.S. Binkley, M.S. Gordon,<br /> ! D.J. DeFrees and J.A. Pople, J. Chem. Phys. 77, 3654 (1982)<br /> ! K - Zn: V. Rassolov, J.A. Pople, M. Ratner and T.L. Windus, J. Chem. Phys.<br /> ! 109, 1223 (1998)<br /> ! Note: He and Ne are unpublished basis sets taken from the Gaussian<br /> ! program<br /> ! <br /> $DATA<br /><br /> CARBON<br /> S 6<br /> 1 3047.5249000 0.0018347 <br /> 2 457.3695100 0.0140373 <br /> 3 103.9486900 0.0688426 <br /> 4 29.2101550 0.2321844 <br /> 5 9.2866630 0.4679413 <br /> 6 3.1639270 0.3623120 <br /> L 3<br /> 1 7.8682724 -0.1193324 0.0689991 <br /> 2 1.8812885 -0.1608542 0.3164240 <br /> 3 0.5442493 1.1434564 0.7443083 <br /> L 1<br /> 1 0.1687144 1.0000000 1.0000000<br /> <br /> OXYGEN<br /> S 6<br /> 1 5484.6717000 0.0018311 <br /> 2 825.2349500 0.0139501 <br /> 3 188.0469600 0.0684451 <br /> 4 52.9645000 0.2327143 <br /> 5 16.8975700 0.4701930 <br /> 6 5.7996353 0.3585209 <br /> L 3<br /> 1 15.5396160 -0.1107775 0.0708743 <br /> 2 3.5999336 -0.1480263 0.3397528 <br /> 3 1.0137618 1.1307670 0.7271586 <br /> L 1<br /> 1 0.2700058 1.0000000 1.0000000 <br /> $END<br /> <br /> 3. Now, copy and paste the text between the $DATA and $END headings onto our external text file, CUSTOMMO. We also need to change the change the name of each element to it symbol in the periodic table. Finally, we need to add the name of the external file next to the element symbol, separated by one space. Note that there should be a blank line separating the basis set information and the first, commented-out line (The line starting with the '!'). The CUSTOMMO should look like this:<br /> <br /> ! 6-31G on C and O and LANL2D2 ECP on Mo<br /><br /> C CUSTOMMO<br /> S 6<br /> 1 3047.5249000 0.0018347 <br /> 2 457.3695100 0.0140373 <br /> 3 103.9486900 0.0688426 <br /> 4 29.2101550 0.2321844 <br /> 5 9.2866630 0.4679413 <br /> 6 3.1639270 0.3623120 <br /> L 3<br /> 1 7.8682724 -0.1193324 0.0689991 <br /> 2 1.8812885 -0.1608542 0.3164240 <br /> 3 0.5442493 1.1434564 0.7443083 <br /> L 1<br /> 1 0.1687144 1.0000000 1.0000000<br /> <br /> O CUSTOMMO<br /> S 6<br /> 1 5484.6717000 0.0018311 <br /> 2 825.2349500 0.0139501 <br /> 3 188.0469600 0.0684451 <br /> 4 52.9645000 0.2327143 <br /> 5 16.8975700 0.4701930 <br /> 6 5.7996353 0.3585209 <br /> L 3<br /> 1 15.5396160 -0.1107775 0.0708743 <br /> 2 3.5999336 -0.1480263 0.3397528 <br /> 3 1.0137618 1.1307670 0.7271586 <br /> L 1<br /> 1 0.2700058 1.0000000 1.0000000 <br /> <br /> 4. Repeat Step 3 above but choose Mo and select the LANL2DZ ECP instead. A new window will pop up with the basis set information as well as the ECP data we need, since we specified the LANL2DZ '''ECP'''. The ECP data is not inserted into the external file, rather it is placed into the input file itself (More on this later). <br /> <br /> [[File:Mo_LANL2DZ_basisset.JPG|centre]]<br /> <br /> 5. After copying the molybdenum basis set information, your fiished external basis set file should look like this:<br /> ! 6-31G on C and O and LANL2D2 ECP on Mo<br /><br /> C CUSTOMMO<br /> S 6<br /> 1 3047.5249000 0.0018347 <br /> 2 457.3695100 0.0140373 <br /> 3 103.9486900 0.0688426 <br /> 4 29.2101550 0.2321844 <br /> 5 9.2866630 0.4679413 <br /> 6 3.1639270 0.3623120 <br /> L 3<br /> 1 7.8682724 -0.1193324 0.0689991 <br /> 2 1.8812885 -0.1608542 0.3164240 <br /> 3 0.5442493 1.1434564 0.7443083 <br /> L 1<br /> 1 0.1687144 1.0000000 1.0000000<br /> <br /> O CUSTOMMO<br /> S 6<br /> 1 5484.6717000 0.0018311 <br /> 2 825.2349500 0.0139501 <br /> 3 188.0469600 0.0684451 <br /> 4 52.9645000 0.2327143 <br /> 5 16.8975700 0.4701930 <br /> 6 5.7996353 0.3585209 <br /> L 3<br /> 1 15.5396160 -0.1107775 0.0708743 <br /> 2 3.5999336 -0.1480263 0.3397528 <br /> 3 1.0137618 1.1307670 0.7271586 <br /> L 1<br /> 1 0.2700058 1.0000000 1.0000000<br /> <br /> Mo CUSTOMO<br /> S 3<br /> 1 2.3610000 -0.9121760 <br /> 2 1.3090000 1.1477453 <br /> 3 0.4500000 0.6097109 <br /> S 4<br /> 1 2.3610000 0.8139259 <br /> 2 1.3090000 -1.1360084 <br /> 3 0.4500000 -1.1611592 <br /> 4 0.1681000 1.0064786 <br /> S 1<br /> 1 0.0423000 1.0000000 <br /> P 3<br /> 1 4.8950000 -0.0908258 <br /> 2 1.0440000 0.7042899 <br /> 3 0.3877000 0.3973179 <br /> P 2<br /> 1 0.4995000 -0.1081945 <br /> 2 0.0780000 1.0368093 <br /> P 1<br /> 1 0.0247000 1.0000000 <br /> D 3<br /> 1 2.9930000 0.0527063 <br /> 2 1.0630000 0.5003907 <br /> 3 0.3721000 0.5794024 <br /> D 1<br /> 1 0.1178000 1.0000000<br /> <br /> -- [[User:M.Zimmer-De Iuliis|mzd]] 13 October 2009<br /> <br /> ==Climate Modelling==<br /> <br /> ==Medicine/Bio==<br /> <br /> ==High Energy Physics==<br /> <br /> ==Structural Biology==<br /> Molecular simulation of proteins, lipids, carbohydrates, and other biologically relevant molecules.<br /> ===Molecular Dynamics (MD) simulation===<br /> ====GROMACS====<br /> Please refer to the [[gromacs|GROMACS]] page<br /> ====AMBER====<br /> Please refer to the [[amber|AMBER]] page<br /> ====NAMD====<br /> NAMD is one of the better scaling MD packages out there. With sufficiently large systems, it is able to scale to hundreds or thousands of cores on Scinet. Below are details for compiling and running NAMD on Scinet.<br /> <br /> More information regarding performance and different compile options coming soon...<br /> <br /> =====Compiling NAMD for GPC=====<br /> Ensure the proper compiler/mpi modules are loaded.<br /> <source lang="sh"><br /> module load intel<br /> module load openmpi/1.3.3-intel-v11.0-ofed<br /> </source><br /> <br /> '''Compile Charm++ and NAMD'''<br /> <source lang="sh"><br /> #Unpack source files and get required support libraries<br /> tar -xzf NAMD_2.7b1_Source.tar.gz<br /> cd NAMD_2.7b1_Source<br /> tar -xf charm-6.1.tar<br /> wget http://www.ks.uiuc.edu/Research/namd/libraries/fftw-linux-x86_64.tar.gz<br /> wget http://www.ks.uiuc.edu/Research/namd/libraries/tcl-linux-x86_64.tar.gz<br /> tar -xzf fftw-linux-x86_64.tar.gz; mv linux-x86_64 fftw<br /> tar -xzf tcl-linux-x86_64.tar.gz; mv linux-x86_64 tcl<br /> #Compile Charm++<br /> cd charm-6.1<br /> ./build charm++ mpi-linux-x86_64 icc --basedir /scinet/gpc/mpi/openmpi/1.3.3-intel-v11.0-ofed/ --no-shared -O -DCMK_OPTIMIZE=1<br /> cd ..<br /> #Compile NAMD. <br /> #Edit arch/Linux-x86_64-icc.arch and add "-lmpi" to the end of the CXXOPTS and COPTS line.<br /> #Make a builds directory if you want different versions of NAMD compiled at the same time.<br /> mkdir builds<br /> ./config builds/Linux-x86_64-icc --charm-arch mpi-linux-x86_64-icc<br /> cd builds/Linux-x86_64-icc/<br /> make -j4 namd2 # Adjust value of j as desired to specify number of simultaneous make targets. <br /> </source><br /> --[[User:Cmadill|Cmadill]] 16:18, 27 August 2009 (UTC)<br /> <br /> =====Running Fortran=====<br /> On the development nodes, there is an old gcc. The associated libraries are not on the compute nodes. Ensure the line:<br /> <br /> module load gcc<br /> <br /> is in your .bashrc file.<br /> <br /> ====LAMMPS====<br /> [[Image:StrongScalingLAMMPS.png|thumb|320px|right|Strong scaling test on GPC with OpenMPI and IntelMPI on Ethernet and InfiniBand]]<br /> [[Image:WeakScalingLAMMPS.png|thumb|320px|right|Weak scaling test on GPC with OpenMPI and IntelMPI on Ethernet and InfiniBand]]<br /> Results from strong scaling tests for LAMMPS using EAM potentials on GPC are shown in the graph on the right. The test simulation ran 500 timesteps for 4,000,000 atoms.<br /> <br /> Results from weak scaling tests for LAMMPS using EAM potentials on GPC are shown in the graph on the right. The test simulation ran 500 timesteps for 32,000 atoms per processor.<br /> <br /> OpenMPI version used: openmpi/1.4.1-intel-v11.0-ofed<br /> <br /> IntelMPI version used: intelmpi/impi-4.0.0.013<br /> <br /> Results show good scaling for both OpenMPI and IntelMPI up to 16 processors, after which performance begins to suffer. On Infiniband, excellent scaling is maintained to 512 processors.<br /> <br /> IntelMPI shows slightly better performance compared to OpenMPI.<br /> <br /> --[[User:jchu|jchu]] Feb 2, 2010<br /> <br /> ===Monte Carlo (MC) simulation===</div>

Jchu https://oldwiki.scinet.utoronto.ca/index.php?title=User_Codes&diff=877 User Codes 2010-02-02T18:59:12Z

<p>Jchu: /* LAMMPS */</p> <hr /> <div>__FORCETOC__<br /> <br /> ==Astrophysics==<br /> <br /> ===Athena (explicit, uniform grid MHD code)===<br /> <br /> [[Image:StrongScalingAthenaGPC.png|thumb|right|320px|Athena scaling on GPC with OpenMPI and MVAPICH2 on GigE, and OpenMPI on InfiniBand]]<br /> <br /> [http://www.astro.princeton.edu/~jstone/athena.html Athena] is a straightforward C code which doesn't use a lot of libraries so it is pretty straightforward to build and compile on new machines. <br /> <br /> It encapsulates its compiler flags, etc in an <tt>Makeoptions.in</tt> file which is then processed by <tt>configure</tt>. I've used the following additions to <tt>Makeoptions.in</tt> on TCS and GPC:<br /> <br /> <source lang="sh"><br /> ifeq ($(MACHINE),scinettcs)<br /> CC = mpcc_r<br /> LDR = mpcc_r<br /> OPT = -O5 -q64 -qarch=pwr6 -qtune=pwr6 -qcache=auto -qlargepage -qstrict<br /> MPIINC =<br /> MPILIB =<br /> CFLAGS = $(OPT)<br /> LIB = -ldl -lm<br /> else<br /> ifeq ($(MACHINE),scinetgpc)<br /> CC = mpicc<br /> LDR = mpicc<br /> OPT = -O3<br /> MPIINC =<br /> MPILIB =<br /> CFLAGS = $(OPT)<br /> LIB = -lm<br /> else<br /> ...<br /> endif<br /> endif<br /> </source><br /> It performs quite well on the GPC, scaling extremely well even on a strong scaling test out to about 256 cores (32 nodes) on Gigabit ethernet, and performing beautifully on InfiniBand out to 512 cores (64 nodes). <br /> <br /> <br /> -- [[User:Ljdursi|ljdursi]] 19:20, 13 August 2009 (UTC)<br /> <br /> ===FLASH3 (Adaptive Mesh reactive hydrodynamics; explict hydro/MHD)===<br /> <br /> [[Image:weak-scaling-example.png|thumb|right|320px|Weak scaling test of the 2d sod problem on both the GPC and TCS. The results are actually somewhat faster on the GPC; in both cases (weak) scaling is very good out at least to 256 cores]]<br /> <br /> [http://flash.uchicago.edu FLASH] encapsulates its machine-dependant information in the <tt>FLASH3/sites</tt> directory. For the GPC, you'll have to<br /> <pre><br /> module load intel<br /> module load openmpi<br /> module load hdf5/183-v16-openmpi<br /> </pre><br /> <br /> and with that, the following file (<tt>sites/scinetgpc/Makefile.h</tt>) works for me:<br /> <source lang="sh"><br /> ## Must do module load hdf5/183-v16-openmpi<br /> HDF5_PATH = ${SCINET_HDF5_BASE}<br /> ZLIB_PATH = /usr/local<br /> <br /> #----------------------------------------------------------------------------<br /> # Compiler and linker commands<br /> #<br /> # We use the f90 compiler as the linker, so some C libraries may explicitly<br /> # need to be added into the link line.<br /> #----------------------------------------------------------------------------<br /> <br /> ## modules will put the right mpi in our path<br /> FCOMP = mpif77<br /> CCOMP = mpicc<br /> CPPCOMP = mpiCC<br /> LINK = mpif77<br /> <br /> #----------------------------------------------------------------------------<br /> # Compilation flags<br /> #<br /> # Three sets of compilation/linking flags are defined: one for optimized<br /> # code, one for testing, and one for debugging. The default is to use the <br /> # _OPT version. Specifying -debug to setup will pick the _DEBUG version,<br /> # these should enable bounds checking. Specifying -test is used for <br /> # flash_test, and is set for quick code generation, and (sometimes) <br /> # profiling. The Makefile generated by setup will assign the generic token <br /> # (ex. FFLAGS) to the proper set of flags (ex. FFLAGS_OPT).<br /> #----------------------------------------------------------------------------<br /> <br /> FFLAGS_OPT = -c -r8 -i4 -O3 -xSSE4.2<br /> FFLAGS_DEBUG = -c -g -r8 -i4 -O0<br /> FFLAGS_TEST = -c -r8 -i4<br /> <br /> <br /> # if we are using HDF5, we need to specify the path to the include files<br /> CFLAGS_HDF5 = -I${HDF5_PATH}/include<br /> <br /> CFLAGS_OPT = -c -O3 -xSSE4.2<br /> CFLAGS_TEST = -c -O2 <br /> CFLAGS_DEBUG = -c -g <br /> <br /> MDEFS = <br /> <br /> .SUFFIXES: .o .c .f .F .h .fh .F90 .f90<br /> <br /> #----------------------------------------------------------------------------<br /> # Linker flags<br /> #<br /> # There is a seperate version of the linker flags for each of the _OPT, <br /> # _DEBUG, and _TEST cases.<br /> #----------------------------------------------------------------------------<br /> <br /> LFLAGS_OPT = -o<br /> LFLAGS_TEST = -o<br /> LFLAGS_DEBUG = -g -o<br /> <br /> MACHOBJ = <br /> <br /> <br /> MV = mv -f<br /> AR = ar -r<br /> RM = rm -f<br /> CD = cd<br /> RL = ranlib<br /> ECHO = echo<br /> </source><br /> <br /> <br /> -- [[User:Ljdursi|ljdursi]] 22:11, 13 August 2009 (UTC)<br /> <br /> ==Aeronautics==<br /> <br /> ==Chemistry==<br /> <br /> ===GAMESS (US)===<br /> The GAMESS version January 12, 2009 R3 was built using the Intel v11.1 compilers and v3.2.2 MPI library, according to the instructions in http://software.intel.com/en-us/articles/building-gamess-with-intel-compilers-intel-mkl-and-intel-mpi-on-linux/<br /> <br /> The required build scripts - comp, compall, lked - and run script - rungms - were modified to account for our own installation. In order to build GAMESS one first must ensure that the intel and intelmpi modules are loaded ("module load intel intelmpi"). This applies to running GAMESS as well.<br /> The module "gamess" must also be loaded in order to run GAMESS ("module load gamess").<br /> <br /> The modified scripts are in the file /scinet/gpc/src/gamess-on-scinet.tar.gz<br /> <br /> ====Running GAMESS====<br /> - Make sure the directory /scratch/$USER/gamess-scratch exists (the $SCINET_RUNGMS script will create it if it does not exist)<br /> <br /> - Make sure the modules: intel, intelmpi, gamess are loaded (in your .bashrc: "module load intel intelmpi gamess").<br /> <br /> - Create a torque script to run GAMESS. Here is an example:<br /> <br /> - The GAMESS executable is in $SCINET_GAMESS_HOME/gamess.00.x<br /> - The rungms script is in $SCINET_GAMESS_HOME/rungms (actually it is $SCINET_RUNGMS)<br /> <br /> - For IB multinode runs, use the $SCINET_RUNGMS_IB script<br /> <br /> - The rungms script takes 4 arguments: input file, executable number, number of compute processes, processors per node<br /> <br /> For example, in order to run with the input file /scratch/$USER/gamesstest01, on 8 cpus, and the default version (00)<br /> of the executable on a machine with 8 cores:<br /> <br /> # load the gamess module in .bashrc<br /> module load gamess <br /> <br /> # run the program<br /> $SCINET_RUNGMS /scratch/$USER/gamesstest01 00 8 8<br /> <br /> Here is a sample torque script for running a GAMESS calculation, on a single 8-core node:<br /> <br /> <source lang="bash"><br /> #!/bin/bash<br /> #PBS -l nodes=1:ppn=8,walltime=48:00:00,os=centos53computeA<br /> #PBS -N gamessjob<br /> <br /> ## To submit type: qsub gms.sh<br /> <br /> # If not an interactive job (i.e. -I), then cd into the directory where<br /> # I typed qsub.<br /> if [ "$PBS_ENVIRONMENT" != "PBS_INTERACTIVE" ]; then<br /> if [ -n "$PBS_O_WORKDIR" ]; then<br /> cd $PBS_O_WORKDIR<br /> fi<br /> fi<br /> <br /> # the input file is typically named something like "gamesjob.inp"<br /> # so the script will be run like "$SCINET_RUNGMS gamessjob 00 8 8"<br /> <br /> # load the gamess module if not in .bashrc already<br /> # actually, it MUST be in .bashrc<br /> # module load gamess<br /> <br /> # run the program<br /> <br /> $SCINET_RUNGMS gamessjob 00 8 8<br /> </source><br /> <br /> Here is a similar script, but this one uses 2 InfiniBand-connected nodes,<br /> and runs the appropriate $SCINET_RUNGMS_IB script to actually run the job:<br /> <br /> <source lang="bash"><br /> #!/bin/bash<br /> #PBS -l nodes=2:ib:ppn=8,walltime=48:00:00,os=centos53computeA<br /> #PBS -N gamessjob<br /> <br /> ## To submit type: qsub gmsib.sh<br /> <br /> # If not an interactive job (i.e. -I), then cd into the directory where<br /> # I typed qsub.<br /> if [ "$PBS_ENVIRONMENT" != "PBS_INTERACTIVE" ]; then<br /> if [ -n "$PBS_O_WORKDIR" ]; then<br /> cd $PBS_O_WORKDIR<br /> fi<br /> fi<br /> <br /> # the input file is typically named something like "gamesjob.inp"<br /> # so the script will be run like "$SCINET_RUNGMS gamessjob 00 8 8"<br /> <br /> # load the gamess module if not in .bashrc already<br /> # actually, it MUST be in .bashrc<br /> # module load gamess<br /> <br /> # This script requests InfiniBand-connected nodes (:ib above)<br /> # so it must run with the IB version of the rungms script,<br /> # $SCINET_RUNGMS_IB<br /> <br /> $SCINET_RUNGMS_IB gamessjob 00 16 8<br /> </source><br /> <br /> <br /> -- [[User:Dgruner|dgruner]] 5 October 2009<br /> <br /> ===Tips from the Fekl Lab===<br /> <br /> Through trial and error, we have found a few useful things that we would like to share:<br /> <br /> 1. Two very useful, open-source programs for visualization of output files from GAMESS(US) and for generation of input files are [http://www.scl.ameslab.gov/MacMolPlt/ MacMolPlt]and [http://avogadro.openmolecules.net/wiki/Main_Page Avogadro]. The are available for UNIX/LINUX, Windows and Mac based machines, HOWEVER: any input files that we have generated with these programs on a Windows-based machine do not run on Mac based machines. We don't know why.<br /> <br /> 2. [http://winscp.net/eng/index.php WinSCP] is a very useful tool that has a graphical user interface for moving files from a local machine to SCINET and vice versa. It also has text editing capabilities.<br /> <br /> 3. The [https://bse.pnl.gov/bse/portal ESML Basis Set Exchange] is an excellent source for custom basis set or effective core potential parameters. Make sure that you specify "Gamess-US" in the format drop-down box.<br /> <br /> 4. The commercial program [http://www.chemcraftprog.com/ ChemCraft] is a highly useful visualization program that has the ability to edit molecules in a very similar fashion to GaussView. It can also be customized to build GAMESS(US) input files.<br /> <br /> ====Anatomy of a GAMESS(US) Input File with Basis Set Info in an External File====<br /> <br /> $CONTRL SCFTYP=RHF RUNTYP=OPTIMIZE DFTTYP=M06-L MAXIT=199 MULT=1 NOSYM=1<br /> ECP=READ $END<br /> $SYSTEM TIMLIM=525600 MWORDS=1750 PARALL=.TRUE. $END<br /> $BASIS GBASIS=CUSTOMNI EXTFIL=.t. $END<br /> $SCF DIRSCF=.TRUE. FDIFF=.f. $END<br /> $STATPT OPTTOL=0.0001 NSTEP=500 HSSEND=.t. $END<br /> $DATA<br /> Mo_BDT3<br /> C1<br /> MOLYBDENUM 42.0 5.7556500000 4.4039600000 16.5808400000<br /> SULFUR 16.0 7.4169700000 3.1956300000 15.2089300000<br /> SULFUR 16.0 4.0966800000 3.2258300000 15.1761100000<br /> SULFUR 16.0 3.9677300000 4.4940500000 18.3266100000<br /> SULFUR 16.0 7.1776900000 3.5815000000 18.4485200000<br /> SULFUR 16.0 4.3776600000 6.2447400000 15.6786900000<br /> SULFUR 16.0 7.5478700000 6.0679800000 16.2223700000<br /> CARBON 6.0 6.4716900000 2.1004800000 14.1902300000<br /> CARBON 6.0 5.0690300000 2.1781400000 14.1080700000<br /> CARBON 6.0 4.8421800000 4.2701300000 19.8855500000<br /> CARBON 6.0 6.1969000000 3.9249600000 19.9397400000<br /> CARBON 6.0 6.8280600000 3.7834200000 21.1913200000<br /> CARBON 6.0 5.7697600000 7.6933500000 17.4241800000<br /> CARBON 6.0 7.2043100000 7.9413600000 17.8281100000<br /> CARBON 6.0 5.5051400000 7.0409700000 14.5903800000<br /> CARBON 6.0 6.8905200000 6.9194700000 14.7626200000<br /> CARBON 6.0 7.7396400000 7.5379800000 13.8285700000<br /> HYDROGEN 1.0 8.8190700000 7.4520600000 13.9252200000<br /> CARBON 6.0 7.2169400000 8.2960300000 12.7704100000<br /> HYDROGEN 1.0 7.8667000000 8.7825100000 12.0575600000<br /> CARBON 6.0 5.8260300000 8.4502300000 12.6467800000<br /> HYDROGEN 1.0 5.4143000000 9.0544300000 11.8493100000<br /> CARBON 6.0 4.9881500000 7.8192300000 13.5528400000<br /> HYDROGEN 1.0 3.9090500000 7.9420000000 13.4583700000<br /> CARBON 6.0 7.1538500000 1.1569600000 13.4143900000<br /> CARBON 6.0 4.4018100000 1.3603900000 13.1919900000<br /> CARBON 6.0 6.4791600000 0.3185500000 12.5353300000<br /> CARBON 6.0 5.0837400000 0.4369500000 12.4084900000<br /> HYDROGEN 1.0 7.0116000000 -0.4099400000 11.9434600000<br /> HYDROGEN 1.0 8.2399000000 1.0702400000 13.4937600000<br /> HYDROGEN 1.0 3.3185600000 1.4368700000 13.0953100000<br /> HYDROGEN 1.0 4.5549800000 -0.1997300000 11.7165200000<br /> CARBON 6.0 6.1105700000 3.9639000000 22.3866100000<br /> CARBON 6.0 4.1216300000 4.4424400000 21.1020100000<br /> HYDROGEN 1.0 7.8732900000 3.5217100000 21.2520500000<br /> CARBON 6.0 4.7606000000 4.2868500000 22.3363800000<br /> HYDROGEN 1.0 6.6064200000 3.8406000000 23.3428500000<br /> HYDROGEN 1.0 4.2065000000 4.4170700000 23.2667100000<br /> HYDROGEN 1.0 3.0674000000 4.6893500000 21.0889000000<br /> HYDROGEN 1.0 7.4249200000 7.7545300000 18.8583200000<br /> HYDROGEN 1.0 7.6651700000 8.9049700000 17.7652100000<br /> HYDROGEN 1.0 5.3324000000 8.6487800000 17.2222700000<br /> HYDROGEN 1.0 5.5015000000 7.1039000000 18.2759400000<br /> $END<br /> $ECP<br /> MO-ECP GEN 28 3<br /> 5 ----- f potential -----<br /> -0.0469492 0 537.9667807 <br /> -20.2080084 1 147.8982938 <br /> -106.2116302 2 45.7358898 <br /> -41.8107368 2 13.2911467 <br /> -4.2054103 2 4.7059961 <br /> 3 ----- s-f potential -----<br /> 2.8063717 0 110.2991760 <br /> 44.5162012 1 23.2014645 <br /> 82.7785227 2 5.3530131 <br /> 4 ----- p-f potential -----<br /> 4.9420876 0 63.2901397 <br /> 25.8604976 1 23.3315302 <br /> 132.4708742 2 24.6759423 <br /> 57.3149794 2 4.6493040 <br /> 5 ----- d-f potential -----<br /> 3.0054591 0 104.4839977 <br /> 26.3637851 1 66.2307245 <br /> 183.3849199 2 39.1283176 <br /> 98.4453068 2 13.1164437 <br /> 22.4901377 2 3.6280263 <br /> S NONE<br /> S NONE<br /> S NONE<br /> S NONE<br /> S NONE<br /> S NONE<br /> C NONE<br /> C NONE<br /> C NONE<br /> C NONE<br /> C NONE<br /> C NONE<br /> C NONE<br /> C NONE<br /> C NONE<br /> C NONE<br /> H NONE<br /> C NONE<br /> H NONE<br /> C NONE<br /> H NONE<br /> C NONE<br /> H NONE<br /> C NONE<br /> C NONE<br /> C NONE<br /> C NONE<br /> H NONE<br /> H NONE<br /> H NONE<br /> H NONE<br /> C NONE<br /> C NONE<br /> H NONE<br /> C NONE<br /> H NONE<br /> H NONE<br /> H NONE<br /> H NONE<br /> H NONE<br /> H NONE<br /> H NONE<br /> $END<br /> <br /> =====The Input Deck=====<br /> <br /> Below is the input deck. It is where you tell GAMESS(US) what job type to execute and where all you individual parameters are entered for your specific job type. The example input deck below is for a geometry optimization and frequency calculation. This input deck is equivalent to the Gaussian job with "opt" and "freq" in the route section.<br /> <br /> $CONTRL SCFTYP=RHF RUNTYP=OPTIMIZE DFTTYP=M06-L MAXIT=199 MULT=1 NOSYM=1<br /> ECP=READ $END<br /> $SYSTEM TIMLIM=2850 MWORDS=1750 MEMDDI=20 PARALL=.TRUE. $END<br /> $BASIS GBASIS=CUSTOMNI EXTFIL=.t. $END<br /> $SCF DIRSCF=.TRUE. FDIFF=.f. $END<br /> $STATPT OPTTOL=0.0001 NSTEP=500 HSSEND=.t. $END<br /> $DATA<br /> <br /> An important thing to note is the spacing. In the input deck, there must be 1 space at the beginning of each line of the input deck. If not, the job will fail. Most builders will insert this space anyway, but it helps to double check.<br /> <br /> The end of the input deck is marked by the "$DATA" line.<br /> <br /> =====Job Title Line=====<br /> <br /> The next line of the file is the job title. It can be anthing you wish, however, we have found that to be on the safe side, we avoide using symbols or spaces<br /> <br /> Mo_BDT3<br /> <br /> =====Symmetry Point Group=====<br /> <br /> The next line of the file is the symmetry point group of your molecule. Note that there is no leading space before the point group.<br /> <br /> C1<br /> <br /> =====Coordinates=====<br /> <br /> The next block of text is set aside for the coordinates of the molecule. This can be in internal (or z-matrix) format or cartesian coordinates. Note that there is no leading space before the coordinates. One may use the chemical symbol or the full name of each atom in the molecule. Note that the end of the coordinates is signified by an "$END", which MUST have one space preceding it. The coordinates below do NOT have any basis set information inserted. It is possible to insert basis set information directly into the input file. This is accomplished by obtaining the desired basis set parameters from the EMSL and then inserting them below each relevant atom. An example input file with inserted basis set information will be shown later.<br /> <br /> MOLYBDENUM 42.0 5.7556500000 4.4039600000 16.5808400000<br /> SULFUR 16.0 7.4169700000 3.1956300000 15.2089300000<br /> SULFUR 16.0 4.0966800000 3.2258300000 15.1761100000<br /> SULFUR 16.0 3.9677300000 4.4940500000 18.3266100000<br /> SULFUR 16.0 7.1776900000 3.5815000000 18.4485200000<br /> SULFUR 16.0 4.3776600000 6.2447400000 15.6786900000<br /> SULFUR 16.0 7.5478700000 6.0679800000 16.2223700000<br /> CARBON 6.0 6.4716900000 2.1004800000 14.1902300000<br /> CARBON 6.0 5.0690300000 2.1781400000 14.1080700000<br /> CARBON 6.0 4.8421800000 4.2701300000 19.8855500000<br /> CARBON 6.0 6.1969000000 3.9249600000 19.9397400000<br /> CARBON 6.0 6.8280600000 3.7834200000 21.1913200000<br /> CARBON 6.0 5.7697600000 7.6933500000 17.4241800000<br /> CARBON 6.0 7.2043100000 7.9413600000 17.8281100000<br /> CARBON 6.0 5.5051400000 7.0409700000 14.5903800000<br /> CARBON 6.0 6.8905200000 6.9194700000 14.7626200000<br /> CARBON 6.0 7.7396400000 7.5379800000 13.8285700000<br /> HYDROGEN 1.0 8.8190700000 7.4520600000 13.9252200000<br /> CARBON 6.0 7.2169400000 8.2960300000 12.7704100000<br /> HYDROGEN 1.0 7.8667000000 8.7825100000 12.0575600000<br /> CARBON 6.0 5.8260300000 8.4502300000 12.6467800000<br /> HYDROGEN 1.0 5.4143000000 9.0544300000 11.8493100000<br /> CARBON 6.0 4.9881500000 7.8192300000 13.5528400000<br /> HYDROGEN 1.0 3.9090500000 7.9420000000 13.4583700000<br /> CARBON 6.0 7.1538500000 1.1569600000 13.4143900000<br /> CARBON 6.0 4.4018100000 1.3603900000 13.1919900000<br /> CARBON 6.0 6.4791600000 0.3185500000 12.5353300000<br /> CARBON 6.0 5.0837400000 0.4369500000 12.4084900000<br /> HYDROGEN 1.0 7.0116000000 -0.4099400000 11.9434600000<br /> HYDROGEN 1.0 8.2399000000 1.0702400000 13.4937600000<br /> HYDROGEN 1.0 3.3185600000 1.4368700000 13.0953100000<br /> HYDROGEN 1.0 4.5549800000 -0.1997300000 11.7165200000<br /> CARBON 6.0 6.1105700000 3.9639000000 22.3866100000<br /> CARBON 6.0 4.1216300000 4.4424400000 21.1020100000<br /> HYDROGEN 1.0 7.8732900000 3.5217100000 21.2520500000<br /> CARBON 6.0 4.7606000000 4.2868500000 22.3363800000<br /> HYDROGEN 1.0 6.6064200000 3.8406000000 23.3428500000<br /> HYDROGEN 1.0 4.2065000000 4.4170700000 23.2667100000<br /> HYDROGEN 1.0 3.0674000000 4.6893500000 21.0889000000<br /> HYDROGEN 1.0 7.4249200000 7.7545300000 18.8583200000<br /> HYDROGEN 1.0 7.6651700000 8.9049700000 17.7652100000<br /> HYDROGEN 1.0 5.3324000000 8.6487800000 17.2222700000<br /> HYDROGEN 1.0 5.5015000000 7.1039000000 18.2759400000<br /> $END<br /> <br /> =====Effective Core Potential Data=====<br /> <br /> The effective core potential (ECP) data is entered after the coordinates. It starts with "$ECP", which must be preceded with a space. The atoms of the molecule are listed in the same order as in the coordinates section and the parameters for the ECP are listed after each atom. Note that for any atom that does NOT have an ECP, one must enter "ECP-NONE" or "NONE" after each atom without an ECP.<br /> <br /> $ECP<br /> MO-ECP GEN 28 3<br /> 5 ----- f potential -----<br /> -0.0469492 0 537.9667807 <br /> -20.2080084 1 147.8982938 <br /> -106.2116302 2 45.7358898 <br /> -41.8107368 2 13.2911467 <br /> -4.2054103 2 4.7059961 <br /> 3 ----- s-f potential -----<br /> 2.8063717 0 110.2991760 <br /> 44.5162012 1 23.2014645 <br /> 82.7785227 2 5.3530131 <br /> 4 ----- p-f potential -----<br /> 4.9420876 0 63.2901397 <br /> 25.8604976 1 23.3315302 <br /> 132.4708742 2 24.6759423 <br /> 57.3149794 2 4.6493040 <br /> 5 ----- d-f potential -----<br /> 3.0054591 0 104.4839977 <br /> 26.3637851 1 66.2307245 <br /> 183.3849199 2 39.1283176 <br /> 98.4453068 2 13.1164437 <br /> 22.4901377 2 3.6280263 <br /> S NONE<br /> S NONE<br /> S NONE<br /> S NONE<br /> S NONE<br /> S NONE<br /> C NONE<br /> C NONE<br /> C NONE<br /> C NONE<br /> C NONE<br /> C NONE<br /> C NONE<br /> C NONE<br /> C NONE<br /> C NONE<br /> H NONE<br /> C NONE<br /> H NONE<br /> C NONE<br /> H NONE<br /> C NONE<br /> H NONE<br /> C NONE<br /> C NONE<br /> C NONE<br /> C NONE<br /> H NONE<br /> H NONE<br /> H NONE<br /> H NONE<br /> C NONE<br /> C NONE<br /> H NONE<br /> C NONE<br /> H NONE<br /> H NONE<br /> H NONE<br /> H NONE<br /> H NONE<br /> H NONE<br /> H NONE<br /> $END<br /> <br /> -- [[User:M.Zimmer-De Iuliis|mzd]] 16 November 2009<br /> <br /> ====Using Effective Core Potentials in GAMESS(US)====<br /> <br /> For many metal containing compounds, it is very convenient and time saving to use an effective core potential (ECP) for the core metal electrons, as they are usually not important to the reactivity of the complex or the geometry around the metal. Since GAMESS(US) has a limited number of built-in ECPs, one may want to make GAMESS(US) read an external file that contains the ECP data using the "EXTFIL" keyword in the $GBASIS command line of the input file. In addition, to make GAMESS(US) use this external file, one must copy the "rungms" file and modify it accordingly. The following is a list of instructions with commands that will work from a terminal. One could also use WinSCP to do all of this with a GUI rather than a TUI. <br /> <br /> =====Modifiying rungms to Use Custom Basis Set File=====<br /> 1. Copy "rungms" from /scinet/gpc/Applications/gamess to one's own /scratch/$USER/ directory:<br /> cp /scinet/gpc/Applications/gamess/rungms /scratch/$USER/<br /> <br /> 2. Change to the scratch directory and check to see if "rungms" has copied successfully.<br /> cd /scratch/$USER<br /> ls<br /> <br /> 3. Edit line 147 of the vi. <br /> vi rungms<br /> Move the cursor down to line 147 using the arrow keys. It should say "setenv EXTBAS /dev/null". Using the arrow keys, move the cursor to the first "/" and then hit "i" to insert text. Put the path to your external basis file here. For example, /scratch/$USER/basisset. Then hit "escape". To save the changes and exit vi, type ":" and you should see a colon appear at the bottom of the window. Type "wq" (which should appear at the bottom of the window next to the colon) and then hit enter. Now you are done with vi.<br /> <br /> =====Creating a Custom Basis Set File=====<br /> 1. To create a custom basis set file, you need create a new text document. Our group's common practice is to comment out the first line of this file by inserting an exclamation mark (!) followed by noting the specific basis sets and ECPs that are going to be used for each of the atoms. Let us the molecule Mo(CO)6, Molybdenum hexacarbonyl, as an example. Below is the first line of the the external file, which we will call "CUSTOMMO" (NOTE: you can use any name for the external file that suits you, as long as it has no spaces and is 8 characters or less).<br /> <br /> ! 6-31G on C and O and LANL2D2 ECP on Mo<br /> <br /> 2. The next step is to visit the [https://bse.pnl.gov/bse/portal EMSL Basis Set exchange] and select C and O from the periodic table. Then, on the left of the page, select "6-31G" as the basis set. Finally, make sure the output is in GAMESS(US) format using the drop-down menu and then click "get basis set".<br /> <br /> [[File:C_O_6_31G_basisset.JPG|centre]]<br /> <br /> 3. A new window should appear with text in it. For our example case, the text looks like this:<br /> <br /> ! 6-31G EMSL Basis Set Exchange Library 10/13/09 11:12 AM<br /> ! Elements References<br /> ! -------- ----------<br /> ! H - He: W.J. Hehre, R. Ditchfield and J.A. Pople, J. Chem. Phys. 56,<br /> ! Li - Ne: 2257 (1972). Note: Li and B come from J.D. Dill and J.A.<br /> ! Pople, J. Chem. Phys. 62, 2921 (1975).<br /> ! Na - Ar: M.M. Francl, W.J. Petro, W.J. Hehre, J.S. Binkley, M.S. Gordon,<br /> ! D.J. DeFrees and J.A. Pople, J. Chem. Phys. 77, 3654 (1982)<br /> ! K - Zn: V. Rassolov, J.A. Pople, M. Ratner and T.L. Windus, J. Chem. Phys.<br /> ! 109, 1223 (1998)<br /> ! Note: He and Ne are unpublished basis sets taken from the Gaussian<br /> ! program<br /> ! <br /> $DATA<br /><br /> CARBON<br /> S 6<br /> 1 3047.5249000 0.0018347 <br /> 2 457.3695100 0.0140373 <br /> 3 103.9486900 0.0688426 <br /> 4 29.2101550 0.2321844 <br /> 5 9.2866630 0.4679413 <br /> 6 3.1639270 0.3623120 <br /> L 3<br /> 1 7.8682724 -0.1193324 0.0689991 <br /> 2 1.8812885 -0.1608542 0.3164240 <br /> 3 0.5442493 1.1434564 0.7443083 <br /> L 1<br /> 1 0.1687144 1.0000000 1.0000000<br /> <br /> OXYGEN<br /> S 6<br /> 1 5484.6717000 0.0018311 <br /> 2 825.2349500 0.0139501 <br /> 3 188.0469600 0.0684451 <br /> 4 52.9645000 0.2327143 <br /> 5 16.8975700 0.4701930 <br /> 6 5.7996353 0.3585209 <br /> L 3<br /> 1 15.5396160 -0.1107775 0.0708743 <br /> 2 3.5999336 -0.1480263 0.3397528 <br /> 3 1.0137618 1.1307670 0.7271586 <br /> L 1<br /> 1 0.2700058 1.0000000 1.0000000 <br /> $END<br /> <br /> 3. Now, copy and paste the text between the $DATA and $END headings onto our external text file, CUSTOMMO. We also need to change the change the name of each element to it symbol in the periodic table. Finally, we need to add the name of the external file next to the element symbol, separated by one space. Note that there should be a blank line separating the basis set information and the first, commented-out line (The line starting with the '!'). The CUSTOMMO should look like this:<br /> <br /> ! 6-31G on C and O and LANL2D2 ECP on Mo<br /><br /> C CUSTOMMO<br /> S 6<br /> 1 3047.5249000 0.0018347 <br /> 2 457.3695100 0.0140373 <br /> 3 103.9486900 0.0688426 <br /> 4 29.2101550 0.2321844 <br /> 5 9.2866630 0.4679413 <br /> 6 3.1639270 0.3623120 <br /> L 3<br /> 1 7.8682724 -0.1193324 0.0689991 <br /> 2 1.8812885 -0.1608542 0.3164240 <br /> 3 0.5442493 1.1434564 0.7443083 <br /> L 1<br /> 1 0.1687144 1.0000000 1.0000000<br /> <br /> O CUSTOMMO<br /> S 6<br /> 1 5484.6717000 0.0018311 <br /> 2 825.2349500 0.0139501 <br /> 3 188.0469600 0.0684451 <br /> 4 52.9645000 0.2327143 <br /> 5 16.8975700 0.4701930 <br /> 6 5.7996353 0.3585209 <br /> L 3<br /> 1 15.5396160 -0.1107775 0.0708743 <br /> 2 3.5999336 -0.1480263 0.3397528 <br /> 3 1.0137618 1.1307670 0.7271586 <br /> L 1<br /> 1 0.2700058 1.0000000 1.0000000 <br /> <br /> 4. Repeat Step 3 above but choose Mo and select the LANL2DZ ECP instead. A new window will pop up with the basis set information as well as the ECP data we need, since we specified the LANL2DZ '''ECP'''. The ECP data is not inserted into the external file, rather it is placed into the input file itself (More on this later). <br /> <br /> [[File:Mo_LANL2DZ_basisset.JPG|centre]]<br /> <br /> 5. After copying the molybdenum basis set information, your fiished external basis set file should look like this:<br /> ! 6-31G on C and O and LANL2D2 ECP on Mo<br /><br /> C CUSTOMMO<br /> S 6<br /> 1 3047.5249000 0.0018347 <br /> 2 457.3695100 0.0140373 <br /> 3 103.9486900 0.0688426 <br /> 4 29.2101550 0.2321844 <br /> 5 9.2866630 0.4679413 <br /> 6 3.1639270 0.3623120 <br /> L 3<br /> 1 7.8682724 -0.1193324 0.0689991 <br /> 2 1.8812885 -0.1608542 0.3164240 <br /> 3 0.5442493 1.1434564 0.7443083 <br /> L 1<br /> 1 0.1687144 1.0000000 1.0000000<br /> <br /> O CUSTOMMO<br /> S 6<br /> 1 5484.6717000 0.0018311 <br /> 2 825.2349500 0.0139501 <br /> 3 188.0469600 0.0684451 <br /> 4 52.9645000 0.2327143 <br /> 5 16.8975700 0.4701930 <br /> 6 5.7996353 0.3585209 <br /> L 3<br /> 1 15.5396160 -0.1107775 0.0708743 <br /> 2 3.5999336 -0.1480263 0.3397528 <br /> 3 1.0137618 1.1307670 0.7271586 <br /> L 1<br /> 1 0.2700058 1.0000000 1.0000000<br /> <br /> Mo CUSTOMO<br /> S 3<br /> 1 2.3610000 -0.9121760 <br /> 2 1.3090000 1.1477453 <br /> 3 0.4500000 0.6097109 <br /> S 4<br /> 1 2.3610000 0.8139259 <br /> 2 1.3090000 -1.1360084 <br /> 3 0.4500000 -1.1611592 <br /> 4 0.1681000 1.0064786 <br /> S 1<br /> 1 0.0423000 1.0000000 <br /> P 3<br /> 1 4.8950000 -0.0908258 <br /> 2 1.0440000 0.7042899 <br /> 3 0.3877000 0.3973179 <br /> P 2<br /> 1 0.4995000 -0.1081945 <br /> 2 0.0780000 1.0368093 <br /> P 1<br /> 1 0.0247000 1.0000000 <br /> D 3<br /> 1 2.9930000 0.0527063 <br /> 2 1.0630000 0.5003907 <br /> 3 0.3721000 0.5794024 <br /> D 1<br /> 1 0.1178000 1.0000000<br /> <br /> -- [[User:M.Zimmer-De Iuliis|mzd]] 13 October 2009<br /> <br /> ==Climate Modelling==<br /> <br /> ==Medicine/Bio==<br /> <br /> ==High Energy Physics==<br /> <br /> ==Structural Biology==<br /> Molecular simulation of proteins, lipids, carbohydrates, and other biologically relevant molecules.<br /> ===Molecular Dynamics (MD) simulation===<br /> ====GROMACS====<br /> Please refer to the [[gromacs|GROMACS]] page<br /> ====AMBER====<br /> Please refer to the [[amber|AMBER]] page<br /> ====NAMD====<br /> NAMD is one of the better scaling MD packages out there. With sufficiently large systems, it is able to scale to hundreds or thousands of cores on Scinet. Below are details for compiling and running NAMD on Scinet.<br /> <br /> More information regarding performance and different compile options coming soon...<br /> <br /> =====Compiling NAMD for GPC=====<br /> Ensure the proper compiler/mpi modules are loaded.<br /> <source lang="sh"><br /> module load intel<br /> module load openmpi/1.3.3-intel-v11.0-ofed<br /> </source><br /> <br /> '''Compile Charm++ and NAMD'''<br /> <source lang="sh"><br /> #Unpack source files and get required support libraries<br /> tar -xzf NAMD_2.7b1_Source.tar.gz<br /> cd NAMD_2.7b1_Source<br /> tar -xf charm-6.1.tar<br /> wget http://www.ks.uiuc.edu/Research/namd/libraries/fftw-linux-x86_64.tar.gz<br /> wget http://www.ks.uiuc.edu/Research/namd/libraries/tcl-linux-x86_64.tar.gz<br /> tar -xzf fftw-linux-x86_64.tar.gz; mv linux-x86_64 fftw<br /> tar -xzf tcl-linux-x86_64.tar.gz; mv linux-x86_64 tcl<br /> #Compile Charm++<br /> cd charm-6.1<br /> ./build charm++ mpi-linux-x86_64 icc --basedir /scinet/gpc/mpi/openmpi/1.3.3-intel-v11.0-ofed/ --no-shared -O -DCMK_OPTIMIZE=1<br /> cd ..<br /> #Compile NAMD. <br /> #Edit arch/Linux-x86_64-icc.arch and add "-lmpi" to the end of the CXXOPTS and COPTS line.<br /> #Make a builds directory if you want different versions of NAMD compiled at the same time.<br /> mkdir builds<br /> ./config builds/Linux-x86_64-icc --charm-arch mpi-linux-x86_64-icc<br /> cd builds/Linux-x86_64-icc/<br /> make -j4 namd2 # Adjust value of j as desired to specify number of simultaneous make targets. <br /> </source><br /> --[[User:Cmadill|Cmadill]] 16:18, 27 August 2009 (UTC)<br /> <br /> =====Running Fortran=====<br /> On the development nodes, there is an old gcc. The associated libraries are not on the compute nodes. Ensure the line:<br /> <br /> module load gcc<br /> <br /> is in your .bashrc file.<br /> <br /> ====LAMMPS====<br /> [[Image:StrongScalingLAMMPS.png|thumb|320px|right|Strong scaling test on GPC with OpenMPI and IntelMPI on Ethernet and InfiniBand]]<br /> [[Image:WeakScalingLAMMPS.png|thumb|320px|right|Weak scaling test on GPC with OpenMPI and IntelMPI on Ethernet and InfiniBand]]<br /> Results from strong scaling tests for LAMMPS using EAM potentials on GPC are shown in the graph on the right. The test simulation ran 500 timesteps for 4,000,000 atoms.<br /> <br /> Results from weak scaling tests for LAMMPS using EAM potentials on GPC are shown in the graph on the right. The test simulation ran 500 timesteps for 32,000 atoms per processor.<br /> <br /> OpenMPI version used: openmpi/1.4.1-intel-v11.0-ofed<br /> <br /> IntelMPI version used: intelmpi/impi-4.0.0.013<br /> <br /> Results show good scaling for both OpenMPI and IntelMPI up to 16 processors, after which performance begins to suffer. On Infiniband, excellent scaling is maintained to 512 processors.<br /> <br /> IntelMPI shows slightly better performance compared to OpenMPI.<br /> <br /> ===Monte Carlo (MC) simulation===</div>

Jchu https://oldwiki.scinet.utoronto.ca/index.php?title=User_Codes&diff=876 User Codes 2010-02-02T18:57:44Z

<p>Jchu: /* LAMMPS */</p> <hr /> <div>__FORCETOC__<br /> <br /> ==Astrophysics==<br /> <br /> ===Athena (explicit, uniform grid MHD code)===<br /> <br /> [[Image:StrongScalingAthenaGPC.png|thumb|right|320px|Athena scaling on GPC with OpenMPI and MVAPICH2 on GigE, and OpenMPI on InfiniBand]]<br /> <br /> [http://www.astro.princeton.edu/~jstone/athena.html Athena] is a straightforward C code which doesn't use a lot of libraries so it is pretty straightforward to build and compile on new machines. <br /> <br /> It encapsulates its compiler flags, etc in an <tt>Makeoptions.in</tt> file which is then processed by <tt>configure</tt>. I've used the following additions to <tt>Makeoptions.in</tt> on TCS and GPC:<br /> <br /> <source lang="sh"><br /> ifeq ($(MACHINE),scinettcs)<br /> CC = mpcc_r<br /> LDR = mpcc_r<br /> OPT = -O5 -q64 -qarch=pwr6 -qtune=pwr6 -qcache=auto -qlargepage -qstrict<br /> MPIINC =<br /> MPILIB =<br /> CFLAGS = $(OPT)<br /> LIB = -ldl -lm<br /> else<br /> ifeq ($(MACHINE),scinetgpc)<br /> CC = mpicc<br /> LDR = mpicc<br /> OPT = -O3<br /> MPIINC =<br /> MPILIB =<br /> CFLAGS = $(OPT)<br /> LIB = -lm<br /> else<br /> ...<br /> endif<br /> endif<br /> </source><br /> It performs quite well on the GPC, scaling extremely well even on a strong scaling test out to about 256 cores (32 nodes) on Gigabit ethernet, and performing beautifully on InfiniBand out to 512 cores (64 nodes). <br /> <br /> <br /> -- [[User:Ljdursi|ljdursi]] 19:20, 13 August 2009 (UTC)<br /> <br /> ===FLASH3 (Adaptive Mesh reactive hydrodynamics; explict hydro/MHD)===<br /> <br /> [[Image:weak-scaling-example.png|thumb|right|320px|Weak scaling test of the 2d sod problem on both the GPC and TCS. The results are actually somewhat faster on the GPC; in both cases (weak) scaling is very good out at least to 256 cores]]<br /> <br /> [http://flash.uchicago.edu FLASH] encapsulates its machine-dependant information in the <tt>FLASH3/sites</tt> directory. For the GPC, you'll have to<br /> <pre><br /> module load intel<br /> module load openmpi<br /> module load hdf5/183-v16-openmpi<br /> </pre><br /> <br /> and with that, the following file (<tt>sites/scinetgpc/Makefile.h</tt>) works for me:<br /> <source lang="sh"><br /> ## Must do module load hdf5/183-v16-openmpi<br /> HDF5_PATH = ${SCINET_HDF5_BASE}<br /> ZLIB_PATH = /usr/local<br /> <br /> #----------------------------------------------------------------------------<br /> # Compiler and linker commands<br /> #<br /> # We use the f90 compiler as the linker, so some C libraries may explicitly<br /> # need to be added into the link line.<br /> #----------------------------------------------------------------------------<br /> <br /> ## modules will put the right mpi in our path<br /> FCOMP = mpif77<br /> CCOMP = mpicc<br /> CPPCOMP = mpiCC<br /> LINK = mpif77<br /> <br /> #----------------------------------------------------------------------------<br /> # Compilation flags<br /> #<br /> # Three sets of compilation/linking flags are defined: one for optimized<br /> # code, one for testing, and one for debugging. The default is to use the <br /> # _OPT version. Specifying -debug to setup will pick the _DEBUG version,<br /> # these should enable bounds checking. Specifying -test is used for <br /> # flash_test, and is set for quick code generation, and (sometimes) <br /> # profiling. The Makefile generated by setup will assign the generic token <br /> # (ex. FFLAGS) to the proper set of flags (ex. FFLAGS_OPT).<br /> #----------------------------------------------------------------------------<br /> <br /> FFLAGS_OPT = -c -r8 -i4 -O3 -xSSE4.2<br /> FFLAGS_DEBUG = -c -g -r8 -i4 -O0<br /> FFLAGS_TEST = -c -r8 -i4<br /> <br /> <br /> # if we are using HDF5, we need to specify the path to the include files<br /> CFLAGS_HDF5 = -I${HDF5_PATH}/include<br /> <br /> CFLAGS_OPT = -c -O3 -xSSE4.2<br /> CFLAGS_TEST = -c -O2 <br /> CFLAGS_DEBUG = -c -g <br /> <br /> MDEFS = <br /> <br /> .SUFFIXES: .o .c .f .F .h .fh .F90 .f90<br /> <br /> #----------------------------------------------------------------------------<br /> # Linker flags<br /> #<br /> # There is a seperate version of the linker flags for each of the _OPT, <br /> # _DEBUG, and _TEST cases.<br /> #----------------------------------------------------------------------------<br /> <br /> LFLAGS_OPT = -o<br /> LFLAGS_TEST = -o<br /> LFLAGS_DEBUG = -g -o<br /> <br /> MACHOBJ = <br /> <br /> <br /> MV = mv -f<br /> AR = ar -r<br /> RM = rm -f<br /> CD = cd<br /> RL = ranlib<br /> ECHO = echo<br /> </source><br /> <br /> <br /> -- [[User:Ljdursi|ljdursi]] 22:11, 13 August 2009 (UTC)<br /> <br /> ==Aeronautics==<br /> <br /> ==Chemistry==<br /> <br /> ===GAMESS (US)===<br /> The GAMESS version January 12, 2009 R3 was built using the Intel v11.1 compilers and v3.2.2 MPI library, according to the instructions in http://software.intel.com/en-us/articles/building-gamess-with-intel-compilers-intel-mkl-and-intel-mpi-on-linux/<br /> <br /> The required build scripts - comp, compall, lked - and run script - rungms - were modified to account for our own installation. In order to build GAMESS one first must ensure that the intel and intelmpi modules are loaded ("module load intel intelmpi"). This applies to running GAMESS as well.<br /> The module "gamess" must also be loaded in order to run GAMESS ("module load gamess").<br /> <br /> The modified scripts are in the file /scinet/gpc/src/gamess-on-scinet.tar.gz<br /> <br /> ====Running GAMESS====<br /> - Make sure the directory /scratch/$USER/gamess-scratch exists (the $SCINET_RUNGMS script will create it if it does not exist)<br /> <br /> - Make sure the modules: intel, intelmpi, gamess are loaded (in your .bashrc: "module load intel intelmpi gamess").<br /> <br /> - Create a torque script to run GAMESS. Here is an example:<br /> <br /> - The GAMESS executable is in $SCINET_GAMESS_HOME/gamess.00.x<br /> - The rungms script is in $SCINET_GAMESS_HOME/rungms (actually it is $SCINET_RUNGMS)<br /> <br /> - For IB multinode runs, use the $SCINET_RUNGMS_IB script<br /> <br /> - The rungms script takes 4 arguments: input file, executable number, number of compute processes, processors per node<br /> <br /> For example, in order to run with the input file /scratch/$USER/gamesstest01, on 8 cpus, and the default version (00)<br /> of the executable on a machine with 8 cores:<br /> <br /> # load the gamess module in .bashrc<br /> module load gamess <br /> <br /> # run the program<br /> $SCINET_RUNGMS /scratch/$USER/gamesstest01 00 8 8<br /> <br /> Here is a sample torque script for running a GAMESS calculation, on a single 8-core node:<br /> <br /> <source lang="bash"><br /> #!/bin/bash<br /> #PBS -l nodes=1:ppn=8,walltime=48:00:00,os=centos53computeA<br /> #PBS -N gamessjob<br /> <br /> ## To submit type: qsub gms.sh<br /> <br /> # If not an interactive job (i.e. -I), then cd into the directory where<br /> # I typed qsub.<br /> if [ "$PBS_ENVIRONMENT" != "PBS_INTERACTIVE" ]; then<br /> if [ -n "$PBS_O_WORKDIR" ]; then<br /> cd $PBS_O_WORKDIR<br /> fi<br /> fi<br /> <br /> # the input file is typically named something like "gamesjob.inp"<br /> # so the script will be run like "$SCINET_RUNGMS gamessjob 00 8 8"<br /> <br /> # load the gamess module if not in .bashrc already<br /> # actually, it MUST be in .bashrc<br /> # module load gamess<br /> <br /> # run the program<br /> <br /> $SCINET_RUNGMS gamessjob 00 8 8<br /> </source><br /> <br /> Here is a similar script, but this one uses 2 InfiniBand-connected nodes,<br /> and runs the appropriate $SCINET_RUNGMS_IB script to actually run the job:<br /> <br /> <source lang="bash"><br /> #!/bin/bash<br /> #PBS -l nodes=2:ib:ppn=8,walltime=48:00:00,os=centos53computeA<br /> #PBS -N gamessjob<br /> <br /> ## To submit type: qsub gmsib.sh<br /> <br /> # If not an interactive job (i.e. -I), then cd into the directory where<br /> # I typed qsub.<br /> if [ "$PBS_ENVIRONMENT" != "PBS_INTERACTIVE" ]; then<br /> if [ -n "$PBS_O_WORKDIR" ]; then<br /> cd $PBS_O_WORKDIR<br /> fi<br /> fi<br /> <br /> # the input file is typically named something like "gamesjob.inp"<br /> # so the script will be run like "$SCINET_RUNGMS gamessjob 00 8 8"<br /> <br /> # load the gamess module if not in .bashrc already<br /> # actually, it MUST be in .bashrc<br /> # module load gamess<br /> <br /> # This script requests InfiniBand-connected nodes (:ib above)<br /> # so it must run with the IB version of the rungms script,<br /> # $SCINET_RUNGMS_IB<br /> <br /> $SCINET_RUNGMS_IB gamessjob 00 16 8<br /> </source><br /> <br /> <br /> -- [[User:Dgruner|dgruner]] 5 October 2009<br /> <br /> ===Tips from the Fekl Lab===<br /> <br /> Through trial and error, we have found a few useful things that we would like to share:<br /> <br /> 1. Two very useful, open-source programs for visualization of output files from GAMESS(US) and for generation of input files are [http://www.scl.ameslab.gov/MacMolPlt/ MacMolPlt]and [http://avogadro.openmolecules.net/wiki/Main_Page Avogadro]. The are available for UNIX/LINUX, Windows and Mac based machines, HOWEVER: any input files that we have generated with these programs on a Windows-based machine do not run on Mac based machines. We don't know why.<br /> <br /> 2. [http://winscp.net/eng/index.php WinSCP] is a very useful tool that has a graphical user interface for moving files from a local machine to SCINET and vice versa. It also has text editing capabilities.<br /> <br /> 3. The [https://bse.pnl.gov/bse/portal ESML Basis Set Exchange] is an excellent source for custom basis set or effective core potential parameters. Make sure that you specify "Gamess-US" in the format drop-down box.<br /> <br /> 4. The commercial program [http://www.chemcraftprog.com/ ChemCraft] is a highly useful visualization program that has the ability to edit molecules in a very similar fashion to GaussView. It can also be customized to build GAMESS(US) input files.<br /> <br /> ====Anatomy of a GAMESS(US) Input File with Basis Set Info in an External File====<br /> <br /> $CONTRL SCFTYP=RHF RUNTYP=OPTIMIZE DFTTYP=M06-L MAXIT=199 MULT=1 NOSYM=1<br /> ECP=READ $END<br /> $SYSTEM TIMLIM=525600 MWORDS=1750 PARALL=.TRUE. $END<br /> $BASIS GBASIS=CUSTOMNI EXTFIL=.t. $END<br /> $SCF DIRSCF=.TRUE. FDIFF=.f. $END<br /> $STATPT OPTTOL=0.0001 NSTEP=500 HSSEND=.t. $END<br /> $DATA<br /> Mo_BDT3<br /> C1<br /> MOLYBDENUM 42.0 5.7556500000 4.4039600000 16.5808400000<br /> SULFUR 16.0 7.4169700000 3.1956300000 15.2089300000<br /> SULFUR 16.0 4.0966800000 3.2258300000 15.1761100000<br /> SULFUR 16.0 3.9677300000 4.4940500000 18.3266100000<br /> SULFUR 16.0 7.1776900000 3.5815000000 18.4485200000<br /> SULFUR 16.0 4.3776600000 6.2447400000 15.6786900000<br /> SULFUR 16.0 7.5478700000 6.0679800000 16.2223700000<br /> CARBON 6.0 6.4716900000 2.1004800000 14.1902300000<br /> CARBON 6.0 5.0690300000 2.1781400000 14.1080700000<br /> CARBON 6.0 4.8421800000 4.2701300000 19.8855500000<br /> CARBON 6.0 6.1969000000 3.9249600000 19.9397400000<br /> CARBON 6.0 6.8280600000 3.7834200000 21.1913200000<br /> CARBON 6.0 5.7697600000 7.6933500000 17.4241800000<br /> CARBON 6.0 7.2043100000 7.9413600000 17.8281100000<br /> CARBON 6.0 5.5051400000 7.0409700000 14.5903800000<br /> CARBON 6.0 6.8905200000 6.9194700000 14.7626200000<br /> CARBON 6.0 7.7396400000 7.5379800000 13.8285700000<br /> HYDROGEN 1.0 8.8190700000 7.4520600000 13.9252200000<br /> CARBON 6.0 7.2169400000 8.2960300000 12.7704100000<br /> HYDROGEN 1.0 7.8667000000 8.7825100000 12.0575600000<br /> CARBON 6.0 5.8260300000 8.4502300000 12.6467800000<br /> HYDROGEN 1.0 5.4143000000 9.0544300000 11.8493100000<br /> CARBON 6.0 4.9881500000 7.8192300000 13.5528400000<br /> HYDROGEN 1.0 3.9090500000 7.9420000000 13.4583700000<br /> CARBON 6.0 7.1538500000 1.1569600000 13.4143900000<br /> CARBON 6.0 4.4018100000 1.3603900000 13.1919900000<br /> CARBON 6.0 6.4791600000 0.3185500000 12.5353300000<br /> CARBON 6.0 5.0837400000 0.4369500000 12.4084900000<br /> HYDROGEN 1.0 7.0116000000 -0.4099400000 11.9434600000<br /> HYDROGEN 1.0 8.2399000000 1.0702400000 13.4937600000<br /> HYDROGEN 1.0 3.3185600000 1.4368700000 13.0953100000<br /> HYDROGEN 1.0 4.5549800000 -0.1997300000 11.7165200000<br /> CARBON 6.0 6.1105700000 3.9639000000 22.3866100000<br /> CARBON 6.0 4.1216300000 4.4424400000 21.1020100000<br /> HYDROGEN 1.0 7.8732900000 3.5217100000 21.2520500000<br /> CARBON 6.0 4.7606000000 4.2868500000 22.3363800000<br /> HYDROGEN 1.0 6.6064200000 3.8406000000 23.3428500000<br /> HYDROGEN 1.0 4.2065000000 4.4170700000 23.2667100000<br /> HYDROGEN 1.0 3.0674000000 4.6893500000 21.0889000000<br /> HYDROGEN 1.0 7.4249200000 7.7545300000 18.8583200000<br /> HYDROGEN 1.0 7.6651700000 8.9049700000 17.7652100000<br /> HYDROGEN 1.0 5.3324000000 8.6487800000 17.2222700000<br /> HYDROGEN 1.0 5.5015000000 7.1039000000 18.2759400000<br /> $END<br /> $ECP<br /> MO-ECP GEN 28 3<br /> 5 ----- f potential -----<br /> -0.0469492 0 537.9667807 <br /> -20.2080084 1 147.8982938 <br /> -106.2116302 2 45.7358898 <br /> -41.8107368 2 13.2911467 <br /> -4.2054103 2 4.7059961 <br /> 3 ----- s-f potential -----<br /> 2.8063717 0 110.2991760 <br /> 44.5162012 1 23.2014645 <br /> 82.7785227 2 5.3530131 <br /> 4 ----- p-f potential -----<br /> 4.9420876 0 63.2901397 <br /> 25.8604976 1 23.3315302 <br /> 132.4708742 2 24.6759423 <br /> 57.3149794 2 4.6493040 <br /> 5 ----- d-f potential -----<br /> 3.0054591 0 104.4839977 <br /> 26.3637851 1 66.2307245 <br /> 183.3849199 2 39.1283176 <br /> 98.4453068 2 13.1164437 <br /> 22.4901377 2 3.6280263 <br /> S NONE<br /> S NONE<br /> S NONE<br /> S NONE<br /> S NONE<br /> S NONE<br /> C NONE<br /> C NONE<br /> C NONE<br /> C NONE<br /> C NONE<br /> C NONE<br /> C NONE<br /> C NONE<br /> C NONE<br /> C NONE<br /> H NONE<br /> C NONE<br /> H NONE<br /> C NONE<br /> H NONE<br /> C NONE<br /> H NONE<br /> C NONE<br /> C NONE<br /> C NONE<br /> C NONE<br /> H NONE<br /> H NONE<br /> H NONE<br /> H NONE<br /> C NONE<br /> C NONE<br /> H NONE<br /> C NONE<br /> H NONE<br /> H NONE<br /> H NONE<br /> H NONE<br /> H NONE<br /> H NONE<br /> H NONE<br /> $END<br /> <br /> =====The Input Deck=====<br /> <br /> Below is the input deck. It is where you tell GAMESS(US) what job type to execute and where all you individual parameters are entered for your specific job type. The example input deck below is for a geometry optimization and frequency calculation. This input deck is equivalent to the Gaussian job with "opt" and "freq" in the route section.<br /> <br /> $CONTRL SCFTYP=RHF RUNTYP=OPTIMIZE DFTTYP=M06-L MAXIT=199 MULT=1 NOSYM=1<br /> ECP=READ $END<br /> $SYSTEM TIMLIM=2850 MWORDS=1750 MEMDDI=20 PARALL=.TRUE. $END<br /> $BASIS GBASIS=CUSTOMNI EXTFIL=.t. $END<br /> $SCF DIRSCF=.TRUE. FDIFF=.f. $END<br /> $STATPT OPTTOL=0.0001 NSTEP=500 HSSEND=.t. $END<br /> $DATA<br /> <br /> An important thing to note is the spacing. In the input deck, there must be 1 space at the beginning of each line of the input deck. If not, the job will fail. Most builders will insert this space anyway, but it helps to double check.<br /> <br /> The end of the input deck is marked by the "$DATA" line.<br /> <br /> =====Job Title Line=====<br /> <br /> The next line of the file is the job title. It can be anthing you wish, however, we have found that to be on the safe side, we avoide using symbols or spaces<br /> <br /> Mo_BDT3<br /> <br /> =====Symmetry Point Group=====<br /> <br /> The next line of the file is the symmetry point group of your molecule. Note that there is no leading space before the point group.<br /> <br /> C1<br /> <br /> =====Coordinates=====<br /> <br /> The next block of text is set aside for the coordinates of the molecule. This can be in internal (or z-matrix) format or cartesian coordinates. Note that there is no leading space before the coordinates. One may use the chemical symbol or the full name of each atom in the molecule. Note that the end of the coordinates is signified by an "$END", which MUST have one space preceding it. The coordinates below do NOT have any basis set information inserted. It is possible to insert basis set information directly into the input file. This is accomplished by obtaining the desired basis set parameters from the EMSL and then inserting them below each relevant atom. An example input file with inserted basis set information will be shown later.<br /> <br /> MOLYBDENUM 42.0 5.7556500000 4.4039600000 16.5808400000<br /> SULFUR 16.0 7.4169700000 3.1956300000 15.2089300000<br /> SULFUR 16.0 4.0966800000 3.2258300000 15.1761100000<br /> SULFUR 16.0 3.9677300000 4.4940500000 18.3266100000<br /> SULFUR 16.0 7.1776900000 3.5815000000 18.4485200000<br /> SULFUR 16.0 4.3776600000 6.2447400000 15.6786900000<br /> SULFUR 16.0 7.5478700000 6.0679800000 16.2223700000<br /> CARBON 6.0 6.4716900000 2.1004800000 14.1902300000<br /> CARBON 6.0 5.0690300000 2.1781400000 14.1080700000<br /> CARBON 6.0 4.8421800000 4.2701300000 19.8855500000<br /> CARBON 6.0 6.1969000000 3.9249600000 19.9397400000<br /> CARBON 6.0 6.8280600000 3.7834200000 21.1913200000<br /> CARBON 6.0 5.7697600000 7.6933500000 17.4241800000<br /> CARBON 6.0 7.2043100000 7.9413600000 17.8281100000<br /> CARBON 6.0 5.5051400000 7.0409700000 14.5903800000<br /> CARBON 6.0 6.8905200000 6.9194700000 14.7626200000<br /> CARBON 6.0 7.7396400000 7.5379800000 13.8285700000<br /> HYDROGEN 1.0 8.8190700000 7.4520600000 13.9252200000<br /> CARBON 6.0 7.2169400000 8.2960300000 12.7704100000<br /> HYDROGEN 1.0 7.8667000000 8.7825100000 12.0575600000<br /> CARBON 6.0 5.8260300000 8.4502300000 12.6467800000<br /> HYDROGEN 1.0 5.4143000000 9.0544300000 11.8493100000<br /> CARBON 6.0 4.9881500000 7.8192300000 13.5528400000<br /> HYDROGEN 1.0 3.9090500000 7.9420000000 13.4583700000<br /> CARBON 6.0 7.1538500000 1.1569600000 13.4143900000<br /> CARBON 6.0 4.4018100000 1.3603900000 13.1919900000<br /> CARBON 6.0 6.4791600000 0.3185500000 12.5353300000<br /> CARBON 6.0 5.0837400000 0.4369500000 12.4084900000<br /> HYDROGEN 1.0 7.0116000000 -0.4099400000 11.9434600000<br /> HYDROGEN 1.0 8.2399000000 1.0702400000 13.4937600000<br /> HYDROGEN 1.0 3.3185600000 1.4368700000 13.0953100000<br /> HYDROGEN 1.0 4.5549800000 -0.1997300000 11.7165200000<br /> CARBON 6.0 6.1105700000 3.9639000000 22.3866100000<br /> CARBON 6.0 4.1216300000 4.4424400000 21.1020100000<br /> HYDROGEN 1.0 7.8732900000 3.5217100000 21.2520500000<br /> CARBON 6.0 4.7606000000 4.2868500000 22.3363800000<br /> HYDROGEN 1.0 6.6064200000 3.8406000000 23.3428500000<br /> HYDROGEN 1.0 4.2065000000 4.4170700000 23.2667100000<br /> HYDROGEN 1.0 3.0674000000 4.6893500000 21.0889000000<br /> HYDROGEN 1.0 7.4249200000 7.7545300000 18.8583200000<br /> HYDROGEN 1.0 7.6651700000 8.9049700000 17.7652100000<br /> HYDROGEN 1.0 5.3324000000 8.6487800000 17.2222700000<br /> HYDROGEN 1.0 5.5015000000 7.1039000000 18.2759400000<br /> $END<br /> <br /> =====Effective Core Potential Data=====<br /> <br /> The effective core potential (ECP) data is entered after the coordinates. It starts with "$ECP", which must be preceded with a space. The atoms of the molecule are listed in the same order as in the coordinates section and the parameters for the ECP are listed after each atom. Note that for any atom that does NOT have an ECP, one must enter "ECP-NONE" or "NONE" after each atom without an ECP.<br /> <br /> $ECP<br /> MO-ECP GEN 28 3<br /> 5 ----- f potential -----<br /> -0.0469492 0 537.9667807 <br /> -20.2080084 1 147.8982938 <br /> -106.2116302 2 45.7358898 <br /> -41.8107368 2 13.2911467 <br /> -4.2054103 2 4.7059961 <br /> 3 ----- s-f potential -----<br /> 2.8063717 0 110.2991760 <br /> 44.5162012 1 23.2014645 <br /> 82.7785227 2 5.3530131 <br /> 4 ----- p-f potential -----<br /> 4.9420876 0 63.2901397 <br /> 25.8604976 1 23.3315302 <br /> 132.4708742 2 24.6759423 <br /> 57.3149794 2 4.6493040 <br /> 5 ----- d-f potential -----<br /> 3.0054591 0 104.4839977 <br /> 26.3637851 1 66.2307245 <br /> 183.3849199 2 39.1283176 <br /> 98.4453068 2 13.1164437 <br /> 22.4901377 2 3.6280263 <br /> S NONE<br /> S NONE<br /> S NONE<br /> S NONE<br /> S NONE<br /> S NONE<br /> C NONE<br /> C NONE<br /> C NONE<br /> C NONE<br /> C NONE<br /> C NONE<br /> C NONE<br /> C NONE<br /> C NONE<br /> C NONE<br /> H NONE<br /> C NONE<br /> H NONE<br /> C NONE<br /> H NONE<br /> C NONE<br /> H NONE<br /> C NONE<br /> C NONE<br /> C NONE<br /> C NONE<br /> H NONE<br /> H NONE<br /> H NONE<br /> H NONE<br /> C NONE<br /> C NONE<br /> H NONE<br /> C NONE<br /> H NONE<br /> H NONE<br /> H NONE<br /> H NONE<br /> H NONE<br /> H NONE<br /> H NONE<br /> $END<br /> <br /> -- [[User:M.Zimmer-De Iuliis|mzd]] 16 November 2009<br /> <br /> ====Using Effective Core Potentials in GAMESS(US)====<br /> <br /> For many metal containing compounds, it is very convenient and time saving to use an effective core potential (ECP) for the core metal electrons, as they are usually not important to the reactivity of the complex or the geometry around the metal. Since GAMESS(US) has a limited number of built-in ECPs, one may want to make GAMESS(US) read an external file that contains the ECP data using the "EXTFIL" keyword in the $GBASIS command line of the input file. In addition, to make GAMESS(US) use this external file, one must copy the "rungms" file and modify it accordingly. The following is a list of instructions with commands that will work from a terminal. One could also use WinSCP to do all of this with a GUI rather than a TUI. <br /> <br /> =====Modifiying rungms to Use Custom Basis Set File=====<br /> 1. Copy "rungms" from /scinet/gpc/Applications/gamess to one's own /scratch/$USER/ directory:<br /> cp /scinet/gpc/Applications/gamess/rungms /scratch/$USER/<br /> <br /> 2. Change to the scratch directory and check to see if "rungms" has copied successfully.<br /> cd /scratch/$USER<br /> ls<br /> <br /> 3. Edit line 147 of the vi. <br /> vi rungms<br /> Move the cursor down to line 147 using the arrow keys. It should say "setenv EXTBAS /dev/null". Using the arrow keys, move the cursor to the first "/" and then hit "i" to insert text. Put the path to your external basis file here. For example, /scratch/$USER/basisset. Then hit "escape". To save the changes and exit vi, type ":" and you should see a colon appear at the bottom of the window. Type "wq" (which should appear at the bottom of the window next to the colon) and then hit enter. Now you are done with vi.<br /> <br /> =====Creating a Custom Basis Set File=====<br /> 1. To create a custom basis set file, you need create a new text document. Our group's common practice is to comment out the first line of this file by inserting an exclamation mark (!) followed by noting the specific basis sets and ECPs that are going to be used for each of the atoms. Let us the molecule Mo(CO)6, Molybdenum hexacarbonyl, as an example. Below is the first line of the the external file, which we will call "CUSTOMMO" (NOTE: you can use any name for the external file that suits you, as long as it has no spaces and is 8 characters or less).<br /> <br /> ! 6-31G on C and O and LANL2D2 ECP on Mo<br /> <br /> 2. The next step is to visit the [https://bse.pnl.gov/bse/portal EMSL Basis Set exchange] and select C and O from the periodic table. Then, on the left of the page, select "6-31G" as the basis set. Finally, make sure the output is in GAMESS(US) format using the drop-down menu and then click "get basis set".<br /> <br /> [[File:C_O_6_31G_basisset.JPG|centre]]<br /> <br /> 3. A new window should appear with text in it. For our example case, the text looks like this:<br /> <br /> ! 6-31G EMSL Basis Set Exchange Library 10/13/09 11:12 AM<br /> ! Elements References<br /> ! -------- ----------<br /> ! H - He: W.J. Hehre, R. Ditchfield and J.A. Pople, J. Chem. Phys. 56,<br /> ! Li - Ne: 2257 (1972). Note: Li and B come from J.D. Dill and J.A.<br /> ! Pople, J. Chem. Phys. 62, 2921 (1975).<br /> ! Na - Ar: M.M. Francl, W.J. Petro, W.J. Hehre, J.S. Binkley, M.S. Gordon,<br /> ! D.J. DeFrees and J.A. Pople, J. Chem. Phys. 77, 3654 (1982)<br /> ! K - Zn: V. Rassolov, J.A. Pople, M. Ratner and T.L. Windus, J. Chem. Phys.<br /> ! 109, 1223 (1998)<br /> ! Note: He and Ne are unpublished basis sets taken from the Gaussian<br /> ! program<br /> ! <br /> $DATA<br /><br /> CARBON<br /> S 6<br /> 1 3047.5249000 0.0018347 <br /> 2 457.3695100 0.0140373 <br /> 3 103.9486900 0.0688426 <br /> 4 29.2101550 0.2321844 <br /> 5 9.2866630 0.4679413 <br /> 6 3.1639270 0.3623120 <br /> L 3<br /> 1 7.8682724 -0.1193324 0.0689991 <br /> 2 1.8812885 -0.1608542 0.3164240 <br /> 3 0.5442493 1.1434564 0.7443083 <br /> L 1<br /> 1 0.1687144 1.0000000 1.0000000<br /> <br /> OXYGEN<br /> S 6<br /> 1 5484.6717000 0.0018311 <br /> 2 825.2349500 0.0139501 <br /> 3 188.0469600 0.0684451 <br /> 4 52.9645000 0.2327143 <br /> 5 16.8975700 0.4701930 <br /> 6 5.7996353 0.3585209 <br /> L 3<br /> 1 15.5396160 -0.1107775 0.0708743 <br /> 2 3.5999336 -0.1480263 0.3397528 <br /> 3 1.0137618 1.1307670 0.7271586 <br /> L 1<br /> 1 0.2700058 1.0000000 1.0000000 <br /> $END<br /> <br /> 3. Now, copy and paste the text between the $DATA and $END headings onto our external text file, CUSTOMMO. We also need to change the change the name of each element to it symbol in the periodic table. Finally, we need to add the name of the external file next to the element symbol, separated by one space. Note that there should be a blank line separating the basis set information and the first, commented-out line (The line starting with the '!'). The CUSTOMMO should look like this:<br /> <br /> ! 6-31G on C and O and LANL2D2 ECP on Mo<br /><br /> C CUSTOMMO<br /> S 6<br /> 1 3047.5249000 0.0018347 <br /> 2 457.3695100 0.0140373 <br /> 3 103.9486900 0.0688426 <br /> 4 29.2101550 0.2321844 <br /> 5 9.2866630 0.4679413 <br /> 6 3.1639270 0.3623120 <br /> L 3<br /> 1 7.8682724 -0.1193324 0.0689991 <br /> 2 1.8812885 -0.1608542 0.3164240 <br /> 3 0.5442493 1.1434564 0.7443083 <br /> L 1<br /> 1 0.1687144 1.0000000 1.0000000<br /> <br /> O CUSTOMMO<br /> S 6<br /> 1 5484.6717000 0.0018311 <br /> 2 825.2349500 0.0139501 <br /> 3 188.0469600 0.0684451 <br /> 4 52.9645000 0.2327143 <br /> 5 16.8975700 0.4701930 <br /> 6 5.7996353 0.3585209 <br /> L 3<br /> 1 15.5396160 -0.1107775 0.0708743 <br /> 2 3.5999336 -0.1480263 0.3397528 <br /> 3 1.0137618 1.1307670 0.7271586 <br /> L 1<br /> 1 0.2700058 1.0000000 1.0000000 <br /> <br /> 4. Repeat Step 3 above but choose Mo and select the LANL2DZ ECP instead. A new window will pop up with the basis set information as well as the ECP data we need, since we specified the LANL2DZ '''ECP'''. The ECP data is not inserted into the external file, rather it is placed into the input file itself (More on this later). <br /> <br /> [[File:Mo_LANL2DZ_basisset.JPG|centre]]<br /> <br /> 5. After copying the molybdenum basis set information, your fiished external basis set file should look like this:<br /> ! 6-31G on C and O and LANL2D2 ECP on Mo<br /><br /> C CUSTOMMO<br /> S 6<br /> 1 3047.5249000 0.0018347 <br /> 2 457.3695100 0.0140373 <br /> 3 103.9486900 0.0688426 <br /> 4 29.2101550 0.2321844 <br /> 5 9.2866630 0.4679413 <br /> 6 3.1639270 0.3623120 <br /> L 3<br /> 1 7.8682724 -0.1193324 0.0689991 <br /> 2 1.8812885 -0.1608542 0.3164240 <br /> 3 0.5442493 1.1434564 0.7443083 <br /> L 1<br /> 1 0.1687144 1.0000000 1.0000000<br /> <br /> O CUSTOMMO<br /> S 6<br /> 1 5484.6717000 0.0018311 <br /> 2 825.2349500 0.0139501 <br /> 3 188.0469600 0.0684451 <br /> 4 52.9645000 0.2327143 <br /> 5 16.8975700 0.4701930 <br /> 6 5.7996353 0.3585209 <br /> L 3<br /> 1 15.5396160 -0.1107775 0.0708743 <br /> 2 3.5999336 -0.1480263 0.3397528 <br /> 3 1.0137618 1.1307670 0.7271586 <br /> L 1<br /> 1 0.2700058 1.0000000 1.0000000<br /> <br /> Mo CUSTOMO<br /> S 3<br /> 1 2.3610000 -0.9121760 <br /> 2 1.3090000 1.1477453 <br /> 3 0.4500000 0.6097109 <br /> S 4<br /> 1 2.3610000 0.8139259 <br /> 2 1.3090000 -1.1360084 <br /> 3 0.4500000 -1.1611592 <br /> 4 0.1681000 1.0064786 <br /> S 1<br /> 1 0.0423000 1.0000000 <br /> P 3<br /> 1 4.8950000 -0.0908258 <br /> 2 1.0440000 0.7042899 <br /> 3 0.3877000 0.3973179 <br /> P 2<br /> 1 0.4995000 -0.1081945 <br /> 2 0.0780000 1.0368093 <br /> P 1<br /> 1 0.0247000 1.0000000 <br /> D 3<br /> 1 2.9930000 0.0527063 <br /> 2 1.0630000 0.5003907 <br /> 3 0.3721000 0.5794024 <br /> D 1<br /> 1 0.1178000 1.0000000<br /> <br /> -- [[User:M.Zimmer-De Iuliis|mzd]] 13 October 2009<br /> <br /> ==Climate Modelling==<br /> <br /> ==Medicine/Bio==<br /> <br /> ==High Energy Physics==<br /> <br /> ==Structural Biology==<br /> Molecular simulation of proteins, lipids, carbohydrates, and other biologically relevant molecules.<br /> ===Molecular Dynamics (MD) simulation===<br /> ====GROMACS====<br /> Please refer to the [[gromacs|GROMACS]] page<br /> ====AMBER====<br /> Please refer to the [[amber|AMBER]] page<br /> ====NAMD====<br /> NAMD is one of the better scaling MD packages out there. With sufficiently large systems, it is able to scale to hundreds or thousands of cores on Scinet. Below are details for compiling and running NAMD on Scinet.<br /> <br /> More information regarding performance and different compile options coming soon...<br /> <br /> =====Compiling NAMD for GPC=====<br /> Ensure the proper compiler/mpi modules are loaded.<br /> <source lang="sh"><br /> module load intel<br /> module load openmpi/1.3.3-intel-v11.0-ofed<br /> </source><br /> <br /> '''Compile Charm++ and NAMD'''<br /> <source lang="sh"><br /> #Unpack source files and get required support libraries<br /> tar -xzf NAMD_2.7b1_Source.tar.gz<br /> cd NAMD_2.7b1_Source<br /> tar -xf charm-6.1.tar<br /> wget http://www.ks.uiuc.edu/Research/namd/libraries/fftw-linux-x86_64.tar.gz<br /> wget http://www.ks.uiuc.edu/Research/namd/libraries/tcl-linux-x86_64.tar.gz<br /> tar -xzf fftw-linux-x86_64.tar.gz; mv linux-x86_64 fftw<br /> tar -xzf tcl-linux-x86_64.tar.gz; mv linux-x86_64 tcl<br /> #Compile Charm++<br /> cd charm-6.1<br /> ./build charm++ mpi-linux-x86_64 icc --basedir /scinet/gpc/mpi/openmpi/1.3.3-intel-v11.0-ofed/ --no-shared -O -DCMK_OPTIMIZE=1<br /> cd ..<br /> #Compile NAMD. <br /> #Edit arch/Linux-x86_64-icc.arch and add "-lmpi" to the end of the CXXOPTS and COPTS line.<br /> #Make a builds directory if you want different versions of NAMD compiled at the same time.<br /> mkdir builds<br /> ./config builds/Linux-x86_64-icc --charm-arch mpi-linux-x86_64-icc<br /> cd builds/Linux-x86_64-icc/<br /> make -j4 namd2 # Adjust value of j as desired to specify number of simultaneous make targets. <br /> </source><br /> --[[User:Cmadill|Cmadill]] 16:18, 27 August 2009 (UTC)<br /> <br /> =====Running Fortran=====<br /> On the development nodes, there is an old gcc. The associated libraries are not on the compute nodes. Ensure the line:<br /> <br /> module load gcc<br /> <br /> is in your .bashrc file.<br /> <br /> ====LAMMPS====<br /> [[Image:StrongScalingLAMMPS.png|thumb|320px|right|Strong scaling test on GPC with OpenMPI and IntelMPI on Ethernet and InfiniBand]]<br /> [[Image:WeakScalingLAMMPS.png|thumb|320px|right|Weak scaling test on GPC with OpenMPI and IntelMPI on Ethernet and InfiniBand]]<br /> Results from strong scaling tests for LAMMPS using EAM potentials on GPC are shown in the graph on the right. The test simulation ran 500 timesteps for 4,000,000 atoms.<br /> <br /> Results from weak scaling tests for LAMMPS using EAM potentials on GPC are shown in the graph on the right. The test simulation ran 500 timesteps for 32,000 atoms per processor.<br /> <br /> OpenMPI version used: openmpi/1.4.1-intel-v11.0-ofed<br /> IntelMPI version used: intelmpi/impi-4.0.0.013<br /> <br /> Results show good scaling for both OpenMPI and IntelMPI up to 16 processors, after which performance begins to suffer. On Infiniband, excellent scaling is maintained to 512 processors.<br /> <br /> IntelMPI is found to have a slight edge over OpenMPI.<br /> <br /> ===Monte Carlo (MC) simulation===</div>

Jchu https://oldwiki.scinet.utoronto.ca/index.php?title=User_Codes&diff=875 User Codes 2010-02-02T18:54:11Z

<p>Jchu: /* LAMMPS */</p> <hr /> <div>__FORCETOC__<br /> <br /> ==Astrophysics==<br /> <br /> ===Athena (explicit, uniform grid MHD code)===<br /> <br /> [[Image:StrongScalingAthenaGPC.png|thumb|right|320px|Athena scaling on GPC with OpenMPI and MVAPICH2 on GigE, and OpenMPI on InfiniBand]]<br /> <br /> [http://www.astro.princeton.edu/~jstone/athena.html Athena] is a straightforward C code which doesn't use a lot of libraries so it is pretty straightforward to build and compile on new machines. <br /> <br /> It encapsulates its compiler flags, etc in an <tt>Makeoptions.in</tt> file which is then processed by <tt>configure</tt>. I've used the following additions to <tt>Makeoptions.in</tt> on TCS and GPC:<br /> <br /> <source lang="sh"><br /> ifeq ($(MACHINE),scinettcs)<br /> CC = mpcc_r<br /> LDR = mpcc_r<br /> OPT = -O5 -q64 -qarch=pwr6 -qtune=pwr6 -qcache=auto -qlargepage -qstrict<br /> MPIINC =<br /> MPILIB =<br /> CFLAGS = $(OPT)<br /> LIB = -ldl -lm<br /> else<br /> ifeq ($(MACHINE),scinetgpc)<br /> CC = mpicc<br /> LDR = mpicc<br /> OPT = -O3<br /> MPIINC =<br /> MPILIB =<br /> CFLAGS = $(OPT)<br /> LIB = -lm<br /> else<br /> ...<br /> endif<br /> endif<br /> </source><br /> It performs quite well on the GPC, scaling extremely well even on a strong scaling test out to about 256 cores (32 nodes) on Gigabit ethernet, and performing beautifully on InfiniBand out to 512 cores (64 nodes). <br /> <br /> <br /> -- [[User:Ljdursi|ljdursi]] 19:20, 13 August 2009 (UTC)<br /> <br /> ===FLASH3 (Adaptive Mesh reactive hydrodynamics; explict hydro/MHD)===<br /> <br /> [[Image:weak-scaling-example.png|thumb|right|320px|Weak scaling test of the 2d sod problem on both the GPC and TCS. The results are actually somewhat faster on the GPC; in both cases (weak) scaling is very good out at least to 256 cores]]<br /> <br /> [http://flash.uchicago.edu FLASH] encapsulates its machine-dependant information in the <tt>FLASH3/sites</tt> directory. For the GPC, you'll have to<br /> <pre><br /> module load intel<br /> module load openmpi<br /> module load hdf5/183-v16-openmpi<br /> </pre><br /> <br /> and with that, the following file (<tt>sites/scinetgpc/Makefile.h</tt>) works for me:<br /> <source lang="sh"><br /> ## Must do module load hdf5/183-v16-openmpi<br /> HDF5_PATH = ${SCINET_HDF5_BASE}<br /> ZLIB_PATH = /usr/local<br /> <br /> #----------------------------------------------------------------------------<br /> # Compiler and linker commands<br /> #<br /> # We use the f90 compiler as the linker, so some C libraries may explicitly<br /> # need to be added into the link line.<br /> #----------------------------------------------------------------------------<br /> <br /> ## modules will put the right mpi in our path<br /> FCOMP = mpif77<br /> CCOMP = mpicc<br /> CPPCOMP = mpiCC<br /> LINK = mpif77<br /> <br /> #----------------------------------------------------------------------------<br /> # Compilation flags<br /> #<br /> # Three sets of compilation/linking flags are defined: one for optimized<br /> # code, one for testing, and one for debugging. The default is to use the <br /> # _OPT version. Specifying -debug to setup will pick the _DEBUG version,<br /> # these should enable bounds checking. Specifying -test is used for <br /> # flash_test, and is set for quick code generation, and (sometimes) <br /> # profiling. The Makefile generated by setup will assign the generic token <br /> # (ex. FFLAGS) to the proper set of flags (ex. FFLAGS_OPT).<br /> #----------------------------------------------------------------------------<br /> <br /> FFLAGS_OPT = -c -r8 -i4 -O3 -xSSE4.2<br /> FFLAGS_DEBUG = -c -g -r8 -i4 -O0<br /> FFLAGS_TEST = -c -r8 -i4<br /> <br /> <br /> # if we are using HDF5, we need to specify the path to the include files<br /> CFLAGS_HDF5 = -I${HDF5_PATH}/include<br /> <br /> CFLAGS_OPT = -c -O3 -xSSE4.2<br /> CFLAGS_TEST = -c -O2 <br /> CFLAGS_DEBUG = -c -g <br /> <br /> MDEFS = <br /> <br /> .SUFFIXES: .o .c .f .F .h .fh .F90 .f90<br /> <br /> #----------------------------------------------------------------------------<br /> # Linker flags<br /> #<br /> # There is a seperate version of the linker flags for each of the _OPT, <br /> # _DEBUG, and _TEST cases.<br /> #----------------------------------------------------------------------------<br /> <br /> LFLAGS_OPT = -o<br /> LFLAGS_TEST = -o<br /> LFLAGS_DEBUG = -g -o<br /> <br /> MACHOBJ = <br /> <br /> <br /> MV = mv -f<br /> AR = ar -r<br /> RM = rm -f<br /> CD = cd<br /> RL = ranlib<br /> ECHO = echo<br /> </source><br /> <br /> <br /> -- [[User:Ljdursi|ljdursi]] 22:11, 13 August 2009 (UTC)<br /> <br /> ==Aeronautics==<br /> <br /> ==Chemistry==<br /> <br /> ===GAMESS (US)===<br /> The GAMESS version January 12, 2009 R3 was built using the Intel v11.1 compilers and v3.2.2 MPI library, according to the instructions in http://software.intel.com/en-us/articles/building-gamess-with-intel-compilers-intel-mkl-and-intel-mpi-on-linux/<br /> <br /> The required build scripts - comp, compall, lked - and run script - rungms - were modified to account for our own installation. In order to build GAMESS one first must ensure that the intel and intelmpi modules are loaded ("module load intel intelmpi"). This applies to running GAMESS as well.<br /> The module "gamess" must also be loaded in order to run GAMESS ("module load gamess").<br /> <br /> The modified scripts are in the file /scinet/gpc/src/gamess-on-scinet.tar.gz<br /> <br /> ====Running GAMESS====<br /> - Make sure the directory /scratch/$USER/gamess-scratch exists (the $SCINET_RUNGMS script will create it if it does not exist)<br /> <br /> - Make sure the modules: intel, intelmpi, gamess are loaded (in your .bashrc: "module load intel intelmpi gamess").<br /> <br /> - Create a torque script to run GAMESS. Here is an example:<br /> <br /> - The GAMESS executable is in $SCINET_GAMESS_HOME/gamess.00.x<br /> - The rungms script is in $SCINET_GAMESS_HOME/rungms (actually it is $SCINET_RUNGMS)<br /> <br /> - For IB multinode runs, use the $SCINET_RUNGMS_IB script<br /> <br /> - The rungms script takes 4 arguments: input file, executable number, number of compute processes, processors per node<br /> <br /> For example, in order to run with the input file /scratch/$USER/gamesstest01, on 8 cpus, and the default version (00)<br /> of the executable on a machine with 8 cores:<br /> <br /> # load the gamess module in .bashrc<br /> module load gamess <br /> <br /> # run the program<br /> $SCINET_RUNGMS /scratch/$USER/gamesstest01 00 8 8<br /> <br /> Here is a sample torque script for running a GAMESS calculation, on a single 8-core node:<br /> <br /> <source lang="bash"><br /> #!/bin/bash<br /> #PBS -l nodes=1:ppn=8,walltime=48:00:00,os=centos53computeA<br /> #PBS -N gamessjob<br /> <br /> ## To submit type: qsub gms.sh<br /> <br /> # If not an interactive job (i.e. -I), then cd into the directory where<br /> # I typed qsub.<br /> if [ "$PBS_ENVIRONMENT" != "PBS_INTERACTIVE" ]; then<br /> if [ -n "$PBS_O_WORKDIR" ]; then<br /> cd $PBS_O_WORKDIR<br /> fi<br /> fi<br /> <br /> # the input file is typically named something like "gamesjob.inp"<br /> # so the script will be run like "$SCINET_RUNGMS gamessjob 00 8 8"<br /> <br /> # load the gamess module if not in .bashrc already<br /> # actually, it MUST be in .bashrc<br /> # module load gamess<br /> <br /> # run the program<br /> <br /> $SCINET_RUNGMS gamessjob 00 8 8<br /> </source><br /> <br /> Here is a similar script, but this one uses 2 InfiniBand-connected nodes,<br /> and runs the appropriate $SCINET_RUNGMS_IB script to actually run the job:<br /> <br /> <source lang="bash"><br /> #!/bin/bash<br /> #PBS -l nodes=2:ib:ppn=8,walltime=48:00:00,os=centos53computeA<br /> #PBS -N gamessjob<br /> <br /> ## To submit type: qsub gmsib.sh<br /> <br /> # If not an interactive job (i.e. -I), then cd into the directory where<br /> # I typed qsub.<br /> if [ "$PBS_ENVIRONMENT" != "PBS_INTERACTIVE" ]; then<br /> if [ -n "$PBS_O_WORKDIR" ]; then<br /> cd $PBS_O_WORKDIR<br /> fi<br /> fi<br /> <br /> # the input file is typically named something like "gamesjob.inp"<br /> # so the script will be run like "$SCINET_RUNGMS gamessjob 00 8 8"<br /> <br /> # load the gamess module if not in .bashrc already<br /> # actually, it MUST be in .bashrc<br /> # module load gamess<br /> <br /> # This script requests InfiniBand-connected nodes (:ib above)<br /> # so it must run with the IB version of the rungms script,<br /> # $SCINET_RUNGMS_IB<br /> <br /> $SCINET_RUNGMS_IB gamessjob 00 16 8<br /> </source><br /> <br /> <br /> -- [[User:Dgruner|dgruner]] 5 October 2009<br /> <br /> ===Tips from the Fekl Lab===<br /> <br /> Through trial and error, we have found a few useful things that we would like to share:<br /> <br /> 1. Two very useful, open-source programs for visualization of output files from GAMESS(US) and for generation of input files are [http://www.scl.ameslab.gov/MacMolPlt/ MacMolPlt]and [http://avogadro.openmolecules.net/wiki/Main_Page Avogadro]. The are available for UNIX/LINUX, Windows and Mac based machines, HOWEVER: any input files that we have generated with these programs on a Windows-based machine do not run on Mac based machines. We don't know why.<br /> <br /> 2. [http://winscp.net/eng/index.php WinSCP] is a very useful tool that has a graphical user interface for moving files from a local machine to SCINET and vice versa. It also has text editing capabilities.<br /> <br /> 3. The [https://bse.pnl.gov/bse/portal ESML Basis Set Exchange] is an excellent source for custom basis set or effective core potential parameters. Make sure that you specify "Gamess-US" in the format drop-down box.<br /> <br /> 4. The commercial program [http://www.chemcraftprog.com/ ChemCraft] is a highly useful visualization program that has the ability to edit molecules in a very similar fashion to GaussView. It can also be customized to build GAMESS(US) input files.<br /> <br /> ====Anatomy of a GAMESS(US) Input File with Basis Set Info in an External File====<br /> <br /> $CONTRL SCFTYP=RHF RUNTYP=OPTIMIZE DFTTYP=M06-L MAXIT=199 MULT=1 NOSYM=1<br /> ECP=READ $END<br /> $SYSTEM TIMLIM=525600 MWORDS=1750 PARALL=.TRUE. $END<br /> $BASIS GBASIS=CUSTOMNI EXTFIL=.t. $END<br /> $SCF DIRSCF=.TRUE. FDIFF=.f. $END<br /> $STATPT OPTTOL=0.0001 NSTEP=500 HSSEND=.t. $END<br /> $DATA<br /> Mo_BDT3<br /> C1<br /> MOLYBDENUM 42.0 5.7556500000 4.4039600000 16.5808400000<br /> SULFUR 16.0 7.4169700000 3.1956300000 15.2089300000<br /> SULFUR 16.0 4.0966800000 3.2258300000 15.1761100000<br /> SULFUR 16.0 3.9677300000 4.4940500000 18.3266100000<br /> SULFUR 16.0 7.1776900000 3.5815000000 18.4485200000<br /> SULFUR 16.0 4.3776600000 6.2447400000 15.6786900000<br /> SULFUR 16.0 7.5478700000 6.0679800000 16.2223700000<br /> CARBON 6.0 6.4716900000 2.1004800000 14.1902300000<br /> CARBON 6.0 5.0690300000 2.1781400000 14.1080700000<br /> CARBON 6.0 4.8421800000 4.2701300000 19.8855500000<br /> CARBON 6.0 6.1969000000 3.9249600000 19.9397400000<br /> CARBON 6.0 6.8280600000 3.7834200000 21.1913200000<br /> CARBON 6.0 5.7697600000 7.6933500000 17.4241800000<br /> CARBON 6.0 7.2043100000 7.9413600000 17.8281100000<br /> CARBON 6.0 5.5051400000 7.0409700000 14.5903800000<br /> CARBON 6.0 6.8905200000 6.9194700000 14.7626200000<br /> CARBON 6.0 7.7396400000 7.5379800000 13.8285700000<br /> HYDROGEN 1.0 8.8190700000 7.4520600000 13.9252200000<br /> CARBON 6.0 7.2169400000 8.2960300000 12.7704100000<br /> HYDROGEN 1.0 7.8667000000 8.7825100000 12.0575600000<br /> CARBON 6.0 5.8260300000 8.4502300000 12.6467800000<br /> HYDROGEN 1.0 5.4143000000 9.0544300000 11.8493100000<br /> CARBON 6.0 4.9881500000 7.8192300000 13.5528400000<br /> HYDROGEN 1.0 3.9090500000 7.9420000000 13.4583700000<br /> CARBON 6.0 7.1538500000 1.1569600000 13.4143900000<br /> CARBON 6.0 4.4018100000 1.3603900000 13.1919900000<br /> CARBON 6.0 6.4791600000 0.3185500000 12.5353300000<br /> CARBON 6.0 5.0837400000 0.4369500000 12.4084900000<br /> HYDROGEN 1.0 7.0116000000 -0.4099400000 11.9434600000<br /> HYDROGEN 1.0 8.2399000000 1.0702400000 13.4937600000<br /> HYDROGEN 1.0 3.3185600000 1.4368700000 13.0953100000<br /> HYDROGEN 1.0 4.5549800000 -0.1997300000 11.7165200000<br /> CARBON 6.0 6.1105700000 3.9639000000 22.3866100000<br /> CARBON 6.0 4.1216300000 4.4424400000 21.1020100000<br /> HYDROGEN 1.0 7.8732900000 3.5217100000 21.2520500000<br /> CARBON 6.0 4.7606000000 4.2868500000 22.3363800000<br /> HYDROGEN 1.0 6.6064200000 3.8406000000 23.3428500000<br /> HYDROGEN 1.0 4.2065000000 4.4170700000 23.2667100000<br /> HYDROGEN 1.0 3.0674000000 4.6893500000 21.0889000000<br /> HYDROGEN 1.0 7.4249200000 7.7545300000 18.8583200000<br /> HYDROGEN 1.0 7.6651700000 8.9049700000 17.7652100000<br /> HYDROGEN 1.0 5.3324000000 8.6487800000 17.2222700000<br /> HYDROGEN 1.0 5.5015000000 7.1039000000 18.2759400000<br /> $END<br /> $ECP<br /> MO-ECP GEN 28 3<br /> 5 ----- f potential -----<br /> -0.0469492 0 537.9667807 <br /> -20.2080084 1 147.8982938 <br /> -106.2116302 2 45.7358898 <br /> -41.8107368 2 13.2911467 <br /> -4.2054103 2 4.7059961 <br /> 3 ----- s-f potential -----<br /> 2.8063717 0 110.2991760 <br /> 44.5162012 1 23.2014645 <br /> 82.7785227 2 5.3530131 <br /> 4 ----- p-f potential -----<br /> 4.9420876 0 63.2901397 <br /> 25.8604976 1 23.3315302 <br /> 132.4708742 2 24.6759423 <br /> 57.3149794 2 4.6493040 <br /> 5 ----- d-f potential -----<br /> 3.0054591 0 104.4839977 <br /> 26.3637851 1 66.2307245 <br /> 183.3849199 2 39.1283176 <br /> 98.4453068 2 13.1164437 <br /> 22.4901377 2 3.6280263 <br /> S NONE<br /> S NONE<br /> S NONE<br /> S NONE<br /> S NONE<br /> S NONE<br /> C NONE<br /> C NONE<br /> C NONE<br /> C NONE<br /> C NONE<br /> C NONE<br /> C NONE<br /> C NONE<br /> C NONE<br /> C NONE<br /> H NONE<br /> C NONE<br /> H NONE<br /> C NONE<br /> H NONE<br /> C NONE<br /> H NONE<br /> C NONE<br /> C NONE<br /> C NONE<br /> C NONE<br /> H NONE<br /> H NONE<br /> H NONE<br /> H NONE<br /> C NONE<br /> C NONE<br /> H NONE<br /> C NONE<br /> H NONE<br /> H NONE<br /> H NONE<br /> H NONE<br /> H NONE<br /> H NONE<br /> H NONE<br /> $END<br /> <br /> =====The Input Deck=====<br /> <br /> Below is the input deck. It is where you tell GAMESS(US) what job type to execute and where all you individual parameters are entered for your specific job type. The example input deck below is for a geometry optimization and frequency calculation. This input deck is equivalent to the Gaussian job with "opt" and "freq" in the route section.<br /> <br /> $CONTRL SCFTYP=RHF RUNTYP=OPTIMIZE DFTTYP=M06-L MAXIT=199 MULT=1 NOSYM=1<br /> ECP=READ $END<br /> $SYSTEM TIMLIM=2850 MWORDS=1750 MEMDDI=20 PARALL=.TRUE. $END<br /> $BASIS GBASIS=CUSTOMNI EXTFIL=.t. $END<br /> $SCF DIRSCF=.TRUE. FDIFF=.f. $END<br /> $STATPT OPTTOL=0.0001 NSTEP=500 HSSEND=.t. $END<br /> $DATA<br /> <br /> An important thing to note is the spacing. In the input deck, there must be 1 space at the beginning of each line of the input deck. If not, the job will fail. Most builders will insert this space anyway, but it helps to double check.<br /> <br /> The end of the input deck is marked by the "$DATA" line.<br /> <br /> =====Job Title Line=====<br /> <br /> The next line of the file is the job title. It can be anthing you wish, however, we have found that to be on the safe side, we avoide using symbols or spaces<br /> <br /> Mo_BDT3<br /> <br /> =====Symmetry Point Group=====<br /> <br /> The next line of the file is the symmetry point group of your molecule. Note that there is no leading space before the point group.<br /> <br /> C1<br /> <br /> =====Coordinates=====<br /> <br /> The next block of text is set aside for the coordinates of the molecule. This can be in internal (or z-matrix) format or cartesian coordinates. Note that there is no leading space before the coordinates. One may use the chemical symbol or the full name of each atom in the molecule. Note that the end of the coordinates is signified by an "$END", which MUST have one space preceding it. The coordinates below do NOT have any basis set information inserted. It is possible to insert basis set information directly into the input file. This is accomplished by obtaining the desired basis set parameters from the EMSL and then inserting them below each relevant atom. An example input file with inserted basis set information will be shown later.<br /> <br /> MOLYBDENUM 42.0 5.7556500000 4.4039600000 16.5808400000<br /> SULFUR 16.0 7.4169700000 3.1956300000 15.2089300000<br /> SULFUR 16.0 4.0966800000 3.2258300000 15.1761100000<br /> SULFUR 16.0 3.9677300000 4.4940500000 18.3266100000<br /> SULFUR 16.0 7.1776900000 3.5815000000 18.4485200000<br /> SULFUR 16.0 4.3776600000 6.2447400000 15.6786900000<br /> SULFUR 16.0 7.5478700000 6.0679800000 16.2223700000<br /> CARBON 6.0 6.4716900000 2.1004800000 14.1902300000<br /> CARBON 6.0 5.0690300000 2.1781400000 14.1080700000<br /> CARBON 6.0 4.8421800000 4.2701300000 19.8855500000<br /> CARBON 6.0 6.1969000000 3.9249600000 19.9397400000<br /> CARBON 6.0 6.8280600000 3.7834200000 21.1913200000<br /> CARBON 6.0 5.7697600000 7.6933500000 17.4241800000<br /> CARBON 6.0 7.2043100000 7.9413600000 17.8281100000<br /> CARBON 6.0 5.5051400000 7.0409700000 14.5903800000<br /> CARBON 6.0 6.8905200000 6.9194700000 14.7626200000<br /> CARBON 6.0 7.7396400000 7.5379800000 13.8285700000<br /> HYDROGEN 1.0 8.8190700000 7.4520600000 13.9252200000<br /> CARBON 6.0 7.2169400000 8.2960300000 12.7704100000<br /> HYDROGEN 1.0 7.8667000000 8.7825100000 12.0575600000<br /> CARBON 6.0 5.8260300000 8.4502300000 12.6467800000<br /> HYDROGEN 1.0 5.4143000000 9.0544300000 11.8493100000<br /> CARBON 6.0 4.9881500000 7.8192300000 13.5528400000<br /> HYDROGEN 1.0 3.9090500000 7.9420000000 13.4583700000<br /> CARBON 6.0 7.1538500000 1.1569600000 13.4143900000<br /> CARBON 6.0 4.4018100000 1.3603900000 13.1919900000<br /> CARBON 6.0 6.4791600000 0.3185500000 12.5353300000<br /> CARBON 6.0 5.0837400000 0.4369500000 12.4084900000<br /> HYDROGEN 1.0 7.0116000000 -0.4099400000 11.9434600000<br /> HYDROGEN 1.0 8.2399000000 1.0702400000 13.4937600000<br /> HYDROGEN 1.0 3.3185600000 1.4368700000 13.0953100000<br /> HYDROGEN 1.0 4.5549800000 -0.1997300000 11.7165200000<br /> CARBON 6.0 6.1105700000 3.9639000000 22.3866100000<br /> CARBON 6.0 4.1216300000 4.4424400000 21.1020100000<br /> HYDROGEN 1.0 7.8732900000 3.5217100000 21.2520500000<br /> CARBON 6.0 4.7606000000 4.2868500000 22.3363800000<br /> HYDROGEN 1.0 6.6064200000 3.8406000000 23.3428500000<br /> HYDROGEN 1.0 4.2065000000 4.4170700000 23.2667100000<br /> HYDROGEN 1.0 3.0674000000 4.6893500000 21.0889000000<br /> HYDROGEN 1.0 7.4249200000 7.7545300000 18.8583200000<br /> HYDROGEN 1.0 7.6651700000 8.9049700000 17.7652100000<br /> HYDROGEN 1.0 5.3324000000 8.6487800000 17.2222700000<br /> HYDROGEN 1.0 5.5015000000 7.1039000000 18.2759400000<br /> $END<br /> <br /> =====Effective Core Potential Data=====<br /> <br /> The effective core potential (ECP) data is entered after the coordinates. It starts with "$ECP", which must be preceded with a space. The atoms of the molecule are listed in the same order as in the coordinates section and the parameters for the ECP are listed after each atom. Note that for any atom that does NOT have an ECP, one must enter "ECP-NONE" or "NONE" after each atom without an ECP.<br /> <br /> $ECP<br /> MO-ECP GEN 28 3<br /> 5 ----- f potential -----<br /> -0.0469492 0 537.9667807 <br /> -20.2080084 1 147.8982938 <br /> -106.2116302 2 45.7358898 <br /> -41.8107368 2 13.2911467 <br /> -4.2054103 2 4.7059961 <br /> 3 ----- s-f potential -----<br /> 2.8063717 0 110.2991760 <br /> 44.5162012 1 23.2014645 <br /> 82.7785227 2 5.3530131 <br /> 4 ----- p-f potential -----<br /> 4.9420876 0 63.2901397 <br /> 25.8604976 1 23.3315302 <br /> 132.4708742 2 24.6759423 <br /> 57.3149794 2 4.6493040 <br /> 5 ----- d-f potential -----<br /> 3.0054591 0 104.4839977 <br /> 26.3637851 1 66.2307245 <br /> 183.3849199 2 39.1283176 <br /> 98.4453068 2 13.1164437 <br /> 22.4901377 2 3.6280263 <br /> S NONE<br /> S NONE<br /> S NONE<br /> S NONE<br /> S NONE<br /> S NONE<br /> C NONE<br /> C NONE<br /> C NONE<br /> C NONE<br /> C NONE<br /> C NONE<br /> C NONE<br /> C NONE<br /> C NONE<br /> C NONE<br /> H NONE<br /> C NONE<br /> H NONE<br /> C NONE<br /> H NONE<br /> C NONE<br /> H NONE<br /> C NONE<br /> C NONE<br /> C NONE<br /> C NONE<br /> H NONE<br /> H NONE<br /> H NONE<br /> H NONE<br /> C NONE<br /> C NONE<br /> H NONE<br /> C NONE<br /> H NONE<br /> H NONE<br /> H NONE<br /> H NONE<br /> H NONE<br /> H NONE<br /> H NONE<br /> $END<br /> <br /> -- [[User:M.Zimmer-De Iuliis|mzd]] 16 November 2009<br /> <br /> ====Using Effective Core Potentials in GAMESS(US)====<br /> <br /> For many metal containing compounds, it is very convenient and time saving to use an effective core potential (ECP) for the core metal electrons, as they are usually not important to the reactivity of the complex or the geometry around the metal. Since GAMESS(US) has a limited number of built-in ECPs, one may want to make GAMESS(US) read an external file that contains the ECP data using the "EXTFIL" keyword in the $GBASIS command line of the input file. In addition, to make GAMESS(US) use this external file, one must copy the "rungms" file and modify it accordingly. The following is a list of instructions with commands that will work from a terminal. One could also use WinSCP to do all of this with a GUI rather than a TUI. <br /> <br /> =====Modifiying rungms to Use Custom Basis Set File=====<br /> 1. Copy "rungms" from /scinet/gpc/Applications/gamess to one's own /scratch/$USER/ directory:<br /> cp /scinet/gpc/Applications/gamess/rungms /scratch/$USER/<br /> <br /> 2. Change to the scratch directory and check to see if "rungms" has copied successfully.<br /> cd /scratch/$USER<br /> ls<br /> <br /> 3. Edit line 147 of the vi. <br /> vi rungms<br /> Move the cursor down to line 147 using the arrow keys. It should say "setenv EXTBAS /dev/null". Using the arrow keys, move the cursor to the first "/" and then hit "i" to insert text. Put the path to your external basis file here. For example, /scratch/$USER/basisset. Then hit "escape". To save the changes and exit vi, type ":" and you should see a colon appear at the bottom of the window. Type "wq" (which should appear at the bottom of the window next to the colon) and then hit enter. Now you are done with vi.<br /> <br /> =====Creating a Custom Basis Set File=====<br /> 1. To create a custom basis set file, you need create a new text document. Our group's common practice is to comment out the first line of this file by inserting an exclamation mark (!) followed by noting the specific basis sets and ECPs that are going to be used for each of the atoms. Let us the molecule Mo(CO)6, Molybdenum hexacarbonyl, as an example. Below is the first line of the the external file, which we will call "CUSTOMMO" (NOTE: you can use any name for the external file that suits you, as long as it has no spaces and is 8 characters or less).<br /> <br /> ! 6-31G on C and O and LANL2D2 ECP on Mo<br /> <br /> 2. The next step is to visit the [https://bse.pnl.gov/bse/portal EMSL Basis Set exchange] and select C and O from the periodic table. Then, on the left of the page, select "6-31G" as the basis set. Finally, make sure the output is in GAMESS(US) format using the drop-down menu and then click "get basis set".<br /> <br /> [[File:C_O_6_31G_basisset.JPG|centre]]<br /> <br /> 3. A new window should appear with text in it. For our example case, the text looks like this:<br /> <br /> ! 6-31G EMSL Basis Set Exchange Library 10/13/09 11:12 AM<br /> ! Elements References<br /> ! -------- ----------<br /> ! H - He: W.J. Hehre, R. Ditchfield and J.A. Pople, J. Chem. Phys. 56,<br /> ! Li - Ne: 2257 (1972). Note: Li and B come from J.D. Dill and J.A.<br /> ! Pople, J. Chem. Phys. 62, 2921 (1975).<br /> ! Na - Ar: M.M. Francl, W.J. Petro, W.J. Hehre, J.S. Binkley, M.S. Gordon,<br /> ! D.J. DeFrees and J.A. Pople, J. Chem. Phys. 77, 3654 (1982)<br /> ! K - Zn: V. Rassolov, J.A. Pople, M. Ratner and T.L. Windus, J. Chem. Phys.<br /> ! 109, 1223 (1998)<br /> ! Note: He and Ne are unpublished basis sets taken from the Gaussian<br /> ! program<br /> ! <br /> $DATA<br /><br /> CARBON<br /> S 6<br /> 1 3047.5249000 0.0018347 <br /> 2 457.3695100 0.0140373 <br /> 3 103.9486900 0.0688426 <br /> 4 29.2101550 0.2321844 <br /> 5 9.2866630 0.4679413 <br /> 6 3.1639270 0.3623120 <br /> L 3<br /> 1 7.8682724 -0.1193324 0.0689991 <br /> 2 1.8812885 -0.1608542 0.3164240 <br /> 3 0.5442493 1.1434564 0.7443083 <br /> L 1<br /> 1 0.1687144 1.0000000 1.0000000<br /> <br /> OXYGEN<br /> S 6<br /> 1 5484.6717000 0.0018311 <br /> 2 825.2349500 0.0139501 <br /> 3 188.0469600 0.0684451 <br /> 4 52.9645000 0.2327143 <br /> 5 16.8975700 0.4701930 <br /> 6 5.7996353 0.3585209 <br /> L 3<br /> 1 15.5396160 -0.1107775 0.0708743 <br /> 2 3.5999336 -0.1480263 0.3397528 <br /> 3 1.0137618 1.1307670 0.7271586 <br /> L 1<br /> 1 0.2700058 1.0000000 1.0000000 <br /> $END<br /> <br /> 3. Now, copy and paste the text between the $DATA and $END headings onto our external text file, CUSTOMMO. We also need to change the change the name of each element to it symbol in the periodic table. Finally, we need to add the name of the external file next to the element symbol, separated by one space. Note that there should be a blank line separating the basis set information and the first, commented-out line (The line starting with the '!'). The CUSTOMMO should look like this:<br /> <br /> ! 6-31G on C and O and LANL2D2 ECP on Mo<br /><br /> C CUSTOMMO<br /> S 6<br /> 1 3047.5249000 0.0018347 <br /> 2 457.3695100 0.0140373 <br /> 3 103.9486900 0.0688426 <br /> 4 29.2101550 0.2321844 <br /> 5 9.2866630 0.4679413 <br /> 6 3.1639270 0.3623120 <br /> L 3<br /> 1 7.8682724 -0.1193324 0.0689991 <br /> 2 1.8812885 -0.1608542 0.3164240 <br /> 3 0.5442493 1.1434564 0.7443083 <br /> L 1<br /> 1 0.1687144 1.0000000 1.0000000<br /> <br /> O CUSTOMMO<br /> S 6<br /> 1 5484.6717000 0.0018311 <br /> 2 825.2349500 0.0139501 <br /> 3 188.0469600 0.0684451 <br /> 4 52.9645000 0.2327143 <br /> 5 16.8975700 0.4701930 <br /> 6 5.7996353 0.3585209 <br /> L 3<br /> 1 15.5396160 -0.1107775 0.0708743 <br /> 2 3.5999336 -0.1480263 0.3397528 <br /> 3 1.0137618 1.1307670 0.7271586 <br /> L 1<br /> 1 0.2700058 1.0000000 1.0000000 <br /> <br /> 4. Repeat Step 3 above but choose Mo and select the LANL2DZ ECP instead. A new window will pop up with the basis set information as well as the ECP data we need, since we specified the LANL2DZ '''ECP'''. The ECP data is not inserted into the external file, rather it is placed into the input file itself (More on this later). <br /> <br /> [[File:Mo_LANL2DZ_basisset.JPG|centre]]<br /> <br /> 5. After copying the molybdenum basis set information, your fiished external basis set file should look like this:<br /> ! 6-31G on C and O and LANL2D2 ECP on Mo<br /><br /> C CUSTOMMO<br /> S 6<br /> 1 3047.5249000 0.0018347 <br /> 2 457.3695100 0.0140373 <br /> 3 103.9486900 0.0688426 <br /> 4 29.2101550 0.2321844 <br /> 5 9.2866630 0.4679413 <br /> 6 3.1639270 0.3623120 <br /> L 3<br /> 1 7.8682724 -0.1193324 0.0689991 <br /> 2 1.8812885 -0.1608542 0.3164240 <br /> 3 0.5442493 1.1434564 0.7443083 <br /> L 1<br /> 1 0.1687144 1.0000000 1.0000000<br /> <br /> O CUSTOMMO<br /> S 6<br /> 1 5484.6717000 0.0018311 <br /> 2 825.2349500 0.0139501 <br /> 3 188.0469600 0.0684451 <br /> 4 52.9645000 0.2327143 <br /> 5 16.8975700 0.4701930 <br /> 6 5.7996353 0.3585209 <br /> L 3<br /> 1 15.5396160 -0.1107775 0.0708743 <br /> 2 3.5999336 -0.1480263 0.3397528 <br /> 3 1.0137618 1.1307670 0.7271586 <br /> L 1<br /> 1 0.2700058 1.0000000 1.0000000<br /> <br /> Mo CUSTOMO<br /> S 3<br /> 1 2.3610000 -0.9121760 <br /> 2 1.3090000 1.1477453 <br /> 3 0.4500000 0.6097109 <br /> S 4<br /> 1 2.3610000 0.8139259 <br /> 2 1.3090000 -1.1360084 <br /> 3 0.4500000 -1.1611592 <br /> 4 0.1681000 1.0064786 <br /> S 1<br /> 1 0.0423000 1.0000000 <br /> P 3<br /> 1 4.8950000 -0.0908258 <br /> 2 1.0440000 0.7042899 <br /> 3 0.3877000 0.3973179 <br /> P 2<br /> 1 0.4995000 -0.1081945 <br /> 2 0.0780000 1.0368093 <br /> P 1<br /> 1 0.0247000 1.0000000 <br /> D 3<br /> 1 2.9930000 0.0527063 <br /> 2 1.0630000 0.5003907 <br /> 3 0.3721000 0.5794024 <br /> D 1<br /> 1 0.1178000 1.0000000<br /> <br /> -- [[User:M.Zimmer-De Iuliis|mzd]] 13 October 2009<br /> <br /> ==Climate Modelling==<br /> <br /> ==Medicine/Bio==<br /> <br /> ==High Energy Physics==<br /> <br /> ==Structural Biology==<br /> Molecular simulation of proteins, lipids, carbohydrates, and other biologically relevant molecules.<br /> ===Molecular Dynamics (MD) simulation===<br /> ====GROMACS====<br /> Please refer to the [[gromacs|GROMACS]] page<br /> ====AMBER====<br /> Please refer to the [[amber|AMBER]] page<br /> ====NAMD====<br /> NAMD is one of the better scaling MD packages out there. With sufficiently large systems, it is able to scale to hundreds or thousands of cores on Scinet. Below are details for compiling and running NAMD on Scinet.<br /> <br /> More information regarding performance and different compile options coming soon...<br /> <br /> =====Compiling NAMD for GPC=====<br /> Ensure the proper compiler/mpi modules are loaded.<br /> <source lang="sh"><br /> module load intel<br /> module load openmpi/1.3.3-intel-v11.0-ofed<br /> </source><br /> <br /> '''Compile Charm++ and NAMD'''<br /> <source lang="sh"><br /> #Unpack source files and get required support libraries<br /> tar -xzf NAMD_2.7b1_Source.tar.gz<br /> cd NAMD_2.7b1_Source<br /> tar -xf charm-6.1.tar<br /> wget http://www.ks.uiuc.edu/Research/namd/libraries/fftw-linux-x86_64.tar.gz<br /> wget http://www.ks.uiuc.edu/Research/namd/libraries/tcl-linux-x86_64.tar.gz<br /> tar -xzf fftw-linux-x86_64.tar.gz; mv linux-x86_64 fftw<br /> tar -xzf tcl-linux-x86_64.tar.gz; mv linux-x86_64 tcl<br /> #Compile Charm++<br /> cd charm-6.1<br /> ./build charm++ mpi-linux-x86_64 icc --basedir /scinet/gpc/mpi/openmpi/1.3.3-intel-v11.0-ofed/ --no-shared -O -DCMK_OPTIMIZE=1<br /> cd ..<br /> #Compile NAMD. <br /> #Edit arch/Linux-x86_64-icc.arch and add "-lmpi" to the end of the CXXOPTS and COPTS line.<br /> #Make a builds directory if you want different versions of NAMD compiled at the same time.<br /> mkdir builds<br /> ./config builds/Linux-x86_64-icc --charm-arch mpi-linux-x86_64-icc<br /> cd builds/Linux-x86_64-icc/<br /> make -j4 namd2 # Adjust value of j as desired to specify number of simultaneous make targets. <br /> </source><br /> --[[User:Cmadill|Cmadill]] 16:18, 27 August 2009 (UTC)<br /> <br /> =====Running Fortran=====<br /> On the development nodes, there is an old gcc. The associated libraries are not on the compute nodes. Ensure the line:<br /> <br /> module load gcc<br /> <br /> is in your .bashrc file.<br /> <br /> ====LAMMPS====<br /> [[Image:StrongScalingLAMMPS.png|thumb|320px|Strong scaling test on GPC with OpenMPI and IntelMPI on Ethernet and InfiniBand]]<br /> [[Image:WeakScalingLAMMPS.png|thumb|320px|Weak scaling test on GPC with OpenMPI and IntelMPI on Ethernet and InfiniBand]]<br /> Results from strong scaling tests for LAMMPS using EAM potentials on GPC are shown in the graph above. The test simulation ran 500 timesteps for 4,000,000 atoms.<br /> Results from weak scaling tests for LAMMPS using EAM potentials on GPC are shown in the graph above. The test simulation ran 500 timesteps for 32,000 atoms per processor.<br /> <br /> ===Monte Carlo (MC) simulation===</div>

Jchu https://oldwiki.scinet.utoronto.ca/index.php?title=User_Codes&diff=874 User Codes 2010-02-02T18:51:37Z

<p>Jchu: /* LAMMPS */</p> <hr /> <div>__FORCETOC__<br /> <br /> ==Astrophysics==<br /> <br /> ===Athena (explicit, uniform grid MHD code)===<br /> <br /> [[Image:StrongScalingAthenaGPC.png|thumb|right|320px|Athena scaling on GPC with OpenMPI and MVAPICH2 on GigE, and OpenMPI on InfiniBand]]<br /> <br /> [http://www.astro.princeton.edu/~jstone/athena.html Athena] is a straightforward C code which doesn't use a lot of libraries so it is pretty straightforward to build and compile on new machines. <br /> <br /> It encapsulates its compiler flags, etc in an <tt>Makeoptions.in</tt> file which is then processed by <tt>configure</tt>. I've used the following additions to <tt>Makeoptions.in</tt> on TCS and GPC:<br /> <br /> <source lang="sh"><br /> ifeq ($(MACHINE),scinettcs)<br /> CC = mpcc_r<br /> LDR = mpcc_r<br /> OPT = -O5 -q64 -qarch=pwr6 -qtune=pwr6 -qcache=auto -qlargepage -qstrict<br /> MPIINC =<br /> MPILIB =<br /> CFLAGS = $(OPT)<br /> LIB = -ldl -lm<br /> else<br /> ifeq ($(MACHINE),scinetgpc)<br /> CC = mpicc<br /> LDR = mpicc<br /> OPT = -O3<br /> MPIINC =<br /> MPILIB =<br /> CFLAGS = $(OPT)<br /> LIB = -lm<br /> else<br /> ...<br /> endif<br /> endif<br /> </source><br /> It performs quite well on the GPC, scaling extremely well even on a strong scaling test out to about 256 cores (32 nodes) on Gigabit ethernet, and performing beautifully on InfiniBand out to 512 cores (64 nodes). <br /> <br /> <br /> -- [[User:Ljdursi|ljdursi]] 19:20, 13 August 2009 (UTC)<br /> <br /> ===FLASH3 (Adaptive Mesh reactive hydrodynamics; explict hydro/MHD)===<br /> <br /> [[Image:weak-scaling-example.png|thumb|right|320px|Weak scaling test of the 2d sod problem on both the GPC and TCS. The results are actually somewhat faster on the GPC; in both cases (weak) scaling is very good out at least to 256 cores]]<br /> <br /> [http://flash.uchicago.edu FLASH] encapsulates its machine-dependant information in the <tt>FLASH3/sites</tt> directory. For the GPC, you'll have to<br /> <pre><br /> module load intel<br /> module load openmpi<br /> module load hdf5/183-v16-openmpi<br /> </pre><br /> <br /> and with that, the following file (<tt>sites/scinetgpc/Makefile.h</tt>) works for me:<br /> <source lang="sh"><br /> ## Must do module load hdf5/183-v16-openmpi<br /> HDF5_PATH = ${SCINET_HDF5_BASE}<br /> ZLIB_PATH = /usr/local<br /> <br /> #----------------------------------------------------------------------------<br /> # Compiler and linker commands<br /> #<br /> # We use the f90 compiler as the linker, so some C libraries may explicitly<br /> # need to be added into the link line.<br /> #----------------------------------------------------------------------------<br /> <br /> ## modules will put the right mpi in our path<br /> FCOMP = mpif77<br /> CCOMP = mpicc<br /> CPPCOMP = mpiCC<br /> LINK = mpif77<br /> <br /> #----------------------------------------------------------------------------<br /> # Compilation flags<br /> #<br /> # Three sets of compilation/linking flags are defined: one for optimized<br /> # code, one for testing, and one for debugging. The default is to use the <br /> # _OPT version. Specifying -debug to setup will pick the _DEBUG version,<br /> # these should enable bounds checking. Specifying -test is used for <br /> # flash_test, and is set for quick code generation, and (sometimes) <br /> # profiling. The Makefile generated by setup will assign the generic token <br /> # (ex. FFLAGS) to the proper set of flags (ex. FFLAGS_OPT).<br /> #----------------------------------------------------------------------------<br /> <br /> FFLAGS_OPT = -c -r8 -i4 -O3 -xSSE4.2<br /> FFLAGS_DEBUG = -c -g -r8 -i4 -O0<br /> FFLAGS_TEST = -c -r8 -i4<br /> <br /> <br /> # if we are using HDF5, we need to specify the path to the include files<br /> CFLAGS_HDF5 = -I${HDF5_PATH}/include<br /> <br /> CFLAGS_OPT = -c -O3 -xSSE4.2<br /> CFLAGS_TEST = -c -O2 <br /> CFLAGS_DEBUG = -c -g <br /> <br /> MDEFS = <br /> <br /> .SUFFIXES: .o .c .f .F .h .fh .F90 .f90<br /> <br /> #----------------------------------------------------------------------------<br /> # Linker flags<br /> #<br /> # There is a seperate version of the linker flags for each of the _OPT, <br /> # _DEBUG, and _TEST cases.<br /> #----------------------------------------------------------------------------<br /> <br /> LFLAGS_OPT = -o<br /> LFLAGS_TEST = -o<br /> LFLAGS_DEBUG = -g -o<br /> <br /> MACHOBJ = <br /> <br /> <br /> MV = mv -f<br /> AR = ar -r<br /> RM = rm -f<br /> CD = cd<br /> RL = ranlib<br /> ECHO = echo<br /> </source><br /> <br /> <br /> -- [[User:Ljdursi|ljdursi]] 22:11, 13 August 2009 (UTC)<br /> <br /> ==Aeronautics==<br /> <br /> ==Chemistry==<br /> <br /> ===GAMESS (US)===<br /> The GAMESS version January 12, 2009 R3 was built using the Intel v11.1 compilers and v3.2.2 MPI library, according to the instructions in http://software.intel.com/en-us/articles/building-gamess-with-intel-compilers-intel-mkl-and-intel-mpi-on-linux/<br /> <br /> The required build scripts - comp, compall, lked - and run script - rungms - were modified to account for our own installation. In order to build GAMESS one first must ensure that the intel and intelmpi modules are loaded ("module load intel intelmpi"). This applies to running GAMESS as well.<br /> The module "gamess" must also be loaded in order to run GAMESS ("module load gamess").<br /> <br /> The modified scripts are in the file /scinet/gpc/src/gamess-on-scinet.tar.gz<br /> <br /> ====Running GAMESS====<br /> - Make sure the directory /scratch/$USER/gamess-scratch exists (the $SCINET_RUNGMS script will create it if it does not exist)<br /> <br /> - Make sure the modules: intel, intelmpi, gamess are loaded (in your .bashrc: "module load intel intelmpi gamess").<br /> <br /> - Create a torque script to run GAMESS. Here is an example:<br /> <br /> - The GAMESS executable is in $SCINET_GAMESS_HOME/gamess.00.x<br /> - The rungms script is in $SCINET_GAMESS_HOME/rungms (actually it is $SCINET_RUNGMS)<br /> <br /> - For IB multinode runs, use the $SCINET_RUNGMS_IB script<br /> <br /> - The rungms script takes 4 arguments: input file, executable number, number of compute processes, processors per node<br /> <br /> For example, in order to run with the input file /scratch/$USER/gamesstest01, on 8 cpus, and the default version (00)<br /> of the executable on a machine with 8 cores:<br /> <br /> # load the gamess module in .bashrc<br /> module load gamess <br /> <br /> # run the program<br /> $SCINET_RUNGMS /scratch/$USER/gamesstest01 00 8 8<br /> <br /> Here is a sample torque script for running a GAMESS calculation, on a single 8-core node:<br /> <br /> <source lang="bash"><br /> #!/bin/bash<br /> #PBS -l nodes=1:ppn=8,walltime=48:00:00,os=centos53computeA<br /> #PBS -N gamessjob<br /> <br /> ## To submit type: qsub gms.sh<br /> <br /> # If not an interactive job (i.e. -I), then cd into the directory where<br /> # I typed qsub.<br /> if [ "$PBS_ENVIRONMENT" != "PBS_INTERACTIVE" ]; then<br /> if [ -n "$PBS_O_WORKDIR" ]; then<br /> cd $PBS_O_WORKDIR<br /> fi<br /> fi<br /> <br /> # the input file is typically named something like "gamesjob.inp"<br /> # so the script will be run like "$SCINET_RUNGMS gamessjob 00 8 8"<br /> <br /> # load the gamess module if not in .bashrc already<br /> # actually, it MUST be in .bashrc<br /> # module load gamess<br /> <br /> # run the program<br /> <br /> $SCINET_RUNGMS gamessjob 00 8 8<br /> </source><br /> <br /> Here is a similar script, but this one uses 2 InfiniBand-connected nodes,<br /> and runs the appropriate $SCINET_RUNGMS_IB script to actually run the job:<br /> <br /> <source lang="bash"><br /> #!/bin/bash<br /> #PBS -l nodes=2:ib:ppn=8,walltime=48:00:00,os=centos53computeA<br /> #PBS -N gamessjob<br /> <br /> ## To submit type: qsub gmsib.sh<br /> <br /> # If not an interactive job (i.e. -I), then cd into the directory where<br /> # I typed qsub.<br /> if [ "$PBS_ENVIRONMENT" != "PBS_INTERACTIVE" ]; then<br /> if [ -n "$PBS_O_WORKDIR" ]; then<br /> cd $PBS_O_WORKDIR<br /> fi<br /> fi<br /> <br /> # the input file is typically named something like "gamesjob.inp"<br /> # so the script will be run like "$SCINET_RUNGMS gamessjob 00 8 8"<br /> <br /> # load the gamess module if not in .bashrc already<br /> # actually, it MUST be in .bashrc<br /> # module load gamess<br /> <br /> # This script requests InfiniBand-connected nodes (:ib above)<br /> # so it must run with the IB version of the rungms script,<br /> # $SCINET_RUNGMS_IB<br /> <br /> $SCINET_RUNGMS_IB gamessjob 00 16 8<br /> </source><br /> <br /> <br /> -- [[User:Dgruner|dgruner]] 5 October 2009<br /> <br /> ===Tips from the Fekl Lab===<br /> <br /> Through trial and error, we have found a few useful things that we would like to share:<br /> <br /> 1. Two very useful, open-source programs for visualization of output files from GAMESS(US) and for generation of input files are [http://www.scl.ameslab.gov/MacMolPlt/ MacMolPlt]and [http://avogadro.openmolecules.net/wiki/Main_Page Avogadro]. The are available for UNIX/LINUX, Windows and Mac based machines, HOWEVER: any input files that we have generated with these programs on a Windows-based machine do not run on Mac based machines. We don't know why.<br /> <br /> 2. [http://winscp.net/eng/index.php WinSCP] is a very useful tool that has a graphical user interface for moving files from a local machine to SCINET and vice versa. It also has text editing capabilities.<br /> <br /> 3. The [https://bse.pnl.gov/bse/portal ESML Basis Set Exchange] is an excellent source for custom basis set or effective core potential parameters. Make sure that you specify "Gamess-US" in the format drop-down box.<br /> <br /> 4. The commercial program [http://www.chemcraftprog.com/ ChemCraft] is a highly useful visualization program that has the ability to edit molecules in a very similar fashion to GaussView. It can also be customized to build GAMESS(US) input files.<br /> <br /> ====Anatomy of a GAMESS(US) Input File with Basis Set Info in an External File====<br /> <br /> $CONTRL SCFTYP=RHF RUNTYP=OPTIMIZE DFTTYP=M06-L MAXIT=199 MULT=1 NOSYM=1<br /> ECP=READ $END<br /> $SYSTEM TIMLIM=525600 MWORDS=1750 PARALL=.TRUE. $END<br /> $BASIS GBASIS=CUSTOMNI EXTFIL=.t. $END<br /> $SCF DIRSCF=.TRUE. FDIFF=.f. $END<br /> $STATPT OPTTOL=0.0001 NSTEP=500 HSSEND=.t. $END<br /> $DATA<br /> Mo_BDT3<br /> C1<br /> MOLYBDENUM 42.0 5.7556500000 4.4039600000 16.5808400000<br /> SULFUR 16.0 7.4169700000 3.1956300000 15.2089300000<br /> SULFUR 16.0 4.0966800000 3.2258300000 15.1761100000<br /> SULFUR 16.0 3.9677300000 4.4940500000 18.3266100000<br /> SULFUR 16.0 7.1776900000 3.5815000000 18.4485200000<br /> SULFUR 16.0 4.3776600000 6.2447400000 15.6786900000<br /> SULFUR 16.0 7.5478700000 6.0679800000 16.2223700000<br /> CARBON 6.0 6.4716900000 2.1004800000 14.1902300000<br /> CARBON 6.0 5.0690300000 2.1781400000 14.1080700000<br /> CARBON 6.0 4.8421800000 4.2701300000 19.8855500000<br /> CARBON 6.0 6.1969000000 3.9249600000 19.9397400000<br /> CARBON 6.0 6.8280600000 3.7834200000 21.1913200000<br /> CARBON 6.0 5.7697600000 7.6933500000 17.4241800000<br /> CARBON 6.0 7.2043100000 7.9413600000 17.8281100000<br /> CARBON 6.0 5.5051400000 7.0409700000 14.5903800000<br /> CARBON 6.0 6.8905200000 6.9194700000 14.7626200000<br /> CARBON 6.0 7.7396400000 7.5379800000 13.8285700000<br /> HYDROGEN 1.0 8.8190700000 7.4520600000 13.9252200000<br /> CARBON 6.0 7.2169400000 8.2960300000 12.7704100000<br /> HYDROGEN 1.0 7.8667000000 8.7825100000 12.0575600000<br /> CARBON 6.0 5.8260300000 8.4502300000 12.6467800000<br /> HYDROGEN 1.0 5.4143000000 9.0544300000 11.8493100000<br /> CARBON 6.0 4.9881500000 7.8192300000 13.5528400000<br /> HYDROGEN 1.0 3.9090500000 7.9420000000 13.4583700000<br /> CARBON 6.0 7.1538500000 1.1569600000 13.4143900000<br /> CARBON 6.0 4.4018100000 1.3603900000 13.1919900000<br /> CARBON 6.0 6.4791600000 0.3185500000 12.5353300000<br /> CARBON 6.0 5.0837400000 0.4369500000 12.4084900000<br /> HYDROGEN 1.0 7.0116000000 -0.4099400000 11.9434600000<br /> HYDROGEN 1.0 8.2399000000 1.0702400000 13.4937600000<br /> HYDROGEN 1.0 3.3185600000 1.4368700000 13.0953100000<br /> HYDROGEN 1.0 4.5549800000 -0.1997300000 11.7165200000<br /> CARBON 6.0 6.1105700000 3.9639000000 22.3866100000<br /> CARBON 6.0 4.1216300000 4.4424400000 21.1020100000<br /> HYDROGEN 1.0 7.8732900000 3.5217100000 21.2520500000<br /> CARBON 6.0 4.7606000000 4.2868500000 22.3363800000<br /> HYDROGEN 1.0 6.6064200000 3.8406000000 23.3428500000<br /> HYDROGEN 1.0 4.2065000000 4.4170700000 23.2667100000<br /> HYDROGEN 1.0 3.0674000000 4.6893500000 21.0889000000<br /> HYDROGEN 1.0 7.4249200000 7.7545300000 18.8583200000<br /> HYDROGEN 1.0 7.6651700000 8.9049700000 17.7652100000<br /> HYDROGEN 1.0 5.3324000000 8.6487800000 17.2222700000<br /> HYDROGEN 1.0 5.5015000000 7.1039000000 18.2759400000<br /> $END<br /> $ECP<br /> MO-ECP GEN 28 3<br /> 5 ----- f potential -----<br /> -0.0469492 0 537.9667807 <br /> -20.2080084 1 147.8982938 <br /> -106.2116302 2 45.7358898 <br /> -41.8107368 2 13.2911467 <br /> -4.2054103 2 4.7059961 <br /> 3 ----- s-f potential -----<br /> 2.8063717 0 110.2991760 <br /> 44.5162012 1 23.2014645 <br /> 82.7785227 2 5.3530131 <br /> 4 ----- p-f potential -----<br /> 4.9420876 0 63.2901397 <br /> 25.8604976 1 23.3315302 <br /> 132.4708742 2 24.6759423 <br /> 57.3149794 2 4.6493040 <br /> 5 ----- d-f potential -----<br /> 3.0054591 0 104.4839977 <br /> 26.3637851 1 66.2307245 <br /> 183.3849199 2 39.1283176 <br /> 98.4453068 2 13.1164437 <br /> 22.4901377 2 3.6280263 <br /> S NONE<br /> S NONE<br /> S NONE<br /> S NONE<br /> S NONE<br /> S NONE<br /> C NONE<br /> C NONE<br /> C NONE<br /> C NONE<br /> C NONE<br /> C NONE<br /> C NONE<br /> C NONE<br /> C NONE<br /> C NONE<br /> H NONE<br /> C NONE<br /> H NONE<br /> C NONE<br /> H NONE<br /> C NONE<br /> H NONE<br /> C NONE<br /> C NONE<br /> C NONE<br /> C NONE<br /> H NONE<br /> H NONE<br /> H NONE<br /> H NONE<br /> C NONE<br /> C NONE<br /> H NONE<br /> C NONE<br /> H NONE<br /> H NONE<br /> H NONE<br /> H NONE<br /> H NONE<br /> H NONE<br /> H NONE<br /> $END<br /> <br /> =====The Input Deck=====<br /> <br /> Below is the input deck. It is where you tell GAMESS(US) what job type to execute and where all you individual parameters are entered for your specific job type. The example input deck below is for a geometry optimization and frequency calculation. This input deck is equivalent to the Gaussian job with "opt" and "freq" in the route section.<br /> <br /> $CONTRL SCFTYP=RHF RUNTYP=OPTIMIZE DFTTYP=M06-L MAXIT=199 MULT=1 NOSYM=1<br /> ECP=READ $END<br /> $SYSTEM TIMLIM=2850 MWORDS=1750 MEMDDI=20 PARALL=.TRUE. $END<br /> $BASIS GBASIS=CUSTOMNI EXTFIL=.t. $END<br /> $SCF DIRSCF=.TRUE. FDIFF=.f. $END<br /> $STATPT OPTTOL=0.0001 NSTEP=500 HSSEND=.t. $END<br /> $DATA<br /> <br /> An important thing to note is the spacing. In the input deck, there must be 1 space at the beginning of each line of the input deck. If not, the job will fail. Most builders will insert this space anyway, but it helps to double check.<br /> <br /> The end of the input deck is marked by the "$DATA" line.<br /> <br /> =====Job Title Line=====<br /> <br /> The next line of the file is the job title. It can be anthing you wish, however, we have found that to be on the safe side, we avoide using symbols or spaces<br /> <br /> Mo_BDT3<br /> <br /> =====Symmetry Point Group=====<br /> <br /> The next line of the file is the symmetry point group of your molecule. Note that there is no leading space before the point group.<br /> <br /> C1<br /> <br /> =====Coordinates=====<br /> <br /> The next block of text is set aside for the coordinates of the molecule. This can be in internal (or z-matrix) format or cartesian coordinates. Note that there is no leading space before the coordinates. One may use the chemical symbol or the full name of each atom in the molecule. Note that the end of the coordinates is signified by an "$END", which MUST have one space preceding it. The coordinates below do NOT have any basis set information inserted. It is possible to insert basis set information directly into the input file. This is accomplished by obtaining the desired basis set parameters from the EMSL and then inserting them below each relevant atom. An example input file with inserted basis set information will be shown later.<br /> <br /> MOLYBDENUM 42.0 5.7556500000 4.4039600000 16.5808400000<br /> SULFUR 16.0 7.4169700000 3.1956300000 15.2089300000<br /> SULFUR 16.0 4.0966800000 3.2258300000 15.1761100000<br /> SULFUR 16.0 3.9677300000 4.4940500000 18.3266100000<br /> SULFUR 16.0 7.1776900000 3.5815000000 18.4485200000<br /> SULFUR 16.0 4.3776600000 6.2447400000 15.6786900000<br /> SULFUR 16.0 7.5478700000 6.0679800000 16.2223700000<br /> CARBON 6.0 6.4716900000 2.1004800000 14.1902300000<br /> CARBON 6.0 5.0690300000 2.1781400000 14.1080700000<br /> CARBON 6.0 4.8421800000 4.2701300000 19.8855500000<br /> CARBON 6.0 6.1969000000 3.9249600000 19.9397400000<br /> CARBON 6.0 6.8280600000 3.7834200000 21.1913200000<br /> CARBON 6.0 5.7697600000 7.6933500000 17.4241800000<br /> CARBON 6.0 7.2043100000 7.9413600000 17.8281100000<br /> CARBON 6.0 5.5051400000 7.0409700000 14.5903800000<br /> CARBON 6.0 6.8905200000 6.9194700000 14.7626200000<br /> CARBON 6.0 7.7396400000 7.5379800000 13.8285700000<br /> HYDROGEN 1.0 8.8190700000 7.4520600000 13.9252200000<br /> CARBON 6.0 7.2169400000 8.2960300000 12.7704100000<br /> HYDROGEN 1.0 7.8667000000 8.7825100000 12.0575600000<br /> CARBON 6.0 5.8260300000 8.4502300000 12.6467800000<br /> HYDROGEN 1.0 5.4143000000 9.0544300000 11.8493100000<br /> CARBON 6.0 4.9881500000 7.8192300000 13.5528400000<br /> HYDROGEN 1.0 3.9090500000 7.9420000000 13.4583700000<br /> CARBON 6.0 7.1538500000 1.1569600000 13.4143900000<br /> CARBON 6.0 4.4018100000 1.3603900000 13.1919900000<br /> CARBON 6.0 6.4791600000 0.3185500000 12.5353300000<br /> CARBON 6.0 5.0837400000 0.4369500000 12.4084900000<br /> HYDROGEN 1.0 7.0116000000 -0.4099400000 11.9434600000<br /> HYDROGEN 1.0 8.2399000000 1.0702400000 13.4937600000<br /> HYDROGEN 1.0 3.3185600000 1.4368700000 13.0953100000<br /> HYDROGEN 1.0 4.5549800000 -0.1997300000 11.7165200000<br /> CARBON 6.0 6.1105700000 3.9639000000 22.3866100000<br /> CARBON 6.0 4.1216300000 4.4424400000 21.1020100000<br /> HYDROGEN 1.0 7.8732900000 3.5217100000 21.2520500000<br /> CARBON 6.0 4.7606000000 4.2868500000 22.3363800000<br /> HYDROGEN 1.0 6.6064200000 3.8406000000 23.3428500000<br /> HYDROGEN 1.0 4.2065000000 4.4170700000 23.2667100000<br /> HYDROGEN 1.0 3.0674000000 4.6893500000 21.0889000000<br /> HYDROGEN 1.0 7.4249200000 7.7545300000 18.8583200000<br /> HYDROGEN 1.0 7.6651700000 8.9049700000 17.7652100000<br /> HYDROGEN 1.0 5.3324000000 8.6487800000 17.2222700000<br /> HYDROGEN 1.0 5.5015000000 7.1039000000 18.2759400000<br /> $END<br /> <br /> =====Effective Core Potential Data=====<br /> <br /> The effective core potential (ECP) data is entered after the coordinates. It starts with "$ECP", which must be preceded with a space. The atoms of the molecule are listed in the same order as in the coordinates section and the parameters for the ECP are listed after each atom. Note that for any atom that does NOT have an ECP, one must enter "ECP-NONE" or "NONE" after each atom without an ECP.<br /> <br /> $ECP<br /> MO-ECP GEN 28 3<br /> 5 ----- f potential -----<br /> -0.0469492 0 537.9667807 <br /> -20.2080084 1 147.8982938 <br /> -106.2116302 2 45.7358898 <br /> -41.8107368 2 13.2911467 <br /> -4.2054103 2 4.7059961 <br /> 3 ----- s-f potential -----<br /> 2.8063717 0 110.2991760 <br /> 44.5162012 1 23.2014645 <br /> 82.7785227 2 5.3530131 <br /> 4 ----- p-f potential -----<br /> 4.9420876 0 63.2901397 <br /> 25.8604976 1 23.3315302 <br /> 132.4708742 2 24.6759423 <br /> 57.3149794 2 4.6493040 <br /> 5 ----- d-f potential -----<br /> 3.0054591 0 104.4839977 <br /> 26.3637851 1 66.2307245 <br /> 183.3849199 2 39.1283176 <br /> 98.4453068 2 13.1164437 <br /> 22.4901377 2 3.6280263 <br /> S NONE<br /> S NONE<br /> S NONE<br /> S NONE<br /> S NONE<br /> S NONE<br /> C NONE<br /> C NONE<br /> C NONE<br /> C NONE<br /> C NONE<br /> C NONE<br /> C NONE<br /> C NONE<br /> C NONE<br /> C NONE<br /> H NONE<br /> C NONE<br /> H NONE<br /> C NONE<br /> H NONE<br /> C NONE<br /> H NONE<br /> C NONE<br /> C NONE<br /> C NONE<br /> C NONE<br /> H NONE<br /> H NONE<br /> H NONE<br /> H NONE<br /> C NONE<br /> C NONE<br /> H NONE<br /> C NONE<br /> H NONE<br /> H NONE<br /> H NONE<br /> H NONE<br /> H NONE<br /> H NONE<br /> H NONE<br /> $END<br /> <br /> -- [[User:M.Zimmer-De Iuliis|mzd]] 16 November 2009<br /> <br /> ====Using Effective Core Potentials in GAMESS(US)====<br /> <br /> For many metal containing compounds, it is very convenient and time saving to use an effective core potential (ECP) for the core metal electrons, as they are usually not important to the reactivity of the complex or the geometry around the metal. Since GAMESS(US) has a limited number of built-in ECPs, one may want to make GAMESS(US) read an external file that contains the ECP data using the "EXTFIL" keyword in the $GBASIS command line of the input file. In addition, to make GAMESS(US) use this external file, one must copy the "rungms" file and modify it accordingly. The following is a list of instructions with commands that will work from a terminal. One could also use WinSCP to do all of this with a GUI rather than a TUI. <br /> <br /> =====Modifiying rungms to Use Custom Basis Set File=====<br /> 1. Copy "rungms" from /scinet/gpc/Applications/gamess to one's own /scratch/$USER/ directory:<br /> cp /scinet/gpc/Applications/gamess/rungms /scratch/$USER/<br /> <br /> 2. Change to the scratch directory and check to see if "rungms" has copied successfully.<br /> cd /scratch/$USER<br /> ls<br /> <br /> 3. Edit line 147 of the vi. <br /> vi rungms<br /> Move the cursor down to line 147 using the arrow keys. It should say "setenv EXTBAS /dev/null". Using the arrow keys, move the cursor to the first "/" and then hit "i" to insert text. Put the path to your external basis file here. For example, /scratch/$USER/basisset. Then hit "escape". To save the changes and exit vi, type ":" and you should see a colon appear at the bottom of the window. Type "wq" (which should appear at the bottom of the window next to the colon) and then hit enter. Now you are done with vi.<br /> <br /> =====Creating a Custom Basis Set File=====<br /> 1. To create a custom basis set file, you need create a new text document. Our group's common practice is to comment out the first line of this file by inserting an exclamation mark (!) followed by noting the specific basis sets and ECPs that are going to be used for each of the atoms. Let us the molecule Mo(CO)6, Molybdenum hexacarbonyl, as an example. Below is the first line of the the external file, which we will call "CUSTOMMO" (NOTE: you can use any name for the external file that suits you, as long as it has no spaces and is 8 characters or less).<br /> <br /> ! 6-31G on C and O and LANL2D2 ECP on Mo<br /> <br /> 2. The next step is to visit the [https://bse.pnl.gov/bse/portal EMSL Basis Set exchange] and select C and O from the periodic table. Then, on the left of the page, select "6-31G" as the basis set. Finally, make sure the output is in GAMESS(US) format using the drop-down menu and then click "get basis set".<br /> <br /> [[File:C_O_6_31G_basisset.JPG|centre]]<br /> <br /> 3. A new window should appear with text in it. For our example case, the text looks like this:<br /> <br /> ! 6-31G EMSL Basis Set Exchange Library 10/13/09 11:12 AM<br /> ! Elements References<br /> ! -------- ----------<br /> ! H - He: W.J. Hehre, R. Ditchfield and J.A. Pople, J. Chem. Phys. 56,<br /> ! Li - Ne: 2257 (1972). Note: Li and B come from J.D. Dill and J.A.<br /> ! Pople, J. Chem. Phys. 62, 2921 (1975).<br /> ! Na - Ar: M.M. Francl, W.J. Petro, W.J. Hehre, J.S. Binkley, M.S. Gordon,<br /> ! D.J. DeFrees and J.A. Pople, J. Chem. Phys. 77, 3654 (1982)<br /> ! K - Zn: V. Rassolov, J.A. Pople, M. Ratner and T.L. Windus, J. Chem. Phys.<br /> ! 109, 1223 (1998)<br /> ! Note: He and Ne are unpublished basis sets taken from the Gaussian<br /> ! program<br /> ! <br /> $DATA<br /><br /> CARBON<br /> S 6<br /> 1 3047.5249000 0.0018347 <br /> 2 457.3695100 0.0140373 <br /> 3 103.9486900 0.0688426 <br /> 4 29.2101550 0.2321844 <br /> 5 9.2866630 0.4679413 <br /> 6 3.1639270 0.3623120 <br /> L 3<br /> 1 7.8682724 -0.1193324 0.0689991 <br /> 2 1.8812885 -0.1608542 0.3164240 <br /> 3 0.5442493 1.1434564 0.7443083 <br /> L 1<br /> 1 0.1687144 1.0000000 1.0000000<br /> <br /> OXYGEN<br /> S 6<br /> 1 5484.6717000 0.0018311 <br /> 2 825.2349500 0.0139501 <br /> 3 188.0469600 0.0684451 <br /> 4 52.9645000 0.2327143 <br /> 5 16.8975700 0.4701930 <br /> 6 5.7996353 0.3585209 <br /> L 3<br /> 1 15.5396160 -0.1107775 0.0708743 <br /> 2 3.5999336 -0.1480263 0.3397528 <br /> 3 1.0137618 1.1307670 0.7271586 <br /> L 1<br /> 1 0.2700058 1.0000000 1.0000000 <br /> $END<br /> <br /> 3. Now, copy and paste the text between the $DATA and $END headings onto our external text file, CUSTOMMO. We also need to change the change the name of each element to it symbol in the periodic table. Finally, we need to add the name of the external file next to the element symbol, separated by one space. Note that there should be a blank line separating the basis set information and the first, commented-out line (The line starting with the '!'). The CUSTOMMO should look like this:<br /> <br /> ! 6-31G on C and O and LANL2D2 ECP on Mo<br /><br /> C CUSTOMMO<br /> S 6<br /> 1 3047.5249000 0.0018347 <br /> 2 457.3695100 0.0140373 <br /> 3 103.9486900 0.0688426 <br /> 4 29.2101550 0.2321844 <br /> 5 9.2866630 0.4679413 <br /> 6 3.1639270 0.3623120 <br /> L 3<br /> 1 7.8682724 -0.1193324 0.0689991 <br /> 2 1.8812885 -0.1608542 0.3164240 <br /> 3 0.5442493 1.1434564 0.7443083 <br /> L 1<br /> 1 0.1687144 1.0000000 1.0000000<br /> <br /> O CUSTOMMO<br /> S 6<br /> 1 5484.6717000 0.0018311 <br /> 2 825.2349500 0.0139501 <br /> 3 188.0469600 0.0684451 <br /> 4 52.9645000 0.2327143 <br /> 5 16.8975700 0.4701930 <br /> 6 5.7996353 0.3585209 <br /> L 3<br /> 1 15.5396160 -0.1107775 0.0708743 <br /> 2 3.5999336 -0.1480263 0.3397528 <br /> 3 1.0137618 1.1307670 0.7271586 <br /> L 1<br /> 1 0.2700058 1.0000000 1.0000000 <br /> <br /> 4. Repeat Step 3 above but choose Mo and select the LANL2DZ ECP instead. A new window will pop up with the basis set information as well as the ECP data we need, since we specified the LANL2DZ '''ECP'''. The ECP data is not inserted into the external file, rather it is placed into the input file itself (More on this later). <br /> <br /> [[File:Mo_LANL2DZ_basisset.JPG|centre]]<br /> <br /> 5. After copying the molybdenum basis set information, your fiished external basis set file should look like this:<br /> ! 6-31G on C and O and LANL2D2 ECP on Mo<br /><br /> C CUSTOMMO<br /> S 6<br /> 1 3047.5249000 0.0018347 <br /> 2 457.3695100 0.0140373 <br /> 3 103.9486900 0.0688426 <br /> 4 29.2101550 0.2321844 <br /> 5 9.2866630 0.4679413 <br /> 6 3.1639270 0.3623120 <br /> L 3<br /> 1 7.8682724 -0.1193324 0.0689991 <br /> 2 1.8812885 -0.1608542 0.3164240 <br /> 3 0.5442493 1.1434564 0.7443083 <br /> L 1<br /> 1 0.1687144 1.0000000 1.0000000<br /> <br /> O CUSTOMMO<br /> S 6<br /> 1 5484.6717000 0.0018311 <br /> 2 825.2349500 0.0139501 <br /> 3 188.0469600 0.0684451 <br /> 4 52.9645000 0.2327143 <br /> 5 16.8975700 0.4701930 <br /> 6 5.7996353 0.3585209 <br /> L 3<br /> 1 15.5396160 -0.1107775 0.0708743 <br /> 2 3.5999336 -0.1480263 0.3397528 <br /> 3 1.0137618 1.1307670 0.7271586 <br /> L 1<br /> 1 0.2700058 1.0000000 1.0000000<br /> <br /> Mo CUSTOMO<br /> S 3<br /> 1 2.3610000 -0.9121760 <br /> 2 1.3090000 1.1477453 <br /> 3 0.4500000 0.6097109 <br /> S 4<br /> 1 2.3610000 0.8139259 <br /> 2 1.3090000 -1.1360084 <br /> 3 0.4500000 -1.1611592 <br /> 4 0.1681000 1.0064786 <br /> S 1<br /> 1 0.0423000 1.0000000 <br /> P 3<br /> 1 4.8950000 -0.0908258 <br /> 2 1.0440000 0.7042899 <br /> 3 0.3877000 0.3973179 <br /> P 2<br /> 1 0.4995000 -0.1081945 <br /> 2 0.0780000 1.0368093 <br /> P 1<br /> 1 0.0247000 1.0000000 <br /> D 3<br /> 1 2.9930000 0.0527063 <br /> 2 1.0630000 0.5003907 <br /> 3 0.3721000 0.5794024 <br /> D 1<br /> 1 0.1178000 1.0000000<br /> <br /> -- [[User:M.Zimmer-De Iuliis|mzd]] 13 October 2009<br /> <br /> ==Climate Modelling==<br /> <br /> ==Medicine/Bio==<br /> <br /> ==High Energy Physics==<br /> <br /> ==Structural Biology==<br /> Molecular simulation of proteins, lipids, carbohydrates, and other biologically relevant molecules.<br /> ===Molecular Dynamics (MD) simulation===<br /> ====GROMACS====<br /> Please refer to the [[gromacs|GROMACS]] page<br /> ====AMBER====<br /> Please refer to the [[amber|AMBER]] page<br /> ====NAMD====<br /> NAMD is one of the better scaling MD packages out there. With sufficiently large systems, it is able to scale to hundreds or thousands of cores on Scinet. Below are details for compiling and running NAMD on Scinet.<br /> <br /> More information regarding performance and different compile options coming soon...<br /> <br /> =====Compiling NAMD for GPC=====<br /> Ensure the proper compiler/mpi modules are loaded.<br /> <source lang="sh"><br /> module load intel<br /> module load openmpi/1.3.3-intel-v11.0-ofed<br /> </source><br /> <br /> '''Compile Charm++ and NAMD'''<br /> <source lang="sh"><br /> #Unpack source files and get required support libraries<br /> tar -xzf NAMD_2.7b1_Source.tar.gz<br /> cd NAMD_2.7b1_Source<br /> tar -xf charm-6.1.tar<br /> wget http://www.ks.uiuc.edu/Research/namd/libraries/fftw-linux-x86_64.tar.gz<br /> wget http://www.ks.uiuc.edu/Research/namd/libraries/tcl-linux-x86_64.tar.gz<br /> tar -xzf fftw-linux-x86_64.tar.gz; mv linux-x86_64 fftw<br /> tar -xzf tcl-linux-x86_64.tar.gz; mv linux-x86_64 tcl<br /> #Compile Charm++<br /> cd charm-6.1<br /> ./build charm++ mpi-linux-x86_64 icc --basedir /scinet/gpc/mpi/openmpi/1.3.3-intel-v11.0-ofed/ --no-shared -O -DCMK_OPTIMIZE=1<br /> cd ..<br /> #Compile NAMD. <br /> #Edit arch/Linux-x86_64-icc.arch and add "-lmpi" to the end of the CXXOPTS and COPTS line.<br /> #Make a builds directory if you want different versions of NAMD compiled at the same time.<br /> mkdir builds<br /> ./config builds/Linux-x86_64-icc --charm-arch mpi-linux-x86_64-icc<br /> cd builds/Linux-x86_64-icc/<br /> make -j4 namd2 # Adjust value of j as desired to specify number of simultaneous make targets. <br /> </source><br /> --[[User:Cmadill|Cmadill]] 16:18, 27 August 2009 (UTC)<br /> <br /> =====Running Fortran=====<br /> On the development nodes, there is an old gcc. The associated libraries are not on the compute nodes. Ensure the line:<br /> <br /> module load gcc<br /> <br /> is in your .bashrc file.<br /> <br /> ====LAMMPS====<br /> [[Image:StrongScalingLAMMPS.png|thumb|right|320px|Strong scaling test on GPC with OpenMPI and IntelMPI on Ethernet and InfiniBand]]<br /> Results from strong scaling tests for LAMMPS using EAM potentials on GPC are shown in graph on the right. The test simulation ran 500 timesteps for 4,000,000 atoms.<br /> [[Image:WeakScalingLAMMPS.png|thumb|right|320px|Weak scaling test on GPC with OpenMPI and IntelMPI on Ethernet and InfiniBand]]<br /> Results from weak scaling tests for LAMMPS using EAM potentials on GPC are shown in graph on the right. The test simulation ran 500 timesteps for 32,000 atoms per processor.<br /> <br /> ===Monte Carlo (MC) simulation===</div>

Jchu https://oldwiki.scinet.utoronto.ca/index.php?title=User_Codes&diff=873 User Codes 2010-02-02T18:46:07Z

<p>Jchu: /* LAMMPS */</p> <hr /> <div>__FORCETOC__<br /> <br /> ==Astrophysics==<br /> <br /> ===Athena (explicit, uniform grid MHD code)===<br /> <br /> [[Image:StrongScalingAthenaGPC.png|thumb|right|320px|Athena scaling on GPC with OpenMPI and MVAPICH2 on GigE, and OpenMPI on InfiniBand]]<br /> <br /> [http://www.astro.princeton.edu/~jstone/athena.html Athena] is a straightforward C code which doesn't use a lot of libraries so it is pretty straightforward to build and compile on new machines. <br /> <br /> It encapsulates its compiler flags, etc in an <tt>Makeoptions.in</tt> file which is then processed by <tt>configure</tt>. I've used the following additions to <tt>Makeoptions.in</tt> on TCS and GPC:<br /> <br /> <source lang="sh"><br /> ifeq ($(MACHINE),scinettcs)<br /> CC = mpcc_r<br /> LDR = mpcc_r<br /> OPT = -O5 -q64 -qarch=pwr6 -qtune=pwr6 -qcache=auto -qlargepage -qstrict<br /> MPIINC =<br /> MPILIB =<br /> CFLAGS = $(OPT)<br /> LIB = -ldl -lm<br /> else<br /> ifeq ($(MACHINE),scinetgpc)<br /> CC = mpicc<br /> LDR = mpicc<br /> OPT = -O3<br /> MPIINC =<br /> MPILIB =<br /> CFLAGS = $(OPT)<br /> LIB = -lm<br /> else<br /> ...<br /> endif<br /> endif<br /> </source><br /> It performs quite well on the GPC, scaling extremely well even on a strong scaling test out to about 256 cores (32 nodes) on Gigabit ethernet, and performing beautifully on InfiniBand out to 512 cores (64 nodes). <br /> <br /> <br /> -- [[User:Ljdursi|ljdursi]] 19:20, 13 August 2009 (UTC)<br /> <br /> ===FLASH3 (Adaptive Mesh reactive hydrodynamics; explict hydro/MHD)===<br /> <br /> [[Image:weak-scaling-example.png|thumb|right|320px|Weak scaling test of the 2d sod problem on both the GPC and TCS. The results are actually somewhat faster on the GPC; in both cases (weak) scaling is very good out at least to 256 cores]]<br /> <br /> [http://flash.uchicago.edu FLASH] encapsulates its machine-dependant information in the <tt>FLASH3/sites</tt> directory. For the GPC, you'll have to<br /> <pre><br /> module load intel<br /> module load openmpi<br /> module load hdf5/183-v16-openmpi<br /> </pre><br /> <br /> and with that, the following file (<tt>sites/scinetgpc/Makefile.h</tt>) works for me:<br /> <source lang="sh"><br /> ## Must do module load hdf5/183-v16-openmpi<br /> HDF5_PATH = ${SCINET_HDF5_BASE}<br /> ZLIB_PATH = /usr/local<br /> <br /> #----------------------------------------------------------------------------<br /> # Compiler and linker commands<br /> #<br /> # We use the f90 compiler as the linker, so some C libraries may explicitly<br /> # need to be added into the link line.<br /> #----------------------------------------------------------------------------<br /> <br /> ## modules will put the right mpi in our path<br /> FCOMP = mpif77<br /> CCOMP = mpicc<br /> CPPCOMP = mpiCC<br /> LINK = mpif77<br /> <br /> #----------------------------------------------------------------------------<br /> # Compilation flags<br /> #<br /> # Three sets of compilation/linking flags are defined: one for optimized<br /> # code, one for testing, and one for debugging. The default is to use the <br /> # _OPT version. Specifying -debug to setup will pick the _DEBUG version,<br /> # these should enable bounds checking. Specifying -test is used for <br /> # flash_test, and is set for quick code generation, and (sometimes) <br /> # profiling. The Makefile generated by setup will assign the generic token <br /> # (ex. FFLAGS) to the proper set of flags (ex. FFLAGS_OPT).<br /> #----------------------------------------------------------------------------<br /> <br /> FFLAGS_OPT = -c -r8 -i4 -O3 -xSSE4.2<br /> FFLAGS_DEBUG = -c -g -r8 -i4 -O0<br /> FFLAGS_TEST = -c -r8 -i4<br /> <br /> <br /> # if we are using HDF5, we need to specify the path to the include files<br /> CFLAGS_HDF5 = -I${HDF5_PATH}/include<br /> <br /> CFLAGS_OPT = -c -O3 -xSSE4.2<br /> CFLAGS_TEST = -c -O2 <br /> CFLAGS_DEBUG = -c -g <br /> <br /> MDEFS = <br /> <br /> .SUFFIXES: .o .c .f .F .h .fh .F90 .f90<br /> <br /> #----------------------------------------------------------------------------<br /> # Linker flags<br /> #<br /> # There is a seperate version of the linker flags for each of the _OPT, <br /> # _DEBUG, and _TEST cases.<br /> #----------------------------------------------------------------------------<br /> <br /> LFLAGS_OPT = -o<br /> LFLAGS_TEST = -o<br /> LFLAGS_DEBUG = -g -o<br /> <br /> MACHOBJ = <br /> <br /> <br /> MV = mv -f<br /> AR = ar -r<br /> RM = rm -f<br /> CD = cd<br /> RL = ranlib<br /> ECHO = echo<br /> </source><br /> <br /> <br /> -- [[User:Ljdursi|ljdursi]] 22:11, 13 August 2009 (UTC)<br /> <br /> ==Aeronautics==<br /> <br /> ==Chemistry==<br /> <br /> ===GAMESS (US)===<br /> The GAMESS version January 12, 2009 R3 was built using the Intel v11.1 compilers and v3.2.2 MPI library, according to the instructions in http://software.intel.com/en-us/articles/building-gamess-with-intel-compilers-intel-mkl-and-intel-mpi-on-linux/<br /> <br /> The required build scripts - comp, compall, lked - and run script - rungms - were modified to account for our own installation. In order to build GAMESS one first must ensure that the intel and intelmpi modules are loaded ("module load intel intelmpi"). This applies to running GAMESS as well.<br /> The module "gamess" must also be loaded in order to run GAMESS ("module load gamess").<br /> <br /> The modified scripts are in the file /scinet/gpc/src/gamess-on-scinet.tar.gz<br /> <br /> ====Running GAMESS====<br /> - Make sure the directory /scratch/$USER/gamess-scratch exists (the $SCINET_RUNGMS script will create it if it does not exist)<br /> <br /> - Make sure the modules: intel, intelmpi, gamess are loaded (in your .bashrc: "module load intel intelmpi gamess").<br /> <br /> - Create a torque script to run GAMESS. Here is an example:<br /> <br /> - The GAMESS executable is in $SCINET_GAMESS_HOME/gamess.00.x<br /> - The rungms script is in $SCINET_GAMESS_HOME/rungms (actually it is $SCINET_RUNGMS)<br /> <br /> - For IB multinode runs, use the $SCINET_RUNGMS_IB script<br /> <br /> - The rungms script takes 4 arguments: input file, executable number, number of compute processes, processors per node<br /> <br /> For example, in order to run with the input file /scratch/$USER/gamesstest01, on 8 cpus, and the default version (00)<br /> of the executable on a machine with 8 cores:<br /> <br /> # load the gamess module in .bashrc<br /> module load gamess <br /> <br /> # run the program<br /> $SCINET_RUNGMS /scratch/$USER/gamesstest01 00 8 8<br /> <br /> Here is a sample torque script for running a GAMESS calculation, on a single 8-core node:<br /> <br /> <source lang="bash"><br /> #!/bin/bash<br /> #PBS -l nodes=1:ppn=8,walltime=48:00:00,os=centos53computeA<br /> #PBS -N gamessjob<br /> <br /> ## To submit type: qsub gms.sh<br /> <br /> # If not an interactive job (i.e. -I), then cd into the directory where<br /> # I typed qsub.<br /> if [ "$PBS_ENVIRONMENT" != "PBS_INTERACTIVE" ]; then<br /> if [ -n "$PBS_O_WORKDIR" ]; then<br /> cd $PBS_O_WORKDIR<br /> fi<br /> fi<br /> <br /> # the input file is typically named something like "gamesjob.inp"<br /> # so the script will be run like "$SCINET_RUNGMS gamessjob 00 8 8"<br /> <br /> # load the gamess module if not in .bashrc already<br /> # actually, it MUST be in .bashrc<br /> # module load gamess<br /> <br /> # run the program<br /> <br /> $SCINET_RUNGMS gamessjob 00 8 8<br /> </source><br /> <br /> Here is a similar script, but this one uses 2 InfiniBand-connected nodes,<br /> and runs the appropriate $SCINET_RUNGMS_IB script to actually run the job:<br /> <br /> <source lang="bash"><br /> #!/bin/bash<br /> #PBS -l nodes=2:ib:ppn=8,walltime=48:00:00,os=centos53computeA<br /> #PBS -N gamessjob<br /> <br /> ## To submit type: qsub gmsib.sh<br /> <br /> # If not an interactive job (i.e. -I), then cd into the directory where<br /> # I typed qsub.<br /> if [ "$PBS_ENVIRONMENT" != "PBS_INTERACTIVE" ]; then<br /> if [ -n "$PBS_O_WORKDIR" ]; then<br /> cd $PBS_O_WORKDIR<br /> fi<br /> fi<br /> <br /> # the input file is typically named something like "gamesjob.inp"<br /> # so the script will be run like "$SCINET_RUNGMS gamessjob 00 8 8"<br /> <br /> # load the gamess module if not in .bashrc already<br /> # actually, it MUST be in .bashrc<br /> # module load gamess<br /> <br /> # This script requests InfiniBand-connected nodes (:ib above)<br /> # so it must run with the IB version of the rungms script,<br /> # $SCINET_RUNGMS_IB<br /> <br /> $SCINET_RUNGMS_IB gamessjob 00 16 8<br /> </source><br /> <br /> <br /> -- [[User:Dgruner|dgruner]] 5 October 2009<br /> <br /> ===Tips from the Fekl Lab===<br /> <br /> Through trial and error, we have found a few useful things that we would like to share:<br /> <br /> 1. Two very useful, open-source programs for visualization of output files from GAMESS(US) and for generation of input files are [http://www.scl.ameslab.gov/MacMolPlt/ MacMolPlt]and [http://avogadro.openmolecules.net/wiki/Main_Page Avogadro]. The are available for UNIX/LINUX, Windows and Mac based machines, HOWEVER: any input files that we have generated with these programs on a Windows-based machine do not run on Mac based machines. We don't know why.<br /> <br /> 2. [http://winscp.net/eng/index.php WinSCP] is a very useful tool that has a graphical user interface for moving files from a local machine to SCINET and vice versa. It also has text editing capabilities.<br /> <br /> 3. The [https://bse.pnl.gov/bse/portal ESML Basis Set Exchange] is an excellent source for custom basis set or effective core potential parameters. Make sure that you specify "Gamess-US" in the format drop-down box.<br /> <br /> 4. The commercial program [http://www.chemcraftprog.com/ ChemCraft] is a highly useful visualization program that has the ability to edit molecules in a very similar fashion to GaussView. It can also be customized to build GAMESS(US) input files.<br /> <br /> ====Anatomy of a GAMESS(US) Input File with Basis Set Info in an External File====<br /> <br /> $CONTRL SCFTYP=RHF RUNTYP=OPTIMIZE DFTTYP=M06-L MAXIT=199 MULT=1 NOSYM=1<br /> ECP=READ $END<br /> $SYSTEM TIMLIM=525600 MWORDS=1750 PARALL=.TRUE. $END<br /> $BASIS GBASIS=CUSTOMNI EXTFIL=.t. $END<br /> $SCF DIRSCF=.TRUE. FDIFF=.f. $END<br /> $STATPT OPTTOL=0.0001 NSTEP=500 HSSEND=.t. $END<br /> $DATA<br /> Mo_BDT3<br /> C1<br /> MOLYBDENUM 42.0 5.7556500000 4.4039600000 16.5808400000<br /> SULFUR 16.0 7.4169700000 3.1956300000 15.2089300000<br /> SULFUR 16.0 4.0966800000 3.2258300000 15.1761100000<br /> SULFUR 16.0 3.9677300000 4.4940500000 18.3266100000<br /> SULFUR 16.0 7.1776900000 3.5815000000 18.4485200000<br /> SULFUR 16.0 4.3776600000 6.2447400000 15.6786900000<br /> SULFUR 16.0 7.5478700000 6.0679800000 16.2223700000<br /> CARBON 6.0 6.4716900000 2.1004800000 14.1902300000<br /> CARBON 6.0 5.0690300000 2.1781400000 14.1080700000<br /> CARBON 6.0 4.8421800000 4.2701300000 19.8855500000<br /> CARBON 6.0 6.1969000000 3.9249600000 19.9397400000<br /> CARBON 6.0 6.8280600000 3.7834200000 21.1913200000<br /> CARBON 6.0 5.7697600000 7.6933500000 17.4241800000<br /> CARBON 6.0 7.2043100000 7.9413600000 17.8281100000<br /> CARBON 6.0 5.5051400000 7.0409700000 14.5903800000<br /> CARBON 6.0 6.8905200000 6.9194700000 14.7626200000<br /> CARBON 6.0 7.7396400000 7.5379800000 13.8285700000<br /> HYDROGEN 1.0 8.8190700000 7.4520600000 13.9252200000<br /> CARBON 6.0 7.2169400000 8.2960300000 12.7704100000<br /> HYDROGEN 1.0 7.8667000000 8.7825100000 12.0575600000<br /> CARBON 6.0 5.8260300000 8.4502300000 12.6467800000<br /> HYDROGEN 1.0 5.4143000000 9.0544300000 11.8493100000<br /> CARBON 6.0 4.9881500000 7.8192300000 13.5528400000<br /> HYDROGEN 1.0 3.9090500000 7.9420000000 13.4583700000<br /> CARBON 6.0 7.1538500000 1.1569600000 13.4143900000<br /> CARBON 6.0 4.4018100000 1.3603900000 13.1919900000<br /> CARBON 6.0 6.4791600000 0.3185500000 12.5353300000<br /> CARBON 6.0 5.0837400000 0.4369500000 12.4084900000<br /> HYDROGEN 1.0 7.0116000000 -0.4099400000 11.9434600000<br /> HYDROGEN 1.0 8.2399000000 1.0702400000 13.4937600000<br /> HYDROGEN 1.0 3.3185600000 1.4368700000 13.0953100000<br /> HYDROGEN 1.0 4.5549800000 -0.1997300000 11.7165200000<br /> CARBON 6.0 6.1105700000 3.9639000000 22.3866100000<br /> CARBON 6.0 4.1216300000 4.4424400000 21.1020100000<br /> HYDROGEN 1.0 7.8732900000 3.5217100000 21.2520500000<br /> CARBON 6.0 4.7606000000 4.2868500000 22.3363800000<br /> HYDROGEN 1.0 6.6064200000 3.8406000000 23.3428500000<br /> HYDROGEN 1.0 4.2065000000 4.4170700000 23.2667100000<br /> HYDROGEN 1.0 3.0674000000 4.6893500000 21.0889000000<br /> HYDROGEN 1.0 7.4249200000 7.7545300000 18.8583200000<br /> HYDROGEN 1.0 7.6651700000 8.9049700000 17.7652100000<br /> HYDROGEN 1.0 5.3324000000 8.6487800000 17.2222700000<br /> HYDROGEN 1.0 5.5015000000 7.1039000000 18.2759400000<br /> $END<br /> $ECP<br /> MO-ECP GEN 28 3<br /> 5 ----- f potential -----<br /> -0.0469492 0 537.9667807 <br /> -20.2080084 1 147.8982938 <br /> -106.2116302 2 45.7358898 <br /> -41.8107368 2 13.2911467 <br /> -4.2054103 2 4.7059961 <br /> 3 ----- s-f potential -----<br /> 2.8063717 0 110.2991760 <br /> 44.5162012 1 23.2014645 <br /> 82.7785227 2 5.3530131 <br /> 4 ----- p-f potential -----<br /> 4.9420876 0 63.2901397 <br /> 25.8604976 1 23.3315302 <br /> 132.4708742 2 24.6759423 <br /> 57.3149794 2 4.6493040 <br /> 5 ----- d-f potential -----<br /> 3.0054591 0 104.4839977 <br /> 26.3637851 1 66.2307245 <br /> 183.3849199 2 39.1283176 <br /> 98.4453068 2 13.1164437 <br /> 22.4901377 2 3.6280263 <br /> S NONE<br /> S NONE<br /> S NONE<br /> S NONE<br /> S NONE<br /> S NONE<br /> C NONE<br /> C NONE<br /> C NONE<br /> C NONE<br /> C NONE<br /> C NONE<br /> C NONE<br /> C NONE<br /> C NONE<br /> C NONE<br /> H NONE<br /> C NONE<br /> H NONE<br /> C NONE<br /> H NONE<br /> C NONE<br /> H NONE<br /> C NONE<br /> C NONE<br /> C NONE<br /> C NONE<br /> H NONE<br /> H NONE<br /> H NONE<br /> H NONE<br /> C NONE<br /> C NONE<br /> H NONE<br /> C NONE<br /> H NONE<br /> H NONE<br /> H NONE<br /> H NONE<br /> H NONE<br /> H NONE<br /> H NONE<br /> $END<br /> <br /> =====The Input Deck=====<br /> <br /> Below is the input deck. It is where you tell GAMESS(US) what job type to execute and where all you individual parameters are entered for your specific job type. The example input deck below is for a geometry optimization and frequency calculation. This input deck is equivalent to the Gaussian job with "opt" and "freq" in the route section.<br /> <br /> $CONTRL SCFTYP=RHF RUNTYP=OPTIMIZE DFTTYP=M06-L MAXIT=199 MULT=1 NOSYM=1<br /> ECP=READ $END<br /> $SYSTEM TIMLIM=2850 MWORDS=1750 MEMDDI=20 PARALL=.TRUE. $END<br /> $BASIS GBASIS=CUSTOMNI EXTFIL=.t. $END<br /> $SCF DIRSCF=.TRUE. FDIFF=.f. $END<br /> $STATPT OPTTOL=0.0001 NSTEP=500 HSSEND=.t. $END<br /> $DATA<br /> <br /> An important thing to note is the spacing. In the input deck, there must be 1 space at the beginning of each line of the input deck. If not, the job will fail. Most builders will insert this space anyway, but it helps to double check.<br /> <br /> The end of the input deck is marked by the "$DATA" line.<br /> <br /> =====Job Title Line=====<br /> <br /> The next line of the file is the job title. It can be anthing you wish, however, we have found that to be on the safe side, we avoide using symbols or spaces<br /> <br /> Mo_BDT3<br /> <br /> =====Symmetry Point Group=====<br /> <br /> The next line of the file is the symmetry point group of your molecule. Note that there is no leading space before the point group.<br /> <br /> C1<br /> <br /> =====Coordinates=====<br /> <br /> The next block of text is set aside for the coordinates of the molecule. This can be in internal (or z-matrix) format or cartesian coordinates. Note that there is no leading space before the coordinates. One may use the chemical symbol or the full name of each atom in the molecule. Note that the end of the coordinates is signified by an "$END", which MUST have one space preceding it. The coordinates below do NOT have any basis set information inserted. It is possible to insert basis set information directly into the input file. This is accomplished by obtaining the desired basis set parameters from the EMSL and then inserting them below each relevant atom. An example input file with inserted basis set information will be shown later.<br /> <br /> MOLYBDENUM 42.0 5.7556500000 4.4039600000 16.5808400000<br /> SULFUR 16.0 7.4169700000 3.1956300000 15.2089300000<br /> SULFUR 16.0 4.0966800000 3.2258300000 15.1761100000<br /> SULFUR 16.0 3.9677300000 4.4940500000 18.3266100000<br /> SULFUR 16.0 7.1776900000 3.5815000000 18.4485200000<br /> SULFUR 16.0 4.3776600000 6.2447400000 15.6786900000<br /> SULFUR 16.0 7.5478700000 6.0679800000 16.2223700000<br /> CARBON 6.0 6.4716900000 2.1004800000 14.1902300000<br /> CARBON 6.0 5.0690300000 2.1781400000 14.1080700000<br /> CARBON 6.0 4.8421800000 4.2701300000 19.8855500000<br /> CARBON 6.0 6.1969000000 3.9249600000 19.9397400000<br /> CARBON 6.0 6.8280600000 3.7834200000 21.1913200000<br /> CARBON 6.0 5.7697600000 7.6933500000 17.4241800000<br /> CARBON 6.0 7.2043100000 7.9413600000 17.8281100000<br /> CARBON 6.0 5.5051400000 7.0409700000 14.5903800000<br /> CARBON 6.0 6.8905200000 6.9194700000 14.7626200000<br /> CARBON 6.0 7.7396400000 7.5379800000 13.8285700000<br /> HYDROGEN 1.0 8.8190700000 7.4520600000 13.9252200000<br /> CARBON 6.0 7.2169400000 8.2960300000 12.7704100000<br /> HYDROGEN 1.0 7.8667000000 8.7825100000 12.0575600000<br /> CARBON 6.0 5.8260300000 8.4502300000 12.6467800000<br /> HYDROGEN 1.0 5.4143000000 9.0544300000 11.8493100000<br /> CARBON 6.0 4.9881500000 7.8192300000 13.5528400000<br /> HYDROGEN 1.0 3.9090500000 7.9420000000 13.4583700000<br /> CARBON 6.0 7.1538500000 1.1569600000 13.4143900000<br /> CARBON 6.0 4.4018100000 1.3603900000 13.1919900000<br /> CARBON 6.0 6.4791600000 0.3185500000 12.5353300000<br /> CARBON 6.0 5.0837400000 0.4369500000 12.4084900000<br /> HYDROGEN 1.0 7.0116000000 -0.4099400000 11.9434600000<br /> HYDROGEN 1.0 8.2399000000 1.0702400000 13.4937600000<br /> HYDROGEN 1.0 3.3185600000 1.4368700000 13.0953100000<br /> HYDROGEN 1.0 4.5549800000 -0.1997300000 11.7165200000<br /> CARBON 6.0 6.1105700000 3.9639000000 22.3866100000<br /> CARBON 6.0 4.1216300000 4.4424400000 21.1020100000<br /> HYDROGEN 1.0 7.8732900000 3.5217100000 21.2520500000<br /> CARBON 6.0 4.7606000000 4.2868500000 22.3363800000<br /> HYDROGEN 1.0 6.6064200000 3.8406000000 23.3428500000<br /> HYDROGEN 1.0 4.2065000000 4.4170700000 23.2667100000<br /> HYDROGEN 1.0 3.0674000000 4.6893500000 21.0889000000<br /> HYDROGEN 1.0 7.4249200000 7.7545300000 18.8583200000<br /> HYDROGEN 1.0 7.6651700000 8.9049700000 17.7652100000<br /> HYDROGEN 1.0 5.3324000000 8.6487800000 17.2222700000<br /> HYDROGEN 1.0 5.5015000000 7.1039000000 18.2759400000<br /> $END<br /> <br /> =====Effective Core Potential Data=====<br /> <br /> The effective core potential (ECP) data is entered after the coordinates. It starts with "$ECP", which must be preceded with a space. The atoms of the molecule are listed in the same order as in the coordinates section and the parameters for the ECP are listed after each atom. Note that for any atom that does NOT have an ECP, one must enter "ECP-NONE" or "NONE" after each atom without an ECP.<br /> <br /> $ECP<br /> MO-ECP GEN 28 3<br /> 5 ----- f potential -----<br /> -0.0469492 0 537.9667807 <br /> -20.2080084 1 147.8982938 <br /> -106.2116302 2 45.7358898 <br /> -41.8107368 2 13.2911467 <br /> -4.2054103 2 4.7059961 <br /> 3 ----- s-f potential -----<br /> 2.8063717 0 110.2991760 <br /> 44.5162012 1 23.2014645 <br /> 82.7785227 2 5.3530131 <br /> 4 ----- p-f potential -----<br /> 4.9420876 0 63.2901397 <br /> 25.8604976 1 23.3315302 <br /> 132.4708742 2 24.6759423 <br /> 57.3149794 2 4.6493040 <br /> 5 ----- d-f potential -----<br /> 3.0054591 0 104.4839977 <br /> 26.3637851 1 66.2307245 <br /> 183.3849199 2 39.1283176 <br /> 98.4453068 2 13.1164437 <br /> 22.4901377 2 3.6280263 <br /> S NONE<br /> S NONE<br /> S NONE<br /> S NONE<br /> S NONE<br /> S NONE<br /> C NONE<br /> C NONE<br /> C NONE<br /> C NONE<br /> C NONE<br /> C NONE<br /> C NONE<br /> C NONE<br /> C NONE<br /> C NONE<br /> H NONE<br /> C NONE<br /> H NONE<br /> C NONE<br /> H NONE<br /> C NONE<br /> H NONE<br /> C NONE<br /> C NONE<br /> C NONE<br /> C NONE<br /> H NONE<br /> H NONE<br /> H NONE<br /> H NONE<br /> C NONE<br /> C NONE<br /> H NONE<br /> C NONE<br /> H NONE<br /> H NONE<br /> H NONE<br /> H NONE<br /> H NONE<br /> H NONE<br /> H NONE<br /> $END<br /> <br /> -- [[User:M.Zimmer-De Iuliis|mzd]] 16 November 2009<br /> <br /> ====Using Effective Core Potentials in GAMESS(US)====<br /> <br /> For many metal containing compounds, it is very convenient and time saving to use an effective core potential (ECP) for the core metal electrons, as they are usually not important to the reactivity of the complex or the geometry around the metal. Since GAMESS(US) has a limited number of built-in ECPs, one may want to make GAMESS(US) read an external file that contains the ECP data using the "EXTFIL" keyword in the $GBASIS command line of the input file. In addition, to make GAMESS(US) use this external file, one must copy the "rungms" file and modify it accordingly. The following is a list of instructions with commands that will work from a terminal. One could also use WinSCP to do all of this with a GUI rather than a TUI. <br /> <br /> =====Modifiying rungms to Use Custom Basis Set File=====<br /> 1. Copy "rungms" from /scinet/gpc/Applications/gamess to one's own /scratch/$USER/ directory:<br /> cp /scinet/gpc/Applications/gamess/rungms /scratch/$USER/<br /> <br /> 2. Change to the scratch directory and check to see if "rungms" has copied successfully.<br /> cd /scratch/$USER<br /> ls<br /> <br /> 3. Edit line 147 of the vi. <br /> vi rungms<br /> Move the cursor down to line 147 using the arrow keys. It should say "setenv EXTBAS /dev/null". Using the arrow keys, move the cursor to the first "/" and then hit "i" to insert text. Put the path to your external basis file here. For example, /scratch/$USER/basisset. Then hit "escape". To save the changes and exit vi, type ":" and you should see a colon appear at the bottom of the window. Type "wq" (which should appear at the bottom of the window next to the colon) and then hit enter. Now you are done with vi.<br /> <br /> =====Creating a Custom Basis Set File=====<br /> 1. To create a custom basis set file, you need create a new text document. Our group's common practice is to comment out the first line of this file by inserting an exclamation mark (!) followed by noting the specific basis sets and ECPs that are going to be used for each of the atoms. Let us the molecule Mo(CO)6, Molybdenum hexacarbonyl, as an example. Below is the first line of the the external file, which we will call "CUSTOMMO" (NOTE: you can use any name for the external file that suits you, as long as it has no spaces and is 8 characters or less).<br /> <br /> ! 6-31G on C and O and LANL2D2 ECP on Mo<br /> <br /> 2. The next step is to visit the [https://bse.pnl.gov/bse/portal EMSL Basis Set exchange] and select C and O from the periodic table. Then, on the left of the page, select "6-31G" as the basis set. Finally, make sure the output is in GAMESS(US) format using the drop-down menu and then click "get basis set".<br /> <br /> [[File:C_O_6_31G_basisset.JPG|centre]]<br /> <br /> 3. A new window should appear with text in it. For our example case, the text looks like this:<br /> <br /> ! 6-31G EMSL Basis Set Exchange Library 10/13/09 11:12 AM<br /> ! Elements References<br /> ! -------- ----------<br /> ! H - He: W.J. Hehre, R. Ditchfield and J.A. Pople, J. Chem. Phys. 56,<br /> ! Li - Ne: 2257 (1972). Note: Li and B come from J.D. Dill and J.A.<br /> ! Pople, J. Chem. Phys. 62, 2921 (1975).<br /> ! Na - Ar: M.M. Francl, W.J. Petro, W.J. Hehre, J.S. Binkley, M.S. Gordon,<br /> ! D.J. DeFrees and J.A. Pople, J. Chem. Phys. 77, 3654 (1982)<br /> ! K - Zn: V. Rassolov, J.A. Pople, M. Ratner and T.L. Windus, J. Chem. Phys.<br /> ! 109, 1223 (1998)<br /> ! Note: He and Ne are unpublished basis sets taken from the Gaussian<br /> ! program<br /> ! <br /> $DATA<br /><br /> CARBON<br /> S 6<br /> 1 3047.5249000 0.0018347 <br /> 2 457.3695100 0.0140373 <br /> 3 103.9486900 0.0688426 <br /> 4 29.2101550 0.2321844 <br /> 5 9.2866630 0.4679413 <br /> 6 3.1639270 0.3623120 <br /> L 3<br /> 1 7.8682724 -0.1193324 0.0689991 <br /> 2 1.8812885 -0.1608542 0.3164240 <br /> 3 0.5442493 1.1434564 0.7443083 <br /> L 1<br /> 1 0.1687144 1.0000000 1.0000000<br /> <br /> OXYGEN<br /> S 6<br /> 1 5484.6717000 0.0018311 <br /> 2 825.2349500 0.0139501 <br /> 3 188.0469600 0.0684451 <br /> 4 52.9645000 0.2327143 <br /> 5 16.8975700 0.4701930 <br /> 6 5.7996353 0.3585209 <br /> L 3<br /> 1 15.5396160 -0.1107775 0.0708743 <br /> 2 3.5999336 -0.1480263 0.3397528 <br /> 3 1.0137618 1.1307670 0.7271586 <br /> L 1<br /> 1 0.2700058 1.0000000 1.0000000 <br /> $END<br /> <br /> 3. Now, copy and paste the text between the $DATA and $END headings onto our external text file, CUSTOMMO. We also need to change the change the name of each element to it symbol in the periodic table. Finally, we need to add the name of the external file next to the element symbol, separated by one space. Note that there should be a blank line separating the basis set information and the first, commented-out line (The line starting with the '!'). The CUSTOMMO should look like this:<br /> <br /> ! 6-31G on C and O and LANL2D2 ECP on Mo<br /><br /> C CUSTOMMO<br /> S 6<br /> 1 3047.5249000 0.0018347 <br /> 2 457.3695100 0.0140373 <br /> 3 103.9486900 0.0688426 <br /> 4 29.2101550 0.2321844 <br /> 5 9.2866630 0.4679413 <br /> 6 3.1639270 0.3623120 <br /> L 3<br /> 1 7.8682724 -0.1193324 0.0689991 <br /> 2 1.8812885 -0.1608542 0.3164240 <br /> 3 0.5442493 1.1434564 0.7443083 <br /> L 1<br /> 1 0.1687144 1.0000000 1.0000000<br /> <br /> O CUSTOMMO<br /> S 6<br /> 1 5484.6717000 0.0018311 <br /> 2 825.2349500 0.0139501 <br /> 3 188.0469600 0.0684451 <br /> 4 52.9645000 0.2327143 <br /> 5 16.8975700 0.4701930 <br /> 6 5.7996353 0.3585209 <br /> L 3<br /> 1 15.5396160 -0.1107775 0.0708743 <br /> 2 3.5999336 -0.1480263 0.3397528 <br /> 3 1.0137618 1.1307670 0.7271586 <br /> L 1<br /> 1 0.2700058 1.0000000 1.0000000 <br /> <br /> 4. Repeat Step 3 above but choose Mo and select the LANL2DZ ECP instead. A new window will pop up with the basis set information as well as the ECP data we need, since we specified the LANL2DZ '''ECP'''. The ECP data is not inserted into the external file, rather it is placed into the input file itself (More on this later). <br /> <br /> [[File:Mo_LANL2DZ_basisset.JPG|centre]]<br /> <br /> 5. After copying the molybdenum basis set information, your fiished external basis set file should look like this:<br /> ! 6-31G on C and O and LANL2D2 ECP on Mo<br /><br /> C CUSTOMMO<br /> S 6<br /> 1 3047.5249000 0.0018347 <br /> 2 457.3695100 0.0140373 <br /> 3 103.9486900 0.0688426 <br /> 4 29.2101550 0.2321844 <br /> 5 9.2866630 0.4679413 <br /> 6 3.1639270 0.3623120 <br /> L 3<br /> 1 7.8682724 -0.1193324 0.0689991 <br /> 2 1.8812885 -0.1608542 0.3164240 <br /> 3 0.5442493 1.1434564 0.7443083 <br /> L 1<br /> 1 0.1687144 1.0000000 1.0000000<br /> <br /> O CUSTOMMO<br /> S 6<br /> 1 5484.6717000 0.0018311 <br /> 2 825.2349500 0.0139501 <br /> 3 188.0469600 0.0684451 <br /> 4 52.9645000 0.2327143 <br /> 5 16.8975700 0.4701930 <br /> 6 5.7996353 0.3585209 <br /> L 3<br /> 1 15.5396160 -0.1107775 0.0708743 <br /> 2 3.5999336 -0.1480263 0.3397528 <br /> 3 1.0137618 1.1307670 0.7271586 <br /> L 1<br /> 1 0.2700058 1.0000000 1.0000000<br /> <br /> Mo CUSTOMO<br /> S 3<br /> 1 2.3610000 -0.9121760 <br /> 2 1.3090000 1.1477453 <br /> 3 0.4500000 0.6097109 <br /> S 4<br /> 1 2.3610000 0.8139259 <br /> 2 1.3090000 -1.1360084 <br /> 3 0.4500000 -1.1611592 <br /> 4 0.1681000 1.0064786 <br /> S 1<br /> 1 0.0423000 1.0000000 <br /> P 3<br /> 1 4.8950000 -0.0908258 <br /> 2 1.0440000 0.7042899 <br /> 3 0.3877000 0.3973179 <br /> P 2<br /> 1 0.4995000 -0.1081945 <br /> 2 0.0780000 1.0368093 <br /> P 1<br /> 1 0.0247000 1.0000000 <br /> D 3<br /> 1 2.9930000 0.0527063 <br /> 2 1.0630000 0.5003907 <br /> 3 0.3721000 0.5794024 <br /> D 1<br /> 1 0.1178000 1.0000000<br /> <br /> -- [[User:M.Zimmer-De Iuliis|mzd]] 13 October 2009<br /> <br /> ==Climate Modelling==<br /> <br /> ==Medicine/Bio==<br /> <br /> ==High Energy Physics==<br /> <br /> ==Structural Biology==<br /> Molecular simulation of proteins, lipids, carbohydrates, and other biologically relevant molecules.<br /> ===Molecular Dynamics (MD) simulation===<br /> ====GROMACS====<br /> Please refer to the [[gromacs|GROMACS]] page<br /> ====AMBER====<br /> Please refer to the [[amber|AMBER]] page<br /> ====NAMD====<br /> NAMD is one of the better scaling MD packages out there. With sufficiently large systems, it is able to scale to hundreds or thousands of cores on Scinet. Below are details for compiling and running NAMD on Scinet.<br /> <br /> More information regarding performance and different compile options coming soon...<br /> <br /> =====Compiling NAMD for GPC=====<br /> Ensure the proper compiler/mpi modules are loaded.<br /> <source lang="sh"><br /> module load intel<br /> module load openmpi/1.3.3-intel-v11.0-ofed<br /> </source><br /> <br /> '''Compile Charm++ and NAMD'''<br /> <source lang="sh"><br /> #Unpack source files and get required support libraries<br /> tar -xzf NAMD_2.7b1_Source.tar.gz<br /> cd NAMD_2.7b1_Source<br /> tar -xf charm-6.1.tar<br /> wget http://www.ks.uiuc.edu/Research/namd/libraries/fftw-linux-x86_64.tar.gz<br /> wget http://www.ks.uiuc.edu/Research/namd/libraries/tcl-linux-x86_64.tar.gz<br /> tar -xzf fftw-linux-x86_64.tar.gz; mv linux-x86_64 fftw<br /> tar -xzf tcl-linux-x86_64.tar.gz; mv linux-x86_64 tcl<br /> #Compile Charm++<br /> cd charm-6.1<br /> ./build charm++ mpi-linux-x86_64 icc --basedir /scinet/gpc/mpi/openmpi/1.3.3-intel-v11.0-ofed/ --no-shared -O -DCMK_OPTIMIZE=1<br /> cd ..<br /> #Compile NAMD. <br /> #Edit arch/Linux-x86_64-icc.arch and add "-lmpi" to the end of the CXXOPTS and COPTS line.<br /> #Make a builds directory if you want different versions of NAMD compiled at the same time.<br /> mkdir builds<br /> ./config builds/Linux-x86_64-icc --charm-arch mpi-linux-x86_64-icc<br /> cd builds/Linux-x86_64-icc/<br /> make -j4 namd2 # Adjust value of j as desired to specify number of simultaneous make targets. <br /> </source><br /> --[[User:Cmadill|Cmadill]] 16:18, 27 August 2009 (UTC)<br /> <br /> =====Running Fortran=====<br /> On the development nodes, there is an old gcc. The associated libraries are not on the compute nodes. Ensure the line:<br /> <br /> module load gcc<br /> <br /> is in your .bashrc file.<br /> <br /> ====LAMMPS====<br /> [[Image:StrongScalingLAMMPS.png|thumb|right|320px|Strong scaling test on GPC with OpenMPI and IntelMPI on Ethernet and InfiniBand]][[Image:WeakScalingLAMMPS.png|thumb|right|320px|Weak scaling test on GPC with OpenMPI and IntelMPI on Ethernet and InfiniBand]]<br /> <br /> ===Monte Carlo (MC) simulation===</div>

Jchu https://oldwiki.scinet.utoronto.ca/index.php?title=User_Codes&diff=872 User Codes 2010-02-02T18:44:42Z

<p>Jchu: /* Molecular Dynamics (MD) simulation */</p> <hr /> <div>__FORCETOC__<br /> <br /> ==Astrophysics==<br /> <br /> ===Athena (explicit, uniform grid MHD code)===<br /> <br /> [[Image:StrongScalingAthenaGPC.png|thumb|right|320px|Athena scaling on GPC with OpenMPI and MVAPICH2 on GigE, and OpenMPI on InfiniBand]]<br /> <br /> [http://www.astro.princeton.edu/~jstone/athena.html Athena] is a straightforward C code which doesn't use a lot of libraries so it is pretty straightforward to build and compile on new machines. <br /> <br /> It encapsulates its compiler flags, etc in an <tt>Makeoptions.in</tt> file which is then processed by <tt>configure</tt>. I've used the following additions to <tt>Makeoptions.in</tt> on TCS and GPC:<br /> <br /> <source lang="sh"><br /> ifeq ($(MACHINE),scinettcs)<br /> CC = mpcc_r<br /> LDR = mpcc_r<br /> OPT = -O5 -q64 -qarch=pwr6 -qtune=pwr6 -qcache=auto -qlargepage -qstrict<br /> MPIINC =<br /> MPILIB =<br /> CFLAGS = $(OPT)<br /> LIB = -ldl -lm<br /> else<br /> ifeq ($(MACHINE),scinetgpc)<br /> CC = mpicc<br /> LDR = mpicc<br /> OPT = -O3<br /> MPIINC =<br /> MPILIB =<br /> CFLAGS = $(OPT)<br /> LIB = -lm<br /> else<br /> ...<br /> endif<br /> endif<br /> </source><br /> It performs quite well on the GPC, scaling extremely well even on a strong scaling test out to about 256 cores (32 nodes) on Gigabit ethernet, and performing beautifully on InfiniBand out to 512 cores (64 nodes). <br /> <br /> <br /> -- [[User:Ljdursi|ljdursi]] 19:20, 13 August 2009 (UTC)<br /> <br /> ===FLASH3 (Adaptive Mesh reactive hydrodynamics; explict hydro/MHD)===<br /> <br /> [[Image:weak-scaling-example.png|thumb|right|320px|Weak scaling test of the 2d sod problem on both the GPC and TCS. The results are actually somewhat faster on the GPC; in both cases (weak) scaling is very good out at least to 256 cores]]<br /> <br /> [http://flash.uchicago.edu FLASH] encapsulates its machine-dependant information in the <tt>FLASH3/sites</tt> directory. For the GPC, you'll have to<br /> <pre><br /> module load intel<br /> module load openmpi<br /> module load hdf5/183-v16-openmpi<br /> </pre><br /> <br /> and with that, the following file (<tt>sites/scinetgpc/Makefile.h</tt>) works for me:<br /> <source lang="sh"><br /> ## Must do module load hdf5/183-v16-openmpi<br /> HDF5_PATH = ${SCINET_HDF5_BASE}<br /> ZLIB_PATH = /usr/local<br /> <br /> #----------------------------------------------------------------------------<br /> # Compiler and linker commands<br /> #<br /> # We use the f90 compiler as the linker, so some C libraries may explicitly<br /> # need to be added into the link line.<br /> #----------------------------------------------------------------------------<br /> <br /> ## modules will put the right mpi in our path<br /> FCOMP = mpif77<br /> CCOMP = mpicc<br /> CPPCOMP = mpiCC<br /> LINK = mpif77<br /> <br /> #----------------------------------------------------------------------------<br /> # Compilation flags<br /> #<br /> # Three sets of compilation/linking flags are defined: one for optimized<br /> # code, one for testing, and one for debugging. The default is to use the <br /> # _OPT version. Specifying -debug to setup will pick the _DEBUG version,<br /> # these should enable bounds checking. Specifying -test is used for <br /> # flash_test, and is set for quick code generation, and (sometimes) <br /> # profiling. The Makefile generated by setup will assign the generic token <br /> # (ex. FFLAGS) to the proper set of flags (ex. FFLAGS_OPT).<br /> #----------------------------------------------------------------------------<br /> <br /> FFLAGS_OPT = -c -r8 -i4 -O3 -xSSE4.2<br /> FFLAGS_DEBUG = -c -g -r8 -i4 -O0<br /> FFLAGS_TEST = -c -r8 -i4<br /> <br /> <br /> # if we are using HDF5, we need to specify the path to the include files<br /> CFLAGS_HDF5 = -I${HDF5_PATH}/include<br /> <br /> CFLAGS_OPT = -c -O3 -xSSE4.2<br /> CFLAGS_TEST = -c -O2 <br /> CFLAGS_DEBUG = -c -g <br /> <br /> MDEFS = <br /> <br /> .SUFFIXES: .o .c .f .F .h .fh .F90 .f90<br /> <br /> #----------------------------------------------------------------------------<br /> # Linker flags<br /> #<br /> # There is a seperate version of the linker flags for each of the _OPT, <br /> # _DEBUG, and _TEST cases.<br /> #----------------------------------------------------------------------------<br /> <br /> LFLAGS_OPT = -o<br /> LFLAGS_TEST = -o<br /> LFLAGS_DEBUG = -g -o<br /> <br /> MACHOBJ = <br /> <br /> <br /> MV = mv -f<br /> AR = ar -r<br /> RM = rm -f<br /> CD = cd<br /> RL = ranlib<br /> ECHO = echo<br /> </source><br /> <br /> <br /> -- [[User:Ljdursi|ljdursi]] 22:11, 13 August 2009 (UTC)<br /> <br /> ==Aeronautics==<br /> <br /> ==Chemistry==<br /> <br /> ===GAMESS (US)===<br /> The GAMESS version January 12, 2009 R3 was built using the Intel v11.1 compilers and v3.2.2 MPI library, according to the instructions in http://software.intel.com/en-us/articles/building-gamess-with-intel-compilers-intel-mkl-and-intel-mpi-on-linux/<br /> <br /> The required build scripts - comp, compall, lked - and run script - rungms - were modified to account for our own installation. In order to build GAMESS one first must ensure that the intel and intelmpi modules are loaded ("module load intel intelmpi"). This applies to running GAMESS as well.<br /> The module "gamess" must also be loaded in order to run GAMESS ("module load gamess").<br /> <br /> The modified scripts are in the file /scinet/gpc/src/gamess-on-scinet.tar.gz<br /> <br /> ====Running GAMESS====<br /> - Make sure the directory /scratch/$USER/gamess-scratch exists (the $SCINET_RUNGMS script will create it if it does not exist)<br /> <br /> - Make sure the modules: intel, intelmpi, gamess are loaded (in your .bashrc: "module load intel intelmpi gamess").<br /> <br /> - Create a torque script to run GAMESS. Here is an example:<br /> <br /> - The GAMESS executable is in $SCINET_GAMESS_HOME/gamess.00.x<br /> - The rungms script is in $SCINET_GAMESS_HOME/rungms (actually it is $SCINET_RUNGMS)<br /> <br /> - For IB multinode runs, use the $SCINET_RUNGMS_IB script<br /> <br /> - The rungms script takes 4 arguments: input file, executable number, number of compute processes, processors per node<br /> <br /> For example, in order to run with the input file /scratch/$USER/gamesstest01, on 8 cpus, and the default version (00)<br /> of the executable on a machine with 8 cores:<br /> <br /> # load the gamess module in .bashrc<br /> module load gamess <br /> <br /> # run the program<br /> $SCINET_RUNGMS /scratch/$USER/gamesstest01 00 8 8<br /> <br /> Here is a sample torque script for running a GAMESS calculation, on a single 8-core node:<br /> <br /> <source lang="bash"><br /> #!/bin/bash<br /> #PBS -l nodes=1:ppn=8,walltime=48:00:00,os=centos53computeA<br /> #PBS -N gamessjob<br /> <br /> ## To submit type: qsub gms.sh<br /> <br /> # If not an interactive job (i.e. -I), then cd into the directory where<br /> # I typed qsub.<br /> if [ "$PBS_ENVIRONMENT" != "PBS_INTERACTIVE" ]; then<br /> if [ -n "$PBS_O_WORKDIR" ]; then<br /> cd $PBS_O_WORKDIR<br /> fi<br /> fi<br /> <br /> # the input file is typically named something like "gamesjob.inp"<br /> # so the script will be run like "$SCINET_RUNGMS gamessjob 00 8 8"<br /> <br /> # load the gamess module if not in .bashrc already<br /> # actually, it MUST be in .bashrc<br /> # module load gamess<br /> <br /> # run the program<br /> <br /> $SCINET_RUNGMS gamessjob 00 8 8<br /> </source><br /> <br /> Here is a similar script, but this one uses 2 InfiniBand-connected nodes,<br /> and runs the appropriate $SCINET_RUNGMS_IB script to actually run the job:<br /> <br /> <source lang="bash"><br /> #!/bin/bash<br /> #PBS -l nodes=2:ib:ppn=8,walltime=48:00:00,os=centos53computeA<br /> #PBS -N gamessjob<br /> <br /> ## To submit type: qsub gmsib.sh<br /> <br /> # If not an interactive job (i.e. -I), then cd into the directory where<br /> # I typed qsub.<br /> if [ "$PBS_ENVIRONMENT" != "PBS_INTERACTIVE" ]; then<br /> if [ -n "$PBS_O_WORKDIR" ]; then<br /> cd $PBS_O_WORKDIR<br /> fi<br /> fi<br /> <br /> # the input file is typically named something like "gamesjob.inp"<br /> # so the script will be run like "$SCINET_RUNGMS gamessjob 00 8 8"<br /> <br /> # load the gamess module if not in .bashrc already<br /> # actually, it MUST be in .bashrc<br /> # module load gamess<br /> <br /> # This script requests InfiniBand-connected nodes (:ib above)<br /> # so it must run with the IB version of the rungms script,<br /> # $SCINET_RUNGMS_IB<br /> <br /> $SCINET_RUNGMS_IB gamessjob 00 16 8<br /> </source><br /> <br /> <br /> -- [[User:Dgruner|dgruner]] 5 October 2009<br /> <br /> ===Tips from the Fekl Lab===<br /> <br /> Through trial and error, we have found a few useful things that we would like to share:<br /> <br /> 1. Two very useful, open-source programs for visualization of output files from GAMESS(US) and for generation of input files are [http://www.scl.ameslab.gov/MacMolPlt/ MacMolPlt]and [http://avogadro.openmolecules.net/wiki/Main_Page Avogadro]. The are available for UNIX/LINUX, Windows and Mac based machines, HOWEVER: any input files that we have generated with these programs on a Windows-based machine do not run on Mac based machines. We don't know why.<br /> <br /> 2. [http://winscp.net/eng/index.php WinSCP] is a very useful tool that has a graphical user interface for moving files from a local machine to SCINET and vice versa. It also has text editing capabilities.<br /> <br /> 3. The [https://bse.pnl.gov/bse/portal ESML Basis Set Exchange] is an excellent source for custom basis set or effective core potential parameters. Make sure that you specify "Gamess-US" in the format drop-down box.<br /> <br /> 4. The commercial program [http://www.chemcraftprog.com/ ChemCraft] is a highly useful visualization program that has the ability to edit molecules in a very similar fashion to GaussView. It can also be customized to build GAMESS(US) input files.<br /> <br /> ====Anatomy of a GAMESS(US) Input File with Basis Set Info in an External File====<br /> <br /> $CONTRL SCFTYP=RHF RUNTYP=OPTIMIZE DFTTYP=M06-L MAXIT=199 MULT=1 NOSYM=1<br /> ECP=READ $END<br /> $SYSTEM TIMLIM=525600 MWORDS=1750 PARALL=.TRUE. $END<br /> $BASIS GBASIS=CUSTOMNI EXTFIL=.t. $END<br /> $SCF DIRSCF=.TRUE. FDIFF=.f. $END<br /> $STATPT OPTTOL=0.0001 NSTEP=500 HSSEND=.t. $END<br /> $DATA<br /> Mo_BDT3<br /> C1<br /> MOLYBDENUM 42.0 5.7556500000 4.4039600000 16.5808400000<br /> SULFUR 16.0 7.4169700000 3.1956300000 15.2089300000<br /> SULFUR 16.0 4.0966800000 3.2258300000 15.1761100000<br /> SULFUR 16.0 3.9677300000 4.4940500000 18.3266100000<br /> SULFUR 16.0 7.1776900000 3.5815000000 18.4485200000<br /> SULFUR 16.0 4.3776600000 6.2447400000 15.6786900000<br /> SULFUR 16.0 7.5478700000 6.0679800000 16.2223700000<br /> CARBON 6.0 6.4716900000 2.1004800000 14.1902300000<br /> CARBON 6.0 5.0690300000 2.1781400000 14.1080700000<br /> CARBON 6.0 4.8421800000 4.2701300000 19.8855500000<br /> CARBON 6.0 6.1969000000 3.9249600000 19.9397400000<br /> CARBON 6.0 6.8280600000 3.7834200000 21.1913200000<br /> CARBON 6.0 5.7697600000 7.6933500000 17.4241800000<br /> CARBON 6.0 7.2043100000 7.9413600000 17.8281100000<br /> CARBON 6.0 5.5051400000 7.0409700000 14.5903800000<br /> CARBON 6.0 6.8905200000 6.9194700000 14.7626200000<br /> CARBON 6.0 7.7396400000 7.5379800000 13.8285700000<br /> HYDROGEN 1.0 8.8190700000 7.4520600000 13.9252200000<br /> CARBON 6.0 7.2169400000 8.2960300000 12.7704100000<br /> HYDROGEN 1.0 7.8667000000 8.7825100000 12.0575600000<br /> CARBON 6.0 5.8260300000 8.4502300000 12.6467800000<br /> HYDROGEN 1.0 5.4143000000 9.0544300000 11.8493100000<br /> CARBON 6.0 4.9881500000 7.8192300000 13.5528400000<br /> HYDROGEN 1.0 3.9090500000 7.9420000000 13.4583700000<br /> CARBON 6.0 7.1538500000 1.1569600000 13.4143900000<br /> CARBON 6.0 4.4018100000 1.3603900000 13.1919900000<br /> CARBON 6.0 6.4791600000 0.3185500000 12.5353300000<br /> CARBON 6.0 5.0837400000 0.4369500000 12.4084900000<br /> HYDROGEN 1.0 7.0116000000 -0.4099400000 11.9434600000<br /> HYDROGEN 1.0 8.2399000000 1.0702400000 13.4937600000<br /> HYDROGEN 1.0 3.3185600000 1.4368700000 13.0953100000<br /> HYDROGEN 1.0 4.5549800000 -0.1997300000 11.7165200000<br /> CARBON 6.0 6.1105700000 3.9639000000 22.3866100000<br /> CARBON 6.0 4.1216300000 4.4424400000 21.1020100000<br /> HYDROGEN 1.0 7.8732900000 3.5217100000 21.2520500000<br /> CARBON 6.0 4.7606000000 4.2868500000 22.3363800000<br /> HYDROGEN 1.0 6.6064200000 3.8406000000 23.3428500000<br /> HYDROGEN 1.0 4.2065000000 4.4170700000 23.2667100000<br /> HYDROGEN 1.0 3.0674000000 4.6893500000 21.0889000000<br /> HYDROGEN 1.0 7.4249200000 7.7545300000 18.8583200000<br /> HYDROGEN 1.0 7.6651700000 8.9049700000 17.7652100000<br /> HYDROGEN 1.0 5.3324000000 8.6487800000 17.2222700000<br /> HYDROGEN 1.0 5.5015000000 7.1039000000 18.2759400000<br /> $END<br /> $ECP<br /> MO-ECP GEN 28 3<br /> 5 ----- f potential -----<br /> -0.0469492 0 537.9667807 <br /> -20.2080084 1 147.8982938 <br /> -106.2116302 2 45.7358898 <br /> -41.8107368 2 13.2911467 <br /> -4.2054103 2 4.7059961 <br /> 3 ----- s-f potential -----<br /> 2.8063717 0 110.2991760 <br /> 44.5162012 1 23.2014645 <br /> 82.7785227 2 5.3530131 <br /> 4 ----- p-f potential -----<br /> 4.9420876 0 63.2901397 <br /> 25.8604976 1 23.3315302 <br /> 132.4708742 2 24.6759423 <br /> 57.3149794 2 4.6493040 <br /> 5 ----- d-f potential -----<br /> 3.0054591 0 104.4839977 <br /> 26.3637851 1 66.2307245 <br /> 183.3849199 2 39.1283176 <br /> 98.4453068 2 13.1164437 <br /> 22.4901377 2 3.6280263 <br /> S NONE<br /> S NONE<br /> S NONE<br /> S NONE<br /> S NONE<br /> S NONE<br /> C NONE<br /> C NONE<br /> C NONE<br /> C NONE<br /> C NONE<br /> C NONE<br /> C NONE<br /> C NONE<br /> C NONE<br /> C NONE<br /> H NONE<br /> C NONE<br /> H NONE<br /> C NONE<br /> H NONE<br /> C NONE<br /> H NONE<br /> C NONE<br /> C NONE<br /> C NONE<br /> C NONE<br /> H NONE<br /> H NONE<br /> H NONE<br /> H NONE<br /> C NONE<br /> C NONE<br /> H NONE<br /> C NONE<br /> H NONE<br /> H NONE<br /> H NONE<br /> H NONE<br /> H NONE<br /> H NONE<br /> H NONE<br /> $END<br /> <br /> =====The Input Deck=====<br /> <br /> Below is the input deck. It is where you tell GAMESS(US) what job type to execute and where all you individual parameters are entered for your specific job type. The example input deck below is for a geometry optimization and frequency calculation. This input deck is equivalent to the Gaussian job with "opt" and "freq" in the route section.<br /> <br /> $CONTRL SCFTYP=RHF RUNTYP=OPTIMIZE DFTTYP=M06-L MAXIT=199 MULT=1 NOSYM=1<br /> ECP=READ $END<br /> $SYSTEM TIMLIM=2850 MWORDS=1750 MEMDDI=20 PARALL=.TRUE. $END<br /> $BASIS GBASIS=CUSTOMNI EXTFIL=.t. $END<br /> $SCF DIRSCF=.TRUE. FDIFF=.f. $END<br /> $STATPT OPTTOL=0.0001 NSTEP=500 HSSEND=.t. $END<br /> $DATA<br /> <br /> An important thing to note is the spacing. In the input deck, there must be 1 space at the beginning of each line of the input deck. If not, the job will fail. Most builders will insert this space anyway, but it helps to double check.<br /> <br /> The end of the input deck is marked by the "$DATA" line.<br /> <br /> =====Job Title Line=====<br /> <br /> The next line of the file is the job title. It can be anthing you wish, however, we have found that to be on the safe side, we avoide using symbols or spaces<br /> <br /> Mo_BDT3<br /> <br /> =====Symmetry Point Group=====<br /> <br /> The next line of the file is the symmetry point group of your molecule. Note that there is no leading space before the point group.<br /> <br /> C1<br /> <br /> =====Coordinates=====<br /> <br /> The next block of text is set aside for the coordinates of the molecule. This can be in internal (or z-matrix) format or cartesian coordinates. Note that there is no leading space before the coordinates. One may use the chemical symbol or the full name of each atom in the molecule. Note that the end of the coordinates is signified by an "$END", which MUST have one space preceding it. The coordinates below do NOT have any basis set information inserted. It is possible to insert basis set information directly into the input file. This is accomplished by obtaining the desired basis set parameters from the EMSL and then inserting them below each relevant atom. An example input file with inserted basis set information will be shown later.<br /> <br /> MOLYBDENUM 42.0 5.7556500000 4.4039600000 16.5808400000<br /> SULFUR 16.0 7.4169700000 3.1956300000 15.2089300000<br /> SULFUR 16.0 4.0966800000 3.2258300000 15.1761100000<br /> SULFUR 16.0 3.9677300000 4.4940500000 18.3266100000<br /> SULFUR 16.0 7.1776900000 3.5815000000 18.4485200000<br /> SULFUR 16.0 4.3776600000 6.2447400000 15.6786900000<br /> SULFUR 16.0 7.5478700000 6.0679800000 16.2223700000<br /> CARBON 6.0 6.4716900000 2.1004800000 14.1902300000<br /> CARBON 6.0 5.0690300000 2.1781400000 14.1080700000<br /> CARBON 6.0 4.8421800000 4.2701300000 19.8855500000<br /> CARBON 6.0 6.1969000000 3.9249600000 19.9397400000<br /> CARBON 6.0 6.8280600000 3.7834200000 21.1913200000<br /> CARBON 6.0 5.7697600000 7.6933500000 17.4241800000<br /> CARBON 6.0 7.2043100000 7.9413600000 17.8281100000<br /> CARBON 6.0 5.5051400000 7.0409700000 14.5903800000<br /> CARBON 6.0 6.8905200000 6.9194700000 14.7626200000<br /> CARBON 6.0 7.7396400000 7.5379800000 13.8285700000<br /> HYDROGEN 1.0 8.8190700000 7.4520600000 13.9252200000<br /> CARBON 6.0 7.2169400000 8.2960300000 12.7704100000<br /> HYDROGEN 1.0 7.8667000000 8.7825100000 12.0575600000<br /> CARBON 6.0 5.8260300000 8.4502300000 12.6467800000<br /> HYDROGEN 1.0 5.4143000000 9.0544300000 11.8493100000<br /> CARBON 6.0 4.9881500000 7.8192300000 13.5528400000<br /> HYDROGEN 1.0 3.9090500000 7.9420000000 13.4583700000<br /> CARBON 6.0 7.1538500000 1.1569600000 13.4143900000<br /> CARBON 6.0 4.4018100000 1.3603900000 13.1919900000<br /> CARBON 6.0 6.4791600000 0.3185500000 12.5353300000<br /> CARBON 6.0 5.0837400000 0.4369500000 12.4084900000<br /> HYDROGEN 1.0 7.0116000000 -0.4099400000 11.9434600000<br /> HYDROGEN 1.0 8.2399000000 1.0702400000 13.4937600000<br /> HYDROGEN 1.0 3.3185600000 1.4368700000 13.0953100000<br /> HYDROGEN 1.0 4.5549800000 -0.1997300000 11.7165200000<br /> CARBON 6.0 6.1105700000 3.9639000000 22.3866100000<br /> CARBON 6.0 4.1216300000 4.4424400000 21.1020100000<br /> HYDROGEN 1.0 7.8732900000 3.5217100000 21.2520500000<br /> CARBON 6.0 4.7606000000 4.2868500000 22.3363800000<br /> HYDROGEN 1.0 6.6064200000 3.8406000000 23.3428500000<br /> HYDROGEN 1.0 4.2065000000 4.4170700000 23.2667100000<br /> HYDROGEN 1.0 3.0674000000 4.6893500000 21.0889000000<br /> HYDROGEN 1.0 7.4249200000 7.7545300000 18.8583200000<br /> HYDROGEN 1.0 7.6651700000 8.9049700000 17.7652100000<br /> HYDROGEN 1.0 5.3324000000 8.6487800000 17.2222700000<br /> HYDROGEN 1.0 5.5015000000 7.1039000000 18.2759400000<br /> $END<br /> <br /> =====Effective Core Potential Data=====<br /> <br /> The effective core potential (ECP) data is entered after the coordinates. It starts with "$ECP", which must be preceded with a space. The atoms of the molecule are listed in the same order as in the coordinates section and the parameters for the ECP are listed after each atom. Note that for any atom that does NOT have an ECP, one must enter "ECP-NONE" or "NONE" after each atom without an ECP.<br /> <br /> $ECP<br /> MO-ECP GEN 28 3<br /> 5 ----- f potential -----<br /> -0.0469492 0 537.9667807 <br /> -20.2080084 1 147.8982938 <br /> -106.2116302 2 45.7358898 <br /> -41.8107368 2 13.2911467 <br /> -4.2054103 2 4.7059961 <br /> 3 ----- s-f potential -----<br /> 2.8063717 0 110.2991760 <br /> 44.5162012 1 23.2014645 <br /> 82.7785227 2 5.3530131 <br /> 4 ----- p-f potential -----<br /> 4.9420876 0 63.2901397 <br /> 25.8604976 1 23.3315302 <br /> 132.4708742 2 24.6759423 <br /> 57.3149794 2 4.6493040 <br /> 5 ----- d-f potential -----<br /> 3.0054591 0 104.4839977 <br /> 26.3637851 1 66.2307245 <br /> 183.3849199 2 39.1283176 <br /> 98.4453068 2 13.1164437 <br /> 22.4901377 2 3.6280263 <br /> S NONE<br /> S NONE<br /> S NONE<br /> S NONE<br /> S NONE<br /> S NONE<br /> C NONE<br /> C NONE<br /> C NONE<br /> C NONE<br /> C NONE<br /> C NONE<br /> C NONE<br /> C NONE<br /> C NONE<br /> C NONE<br /> H NONE<br /> C NONE<br /> H NONE<br /> C NONE<br /> H NONE<br /> C NONE<br /> H NONE<br /> C NONE<br /> C NONE<br /> C NONE<br /> C NONE<br /> H NONE<br /> H NONE<br /> H NONE<br /> H NONE<br /> C NONE<br /> C NONE<br /> H NONE<br /> C NONE<br /> H NONE<br /> H NONE<br /> H NONE<br /> H NONE<br /> H NONE<br /> H NONE<br /> H NONE<br /> $END<br /> <br /> -- [[User:M.Zimmer-De Iuliis|mzd]] 16 November 2009<br /> <br /> ====Using Effective Core Potentials in GAMESS(US)====<br /> <br /> For many metal containing compounds, it is very convenient and time saving to use an effective core potential (ECP) for the core metal electrons, as they are usually not important to the reactivity of the complex or the geometry around the metal. Since GAMESS(US) has a limited number of built-in ECPs, one may want to make GAMESS(US) read an external file that contains the ECP data using the "EXTFIL" keyword in the $GBASIS command line of the input file. In addition, to make GAMESS(US) use this external file, one must copy the "rungms" file and modify it accordingly. The following is a list of instructions with commands that will work from a terminal. One could also use WinSCP to do all of this with a GUI rather than a TUI. <br /> <br /> =====Modifiying rungms to Use Custom Basis Set File=====<br /> 1. Copy "rungms" from /scinet/gpc/Applications/gamess to one's own /scratch/$USER/ directory:<br /> cp /scinet/gpc/Applications/gamess/rungms /scratch/$USER/<br /> <br /> 2. Change to the scratch directory and check to see if "rungms" has copied successfully.<br /> cd /scratch/$USER<br /> ls<br /> <br /> 3. Edit line 147 of the vi. <br /> vi rungms<br /> Move the cursor down to line 147 using the arrow keys. It should say "setenv EXTBAS /dev/null". Using the arrow keys, move the cursor to the first "/" and then hit "i" to insert text. Put the path to your external basis file here. For example, /scratch/$USER/basisset. Then hit "escape". To save the changes and exit vi, type ":" and you should see a colon appear at the bottom of the window. Type "wq" (which should appear at the bottom of the window next to the colon) and then hit enter. Now you are done with vi.<br /> <br /> =====Creating a Custom Basis Set File=====<br /> 1. To create a custom basis set file, you need create a new text document. Our group's common practice is to comment out the first line of this file by inserting an exclamation mark (!) followed by noting the specific basis sets and ECPs that are going to be used for each of the atoms. Let us the molecule Mo(CO)6, Molybdenum hexacarbonyl, as an example. Below is the first line of the the external file, which we will call "CUSTOMMO" (NOTE: you can use any name for the external file that suits you, as long as it has no spaces and is 8 characters or less).<br /> <br /> ! 6-31G on C and O and LANL2D2 ECP on Mo<br /> <br /> 2. The next step is to visit the [https://bse.pnl.gov/bse/portal EMSL Basis Set exchange] and select C and O from the periodic table. Then, on the left of the page, select "6-31G" as the basis set. Finally, make sure the output is in GAMESS(US) format using the drop-down menu and then click "get basis set".<br /> <br /> [[File:C_O_6_31G_basisset.JPG|centre]]<br /> <br /> 3. A new window should appear with text in it. For our example case, the text looks like this:<br /> <br /> ! 6-31G EMSL Basis Set Exchange Library 10/13/09 11:12 AM<br /> ! Elements References<br /> ! -------- ----------<br /> ! H - He: W.J. Hehre, R. Ditchfield and J.A. Pople, J. Chem. Phys. 56,<br /> ! Li - Ne: 2257 (1972). Note: Li and B come from J.D. Dill and J.A.<br /> ! Pople, J. Chem. Phys. 62, 2921 (1975).<br /> ! Na - Ar: M.M. Francl, W.J. Petro, W.J. Hehre, J.S. Binkley, M.S. Gordon,<br /> ! D.J. DeFrees and J.A. Pople, J. Chem. Phys. 77, 3654 (1982)<br /> ! K - Zn: V. Rassolov, J.A. Pople, M. Ratner and T.L. Windus, J. Chem. Phys.<br /> ! 109, 1223 (1998)<br /> ! Note: He and Ne are unpublished basis sets taken from the Gaussian<br /> ! program<br /> ! <br /> $DATA<br /><br /> CARBON<br /> S 6<br /> 1 3047.5249000 0.0018347 <br /> 2 457.3695100 0.0140373 <br /> 3 103.9486900 0.0688426 <br /> 4 29.2101550 0.2321844 <br /> 5 9.2866630 0.4679413 <br /> 6 3.1639270 0.3623120 <br /> L 3<br /> 1 7.8682724 -0.1193324 0.0689991 <br /> 2 1.8812885 -0.1608542 0.3164240 <br /> 3 0.5442493 1.1434564 0.7443083 <br /> L 1<br /> 1 0.1687144 1.0000000 1.0000000<br /> <br /> OXYGEN<br /> S 6<br /> 1 5484.6717000 0.0018311 <br /> 2 825.2349500 0.0139501 <br /> 3 188.0469600 0.0684451 <br /> 4 52.9645000 0.2327143 <br /> 5 16.8975700 0.4701930 <br /> 6 5.7996353 0.3585209 <br /> L 3<br /> 1 15.5396160 -0.1107775 0.0708743 <br /> 2 3.5999336 -0.1480263 0.3397528 <br /> 3 1.0137618 1.1307670 0.7271586 <br /> L 1<br /> 1 0.2700058 1.0000000 1.0000000 <br /> $END<br /> <br /> 3. Now, copy and paste the text between the $DATA and $END headings onto our external text file, CUSTOMMO. We also need to change the change the name of each element to it symbol in the periodic table. Finally, we need to add the name of the external file next to the element symbol, separated by one space. Note that there should be a blank line separating the basis set information and the first, commented-out line (The line starting with the '!'). The CUSTOMMO should look like this:<br /> <br /> ! 6-31G on C and O and LANL2D2 ECP on Mo<br /><br /> C CUSTOMMO<br /> S 6<br /> 1 3047.5249000 0.0018347 <br /> 2 457.3695100 0.0140373 <br /> 3 103.9486900 0.0688426 <br /> 4 29.2101550 0.2321844 <br /> 5 9.2866630 0.4679413 <br /> 6 3.1639270 0.3623120 <br /> L 3<br /> 1 7.8682724 -0.1193324 0.0689991 <br /> 2 1.8812885 -0.1608542 0.3164240 <br /> 3 0.5442493 1.1434564 0.7443083 <br /> L 1<br /> 1 0.1687144 1.0000000 1.0000000<br /> <br /> O CUSTOMMO<br /> S 6<br /> 1 5484.6717000 0.0018311 <br /> 2 825.2349500 0.0139501 <br /> 3 188.0469600 0.0684451 <br /> 4 52.9645000 0.2327143 <br /> 5 16.8975700 0.4701930 <br /> 6 5.7996353 0.3585209 <br /> L 3<br /> 1 15.5396160 -0.1107775 0.0708743 <br /> 2 3.5999336 -0.1480263 0.3397528 <br /> 3 1.0137618 1.1307670 0.7271586 <br /> L 1<br /> 1 0.2700058 1.0000000 1.0000000 <br /> <br /> 4. Repeat Step 3 above but choose Mo and select the LANL2DZ ECP instead. A new window will pop up with the basis set information as well as the ECP data we need, since we specified the LANL2DZ '''ECP'''. The ECP data is not inserted into the external file, rather it is placed into the input file itself (More on this later). <br /> <br /> [[File:Mo_LANL2DZ_basisset.JPG|centre]]<br /> <br /> 5. After copying the molybdenum basis set information, your fiished external basis set file should look like this:<br /> ! 6-31G on C and O and LANL2D2 ECP on Mo<br /><br /> C CUSTOMMO<br /> S 6<br /> 1 3047.5249000 0.0018347 <br /> 2 457.3695100 0.0140373 <br /> 3 103.9486900 0.0688426 <br /> 4 29.2101550 0.2321844 <br /> 5 9.2866630 0.4679413 <br /> 6 3.1639270 0.3623120 <br /> L 3<br /> 1 7.8682724 -0.1193324 0.0689991 <br /> 2 1.8812885 -0.1608542 0.3164240 <br /> 3 0.5442493 1.1434564 0.7443083 <br /> L 1<br /> 1 0.1687144 1.0000000 1.0000000<br /> <br /> O CUSTOMMO<br /> S 6<br /> 1 5484.6717000 0.0018311 <br /> 2 825.2349500 0.0139501 <br /> 3 188.0469600 0.0684451 <br /> 4 52.9645000 0.2327143 <br /> 5 16.8975700 0.4701930 <br /> 6 5.7996353 0.3585209 <br /> L 3<br /> 1 15.5396160 -0.1107775 0.0708743 <br /> 2 3.5999336 -0.1480263 0.3397528 <br /> 3 1.0137618 1.1307670 0.7271586 <br /> L 1<br /> 1 0.2700058 1.0000000 1.0000000<br /> <br /> Mo CUSTOMO<br /> S 3<br /> 1 2.3610000 -0.9121760 <br /> 2 1.3090000 1.1477453 <br /> 3 0.4500000 0.6097109 <br /> S 4<br /> 1 2.3610000 0.8139259 <br /> 2 1.3090000 -1.1360084 <br /> 3 0.4500000 -1.1611592 <br /> 4 0.1681000 1.0064786 <br /> S 1<br /> 1 0.0423000 1.0000000 <br /> P 3<br /> 1 4.8950000 -0.0908258 <br /> 2 1.0440000 0.7042899 <br /> 3 0.3877000 0.3973179 <br /> P 2<br /> 1 0.4995000 -0.1081945 <br /> 2 0.0780000 1.0368093 <br /> P 1<br /> 1 0.0247000 1.0000000 <br /> D 3<br /> 1 2.9930000 0.0527063 <br /> 2 1.0630000 0.5003907 <br /> 3 0.3721000 0.5794024 <br /> D 1<br /> 1 0.1178000 1.0000000<br /> <br /> -- [[User:M.Zimmer-De Iuliis|mzd]] 13 October 2009<br /> <br /> ==Climate Modelling==<br /> <br /> ==Medicine/Bio==<br /> <br /> ==High Energy Physics==<br /> <br /> ==Structural Biology==<br /> Molecular simulation of proteins, lipids, carbohydrates, and other biologically relevant molecules.<br /> ===Molecular Dynamics (MD) simulation===<br /> ====GROMACS====<br /> Please refer to the [[gromacs|GROMACS]] page<br /> ====AMBER====<br /> Please refer to the [[amber|AMBER]] page<br /> ====NAMD====<br /> NAMD is one of the better scaling MD packages out there. With sufficiently large systems, it is able to scale to hundreds or thousands of cores on Scinet. Below are details for compiling and running NAMD on Scinet.<br /> <br /> More information regarding performance and different compile options coming soon...<br /> <br /> =====Compiling NAMD for GPC=====<br /> Ensure the proper compiler/mpi modules are loaded.<br /> <source lang="sh"><br /> module load intel<br /> module load openmpi/1.3.3-intel-v11.0-ofed<br /> </source><br /> <br /> '''Compile Charm++ and NAMD'''<br /> <source lang="sh"><br /> #Unpack source files and get required support libraries<br /> tar -xzf NAMD_2.7b1_Source.tar.gz<br /> cd NAMD_2.7b1_Source<br /> tar -xf charm-6.1.tar<br /> wget http://www.ks.uiuc.edu/Research/namd/libraries/fftw-linux-x86_64.tar.gz<br /> wget http://www.ks.uiuc.edu/Research/namd/libraries/tcl-linux-x86_64.tar.gz<br /> tar -xzf fftw-linux-x86_64.tar.gz; mv linux-x86_64 fftw<br /> tar -xzf tcl-linux-x86_64.tar.gz; mv linux-x86_64 tcl<br /> #Compile Charm++<br /> cd charm-6.1<br /> ./build charm++ mpi-linux-x86_64 icc --basedir /scinet/gpc/mpi/openmpi/1.3.3-intel-v11.0-ofed/ --no-shared -O -DCMK_OPTIMIZE=1<br /> cd ..<br /> #Compile NAMD. <br /> #Edit arch/Linux-x86_64-icc.arch and add "-lmpi" to the end of the CXXOPTS and COPTS line.<br /> #Make a builds directory if you want different versions of NAMD compiled at the same time.<br /> mkdir builds<br /> ./config builds/Linux-x86_64-icc --charm-arch mpi-linux-x86_64-icc<br /> cd builds/Linux-x86_64-icc/<br /> make -j4 namd2 # Adjust value of j as desired to specify number of simultaneous make targets. <br /> </source><br /> --[[User:Cmadill|Cmadill]] 16:18, 27 August 2009 (UTC)<br /> <br /> =====Running Fortran=====<br /> On the development nodes, there is an old gcc. The associated libraries are not on the compute nodes. Ensure the line:<br /> <br /> module load gcc<br /> <br /> is in your .bashrc file.<br /> <br /> ====LAMMPS====<br /> <br /> ===Monte Carlo (MC) simulation===</div>

Jchu https://oldwiki.scinet.utoronto.ca/index.php?title=File:StrongScalingLAMMPS.png&diff=871 File:StrongScalingLAMMPS.png 2010-02-02T18:43:46Z

<p>Jchu: uploaded a new version of "File:StrongScalingLAMMPS.png": Strong scaling test for LAMMPS (using EAM potentials) on GPC with OpenMPI and IntelMPI for both Ethernet and Infiniband</p> <hr /> <div></div>

Jchu https://oldwiki.scinet.utoronto.ca/index.php?title=File:WeakScalingLAMMPS.png&diff=870 File:WeakScalingLAMMPS.png 2010-02-02T18:42:22Z

<p>Jchu: Weak scaling test for LAMMPS (using EAM potentials) on GPC with OpenMPI and IntelMPI using both Ethernet and Infiniband.</p> <hr /> <div>Weak scaling test for LAMMPS (using EAM potentials) on GPC with OpenMPI and IntelMPI using both Ethernet and Infiniband.</div>

Jchu https://oldwiki.scinet.utoronto.ca/index.php?title=File:StrongScalingLAMMPS.png&diff=869 File:StrongScalingLAMMPS.png 2010-02-02T18:40:46Z

<p>Jchu: </p> <hr /> <div></div>

Jchu https://oldwiki.scinet.utoronto.ca/index.php?title=File:Strong.png&diff=868 File:Strong.png 2010-02-02T18:32:29Z

<p>Jchu: Strong scaling test for LAMMPS (using EAM potentials) on GPC with OpenMPI and IntelMPI for both Ethernet and Infiniband</p> <hr /> <div>Strong scaling test for LAMMPS (using EAM potentials) on GPC with OpenMPI and IntelMPI for both Ethernet and Infiniband</div>

Jchu