User Codes

Fluid Dynamics

Athena (explicit, uniform grid MHD code)

Athena is a straightforward C code which doesn't use a lot of libraries so it is pretty straightforward to build and compile on new machines. It encapsulates its compiler flags, etc in an Makeoptions.in file which is then processed by configure. I've used the following additions to Makeoptions.in on TCS and GPC:

<source lang="sh"> ifeq ($(MACHINE),scinettcs)

 CC = mpcc_r
 LDR = mpcc_r
 OPT = -O5 -q64 -qarch=pwr6 -qtune=pwr6 -qcache=auto -qlargepage -qstrict
 MPIINC =
 MPILIB =
 CFLAGS = $(OPT)
 LIB = -ldl -lm

else ifeq ($(MACHINE),scinetgpc)

 CC = mpicc
 LDR = mpicc
 OPT = -O3
 MPIINC =
 MPILIB =
 CFLAGS = $(OPT)
 LIB = -lm

else ... endif endif </source> 3 It performs quite well on the GPC, scaling extremely well even on a strong scaling test out to about 256 cores (32 nodes) on Gigabit ethernet, and performing beautifully on InfiniBand out to 512 cores (64 nodes).

-- ljdursi 19:20, 13 August 2009 (UTC)