Knowledge Base: Tutorials and Manuals

WARNING: SciNet is in the process of replacing this wiki with a new documentation site. The links below may still contain valuable information, but for current information, please go to https://docs.scinet.utoronto.ca

Training material

For upcoming classes, see our Training and Education website!

SciNet Basics

• SciNet User Tutorial
• SciNet Command Line Utilities
• Intro to SciNet May 2014: Video/Slides
• SciNet Resources: Video/ Slides
• Essentials
• Frequenty asked questions
• Ssh
• GPC quickstart
• TCS quickstart
• ARC/GPU quickstart
• ARC/Cell quickstart
• Important .bashrc guidelines
• Workflow Optimization (w/focus on Large Scale BioInformatics)
• Software and libraries
• Installing your own modules
• User-space modules and packages (April 2011 SNUG TechTalk)
• HPSS - SciNet's storage capacity expansion
• BGQ Hardware Overview Slides / Video Recording
• Intro to Using the BGQ Slides /Video Recording

Linux

• Linux Tutorial (warning: the last part of this tutorial is specific to the csh shell, whereas SciNet uses the bash shell).
• Linux Command Line: A Primer (June 2012) Slides, Files
• Introduction to the Linux Shell, SciNet, Mar 2012: Slides and Data files

Batch job management

• Job and Queue Management Tools (Techtalk Mar.11, 2015)
• Workflow Optimization (w/focus on Large Scale BioInformatics)
• GNU Parallel (Techtalk Nov 14, 2012)
• Job Monitoring on SciNet and Job Efficiency
• Wallclock time
• Checkpoints
• Signals
• Moab
• Serial Jobs (including GNU Parallel)
• Ramdisk
• Moab workload manager
• Moab commands
• Torque resource manager
• Torque PBS commands
• Parallel environment
• Cluster information center (with error codes)
• LoadLeveler: using & administering

Programming

General

• Performance Tuning with the IBM XL Compilers: Slides fromt the SciNet Develop Seminar by Kit Barton, Sep 17, 2012.
• Remote Development, slides from TechTalk Jun 13, 2012
• Scientific Software Development Course, part I of the SciNet's Scientific Computing Course
• Software Carpentry Resources
• Version Control: Video/ Slides
• IBM AIX Workshop, SciNet, Nov 2008
• IBM Compiler Workshop, SciNet, Feb 2009
• SNUG Techtalk Dec 2011 Intel Compiler Optimizations

Fortran

• Modern Fortran Course (1 day), SciNet, 19 Apr 2011
  • Slides
  • Source Code
• [http://support.scinet.utoronto.ca/Manuals/xlf-compiler.pdf IBM Fortran compiler language, optimization

C++

• Slides and recording of the SciNet Developer Seminar on C++11, March 20, 2013
• Scientific C++ Course (1 day), SciNet, 15 March 2011
  • Slides (updated on Apr 26, 2012)
  • Example source code
  • Videos of the Scientific C++ class
• Intel C & C++ compiler

ro/en-us/fortran/lin/compiler_f/index.htm Intel Fortran compiler]

• IBM C++ compiler, language, optimization

C

• C refresher: Video/ Slides
• IBM C compiler, language, optimization

Cilk Plus

• CilkPlus - Getting Started (slides of the May 2014 TechTalk by Chris Szalwinski)

Hadoop

• Introduction to Hadoop for HPCers, Part I - MapReduce: Slides, Source Code, Virtual Machine

Perl

• Perl

Python

• Python
• IPython Notebook on GPC (January 2014 TechTalk)
• Research Computing with Python (Modular Course, Fall 2013)
• Python GUIs with Python and TkInter slides code
• f2py: Fortran and Python (June 2011 TechTalk by Pierre de Buyl)

R

• R Statistical Package

Lua

• Scripting HALMD with Lua and Luabind (May 2011 TechTalk by Peter Colberg)

Parallel Programming

• 2014 Ontario Summerschool on High Performance Computing Central (incl. OpenMP and MPI)
• High Performance Scientific Computing Course 2014 (OpenMP and MPI, with video recordings of lectures)
• MPI 3 Developer Seminar
• OpenMP 4 Developer Seminar
• Ontario Summerschool on High Performance Computing Central (2013)
• High Performance Scientific Computing, part 3 of SciNet's Scientific Computing Course (Winter 2012)
• Parallel Programming Course (5 days), SciNet, May 2011
  • Videos, slides and code
• Parallel Computing for Computational Fluid Dynamics (CFD), SciNet, 23 March 2011
  • Slides
  • Source Code
• Intro to Practical Parallel Programming (1 day), SciNet, 22 Sept 2010:
  • Morning Slides, Intro and OpenMP
  • Afternoon Slides, MPI
  • Example source code
• Parallel Scientific Computing Workshop (5 days), SciNet, Aug 2009:
  • Slides
  • Video
• Virtual School for CSE Web courses (Jul/Aug 2010):
  • Petascale programming environments and tools
  • Big data for science
  • Proven algorithmic techniques for many-core processors
• LLNL MPI Tutorial: This was the basis for the MPI workshop at SciNet.
• Intel MPI library
• GPC MPI Versions
• Co-array Fortran on the GPC
• IBM MPI Workshop, SciNet, Feb 2009
• IBM UPC compiler, language, optimization, library, user's guide, programmer's guide

GPU Computing

• 1-day "Introduction to GPGPU Programming with CUDA" Course (May 2015): [1].
• 2-day "Programming GPUs with CUDA" Course (2015 Summer School): [2]
• New Features in CUDA 5 & 6, Nov. 2014
• Intro to GPU Computing Using CUDA (WestGrid Spring 2014 Seminar Series)
• 1.5 hour intro to CUDA, March 2013: Slides and Source Code
• CITA/SciNet CUDA Minicourse, Fall 2012
• SciNet GPU Workshop July 2010
• Intro to GPGPU Programming: Video/ Slides
  (from 5 day parallel programming course at SciNet, May 2011)
• 1-day intro to GPGPU using CUDA Course (Aug 2011): Source Code, Slides.
• NVidia archived courses for GPGPU Programming
• PGI Compiler User's Guide
• PGI Compiler Reference Manual
• PGI Fortran reference
• PGI CUDA Fortran Programming Guide and Reference
• PGI OpenACC Getting Started Guide

Performance Tuning

• Performance and Profiling Course, April 2013
• Introduction To Performance
• Performance tools for GPC and TCS
• Dec 2010 SNUG TechTalk: Profiling Tools on GPC
• Performance tuning
• MPI Tuning Parameters - SNUG TechTalk, Feb 2012
• High Performance Computing Book
  Online version of an older O'Reilly book which covers the basics of (mostly serial) programming for performance. Covers the most important issues today (such as cache) very clearly.
• How to Write Fast Numerical Code
  Good introduction to thinking about performance and cache.

Debugging

• Parallel Debugging with DDT
• Debugging with GDB and DDT, half-day session at the Ontario HPC Summerschool 2012 Central
  Slides, Code.
• TechTalk: Debuggers & Parallel Debugging on SciNet - gdb, ddd, padb], SciNet User Group Meeting, Nov 2010
  Video
• Allinea DDT (Distributed Debugging Tool) User Guide

Math libraries (BLAS, LAPACK, FFT)

• Intel Math Kernel Library (MKL): An overview (TechTalk, March, 2012)
• Numerical Tools for Physical Scientists, part 2 of SciNet's Scientific computing course, covers, random nubers, blas, lapack, fft, ...
• Intel Compiler Floating Point Consistency
• Math Kernel Library (MKL)
• Math Kernel Library's Vector Statistical Library
• Math Kernel Library link line advisor
  ($MKLPATH → ${MKLPATH} in makefiles)
• ESSL high performance math library V4 (V3)
• Parallel ESSL high performance math library V3.3
• Linear Algebra Libraries by Claire Mouton. 2009 INRIA Technical Report on existing linear algebra libraries for C++ (also here: [3])

I/O

• Introduction to NetCDF4 binary files with Python, C++ and R (TechTalk March 2014)
• Using MySQL on the GPC
• Relational Database Basics
• Sep 2012 SNUG TechTalk: Science=Data
• Data management
• Intro to Parallel I/O, SciNet, Oct 6th, 2010:
  • Morning & MPI-IO Slides
  • NetCDF/HDF5 Slides
  • Source Code.
• Intro to Parallel I/O, SciNet, Feb, 2013
  • Video of the Introduction
  • Video of the parallel hdf5 and netcdf part NetCDF/HDF5 Slides
  • Video of the MPI-IO part
• Half-day HPCS2012 Parallel I/O tutorial, covering MPI-IO, HDF5, NetCDF, based on the above: slides (pdf) and source code.
• Sept 2010 SNUG TechTalk: Parallel File System and IO
  Video
• File System and I/O dos and don'ts
• So you have 40TB of Data -- an overview of things to consider with large data sets.
• May 2012 SNUG TechTalk: ADIOS for Parallel IO slides and source code
• Writting / Reading a table in hdf5
• Writting / Reading a table in NetCDF
• Intro to Parallel I/O, SciNet, Sept 215
  • MPI-IO Slides and Sample codes
  • NETCDF & HDF5

Infiniband Networking

• TechTalk on SciNet's Infiniband Network & MPI options

Visualization

• Using Paraview
• VNC on the GPC (see also slides of the TechTalk on VNC)
• Visualization Software on the GPC
• Remote visualization (X-forwarding and VNC)
• Using Paraview
• gnuplot, xmgrace, remote visualization tools (X-forwarding and VNC), python's matplotlib
• Brief overview of ParaView & VisIt
• VisIt Basics
• Scientific Visualization using VisIt (Basics and Advanced), M.Ponce and A.Razoumov
• Intro to Complex Networks Visualization, with Python

• Science Illustrated: Two-day symposium on Visualizing Science, Feb 2011
• Videos of the talks given at Science Illustrated (recorded by KMDI at UoT):
  • Opening remarks by Paul Young
  • Information Visualization and the Myth of Information Overload by Christopher Collins
  • Beyond Basic Visualization by Ramses Van Zon
  • Network Visualization & Analysis by Igor Jurisica
  • Simulation and Visualization of Blood Flow by David Steinman
  • Scientific Visualizations: Does the Science Matter? by Thomas Lucas
  • How can visualization impact public perception of science? Panelists: Jay Ingram, Peter Calamai, Reni Barlow, Hooley McLaughlin
  • How Info Graphics are Created for the Mainstream Media by Peter Calamai
  • Design Boot Camp by Graham Huber
  • Visualization Large Datasets by Jonathan Dursi
  • Visualizing Colliding Black Holes by Herald Pfeiffer
  • Closing remarks by Mubdi Rahman

Applications

Currently, information on the following applications is available:

• Athena
• Amber
• CCSM3/CCSM4
• FLASH3
• Gamess. See also here
• Gromacs
• LAMMPS
• NAMD
• Paraview
• VASP
• See also User Codes

Development Systems

KNL

• SNUG TechTalk October 2016 - Intel Xeon Phi Knights Landing

Manuals

Intel compilers and libraries (GPC)

• C & C++ compiler
• Fortran compiler
• Intel Compiler Floating Point Consistency
• Intel Compiler Optimization Guide
• Math Kernel Library (MKL)
• Math Kernel Library's Vector Statistical Library
• Intel MPI library
• Math Kernel Library link line advisor
  ($MKLPATH → ${MKLPATH} in makefiles)

IBM compilers and libraries (TCS/P7)

• C compiler, language, optimization
• C++ compiler, language, optimization
• Fortran compiler language, optimization
• UPC compiler, language, optimization, library, user's guide, programmer's guide
• ESSL high performance math library V4 (V3)
• ESSL high performance math library V5.1 for Linux on Power
• Performance tuning
• Parallel environment
• Cluster information center (with error codes)
• LoadLeveler: using & administering

PGI compilers (ARC)

• Compiler User's Guide
• Compiler Reference Manual
• Fortran reference
• CUDA Fortran Programming Guide and Reference
• OpenACC Getting Started Guide
  (Note: $PGI/linux86-64/12.5/doc contains a newer version.)

Scheduler (Adaptive Computing/Cluster Resources)

• Moab workload manager
• Moab commands
• Torque resource manager
• Torque PBS commands

DDT Debugger (Allinea)

• Distributed Debugging Tool User Guide