Difference between revisions of "Knowledge Base: Tutorials and Manuals"
Jump to navigation
Jump to search
| Line 257: | Line 257: | ||
==Scheduler (Adaptive Computing/Cluster Resources)== | ==Scheduler (Adaptive Computing/Cluster Resources)== | ||
| − | * [http:// | + | * [http://docs.adaptivecomputing.com/mwm/archive/7-0/help.htm Moab workload manager] |
| − | * [http:// | + | * [http://docs.adaptivecomputing.com/mwm/archive/7-0/Content/a.gcommandoverview.html Moab commands] |
| − | * [http:// | + | * [http://docs.adaptivecomputing.com/torque/archive/4-0-1/help.htm Torque resource manager] |
| − | * [http:// | + | * [http://docs.adaptivecomputing.com/torque/archive/4-0-1/help.htm#topics/12-appendices/commandsOverview.htm Torque PBS commands] |
==DDT Debugger (Allinea)== | ==DDT Debugger (Allinea)== | ||
* [http://www.allinea.com/downloads/userguide.pdf Distributed Debugging Tool User Guide] | * [http://www.allinea.com/downloads/userguide.pdf Distributed Debugging Tool User Guide] | ||
Revision as of 21:38, 2 June 2016
Training material
For upcoming classes, see our Training and Education website!
SciNet Basics
- SciNet User Tutorial
- SciNet Command Line Utilities
- Intro to SciNet May 2014: Video/Slides
- SciNet Resources: Video/ Slides
- Essentials
- Frequenty asked questions
- Ssh
- GPC quickstart
- TCS quickstart
- ARC/GPU quickstart
- ARC/Cell quickstart
- Important .bashrc guidelines
- Workflow Optimization (w/focus on Large Scale BioInformatics)
- Software and libraries
- Installing your own modules
- User-space modules and packages (April 2011 SNUG TechTalk)
- HPSS - SciNet's storage capacity expansion
- BGQ Hardware Overview Slides / Video Recording
- Intro to Using the BGQ Slides /Video Recording
Linux
- Linux Tutorial (warning: the last part of this tutorial is specific to the csh shell, whereas SciNet uses the bash shell).
- Linux Command Line: A Primer (June 2012) Slides, Files
- Introduction to the Linux Shell, SciNet, Mar 2012: Slides and Data files
Batch job management
- Job and Queue Management Tools (Techtalk Mar.11, 2015)
- Workflow Optimization (w/focus on Large Scale BioInformatics)
- GNU Parallel (Techtalk Nov 14, 2012)
- Job Monitoring on SciNet and Job Efficiency
- Wallclock time
- Checkpoints
- Signals
- Moab
- Serial Jobs (including GNU Parallel)
- Ramdisk
- Moab workload manager
- Moab commands
- Torque resource manager
- Torque PBS commands
- Parallel environment
- Cluster information center (with error codes)
- LoadLeveler: using & administering
Programming
General
- Performance Tuning with the IBM XL Compilers: Slides fromt the SciNet Develop Seminar by Kit Barton, Sep 17, 2012.
- Remote Development, slides from TechTalk Jun 13, 2012
- Scientific Software Development Course, part I of the SciNet's Scientific Computing Course
- Software Carpentry Resources
- Version Control: Video/ Slides
- IBM AIX Workshop, SciNet, Nov 2008
- IBM Compiler Workshop, SciNet, Feb 2009
- SNUG Techtalk Dec 2011 Intel Compiler Optimizations
Fortran
- Modern Fortran Course (1 day), SciNet, 19 Apr 2011
- [http://support.scinet.utoronto.ca/Manuals/xlf-compiler.pdf IBM Fortran compiler language, optimization
C++
- Slides and recording of the SciNet Developer Seminar on C++11, March 20, 2013
- Scientific C++ Course (1 day), SciNet, 15 March 2011
- Slides (updated on Apr 26, 2012)
- Example source code
- Videos of the Scientific C++ class
- Intel C & C++ compiler
ro/en-us/fortran/lin/compiler_f/index.htm Intel Fortran compiler]
C
- C refresher: Video/ Slides
- IBM C compiler, language, optimization
Cilk Plus
- CilkPlus - Getting Started (slides of the May 2014 TechTalk by Chris Szalwinski)
Hadoop
- Introduction to Hadoop for HPCers, Part I - MapReduce: Slides, Source Code, Virtual Machine
Perl
Python
- Python
- IPython Notebook on GPC (January 2014 TechTalk)
- Research Computing with Python (Modular Course, Fall 2013)
- Python GUIs with Python and TkInter slides code
- f2py: Fortran and Python (June 2011 TechTalk by Pierre de Buyl)
R
Lua
- Scripting HALMD with Lua and Luabind (May 2011 TechTalk by Peter Colberg)
Parallel Programming
- 2014 Ontario Summerschool on High Performance Computing Central (incl. OpenMP and MPI)
- High Performance Scientific Computing Course 2014 (OpenMP and MPI, with video recordings of lectures)
- MPI 3 Developer Seminar
- OpenMP 4 Developer Seminar
- Ontario Summerschool on High Performance Computing Central (2013)
- High Performance Scientific Computing, part 3 of SciNet's Scientific Computing Course (Winter 2012)
- Parallel Programming Course (5 days), SciNet, May 2011
- Parallel Computing for Computational Fluid Dynamics (CFD), SciNet, 23 March 2011
- Intro to Practical Parallel Programming (1 day), SciNet, 22 Sept 2010:
- Parallel Scientific Computing Workshop (5 days), SciNet, Aug 2009:
- Virtual School for CSE Web courses (Jul/Aug 2010):
- Petascale programming environments and tools
- Big data for science
- Proven algorithmic techniques for many-core processors
- LLNL MPI Tutorial: This was the basis for the MPI workshop at SciNet.
- Intel MPI library
- GPC MPI Versions
- Co-array Fortran on the GPC
- IBM MPI Workshop, SciNet, Feb 2009
- IBM UPC compiler, language, optimization, library, user's guide, programmer's guide
GPU Computing
- New Features in CUDA 5 & 6, Nov. 2014
- Intro to GPU Computing Using CUDA (WestGrid Spring 2014 Seminar Series)
- 1.5 hour intro to CUDA, March 2013: Slides and Source Code
- CITA/SciNet CUDA Minicourse, Fall 2012
- SciNet GPU Workshop July 2010
- Intro to GPGPU Programming: Video/ Slides
(from 5 day parallel programming course at SciNet, May 2011) - 1-day intro to GPGPU using CUDA Course (Aug 2011): Source Code, Slides.
- NVidia archived courses for GPGPU Programming
- PGI Compiler User's Guide
- PGI Compiler Reference Manual
- PGI Fortran reference
- PGI CUDA Fortran Programming Guide and Reference
- PGI OpenACC Getting Started Guide
Performance Tuning
- Performance and Profiling Course, April 2013
- Introduction To Performance
- Performance tools for GPC and TCS
- Dec 2010 SNUG TechTalk: Profiling Tools on GPC
- Performance tuning
- MPI Tuning Parameters - SNUG TechTalk, Feb 2012
- High Performance Computing Book
Online version of an older O'Reilly book which covers the basics of (mostly serial) programming for performance. Covers the most important issues today (such as cache) very clearly. - How to Write Fast Numerical Code
Good introduction to thinking about performance and cache.
Debugging
- Parallel Debugging with DDT
- Debugging with GDB and DDT, half-day session at the Ontario HPC Summerschool 2012 Central
Slides, Code. - TechTalk: Debuggers & Parallel Debugging on SciNet - gdb, ddd, padb], SciNet User Group Meeting, Nov 2010
Video - Allinea DDT (Distributed Debugging Tool) User Guide
Math libraries (BLAS, LAPACK, FFT)
- Intel Math Kernel Library (MKL): An overview (TechTalk, March, 2012)
- Numerical Tools for Physical Scientists, part 2 of SciNet's Scientific computing course, covers, random nubers, blas, lapack, fft, ...
- Intel Compiler Floating Point Consistency
- Math Kernel Library (MKL)
- Math Kernel Library's Vector Statistical Library
- Math Kernel Library link line advisor
($MKLPATH → ${MKLPATH} in makefiles) - ESSL high performance math library V4 (V3)
- Parallel ESSL high performance math library V3.3
- Linear Algebra Libraries by Claire Mouton. 2009 INRIA Technical Report on existing linear algebra libraries for C++ (also here: [1])
I/O
- Introduction to NetCDF4 binary files with Python, C++ and R (TechTalk March 2014)
- Using MySQL on the GPC
- Relational Database Basics
- Sep 2012 SNUG TechTalk: Science=Data
- Data management
- Intro to Parallel I/O, SciNet, Oct 6th, 2010:
- Intro to Parallel I/O, SciNet, Feb, 2013
- Half-day HPCS2012 Parallel I/O tutorial, covering MPI-IO, HDF5, NetCDF, based on the above: slides (pdf) and source code.
- Sept 2010 SNUG TechTalk: Parallel File System and IO
Video - File System and I/O dos and don'ts
- So you have 40TB of Data -- an overview of things to consider with large data sets.
- May 2012 SNUG TechTalk: ADIOS for Parallel IO slides and source code
- Writting / Reading a table in hdf5
- Writting / Reading a table in NetCDF
- Intro to Parallel I/O, SciNet, Sept 215
- MPI-IO Slides and Sample codes
- NETCDF & HDF5
Infiniband Networking
Visualization
- Using Paraview
- VNC on the GPC (see also slides of the TechTalk on VNC)
- Visualization Software on the GPC
- Science Illustrated: Two-day symposium on Visualizing Science, Feb 2011
- Videos of the talks given at Science Illustrated (recorded by KMDI at UoT):
- Opening remarks by Paul Young
- Information Visualization and the Myth of Information Overload by Christopher Collins
- Beyond Basic Visualization by Ramses Van Zon
- Network Visualization & Analysis by Igor Jurisica
- Simulation and Visualization of Blood Flow by David Steinman
- Scientific Visualizations: Does the Science Matter? by Thomas Lucas
- How can visualization impact public perception of science? Panelists: Jay Ingram, Peter Calamai, Reni Barlow, Hooley McLaughlin
- How Info Graphics are Created for the Mainstream Media by Peter Calamai
- Design Boot Camp by Graham Huber
- Visualization Large Datasets by Jonathan Dursi
- Visualizing Colliding Black Holes by Herald Pfeiffer
- Closing remarks by Mubdi Rahman
Applications
Currently, information on the following applications is available:
- Athena
- Amber
- CCSM3/CCSM4
- FLASH3
- Gamess. See also here
- Gromacs
- LAMMPS
- NAMD
- Paraview
- VASP
- See also User Codes
Manuals
Intel compilers and libraries (GPC)
- C & C++ compiler
- Fortran compiler
- Intel Compiler Floating Point Consistency
- Intel Compiler Optimization Guide
- Math Kernel Library (MKL)
- Math Kernel Library's Vector Statistical Library
- Intel MPI library
- Math Kernel Library link line advisor
($MKLPATH → ${MKLPATH} in makefiles)
IBM compilers and libraries (TCS/P7)
- C compiler, language, optimization
- C++ compiler, language, optimization
- Fortran compiler language, optimization
- UPC compiler, language, optimization, library, user's guide, programmer's guide
- ESSL high performance math library V4 (V3)
- ESSL high performance math library V5.1 for Linux on Power
- Performance tuning
- Parallel environment
- Cluster information center (with error codes)
- LoadLeveler: using & administering
PGI compilers (ARC)
- Compiler User's Guide
- Compiler Reference Manual
- Fortran reference
- CUDA Fortran Programming Guide and Reference
- OpenACC Getting Started Guide
(Note: $PGI/linux86-64/12.5/doc contains a newer version.)