Difference between revisions of "Knowledge Base: Tutorials and Manuals"
Jump to navigation
Jump to search
m (→SciNet Basics) |
m |
||
| Line 1: | Line 1: | ||
| − | + | __TOC__ | |
| − | + | =Training material= | |
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | == | ||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
For upcoming classes, see our [https://support.scinet.utoronto.ca/courses/ Courses website]! | For upcoming classes, see our [https://support.scinet.utoronto.ca/courses/ Courses website]! | ||
==SciNet Basics== | ==SciNet Basics== | ||
| Line 226: | Line 182: | ||
{{:Knowledge Base: Applications}} | {{:Knowledge Base: Applications}} | ||
* See also [[User Codes]] | * See also [[User Codes]] | ||
| + | |||
| + | =Manuals= | ||
| + | |||
| + | ==Intel compilers and libraries (GPC)== | ||
| + | * [http://software.intel.com/sites/products/documentation/hpc/compilerpro/en-us/cpp/lin/compiler_c/index.htm C & C++ compiler] | ||
| + | * [http://software.intel.com/sites/products/documentation/hpc/compilerpro/en-us/fortran/lin/compiler_f/index.htm Fortran compiler] | ||
| + | * [http://software.intel.com/sites/products/documentation/hpc/composerxe/en-us/cpp/lin/index.htm C & C++ compiler 12.0] | ||
| + | * [http://software.intel.com/sites/products/documentation/hpc/composerxe/en-us/fortran/lin/index.htm Fortran compiler 12.0] | ||
| + | * [[Media:FP_Consistency.pdf|Intel Compiler Floating Point Consistency]] | ||
| + | * [[Media:Compiler_qrg12.pdf|Intel Compiler Optimization Guide]] | ||
| + | * [http://software.intel.com/sites/products/documentation/hpc/mkl/lin/index.htm Math Kernel Library (MKL)] | ||
| + | * [http://software.intel.com/sites/products/documentation/hpc/mkl/vsl/vslnotes.pdf Math Kernel Library's Vector Statistical Library] | ||
| + | * [http://software.intel.com/sites/products/documentation/hpc/mpi/linux/reference_manual.pdf Intel MPI library] | ||
| + | * [http://software.intel.com/en-us/articles/intel-mkl-link-line-advisor Math Kernel Library link line advisor]<br/>($MKLPATH → ${MKLPATH} in makefiles) | ||
| + | |||
| + | ==IBM compilers and libraries (TCS/P7)== | ||
| + | * [http://support.scinet.utoronto.ca/Manuals/xlc-compiler.pdf C compiler], [http://support.scinet.utoronto.ca/Manuals/xlc-langref.pdf language], [http://support.scinet.utoronto.ca/Manuals/xlc-proguide.pdf optimization] | ||
| + | * [http://support.scinet.utoronto.ca/Manuals/xlC++-compiler.pdf C++ compiler], [http://support.scinet.utoronto.ca/Manuals/xlC++-langref.pdf language], [http://support.scinet.utoronto.ca/Manuals/xlC++-proguide.pdf optimization] | ||
| + | * [http://support.scinet.utoronto.ca/Manuals/xlf-compiler.pdf Fortran compiler] [http://support.scinet.utoronto.ca/Manuals/xlf-langref.pdf language], [http://support.scinet.utoronto.ca/Manuals/xlf-proguide.pdf optimization] | ||
| + | * [http://support.scinet.utoronto.ca/Manuals/UPC/compiler.pdf UPC compiler], [http://support.scinet.utoronto.ca/Manuals/UPC/langref.pdf language], [http://support.scinet.utoronto.ca/Manuals/UPC/upcopt.pdf optimization], [http://support.scinet.utoronto.ca/Manuals/UPC/standlib.pdf library], [http://support.scinet.utoronto.ca/Manuals/UPC/upcusersguide.pdf user's guide], [http://support.scinet.utoronto.ca/Manuals/UPC/proguide.pdf programmer's guide] | ||
| + | * [http://publib.boulder.ibm.com/epubs/pdf/am501405.pdf ESSL high performance math library V4] ([http://publib.boulder.ibm.com/epubs/pdf/am601305.pdf V3]) | ||
| + | * [[Media:essl51.pdf|ESSL high performance math library V5.1 for Linux on Power]] | ||
| + | * [http://support.scinet.utoronto.ca/Manuals/JUMP-AIX-POWER6-AppsPerformanceTuning-wp032008.pdf Performance tuning] | ||
| + | * [http://support.scinet.utoronto.ca/Manuals/PE5.1-operationanduse.pdf Parallel environment] | ||
| + | * [http://publib.boulder.ibm.com/infocenter/clresctr/vxrx/index.jsp Cluster information center] (with error codes) | ||
| + | * [http://support.scinet.utoronto.ca/Manuals/LL-usingandadministering.pdf LoadLeveler: using & administering] | ||
| + | |||
| + | ==PGI compilers (ARC)== | ||
| + | * [http://www.pgroup.com/doc/pgiug.pdf Compiler User's Guide] | ||
| + | * [http://www.pgroup.com/doc/pgiref.pdf Compiler Reference Manual] | ||
| + | * [http://www.pgroup.com/doc/pgifortref.pdf Fortran reference] | ||
| + | * [http://www.pgroup.com/doc/pgicudaforug.pdf CUDA Fortran Programming Guide and Reference] | ||
| + | * [http://www.pgroup.com/doc/openACC_gs.pdf OpenACC Getting Started Guide]<br>(Note: $PGI/linux86-64/12.5/doc contains a newer version.) | ||
| + | |||
| + | ==Scheduler (Adaptive Computing/Cluster Resources)== | ||
| + | * [http://www.clusterresources.com/products/mwm/docs/index.shtml Moab workload manager] | ||
| + | * [http://www.clusterresources.com/products/mwm/docs/a.gcommandoverview.shtml Moab commands] | ||
| + | * [http://www.clusterresources.com/products/torque/docs/ Torque resource manager] | ||
| + | * [http://www.clusterresources.com/products/torque/docs/a.acommands.shtml Torque PBS commands] | ||
| + | |||
| + | ==DDT Debugger (Allinea)== | ||
| + | * [http://www.allinea.com/downloads/userguide.pdf Distributed Debugging Tool User Guide] | ||
Revision as of 11:07, 12 March 2013
Training material
For upcoming classes, see our Courses website!
SciNet Basics
- SciNet User Tutorial
- Intro to SciNet: Video/Slides, SciNet, November 2012
- SciNet Resources: Video/ Slides
- Essentials
- Frequenty asked questions
- Ssh
- GPC quickstart
- TCS quickstart
- ARC/GPU quickstart
- ARC/Cell quickstart
- Important .bashrc guidelines
- Workflow Optimization (w/focus on Large Scale BioInformatics)
- Software and libraries
- Installing your own modules
- User-space modules and packages (April 2011 SNUG TechTalk)
- HPSS - SciNet's storage capacity expansion
- BGQ Hardware Overview Slides / Video Recording
- Intro to Using the BGQ Slides /Video Recording
Linux
- Linux Command Line: A Primer (June 2012) Slides, Files
- Introduction to the Linux Shell, SciNet, Mar 2012: Slides and Data files
Batch job management
- Job Monitoring on SciNet and Job Efficiency
- Wallclock time
- Checkpoints
- Signals
- Moab
- Serial Jobs (including GNU Parallel)
- GNU Parallel (Techtalk Nov 14, 2012)
- Ramdisk
- Moab workload manager
- Moab commands
- Torque resource manager
- Torque PBS commands
- Parallel environment
- Cluster information center (with error codes)
- LoadLeveler: using & administering
Programming
General
- Performance Tuning with the IBM XL Compilers: Slides fromt the SciNet Develop Seminar by Kit Barton, Sep 17, 2012.
- Remote Development, slides from TechTalk Jun 13, 2012
- Scientific Software Development Course, part I of the SciNet's Scientific Computing Course
- Software Carpentry Resources
- Version Control: Video/ Slides
- IBM AIX Workshop, SciNet, Nov 2008
- IBM Compiler Workshop, SciNet, Feb 2009
- SNUG Techtalk Dec 2011 Intel Compiler Optimizations
Fortran
- Modern Fortran Course (1 day), SciNet, 19 Apr 2011
- [http://support.scinet.utoronto.ca/Manuals/xlf-compiler.pdf IBM Fortran compiler language, optimization
C++
- Scientific C++ Course (1 day), SciNet, 15 March 2011
- Slides (updated on Apr 26, 2012)
- Example source code
- Videos of the Scientific C++ class
- Intel C & C++ compiler
ro/en-us/fortran/lin/compiler_f/index.htm Intel Fortran compiler]
C
- C refresher: Video/ Slides
- IBM C compiler, language, optimization
Perl
Python
- Python
- f2py: Fortran and Python (June 2011 TechTalk by Pierre de Buyl)
R
Lua
- Scripting HALMD with Lua and Luabind (May 2011 TechTalk by Peter Colberg)
Parallel Programming
- Ontario Summerschool on High Performance Computing Central
- High Performance Scientific Computing, part 3 of SciNet's Scientific Computing Course (Winter 2012)
- Parallel Programming Course (5 days), SciNet, May 2011
- Parallel Computing for Computational Fluid Dynamics (CFD), SciNet, 23 March 2011
- Intro to Practical Parallel Programming (1 day), SciNet, 22 Sept 2010:
- Parallel Scientific Computing Workshop (5 days), SciNet, Aug 2009:
- Virtual School for CSE Web courses (Jul/Aug 2010):
- Petascale programming environments and tools
- Big data for science
- Proven algorithmic techniques for many-core processors
- LLNL MPI Tutorial: This was the basis for the MPI workshop at SciNet.
- Intel MPI library
- GPC MPI Versions
- Co-array Fortran on the GPC
- IBM MPI Workshop, SciNet, Feb 2009
- IBM UPC compiler, language, optimization, library, user's guide, programmer's guide
GPU Computing
- CITA/SciNet CUDA Minicourse, Fall 2012
- SciNet GPU Workshop July 2010
- Intro to GPGPU Programming: Video/ Slides
(from 5 day parallel programming course at SciNet, May 2011) - 1-day intro to GPGPU using CUDA Course (Aug 2011): Source Code, Slides.
- NVidia archived courses for GPGPU Programming
- PGI Compiler User's Guide
- PGI Compiler Reference Manual
- PGI Fortran reference
- PGI CUDA Fortran Programming Guide and Reference
- PGI OpenACC Getting Started Guide
Performance Tuning
- Introduction To Performance
- Performance tools for GPC and TCS
- Dec 2010 SNUG TechTalk: Profiling Tools on GPC
- High Performance Computing Book
Online version of an older O'Reilly book which covers the basics of (mostly serial) programming for performance. Covers the most important issues today very clearly. - How to Write Fast Numerical Code
Good introduction to thinking about performance. - Performance tuning
- MPI Tuning Parameters - SNUG TechTalk, Feb 2012
Debugging
- Debugging with GDB and DDT, half-day session at the Ontario HPC Summerschool 2012 Central
Slides, Code. - TechTalk: Debuggers & Parallel Debugging on SciNet - gdb, ddd, padb], SciNet User Group Meeting, Nov 2010
Video - Allinea DDT (Distributed Debugging Tool) User Guide
Math libraries (BLAS, LAPACK, FFT)
- Intel Math Kernel Library (MKL): An overview (TechTalk, March, 2012)
- Numerical Tools for Physical Scientists, part 2 of SciNet's Scientific computing course, covers, random nubers, blas, lapack, fft, ...
- Intel Compiler Floating Point Consistency
- Math Kernel Library (MKL)
- Math Kernel Library's Vector Statistical Library
- Math Kernel Library link line advisor
($MKLPATH → ${MKLPATH} in makefiles) - ESSL high performance math library V4 (V3)
- Parallel ESSL high performance math library V3.3
- Linear Algebra Libraries by Claire Mouton. 2009 INRIA Technical Report on existing linear algebra libraries for C++ (also here: [1])
I/O
- Sep 2012 SNUG TechTalk: Science=Data
- Data management
- Intro to Parallel I/O, SciNet, Oct 6th, 2010:
- Half-day HPCS2012 Parallel I/O tutorial, covering MPI-IO, HDF5, NetCDF, based on the above: slides (pdf) and source code.
- Sept 2010 SNUG TechTalk: Parallel File System and IO
Video - File System and I/O dos and don'ts
- So you have 40TB of Data -- an overview of things to consider with large data sets.
- May 2012 SNUG TechTalk: ADIOS for Parallel IO slides and source code
Infiniband Networking
Visualization
- Using Paraview
- Science Illustrated: Two-day symposium on Visualizing Science, Feb 2011
- Videos of the talks given at Science Illustrated (recorded by KMDI at UoT):
- Opening remarks by Paul Young
- Information Visualization and the Myth of Information Overload by Christopher Collins
- Beyond Basic Visualization by Ramses Van Zon
- Network Visualization & Analysis by Igor Jurisica
- Simulation and Visualization of Blood Flow by David Steinman
- Scientific Visualizations: Does the Science Matter? by Thomas Lucas
- How can visualization impact public perception of science? Panelists: Jay Ingram, Peter Calamai, Reni Barlow, Hooley McLaughlin
- How Info Graphics are Created for the Mainstream Media by Peter Calamai
- Design Boot Camp by Graham Huber
- Visualization Large Datasets by Jonathan Dursi
- Visualizing Colliding Black Holes by Herald Pfeiffer
- Closing remarks by Mubdi Rahman
Applications
Currently, information on the following applications is available:
- Athena
- Amber
- CCSM3/CCSM4
- FLASH3
- Gamess. See also here
- Gromacs
- LAMMPS
- NAMD
- Paraview
- VASP
- See also User Codes
Manuals
Intel compilers and libraries (GPC)
- C & C++ compiler
- Fortran compiler
- C & C++ compiler 12.0
- Fortran compiler 12.0
- Intel Compiler Floating Point Consistency
- Intel Compiler Optimization Guide
- Math Kernel Library (MKL)
- Math Kernel Library's Vector Statistical Library
- Intel MPI library
- Math Kernel Library link line advisor
($MKLPATH → ${MKLPATH} in makefiles)
IBM compilers and libraries (TCS/P7)
- C compiler, language, optimization
- C++ compiler, language, optimization
- Fortran compiler language, optimization
- UPC compiler, language, optimization, library, user's guide, programmer's guide
- ESSL high performance math library V4 (V3)
- ESSL high performance math library V5.1 for Linux on Power
- Performance tuning
- Parallel environment
- Cluster information center (with error codes)
- LoadLeveler: using & administering
PGI compilers (ARC)
- Compiler User's Guide
- Compiler Reference Manual
- Fortran reference
- CUDA Fortran Programming Guide and Reference
- OpenACC Getting Started Guide
(Note: $PGI/linux86-64/12.5/doc contains a newer version.)