oldwiki.scinet.utoronto.ca - User contributions [en-gb]

LP Benchmark

2017-02-21T14:15:10Z

Ejspence: /* Nek5000 */

=Benchmark Guidelines=

This site contains the benchmarks required for the 2017 University of Toronto Large Parallel System procurement. The benchmark sources and specific run instructions are provided for each of the benchmarks below. Information on baseline, optimization and reporting of the benchmark scores is described in Schedule 1 of the procurement document.



= Large Parallel Benchmark (LPBM) =

The following six (6) application benchmarks make up the Large Parallel Benchmark (LPBM). Each benchmark includes the source codes, benchmark run requirements and the instructions for reporting.

== HPCG ==

The High Performance Conjugate Gradients [http://www.hpcg-benchmark.org/ HPCG] benchmark is designed to exercise computational and data access patterns that closely match a broad set of important scientific applications, and to give incentive to computer system designers to invest in capabilities that will have impact on the collective performance of these applications.

HPCG is an open source benchmark developed by Mike Heroux, Jack Dongarra and Piotr Luszcze

Source and LBPM specific instructions can be downloaded [https://wiki.scinet.utoronto.ca/wiki/images/5/57/Hpcg-lpbm.tar.gz here]

== Nek5000 ==

[https://nek5000.mcs.anl.gov/ Nek5000] is an open source, highly scalable and portable spectral element code designed to simulate a range of flow physics.

Developed at Argonne National Nek5000 is released under the terms of the GNU GPL license

Source and LBPM specific instructions can be downloaded [https://wiki.scinet.utoronto.ca/wiki/images/a/a5/Nek5000_benchmark.tar.gz here]

== WRF ==

The Weather Research and Forecasting [http://www.wrf-model.org/index.php (WRF)] Model is a next-generation mesoscale numerical weather prediction system designed for both atmospheric research and operational forecasting needs.

WRF is public domain, details [http://www2.mmm.ucar.edu/wrf/users/public.html here]

Source can be downloaded [https://wiki.scinet.utoronto.ca/wiki/images/5/57/WRF3.7.1-LPBM.tar.gz here]

Input files and LBPM specific instructions can be downloaded [https://wiki.scinet.utoronto.ca/wiki/images/5/57/WRF-LPBM-input.tar here] (NOTE: 36G file)

== NAMD ==

[http://www.ks.uiuc.edu/Research/namd/ NAMD] is a parallel molecular dynamics code designed for high-performance simulation of large biomolecular systems.

NAMD is Licensed under the University of Illinois NAMD Molecular Dynamics Software Non-Exclusive, Non-Commercial Use License
http://www.ks.uiuc.edu/Research/namd/license.html

NAMD version 2.12 source, and/or binary packages, must be obtained directly from the Theoretical and Computational Biophysics Group at UIUC [http://www.ks.uiuc.edu/Development/Download/download.cgi?PackageName=NAMD here].

LBPM specific instructions and input files can be downloaded [https://wiki.scinet.utoronto.ca/wiki/images/5/57/NAMD_LPBM.tar.gz here]

== miniDFT ==

[http://www.nersc.gov/research-and-development/apex/apex-benchmarks/minidft/ MiniDFT] is a plane-wave density functional theory (DFT) mini-app for modeling materials. Given an set of atomic coordinates and pseudopotentials, MiniDFT computes self-consistent solutions of the Kohn-Sham equations using either the LDA or PBE exchange-correlation functionals. For each iteration of the self-consistent field cycle, the Fock matrix is constructed and then diagonalized. To build the Fock matrix, Fast Fourier Transforms are used to transform orbitals from the plane wave basis ( where the kinetic energy is most readily computed ) to real space (where the potential is evaluated ) and back. Davidson diagonalization is used to compute the orbital energies and update the orbital coefficients.

The MiniDFT mini-app was excised from the general-purpose Quantum Espresso (QE) code. Quantum Espresso is licensed per the GNU General Public License (GPL). A copy of the GPL is provided in the distribution's 'License' file.

Source and LBPM specific instructions can be downloaded [https://wiki.scinet.utoronto.ca/wiki/images/5/57/MiniDFT-1.1.1.lpbm.tgz here]

== SPEC MPI 2007 ==

SPEC MPI® 2007 focuses on performance of compute intensive applications using the Message-Passing Interface (MPI)

Proponents should report the SPECmpiL_base2017 and optionally the SPECmpiL_peak2017 scores in the Schedule 2 LPBM spreadsheet for the closest number of whole nodes to achieve 2048 MPI ranks.

Available from https://www.spec.org/mpi2007/

File:Nek5000 benchmark.tar.gz

2017-02-21T14:11:54Z

Ejspence: uploaded a new version of "File:Nek5000 benchmark.tar.gz": Nek5000 benchmark files for the LP RFP.

2016 Ontario Summer School for High Performance Computing Central

2016-07-13T13:18:22Z

Ejspence:

===About===

The 2016 Ontario Summer School on High Performance Computing, whose second installment of the year will be hosted by SciNet, provides attendees with opportunities to learn and share knowledge and experience in high performance and technical computing. The format: a five-day workshop with mixed lectures and hands-on sessions on a number of selected subjects, including shared memory programming, distributed memory programming, parallel data processing, and general purpose graphics processing unit programming.

Below are descriptions and course material for the Central version, given from July 11th to July 15th, 2016, in Toronto.

The location of this version of the summer school is:

'''Wallberg Building''' 
'''Rooms 116 and 119''' 
University of Toronto 
St. George Campus 
184-200 College Street 
Toronto, Ontario, M5S 3E5 
Canada

More information can be found below and in the [[Media:handout2016.pdf|handout package]].

__TOC__

===Introduction to HPC and SciNet===

=====Abstract=====
This half-day session will provide an introduction to basic concepts of high-performance computing. It is intended to be a high level primer for those largely new to HPC, and serve as a foundation upon which to build over the coming days. Topics will include motivation for HPC, essential issues, problem characteristics as they apply to parallelism and a high level overview of parallel programming models. Strategies of running large sets of serial processes using e.g. GNU parallel, will also be presented.

=====Details=====
''Time/date''

Monday July 11, 2016, 09:30 - 12:30

''Location''

Wallberg Building, Room 116

''Instructor''

Dr. Mike Nolta (SciNet)

=====Prerequisites=====
Basic Linux skills are useful to follow along. If you do not have these, consider taking the "Intro to the Linux Shell" in the afternoon.

=====Slides and other material=====

* [https://support.scinet.utoronto.ca/~nolta/HPC_intro.pdf Slides]

----

===Programming Shared Memory Systems with OpenMP===

=====Abstract=====
In these two half-day sessions, lectures and hands-on labs are interspersed, and the students will learn the basics of shared memory programming with OpenMP. In particular, we will discuss the OpenMP's threads, memory, and performance, reductions and load balancing. We will also discuss extensions to heterogeneous systems such as offered by the OpenMP 4.0 and OpenACC standards.

=====Details=====

''Time/date''

Monday July 11, 2016, 13:30 - 16:30 
Tuesday July 12, 2016, 09:30 - 12:30

''Location''

Wallberg Building, Room 116

''Instructor''

Dr. Ramses van Zon (SciNet)

=====Prerequisites=====
C/C++ and/or Fortran scientific programming; Experience editing and compiling code in a Linux environment.

=====Slides and other material=====

* [[Media:OpenMP2016.pdf | Slides]]
* [[Media:OpenMP_SS2015_code.tar.gz|Code]]

----

===Introduction to the Linux Shell===

====Abstract====
Working with many of the HPC systems in Ontario involves using the Linux/UNIX command line. This provides a very powerful interface, but it can be quite daunting for the uninitiated. In this half-day session, you can become initiated with this course. This hands on session will cover basic commands and scripting, as well as touching on some powerful constructs like regular expressions. It could be a great boon for your productivity!

====Details====
''Time/date''

Monday July 11, 13:30 - 16:30

''Location''

Wallberg Building, Room 119

''Instructor''

Dr. Mike Nolta (SciNet)

====Prerequisites====
A laptop with a linux-like environment.

====Slides and other material====

* [[Media:intro_to_the_unix_shell.pdf | Slides ]]

----

===R for Data Science===

=====Abstract=====
This session offers a brief introduction to R, with a focus on data analysis and statistics.

=====Details=====

''Date/time''

Tuesday July 12, 2016, 09:30 - 12:30

''Location''

Wallberg Building, Room 119 

''Instructor''

Dr. Erik Spence (SciNet)

=====Prerequisites=====

Some programming experience in any language. You should have used and written functions. Bring a laptop with R or Rstudio installed.

=====Slides and other material=====

* [https://wiki.scinet.utoronto.ca/wiki/images/b/b1/Intro_R.pdf Slides]

----

===Programming Clusters with MPI===

=====Abstract=====
In this 1.5-day session, through lectures interspersed with hands-on labs, the students will learn the basics of MPI programming. Examples and exercises will be based on parallelization of common scientific computing problems.

=====Details=====
''Time/date''

Tuesday July 12, 2016, 13:30 - 16:30 
Wednesday July 13, 2016, 09:30 - 12:30 
Wednesday July 13, 2016, 13:30 - 16:30 

''Location''

Wallberg Building, Room 116

''Instructor''

Dr. Scott Northrup (SciNet)

=====Prerequisites=====
C and/or Fortran scientific programming; experience editing and compiling code in a Linux environment.

=====Slides and other material=====
* [[Media:MPIss2016.pdf | Slides ]]
----

===Parallel R for Data Science===

=====Abstract=====
This half-day session will cover parallel programming in R, with a focus on parallel data analysis. Topics covered will include snow, parallel, and foreach/doparallel.

=====Details=====

''Time/date''

Tuesday July 12, 2016, 13:30 - 16:30

''Location''

Wallberg Building, Room 119

''Instructor''

Dr. Erik Spence (SciNet)

=====Prerequisites=====
Experience with programming in R and a rudimentary understanding of parallel computing.

=====Slides and other material=====

* [https://wiki.scinet.utoronto.ca/wiki/images/0/02/Parallel_R.pdf Slides]

----

===Python for Scientific Computing===

=====Abstract=====
In this half-day session, you will be taught how to use python for research computing purposes. Topics include: the basics of python, automation, numpy, scipy, file i/o, and visualization.

=====Details=====
''Time/date'''

Wednesday July 13, 9:30 - 12:30

''Location''

Wallberg Building, Room 119

''Instructor''

Dr. Erik Spence (SciNet)

=====Prerequisites=====
A bit of coding experience in any programming or scripting language. Command-line experience in Linux is a plus.

=====Slides and other material=====

* [https://wiki.scinet.utoronto.ca/wiki/images/1/1a/Intro_Python.pdf Slides]
----

===Python for High Performance Computing===

=====Abstract=====
This half-day session will cover parallel programming in python, with a focus on parallel data analysis. We will cover subprocess, multiprocessing, pypar and other parallel-enabling python packages.

=====Details=====
''Time/date''

Wednesday July 13, 2016, 13:30 - 16:30

''Location''

Wallberg Building, Room 119

''Instructor''

Dr. Ramses van Zon (SciNet)

=====Prerequisites=====
A bit of coding experience in any programming or scripting language. Command-line experience in Linux is a plus.

=====Slides and other material=====

----

===Programming GPUs with CUDA===

=====Abstract=====
This two-day session is an introductory course covering programming and computing on GPUs - graphics processing units - which are an increasingly common presence in massively parallel computing architectures. This session will cover both of the available C-like programming frameworks: NVIDIA’s CUDA C. The basics of GPU programming will be covered, and students will work through a number of hands on examples. Demonstrations of profiling and debugging applications running on the GPU will also be included. The structuring of data and computations that makes full use of the GPU will be discussed in detail. Students should be able to leave the course with the knowledge necessary to begin developing their own GPU applications.

=====Details=====
''Time/date''

Thursday July 14, 2016, 09:30 - 12:30 
Thursday July 14, 2016, 13:30 - 16:30 
Friday July 15, 2016, 09:30 - 12:30 
Friday July 15, 2016, 13:30 - 16:30

''Location''

Wallberg Building, Room 116

''Instructors''
Dr. Pawel Pomorski and Dr. Sergey Mashchenko (SHARCNET)

=====Prerequisites=====
C/C++ scientific programming, experience editing and compiling code in a Linux environment. Some experience with CUDA and/or OpenMP a plus.

=====Slides and other materials=====

----

===Visualization===

=====Abstract=====
The first part of this whole-day session will be a basic introduction to general-purpose scientific visualization tools such as gnuplot, grace(xmgr), remote tools (tunneling, VNC), python's matplotlib, and examples from other. The afternoon part will be a mix of lecture and hands-on to introduce more specialized scientific visualization software such as vmd, paraview, and visit.

=====Details=====
''Time/date''

Thursday July 14, 2016, 09:30 - 12:30 
Thursday July 14, 2016, 13:30 - 16:30

''Location''

Wallberg Building, Room 119

''Instructor''
Dr. Marcelo Ponce (SciNet)

=====Prerequisites=====
Software installed, for the morning session: gnuplot, xmgr ([http://plasma-gate.weizmann.ac.il/Grace/ Grace],[http://plasma-gate.weizmann.ac.il/Xmgr/ xmgr]), ssh (graphic client), your favorite vnc client (egs. [https://www.realvnc.com/download/viewer/ RealVNC],[http://www.tightvnc.com/download.php TightVNC]), python and matplotlib ([http://matplotlib.org/ matplotlib]); afternoon session: ParaView ([http://www.paraview.org/download/ ParaView]) and VisIt ([https://wci.llnl.gov/simulation/computer-codes/visit/executables VisIt]).

=====Slides and other material=====
======Slides======

[https://support.scinet.utoronto.ca/~mponce/ss2016/ss2016_visualization-I.pdf Morning Session]

[https://support.scinet.utoronto.ca/~mponce/ss2016/ss2016_visualization-II.pdf Afternoon Session]

======Datasets======
Morning datasets and scripts: [[Media:Data_scripts.tar.gz]]

[https://support.scinet.utoronto.ca/~mponce/ss2016/datasets.tar.gz Afternoon dataset]

[https://support.scinet.utoronto.ca/~mponce/ss2016/aneurysm_data.tar.gz Aneurysm dataset]
----

===Debugging===

=====Abstract=====
Debugging is an important step in developing a new code, or porting an old one to a new machine. In this half-day session, we will discuss the debugging of frequently encountered bugs in serial code with gdb and the debugging of parallel (MPI and threaded) codes with DDT on live systems.

=====Details=====
''Date/time''

Friday July 15, 09:30 - 12:30

''Location''

Wallberg Building, Room 119

Note the different room number!

''Instructors''

Dr. Marcelo Ponce (SciNet)

=====Prerequisites=====
Basic familiarity with a high-level language (C/C++/Fortran) as well as compiling and running programs via the command line environment on a HPC system.

=====Slides and other material=====

----

===Bring Your Own Code===

=====Abstract=====
The final afternoon of the summer school is your chance to get help with your own code from the SciNet software analysts. Bring your code, explain your problem or concerns, and we'll give you advice and work with you to try to improve it.

=====Details=====
''Date/time''

Friday July 15, 2016, 13:30 - 16:30

''Location''

Wallberg Building, Room 119

''Instructors''

Various SciNet software analysts.

=====Prerequisites=====
None.

File:Intro Python.pdf

2016-07-13T13:17:40Z

Ejspence: Slides for Into Python session for the 2016 Summer School.

Slides for Into Python session for the 2016 Summer School.

2016 Ontario Summer School for High Performance Computing Central

2016-07-12T17:03:10Z

Ejspence: /* Slides and other material */

===About===

The 2016 Ontario Summer School on High Performance Computing, whose second installment of the year will be hosted by SciNet, provides attendees with opportunities to learn and share knowledge and experience in high performance and technical computing. The format: a five-day workshop with mixed lectures and hands-on sessions on a number of selected subjects, including shared memory programming, distributed memory programming, parallel data processing, and general purpose graphics processing unit programming.

Below are descriptions and course material for the Central version, given from July 11th to July 15th, 2016, in Toronto.

The location of this version of the summer school is:

'''Wallberg Building''' 
'''Rooms 116 and 119''' 
University of Toronto 
St. George Campus 
184-200 College Street 
Toronto, Ontario, M5S 3E5 
Canada

More information can be found below and in the [[Media:handout2016.pdf|handout package]].

__TOC__

===Introduction to HPC and SciNet===

=====Abstract=====
This half-day session will provide an introduction to basic concepts of high-performance computing. It is intended to be a high level primer for those largely new to HPC, and serve as a foundation upon which to build over the coming days. Topics will include motivation for HPC, essential issues, problem characteristics as they apply to parallelism and a high level overview of parallel programming models. Strategies of running large sets of serial processes using e.g. GNU parallel, will also be presented.

=====Details=====
''Time/date''

Monday July 11, 2016, 09:30 - 12:30

''Location''

Wallberg Building, Room 116

''Instructor''

Dr. Mike Nolta (SciNet)

=====Prerequisites=====
Basic Linux skills are useful to follow along. If you do not have these, consider taking the "Intro to the Linux Shell" in the afternoon.

=====Slides and other material=====

* [https://support.scinet.utoronto.ca/~nolta/HPC_intro.pdf Slides]

----

===Programming Shared Memory Systems with OpenMP===

=====Abstract=====
In these two half-day sessions, lectures and hands-on labs are interspersed, and the students will learn the basics of shared memory programming with OpenMP. In particular, we will discuss the OpenMP's threads, memory, and performance, reductions and load balancing. We will also discuss extensions to heterogeneous systems such as offered by the OpenMP 4.0 and OpenACC standards.

=====Details=====

''Time/date''

Monday July 11, 2016, 13:30 - 16:30 
Tuesday July 12, 2016, 09:30 - 12:30

''Location''

Wallberg Building, Room 116

''Instructor''

Dr. Ramses van Zon (SciNet)

=====Prerequisites=====
C/C++ and/or Fortran scientific programming; Experience editing and compiling code in a Linux environment.

=====Slides and other material=====

* [[Media:OpenMP2016.pdf | Slides]]
* [[Media:OpenMP_SS2015_code.tar.gz|Code]]

----

===Introduction to the Linux Shell===

====Abstract====
Working with many of the HPC systems in Ontario involves using the Linux/UNIX command line. This provides a very powerful interface, but it can be quite daunting for the uninitiated. In this half-day session, you can become initiated with this course. This hands on session will cover basic commands and scripting, as well as touching on some powerful constructs like regular expressions. It could be a great boon for your productivity!

====Details====
''Time/date''

Monday July 11, 13:30 - 16:30

''Location''

Wallberg Building, Room 119

''Instructor''

Dr. Mike Nolta (SciNet)

====Prerequisites====
A laptop with a linux-like environment.

====Slides and other material====

* [[Media:intro_to_the_unix_shell.pdf | Slides ]]

----

===R for Data Science===

=====Abstract=====
This session offers a brief introduction to R, with a focus on data analysis and statistics.

=====Details=====

''Date/time''

Tuesday July 12, 2016, 09:30 - 12:30

''Location''

Wallberg Building, Room 119 

''Instructor''

Dr. Erik Spence (SciNet)

=====Prerequisites=====

Some programming experience in any language. You should have used and written functions. Bring a laptop with R or Rstudio installed.

=====Slides and other material=====

* [https://wiki.scinet.utoronto.ca/wiki/images/b/b1/Intro_R.pdf Slides]

----

===Programming Clusters with MPI===

=====Abstract=====
In this 1.5-day session, through lectures interspersed with hands-on labs, the students will learn the basics of MPI programming. Examples and exercises will be based on parallelization of common scientific computing problems.

=====Details=====
''Time/date''

Tuesday July 12, 2016, 13:30 - 16:30 
Wednesday July 13, 2016, 09:30 - 12:30 
Wednesday July 13, 2016, 13:30 - 16:30 

''Location''

Wallberg Building, Room 116

''Instructor''

Dr. Scott Northrup (SciNet)

=====Prerequisites=====
C and/or Fortran scientific programming; experience editing and compiling code in a Linux environment.

=====Slides and other material=====
* [[Media:MPIss2016.pdf | Slides ]]
----

===Parallel R for Data Science===

=====Abstract=====
This half-day session will cover parallel programming in R, with a focus on parallel data analysis. Topics covered will include snow, parallel, and foreach/doparallel.

=====Details=====

''Time/date''

Tuesday July 12, 2016, 13:30 - 16:30

''Location''

Wallberg Building, Room 119

''Instructor''

Dr. Erik Spence (SciNet)

=====Prerequisites=====
Experience with programming in R and a rudimentary understanding of parallel computing.

=====Slides and other material=====

* [https://wiki.scinet.utoronto.ca/wiki/images/0/02/Parallel_R.pdf Slides]

----

===Python for Scientific Computing===

=====Abstract=====
In this half-day session, you will be taught how to use python for research computing purposes. Topics include: the basics of python, automation, numpy, scipy, file i/o, and visualization.

=====Details=====
''Time/date'''

Wednesday July 13, 9:30 - 12:30

''Location''

Wallberg Building, Room 119

''Instructor''

Dr. Erik Spence (SciNet)

=====Prerequisites=====
A bit of coding experience in any programming or scripting language. Command-line experience in Linux is a plus.

=====Slides and other material=====

----

===Python for High Performance Computing===

=====Abstract=====
This half-day session will cover parallel programming in python, with a focus on parallel data analysis. We will cover subprocess, multiprocessing, pypar and other parallel-enabling python packages.

=====Details=====
''Time/date''

Wednesday July 13, 2016, 13:30 - 16:30

''Location''

Wallberg Building, Room 119

''Instructor''

Dr. Ramses van Zon (SciNet)

=====Prerequisites=====
A bit of coding experience in any programming or scripting language. Command-line experience in Linux is a plus.

=====Slides and other material=====

----

===Programming GPUs with CUDA===

=====Abstract=====
This two-day session is an introductory course covering programming and computing on GPUs - graphics processing units - which are an increasingly common presence in massively parallel computing architectures. This session will cover both of the available C-like programming frameworks: NVIDIA’s CUDA C. The basics of GPU programming will be covered, and students will work through a number of hands on examples. Demonstrations of profiling and debugging applications running on the GPU will also be included. The structuring of data and computations that makes full use of the GPU will be discussed in detail. Students should be able to leave the course with the knowledge necessary to begin developing their own GPU applications.

=====Details=====
''Time/date''

Thursday July 14, 2016, 09:30 - 12:30 
Thursday July 14, 2016, 13:30 - 16:30 
Friday July 15, 2016, 09:30 - 12:30 
Friday July 15, 2016, 13:30 - 16:30

''Location''

Wallberg Building, Room 116

''Instructors''
Dr. Pawel Pomorski and Dr. Sergey Mashchenko (SHARCNET)

=====Prerequisites=====
C/C++ scientific programming, experience editing and compiling code in a Linux environment. Some experience with CUDA and/or OpenMP a plus.

=====Slides and other materials=====

----

===Visualization===

=====Abstract=====
The first part of this whole-day session will be a basic introduction to general-purpose scientific visualization tools such as gnuplot, grace(xmgr), remote tools (tunneling, VNC), python's matplotlib, and examples from other. The afternoon part will be a mix of lecture and hands-on to introduce more specialized scientific visualization software such as vmd, paraview, and visit.

=====Details=====
''Time/date''

Thursday July 14, 2016, 09:30 - 12:30 
Thursday July 14, 2016, 13:30 - 16:30

''Location''

Wallberg Building, Room 119

''Instructor''
Dr. Marcelo Ponce (SciNet)

=====Prerequisites=====
Software installed, for the morning session: gnuplot, xmgr ([http://plasma-gate.weizmann.ac.il/Grace/ Grace],[http://plasma-gate.weizmann.ac.il/Xmgr/ xmgr]), ssh (graphic client), your favorite vnc client (egs. [https://www.realvnc.com/download/viewer/ RealVNC],[http://www.tightvnc.com/download.php TightVNC]), python and matplotlib ([http://matplotlib.org/ matplotlib]); afternoon session: ParaView ([http://www.paraview.org/download/ ParaView]) and VisIt ([https://wci.llnl.gov/simulation/computer-codes/visit/executables VisIt]).

=====Slides and other material=====
======Slides======
======Datasets======
[ Morning datasets and scripts]

[https://support.scinet.utoronto.ca/~mponce/ss2016/datasets.tar.gz Afternoon dataset]

[https://support.scinet.utoronto.ca/~mponce/ss2016/aneurysm_data.tar.gz Aneurysm dataset]
----

===Debugging===

=====Abstract=====
Debugging is an important step in developing a new code, or porting an old one to a new machine. In this half-day session, we will discuss the debugging of frequently encountered bugs in serial code with gdb and the debugging of parallel (MPI and threaded) codes with DDT on live systems.

=====Details=====
''Date/time''

Friday July 15, 09:30 - 12:30

''Location''

Wallberg Building, Room 119

Note the different room number!

''Instructors''

Dr. Marcelo Ponce (SciNet)

=====Prerequisites=====
Basic familiarity with a high-level language (C/C++/Fortran) as well as compiling and running programs via the command line environment on a HPC system.

=====Slides and other material=====

----

===Bring Your Own Code===

=====Abstract=====
The final afternoon of the summer school is your chance to get help with your own code from the SciNet software analysts. Bring your code, explain your problem or concerns, and we'll give you advice and work with you to try to improve it.

=====Details=====
''Date/time''

Friday July 15, 2016, 13:30 - 16:30

''Location''

Wallberg Building, Room 119

''Instructors''

Various SciNet software analysts.

=====Prerequisites=====
None.

2016 Ontario Summer School for High Performance Computing Central

2016-07-12T17:02:48Z

Ejspence: /* Slides and other material */

===About===

The 2016 Ontario Summer School on High Performance Computing, whose second installment of the year will be hosted by SciNet, provides attendees with opportunities to learn and share knowledge and experience in high performance and technical computing. The format: a five-day workshop with mixed lectures and hands-on sessions on a number of selected subjects, including shared memory programming, distributed memory programming, parallel data processing, and general purpose graphics processing unit programming.

Below are descriptions and course material for the Central version, given from July 11th to July 15th, 2016, in Toronto.

The location of this version of the summer school is:

'''Wallberg Building''' 
'''Rooms 116 and 119''' 
University of Toronto 
St. George Campus 
184-200 College Street 
Toronto, Ontario, M5S 3E5 
Canada

More information can be found below and in the [[Media:handout2016.pdf|handout package]].

__TOC__

===Introduction to HPC and SciNet===

=====Abstract=====
This half-day session will provide an introduction to basic concepts of high-performance computing. It is intended to be a high level primer for those largely new to HPC, and serve as a foundation upon which to build over the coming days. Topics will include motivation for HPC, essential issues, problem characteristics as they apply to parallelism and a high level overview of parallel programming models. Strategies of running large sets of serial processes using e.g. GNU parallel, will also be presented.

=====Details=====
''Time/date''

Monday July 11, 2016, 09:30 - 12:30

''Location''

Wallberg Building, Room 116

''Instructor''

Dr. Mike Nolta (SciNet)

=====Prerequisites=====
Basic Linux skills are useful to follow along. If you do not have these, consider taking the "Intro to the Linux Shell" in the afternoon.

=====Slides and other material=====

* [https://support.scinet.utoronto.ca/~nolta/HPC_intro.pdf Slides]

----

===Programming Shared Memory Systems with OpenMP===

=====Abstract=====
In these two half-day sessions, lectures and hands-on labs are interspersed, and the students will learn the basics of shared memory programming with OpenMP. In particular, we will discuss the OpenMP's threads, memory, and performance, reductions and load balancing. We will also discuss extensions to heterogeneous systems such as offered by the OpenMP 4.0 and OpenACC standards.

=====Details=====

''Time/date''

Monday July 11, 2016, 13:30 - 16:30 
Tuesday July 12, 2016, 09:30 - 12:30

''Location''

Wallberg Building, Room 116

''Instructor''

Dr. Ramses van Zon (SciNet)

=====Prerequisites=====
C/C++ and/or Fortran scientific programming; Experience editing and compiling code in a Linux environment.

=====Slides and other material=====

* [[Media:OpenMP2016.pdf | Slides]]
* [[Media:OpenMP_SS2015_code.tar.gz|Code]]

----

===Introduction to the Linux Shell===

====Abstract====
Working with many of the HPC systems in Ontario involves using the Linux/UNIX command line. This provides a very powerful interface, but it can be quite daunting for the uninitiated. In this half-day session, you can become initiated with this course. This hands on session will cover basic commands and scripting, as well as touching on some powerful constructs like regular expressions. It could be a great boon for your productivity!

====Details====
''Time/date''

Monday July 11, 13:30 - 16:30

''Location''

Wallberg Building, Room 119

''Instructor''

Dr. Mike Nolta (SciNet)

====Prerequisites====
A laptop with a linux-like environment.

====Slides and other material====

* [[Media:intro_to_the_unix_shell.pdf | Slides ]]

----

===R for Data Science===

=====Abstract=====
This session offers a brief introduction to R, with a focus on data analysis and statistics.

=====Details=====

''Date/time''

Tuesday July 12, 2016, 09:30 - 12:30

''Location''

Wallberg Building, Room 119 

''Instructor''

Dr. Erik Spence (SciNet)

=====Prerequisites=====

Some programming experience in any language. You should have used and written functions. Bring a laptop with R or Rstudio installed.

=====Slides and other material=====

* [https://wiki.scinet.utoronto.ca/wiki/images/b/b1/Intro_R.pdf Slides]

----

===Programming Clusters with MPI===

=====Abstract=====
In this 1.5-day session, through lectures interspersed with hands-on labs, the students will learn the basics of MPI programming. Examples and exercises will be based on parallelization of common scientific computing problems.

=====Details=====
''Time/date''

Tuesday July 12, 2016, 13:30 - 16:30 
Wednesday July 13, 2016, 09:30 - 12:30 
Wednesday July 13, 2016, 13:30 - 16:30 

''Location''

Wallberg Building, Room 116

''Instructor''

Dr. Scott Northrup (SciNet)

=====Prerequisites=====
C and/or Fortran scientific programming; experience editing and compiling code in a Linux environment.

=====Slides and other material=====
* [[Media:MPIss2016.pdf | Slides ]]
----

===Parallel R for Data Science===

=====Abstract=====
This half-day session will cover parallel programming in R, with a focus on parallel data analysis. Topics covered will include snow, parallel, and foreach/doparallel.

=====Details=====

''Time/date''

Tuesday July 12, 2016, 13:30 - 16:30

''Location''

Wallberg Building, Room 119

''Instructor''

Dr. Erik Spence (SciNet)

=====Prerequisites=====
Experience with programming in R and a rudimentary understanding of parallel computing.

=====Slides and other material=====

* [https://wiki.scinet.utoronto.ca/wiki/images/0/02/Parallel_R.pdf Slides]

----

===Python for Scientific Computing===

=====Abstract=====
In this half-day session, you will be taught how to use python for research computing purposes. Topics include: the basics of python, automation, numpy, scipy, file i/o, and visualization.

=====Details=====
''Time/date'''

Wednesday July 13, 9:30 - 12:30

''Location''

Wallberg Building, Room 119

''Instructor''

Dr. Erik Spence (SciNet)

=====Prerequisites=====
A bit of coding experience in any programming or scripting language. Command-line experience in Linux is a plus.

=====Slides and other material=====

----

===Python for High Performance Computing===

=====Abstract=====
This half-day session will cover parallel programming in python, with a focus on parallel data analysis. We will cover subprocess, multiprocessing, pypar and other parallel-enabling python packages.

=====Details=====
''Time/date''

Wednesday July 13, 2016, 13:30 - 16:30

''Location''

Wallberg Building, Room 119

''Instructor''

Dr. Ramses van Zon (SciNet)

=====Prerequisites=====
A bit of coding experience in any programming or scripting language. Command-line experience in Linux is a plus.

=====Slides and other material=====

----

===Programming GPUs with CUDA===

=====Abstract=====
This two-day session is an introductory course covering programming and computing on GPUs - graphics processing units - which are an increasingly common presence in massively parallel computing architectures. This session will cover both of the available C-like programming frameworks: NVIDIA’s CUDA C. The basics of GPU programming will be covered, and students will work through a number of hands on examples. Demonstrations of profiling and debugging applications running on the GPU will also be included. The structuring of data and computations that makes full use of the GPU will be discussed in detail. Students should be able to leave the course with the knowledge necessary to begin developing their own GPU applications.

=====Details=====
''Time/date''

Thursday July 14, 2016, 09:30 - 12:30 
Thursday July 14, 2016, 13:30 - 16:30 
Friday July 15, 2016, 09:30 - 12:30 
Friday July 15, 2016, 13:30 - 16:30

''Location''

Wallberg Building, Room 116

''Instructors''
Dr. Pawel Pomorski and Dr. Sergey Mashchenko (SHARCNET)

=====Prerequisites=====
C/C++ scientific programming, experience editing and compiling code in a Linux environment. Some experience with CUDA and/or OpenMP a plus.

=====Slides and other materials=====

----

===Visualization===

=====Abstract=====
The first part of this whole-day session will be a basic introduction to general-purpose scientific visualization tools such as gnuplot, grace(xmgr), remote tools (tunneling, VNC), python's matplotlib, and examples from other. The afternoon part will be a mix of lecture and hands-on to introduce more specialized scientific visualization software such as vmd, paraview, and visit.

=====Details=====
''Time/date''

Thursday July 14, 2016, 09:30 - 12:30 
Thursday July 14, 2016, 13:30 - 16:30

''Location''

Wallberg Building, Room 119

''Instructor''
Dr. Marcelo Ponce (SciNet)

=====Prerequisites=====
Software installed, for the morning session: gnuplot, xmgr ([http://plasma-gate.weizmann.ac.il/Grace/ Grace],[http://plasma-gate.weizmann.ac.il/Xmgr/ xmgr]), ssh (graphic client), your favorite vnc client (egs. [https://www.realvnc.com/download/viewer/ RealVNC],[http://www.tightvnc.com/download.php TightVNC]), python and matplotlib ([http://matplotlib.org/ matplotlib]); afternoon session: ParaView ([http://www.paraview.org/download/ ParaView]) and VisIt ([https://wci.llnl.gov/simulation/computer-codes/visit/executables VisIt]).

=====Slides and other material=====
======Slides======
======Datasets======
[ Morning datasets and scripts]

[https://support.scinet.utoronto.ca/~mponce/ss2016/datasets.tar.gz Afternoon dataset]

[https://support.scinet.utoronto.ca/~mponce/ss2016/aneurysm_data.tar.gz Aneurysm dataset]
----

===Debugging===

=====Abstract=====
Debugging is an important step in developing a new code, or porting an old one to a new machine. In this half-day session, we will discuss the debugging of frequently encountered bugs in serial code with gdb and the debugging of parallel (MPI and threaded) codes with DDT on live systems.

=====Details=====
''Date/time''

Friday July 15, 09:30 - 12:30

''Location''

Wallberg Building, Room 119

Note the different room number!

''Instructors''

Dr. Marcelo Ponce (SciNet)

=====Prerequisites=====
Basic familiarity with a high-level language (C/C++/Fortran) as well as compiling and running programs via the command line environment on a HPC system.

=====Slides and other material=====

----

===Bring Your Own Code===

=====Abstract=====
The final afternoon of the summer school is your chance to get help with your own code from the SciNet software analysts. Bring your code, explain your problem or concerns, and we'll give you advice and work with you to try to improve it.

=====Details=====
''Date/time''

Friday July 15, 2016, 13:30 - 16:30

''Location''

Wallberg Building, Room 119

''Instructors''

Various SciNet software analysts.

=====Prerequisites=====
None.

2016 Ontario Summer School for High Performance Computing Central

2016-07-12T17:02:27Z

Ejspence: /* Slides and other material */

===About===

The 2016 Ontario Summer School on High Performance Computing, whose second installment of the year will be hosted by SciNet, provides attendees with opportunities to learn and share knowledge and experience in high performance and technical computing. The format: a five-day workshop with mixed lectures and hands-on sessions on a number of selected subjects, including shared memory programming, distributed memory programming, parallel data processing, and general purpose graphics processing unit programming.

Below are descriptions and course material for the Central version, given from July 11th to July 15th, 2016, in Toronto.

The location of this version of the summer school is:

'''Wallberg Building''' 
'''Rooms 116 and 119''' 
University of Toronto 
St. George Campus 
184-200 College Street 
Toronto, Ontario, M5S 3E5 
Canada

More information can be found below and in the [[Media:handout2016.pdf|handout package]].

__TOC__

===Introduction to HPC and SciNet===

=====Abstract=====
This half-day session will provide an introduction to basic concepts of high-performance computing. It is intended to be a high level primer for those largely new to HPC, and serve as a foundation upon which to build over the coming days. Topics will include motivation for HPC, essential issues, problem characteristics as they apply to parallelism and a high level overview of parallel programming models. Strategies of running large sets of serial processes using e.g. GNU parallel, will also be presented.

=====Details=====
''Time/date''

Monday July 11, 2016, 09:30 - 12:30

''Location''

Wallberg Building, Room 116

''Instructor''

Dr. Mike Nolta (SciNet)

=====Prerequisites=====
Basic Linux skills are useful to follow along. If you do not have these, consider taking the "Intro to the Linux Shell" in the afternoon.

=====Slides and other material=====

* [https://support.scinet.utoronto.ca/~nolta/HPC_intro.pdf Slides]

----

===Programming Shared Memory Systems with OpenMP===

=====Abstract=====
In these two half-day sessions, lectures and hands-on labs are interspersed, and the students will learn the basics of shared memory programming with OpenMP. In particular, we will discuss the OpenMP's threads, memory, and performance, reductions and load balancing. We will also discuss extensions to heterogeneous systems such as offered by the OpenMP 4.0 and OpenACC standards.

=====Details=====

''Time/date''

Monday July 11, 2016, 13:30 - 16:30 
Tuesday July 12, 2016, 09:30 - 12:30

''Location''

Wallberg Building, Room 116

''Instructor''

Dr. Ramses van Zon (SciNet)

=====Prerequisites=====
C/C++ and/or Fortran scientific programming; Experience editing and compiling code in a Linux environment.

=====Slides and other material=====

* [[Media:OpenMP2016.pdf | Slides]]
* [[Media:OpenMP_SS2015_code.tar.gz|Code]]

----

===Introduction to the Linux Shell===

====Abstract====
Working with many of the HPC systems in Ontario involves using the Linux/UNIX command line. This provides a very powerful interface, but it can be quite daunting for the uninitiated. In this half-day session, you can become initiated with this course. This hands on session will cover basic commands and scripting, as well as touching on some powerful constructs like regular expressions. It could be a great boon for your productivity!

====Details====
''Time/date''

Monday July 11, 13:30 - 16:30

''Location''

Wallberg Building, Room 119

''Instructor''

Dr. Mike Nolta (SciNet)

====Prerequisites====
A laptop with a linux-like environment.

====Slides and other material====

* [[Media:intro_to_the_unix_shell.pdf | Slides ]]

----

===R for Data Science===

=====Abstract=====
This session offers a brief introduction to R, with a focus on data analysis and statistics.

=====Details=====

''Date/time''

Tuesday July 12, 2016, 09:30 - 12:30

''Location''

Wallberg Building, Room 119 

''Instructor''

Dr. Erik Spence (SciNet)

=====Prerequisites=====

Some programming experience in any language. You should have used and written functions. Bring a laptop with R or Rstudio installed.

=====Slides and other material=====

* [https://wiki.scinet.utoronto.ca/wiki/images/b/b1/Intro_R.pdf Slides]

----

===Programming Clusters with MPI===

=====Abstract=====
In this 1.5-day session, through lectures interspersed with hands-on labs, the students will learn the basics of MPI programming. Examples and exercises will be based on parallelization of common scientific computing problems.

=====Details=====
''Time/date''

Tuesday July 12, 2016, 13:30 - 16:30 
Wednesday July 13, 2016, 09:30 - 12:30 
Wednesday July 13, 2016, 13:30 - 16:30 

''Location''

Wallberg Building, Room 116

''Instructor''

Dr. Scott Northrup (SciNet)

=====Prerequisites=====
C and/or Fortran scientific programming; experience editing and compiling code in a Linux environment.

=====Slides and other material=====
* [[Media:MPIss2016.pdf | Slides ]]
----

===Parallel R for Data Science===

=====Abstract=====
This half-day session will cover parallel programming in R, with a focus on parallel data analysis. Topics covered will include snow, parallel, and foreach/doparallel.

=====Details=====

''Time/date''

Tuesday July 12, 2016, 13:30 - 16:30

''Location''

Wallberg Building, Room 119

''Instructor''

Dr. Erik Spence (SciNet)

=====Prerequisites=====
Experience with programming in R and a rudimentary understanding of parallel computing.

=====Slides and other material=====

* [https://wiki.scinet.utoronto.ca/wiki/images/0/02/Parallel_R.pdf Slides]

----

===Python for Scientific Computing===

=====Abstract=====
In this half-day session, you will be taught how to use python for research computing purposes. Topics include: the basics of python, automation, numpy, scipy, file i/o, and visualization.

=====Details=====
''Time/date'''

Wednesday July 13, 9:30 - 12:30

''Location''

Wallberg Building, Room 119

''Instructor''

Dr. Erik Spence (SciNet)

=====Prerequisites=====
A bit of coding experience in any programming or scripting language. Command-line experience in Linux is a plus.

=====Slides and other material=====

----

===Python for High Performance Computing===

=====Abstract=====
This half-day session will cover parallel programming in python, with a focus on parallel data analysis. We will cover subprocess, multiprocessing, pypar and other parallel-enabling python packages.

=====Details=====
''Time/date''

Wednesday July 13, 2016, 13:30 - 16:30

''Location''

Wallberg Building, Room 119

''Instructor''

Dr. Ramses van Zon (SciNet)

=====Prerequisites=====
A bit of coding experience in any programming or scripting language. Command-line experience in Linux is a plus.

=====Slides and other material=====

----

===Programming GPUs with CUDA===

=====Abstract=====
This two-day session is an introductory course covering programming and computing on GPUs - graphics processing units - which are an increasingly common presence in massively parallel computing architectures. This session will cover both of the available C-like programming frameworks: NVIDIA’s CUDA C. The basics of GPU programming will be covered, and students will work through a number of hands on examples. Demonstrations of profiling and debugging applications running on the GPU will also be included. The structuring of data and computations that makes full use of the GPU will be discussed in detail. Students should be able to leave the course with the knowledge necessary to begin developing their own GPU applications.

=====Details=====
''Time/date''

Thursday July 14, 2016, 09:30 - 12:30 
Thursday July 14, 2016, 13:30 - 16:30 
Friday July 15, 2016, 09:30 - 12:30 
Friday July 15, 2016, 13:30 - 16:30

''Location''

Wallberg Building, Room 116

''Instructors''
Dr. Pawel Pomorski and Dr. Sergey Mashchenko (SHARCNET)

=====Prerequisites=====
C/C++ scientific programming, experience editing and compiling code in a Linux environment. Some experience with CUDA and/or OpenMP a plus.

=====Slides and other materials=====

----

===Visualization===

=====Abstract=====
The first part of this whole-day session will be a basic introduction to general-purpose scientific visualization tools such as gnuplot, grace(xmgr), remote tools (tunneling, VNC), python's matplotlib, and examples from other. The afternoon part will be a mix of lecture and hands-on to introduce more specialized scientific visualization software such as vmd, paraview, and visit.

=====Details=====
''Time/date''

Thursday July 14, 2016, 09:30 - 12:30 
Thursday July 14, 2016, 13:30 - 16:30

''Location''

Wallberg Building, Room 119

''Instructor''
Dr. Marcelo Ponce (SciNet)

=====Prerequisites=====
Software installed, for the morning session: gnuplot, xmgr ([http://plasma-gate.weizmann.ac.il/Grace/ Grace],[http://plasma-gate.weizmann.ac.il/Xmgr/ xmgr]), ssh (graphic client), your favorite vnc client (egs. [https://www.realvnc.com/download/viewer/ RealVNC],[http://www.tightvnc.com/download.php TightVNC]), python and matplotlib ([http://matplotlib.org/ matplotlib]); afternoon session: ParaView ([http://www.paraview.org/download/ ParaView]) and VisIt ([https://wci.llnl.gov/simulation/computer-codes/visit/executables VisIt]).

=====Slides and other material=====
======Slides======
======Datasets======
[ Morning datasets and scripts]

[https://support.scinet.utoronto.ca/~mponce/ss2016/datasets.tar.gz Afternoon dataset]

[https://support.scinet.utoronto.ca/~mponce/ss2016/aneurysm_data.tar.gz Aneurysm dataset]
----

===Debugging===

=====Abstract=====
Debugging is an important step in developing a new code, or porting an old one to a new machine. In this half-day session, we will discuss the debugging of frequently encountered bugs in serial code with gdb and the debugging of parallel (MPI and threaded) codes with DDT on live systems.

=====Details=====
''Date/time''

Friday July 15, 09:30 - 12:30

''Location''

Wallberg Building, Room 119

Note the different room number!

''Instructors''

Dr. Marcelo Ponce (SciNet)

=====Prerequisites=====
Basic familiarity with a high-level language (C/C++/Fortran) as well as compiling and running programs via the command line environment on a HPC system.

=====Slides and other material=====

----

===Bring Your Own Code===

=====Abstract=====
The final afternoon of the summer school is your chance to get help with your own code from the SciNet software analysts. Bring your code, explain your problem or concerns, and we'll give you advice and work with you to try to improve it.

=====Details=====
''Date/time''

Friday July 15, 2016, 13:30 - 16:30

''Location''

Wallberg Building, Room 119

''Instructors''

Various SciNet software analysts.

=====Prerequisites=====
None.

2016 Ontario Summer School for High Performance Computing Central

2016-07-12T17:01:57Z

Ejspence:

===About===

The 2016 Ontario Summer School on High Performance Computing, whose second installment of the year will be hosted by SciNet, provides attendees with opportunities to learn and share knowledge and experience in high performance and technical computing. The format: a five-day workshop with mixed lectures and hands-on sessions on a number of selected subjects, including shared memory programming, distributed memory programming, parallel data processing, and general purpose graphics processing unit programming.

Below are descriptions and course material for the Central version, given from July 11th to July 15th, 2016, in Toronto.

The location of this version of the summer school is:

'''Wallberg Building''' 
'''Rooms 116 and 119''' 
University of Toronto 
St. George Campus 
184-200 College Street 
Toronto, Ontario, M5S 3E5 
Canada

More information can be found below and in the [[Media:handout2016.pdf|handout package]].

__TOC__

===Introduction to HPC and SciNet===

=====Abstract=====
This half-day session will provide an introduction to basic concepts of high-performance computing. It is intended to be a high level primer for those largely new to HPC, and serve as a foundation upon which to build over the coming days. Topics will include motivation for HPC, essential issues, problem characteristics as they apply to parallelism and a high level overview of parallel programming models. Strategies of running large sets of serial processes using e.g. GNU parallel, will also be presented.

=====Details=====
''Time/date''

Monday July 11, 2016, 09:30 - 12:30

''Location''

Wallberg Building, Room 116

''Instructor''

Dr. Mike Nolta (SciNet)

=====Prerequisites=====
Basic Linux skills are useful to follow along. If you do not have these, consider taking the "Intro to the Linux Shell" in the afternoon.

=====Slides and other material=====

* [https://support.scinet.utoronto.ca/~nolta/HPC_intro.pdf Slides]

----

===Programming Shared Memory Systems with OpenMP===

=====Abstract=====
In these two half-day sessions, lectures and hands-on labs are interspersed, and the students will learn the basics of shared memory programming with OpenMP. In particular, we will discuss the OpenMP's threads, memory, and performance, reductions and load balancing. We will also discuss extensions to heterogeneous systems such as offered by the OpenMP 4.0 and OpenACC standards.

=====Details=====

''Time/date''

Monday July 11, 2016, 13:30 - 16:30 
Tuesday July 12, 2016, 09:30 - 12:30

''Location''

Wallberg Building, Room 116

''Instructor''

Dr. Ramses van Zon (SciNet)

=====Prerequisites=====
C/C++ and/or Fortran scientific programming; Experience editing and compiling code in a Linux environment.

=====Slides and other material=====

* [[Media:OpenMP2016.pdf | Slides]]
* [[Media:OpenMP_SS2015_code.tar.gz|Code]]

----

===Introduction to the Linux Shell===

====Abstract====
Working with many of the HPC systems in Ontario involves using the Linux/UNIX command line. This provides a very powerful interface, but it can be quite daunting for the uninitiated. In this half-day session, you can become initiated with this course. This hands on session will cover basic commands and scripting, as well as touching on some powerful constructs like regular expressions. It could be a great boon for your productivity!

====Details====
''Time/date''

Monday July 11, 13:30 - 16:30

''Location''

Wallberg Building, Room 119

''Instructor''

Dr. Mike Nolta (SciNet)

====Prerequisites====
A laptop with a linux-like environment.

====Slides and other material====

* [[Media:intro_to_the_unix_shell.pdf | Slides ]]

----

===R for Data Science===

=====Abstract=====
This session offers a brief introduction to R, with a focus on data analysis and statistics.

=====Details=====

''Date/time''

Tuesday July 12, 2016, 09:30 - 12:30

''Location''

Wallberg Building, Room 119 

''Instructor''

Dr. Erik Spence (SciNet)

=====Prerequisites=====

Some programming experience in any language. You should have used and written functions. Bring a laptop with R or Rstudio installed.

=====Slides and other material=====

* [https://wiki.scinet.utoronto.ca/wiki/images/b/b1/Intro_R.pdf Slides]

----

===Programming Clusters with MPI===

=====Abstract=====
In this 1.5-day session, through lectures interspersed with hands-on labs, the students will learn the basics of MPI programming. Examples and exercises will be based on parallelization of common scientific computing problems.

=====Details=====
''Time/date''

Tuesday July 12, 2016, 13:30 - 16:30 
Wednesday July 13, 2016, 09:30 - 12:30 
Wednesday July 13, 2016, 13:30 - 16:30 

''Location''

Wallberg Building, Room 116

''Instructor''

Dr. Scott Northrup (SciNet)

=====Prerequisites=====
C and/or Fortran scientific programming; experience editing and compiling code in a Linux environment.

=====Slides and other material=====
* [[Media:MPIss2016.pdf | Slides ]]
----

===Parallel R for Data Science===

=====Abstract=====
This half-day session will cover parallel programming in R, with a focus on parallel data analysis. Topics covered will include snow, parallel, and foreach/doparallel.

=====Details=====

''Time/date''

Tuesday July 12, 2016, 13:30 - 16:30

''Location''

Wallberg Building, Room 119

''Instructor''

Dr. Erik Spence (SciNet)

=====Prerequisites=====
Experience with programming in R and a rudimentary understanding of parallel computing.

=====Slides and other material=====

* [https://wiki.scinet.utoronto.ca/wiki/images/0/02/Parallel_R.pdf Slides]
----

===Python for Scientific Computing===

=====Abstract=====
In this half-day session, you will be taught how to use python for research computing purposes. Topics include: the basics of python, automation, numpy, scipy, file i/o, and visualization.

=====Details=====
''Time/date'''

Wednesday July 13, 9:30 - 12:30

''Location''

Wallberg Building, Room 119

''Instructor''

Dr. Erik Spence (SciNet)

=====Prerequisites=====
A bit of coding experience in any programming or scripting language. Command-line experience in Linux is a plus.

=====Slides and other material=====

----

===Python for High Performance Computing===

=====Abstract=====
This half-day session will cover parallel programming in python, with a focus on parallel data analysis. We will cover subprocess, multiprocessing, pypar and other parallel-enabling python packages.

=====Details=====
''Time/date''

Wednesday July 13, 2016, 13:30 - 16:30

''Location''

Wallberg Building, Room 119

''Instructor''

Dr. Ramses van Zon (SciNet)

=====Prerequisites=====
A bit of coding experience in any programming or scripting language. Command-line experience in Linux is a plus.

=====Slides and other material=====

----

===Programming GPUs with CUDA===

=====Abstract=====
This two-day session is an introductory course covering programming and computing on GPUs - graphics processing units - which are an increasingly common presence in massively parallel computing architectures. This session will cover both of the available C-like programming frameworks: NVIDIA’s CUDA C. The basics of GPU programming will be covered, and students will work through a number of hands on examples. Demonstrations of profiling and debugging applications running on the GPU will also be included. The structuring of data and computations that makes full use of the GPU will be discussed in detail. Students should be able to leave the course with the knowledge necessary to begin developing their own GPU applications.

=====Details=====
''Time/date''

Thursday July 14, 2016, 09:30 - 12:30 
Thursday July 14, 2016, 13:30 - 16:30 
Friday July 15, 2016, 09:30 - 12:30 
Friday July 15, 2016, 13:30 - 16:30

''Location''

Wallberg Building, Room 116

''Instructors''
Dr. Pawel Pomorski and Dr. Sergey Mashchenko (SHARCNET)

=====Prerequisites=====
C/C++ scientific programming, experience editing and compiling code in a Linux environment. Some experience with CUDA and/or OpenMP a plus.

=====Slides and other materials=====

----

===Visualization===

=====Abstract=====
The first part of this whole-day session will be a basic introduction to general-purpose scientific visualization tools such as gnuplot, grace(xmgr), remote tools (tunneling, VNC), python's matplotlib, and examples from other. The afternoon part will be a mix of lecture and hands-on to introduce more specialized scientific visualization software such as vmd, paraview, and visit.

=====Details=====
''Time/date''

Thursday July 14, 2016, 09:30 - 12:30 
Thursday July 14, 2016, 13:30 - 16:30

''Location''

Wallberg Building, Room 119

''Instructor''
Dr. Marcelo Ponce (SciNet)

=====Prerequisites=====
Software installed, for the morning session: gnuplot, xmgr ([http://plasma-gate.weizmann.ac.il/Grace/ Grace],[http://plasma-gate.weizmann.ac.il/Xmgr/ xmgr]), ssh (graphic client), your favorite vnc client (egs. [https://www.realvnc.com/download/viewer/ RealVNC],[http://www.tightvnc.com/download.php TightVNC]), python and matplotlib ([http://matplotlib.org/ matplotlib]); afternoon session: ParaView ([http://www.paraview.org/download/ ParaView]) and VisIt ([https://wci.llnl.gov/simulation/computer-codes/visit/executables VisIt]).

=====Slides and other material=====
======Slides======
======Datasets======
[ Morning datasets and scripts]

[https://support.scinet.utoronto.ca/~mponce/ss2016/datasets.tar.gz Afternoon dataset]

[https://support.scinet.utoronto.ca/~mponce/ss2016/aneurysm_data.tar.gz Aneurysm dataset]
----

===Debugging===

=====Abstract=====
Debugging is an important step in developing a new code, or porting an old one to a new machine. In this half-day session, we will discuss the debugging of frequently encountered bugs in serial code with gdb and the debugging of parallel (MPI and threaded) codes with DDT on live systems.

=====Details=====
''Date/time''

Friday July 15, 09:30 - 12:30

''Location''

Wallberg Building, Room 119

Note the different room number!

''Instructors''

Dr. Marcelo Ponce (SciNet)

=====Prerequisites=====
Basic familiarity with a high-level language (C/C++/Fortran) as well as compiling and running programs via the command line environment on a HPC system.

=====Slides and other material=====

----

===Bring Your Own Code===

=====Abstract=====
The final afternoon of the summer school is your chance to get help with your own code from the SciNet software analysts. Bring your code, explain your problem or concerns, and we'll give you advice and work with you to try to improve it.

=====Details=====
''Date/time''

Friday July 15, 2016, 13:30 - 16:30

''Location''

Wallberg Building, Room 119

''Instructors''

Various SciNet software analysts.

=====Prerequisites=====
None.

File:Parallel R.pdf

2016-07-12T17:00:32Z

Ejspence: Slides for the Parallel R session of the 2016 Summer School.

Slides for the Parallel R session of the 2016 Summer School.

2016 Ontario Summer School for High Performance Computing Central

2016-07-12T12:40:44Z

Ejspence: /* Slides and other material */

===About===

The 2016 Ontario Summer School on High Performance Computing, whose second installment of the year will be hosted by SciNet, provides attendees with opportunities to learn and share knowledge and experience in high performance and technical computing. The format: a five-day workshop with mixed lectures and hands-on sessions on a number of selected subjects, including shared memory programming, distributed memory programming, parallel data processing, and general purpose graphics processing unit programming.

Below are descriptions and course material for the Central version, given from July 11th to July 15th, 2016, in Toronto.

The location of this version of the summer school is:

'''Wallberg Building''' 
'''Rooms 116 and 119''' 
University of Toronto 
St. George Campus 
184-200 College Street 
Toronto, Ontario, M5S 3E5 
Canada

More information can be found below and in the [[Media:handout2016.pdf|handout package]].

__TOC__

===Introduction to HPC and SciNet===

=====Abstract=====
This half-day session will provide an introduction to basic concepts of high-performance computing. It is intended to be a high level primer for those largely new to HPC, and serve as a foundation upon which to build over the coming days. Topics will include motivation for HPC, essential issues, problem characteristics as they apply to parallelism and a high level overview of parallel programming models. Strategies of running large sets of serial processes using e.g. GNU parallel, will also be presented.

=====Details=====
''Time/date''

Monday July 11, 2016, 09:30 - 12:30

''Location''

Wallberg Building, Room 116

''Instructor''

Dr. Mike Nolta (SciNet)

=====Prerequisites=====
Basic Linux skills are useful to follow along. If you do not have these, consider taking the "Intro to the Linux Shell" in the afternoon.

=====Slides and other material=====

* [https://support.scinet.utoronto.ca/~nolta/HPC_intro.pdf Slides]

----

===Programming Shared Memory Systems with OpenMP===

=====Abstract=====
In these two half-day sessions, lectures and hands-on labs are interspersed, and the students will learn the basics of shared memory programming with OpenMP. In particular, we will discuss the OpenMP's threads, memory, and performance, reductions and load balancing. We will also discuss extensions to heterogeneous systems such as offered by the OpenMP 4.0 and OpenACC standards.

=====Details=====

''Time/date''

Monday July 11, 2016, 13:30 - 16:30 
Tuesday July 12, 2016, 09:30 - 12:30

''Location''

Wallberg Building, Room 116

''Instructor''

Dr. Ramses van Zon (SciNet)

=====Prerequisites=====
C/C++ and/or Fortran scientific programming; Experience editing and compiling code in a Linux environment.

=====Slides and other material=====

* [[Media:OpenMP2016.pdf | Slides]]
* [[Media:OpenMP_SS2015_code.tar.gz|Code]]

----

===Introduction to the Linux Shell===

====Abstract====
Working with many of the HPC systems in Ontario involves using the Linux/UNIX command line. This provides a very powerful interface, but it can be quite daunting for the uninitiated. In this half-day session, you can become initiated with this course. This hands on session will cover basic commands and scripting, as well as touching on some powerful constructs like regular expressions. It could be a great boon for your productivity!

====Details====
''Time/date''

Monday July 11, 13:30 - 16:30

''Location''

Wallberg Building, Room 119

''Instructor''

Dr. Mike Nolta (SciNet)

====Prerequisites====
A laptop with a linux-like environment.

====Slides and other material====

* [[Media:intro_to_the_unix_shell.pdf | Slides ]]

----

===R for Data Science===

=====Abstract=====
This session offers a brief introduction to R, with a focus on data analysis and statistics.

=====Details=====

''Date/time''

Tuesday July 12, 2016, 09:30 - 12:30

''Location''

Wallberg Building, Room 119 

''Instructor''

Dr. Erik Spence (SciNet)

=====Prerequisites=====

Some programming experience in any language. You should have used and written functions. Bring a laptop with R or Rstudio installed.

=====Slides and other material=====

* [https://wiki.scinet.utoronto.ca/wiki/images/b/b1/Intro_R.pdf Slides]

----

===Programming Clusters with MPI===

=====Abstract=====
In this 1.5-day session, through lectures interspersed with hands-on labs, the students will learn the basics of MPI programming. Examples and exercises will be based on parallelization of common scientific computing problems.

=====Details=====
''Time/date''

Tuesday July 12, 2016, 13:30 - 16:30 
Wednesday July 13, 2016, 09:30 - 12:30 
Wednesday July 13, 2016, 13:30 - 16:30 

''Location''

Wallberg Building, Room 116

''Instructor''

Dr. Scott Northrup (SciNet)

=====Prerequisites=====
C and/or Fortran scientific programming; experience editing and compiling code in a Linux environment.

=====Slides and other material=====
* [https://support.scinet.utoronto.ca/~northrup/SS2016/MPIss2016.pdf Slides]
----

===Parallel R for Data Science===

=====Abstract=====
This half-day session will cover parallel programming in R, with a focus on parallel data analysis. Topics covered will include snow, parallel, and foreach/doparallel.

=====Details=====

''Time/date''

Tuesday July 12, 2016, 13:30 - 16:30

''Location''

Wallberg Building, Room 119

''Instructor''

Dr. Erik Spence (SciNet)

=====Prerequisites=====
Experience with programming in R and a rudimentary understanding of parallel computing.

=====Slides and other material=====

----

===Python for Scientific Computing===

=====Abstract=====
In this half-day session, you will be taught how to use python for research computing purposes. Topics include: the basics of python, automation, numpy, scipy, file i/o, and visualization.

=====Details=====
''Time/date'''

Wednesday July 13, 9:30 - 12:30

''Location''

Wallberg Building, Room 119

''Instructor''

Dr. Erik Spence (SciNet)

=====Prerequisites=====
A bit of coding experience in any programming or scripting language. Command-line experience in Linux is a plus.

=====Slides and other material=====

----

===Python for High Performance Computing===

=====Abstract=====
This half-day session will cover parallel programming in python, with a focus on parallel data analysis. We will cover subprocess, multiprocessing, pypar and other parallel-enabling python packages.

=====Details=====
''Time/date''

Wednesday July 13, 2016, 13:30 - 16:30

''Location''

Wallberg Building, Room 119

''Instructor''

Dr. Ramses van Zon (SciNet)

=====Prerequisites=====
A bit of coding experience in any programming or scripting language. Command-line experience in Linux is a plus.

=====Slides and other material=====

----

===Programming GPUs with CUDA===

=====Abstract=====
This two-day session is an introductory course covering programming and computing on GPUs - graphics processing units - which are an increasingly common presence in massively parallel computing architectures. This session will cover both of the available C-like programming frameworks: NVIDIA’s CUDA C. The basics of GPU programming will be covered, and students will work through a number of hands on examples. Demonstrations of profiling and debugging applications running on the GPU will also be included. The structuring of data and computations that makes full use of the GPU will be discussed in detail. Students should be able to leave the course with the knowledge necessary to begin developing their own GPU applications.

=====Details=====
''Time/date''

Thursday July 14, 2016, 09:30 - 12:30 
Thursday July 14, 2016, 13:30 - 16:30 
Friday July 15, 2016, 09:30 - 12:30 
Friday July 15, 2016, 13:30 - 16:30

''Location''

Wallberg Building, Room 116

''Instructors''
Dr. Pawel Pomorski and Dr. Sergey Mashchenko (SHARCNET)

=====Prerequisites=====
C/C++ scientific programming, experience editing and compiling code in a Linux environment. Some experience with CUDA and/or OpenMP a plus.

=====Slides and other materials=====

----

===Visualization===

=====Abstract=====
The first part of this whole-day session will be a basic introduction to general-purpose scientific visualization tools such as gnuplot, grace(xmgr), remote tools (tunneling, VNC), python's matplotlib, and examples from other. The afternoon part will be a mix of lecture and hands-on to introduce more specialized scientific visualization software such as vmd, paraview, and visit.

=====Details=====
''Time/date''

Thursday July 14, 2016, 09:30 - 12:30 
Thursday July 14, 2016, 13:30 - 16:30

''Location''

Wallberg Building, Room 119

''Instructor''
Dr. Marcelo Ponce (SciNet)

=====Prerequisites=====
Software installed, for the morning session: gnuplot, xmgr ([http://plasma-gate.weizmann.ac.il/Grace/ Grace],[http://plasma-gate.weizmann.ac.il/Xmgr/ xmgr]), ssh (graphic client), your favorite vnc client (egs. [https://www.realvnc.com/download/viewer/ RealVNC],[http://www.tightvnc.com/download.php TightVNC]), python and matplotlib ([http://matplotlib.org/ matplotlib]); afternoon session: ParaView ([http://www.paraview.org/download/ ParaView]) and VisIt ([https://wci.llnl.gov/simulation/computer-codes/visit/executables VisIt]).

=====Slides and other material=====
======Slides======
======Datasets======
[ Morning datasets and scripts]

[https://support.scinet.utoronto.ca/~mponce/ss2016/datasets.tar.gz Afternoon dataset]

[https://support.scinet.utoronto.ca/~mponce/ss2016/aneurysm_data.tar.gz Aneurysm dataset]
----

===Debugging===

=====Abstract=====
Debugging is an important step in developing a new code, or porting an old one to a new machine. In this half-day session, we will discuss the debugging of frequently encountered bugs in serial code with gdb and the debugging of parallel (MPI and threaded) codes with DDT on live systems.

=====Details=====
''Date/time''

Friday July 15, 09:30 - 12:30

''Location''

Wallberg Building, Room 119

Note the different room number!

''Instructors''

Dr. Marcelo Ponce (SciNet)

=====Prerequisites=====
Basic familiarity with a high-level language (C/C++/Fortran) as well as compiling and running programs via the command line environment on a HPC system.

=====Slides and other material=====

----

===Bring Your Own Code===

=====Abstract=====
The final afternoon of the summer school is your chance to get help with your own code from the SciNet software analysts. Bring your code, explain your problem or concerns, and we'll give you advice and work with you to try to improve it.

=====Details=====
''Date/time''

Friday July 15, 2016, 13:30 - 16:30

''Location''

Wallberg Building, Room 119

''Instructors''

Various SciNet software analysts.

=====Prerequisites=====
None.

2016 Ontario Summer School for High Performance Computing Central

2016-07-12T12:39:00Z

Ejspence:

===About===

The 2016 Ontario Summer School on High Performance Computing, whose second installment of the year will be hosted by SciNet, provides attendees with opportunities to learn and share knowledge and experience in high performance and technical computing. The format: a five-day workshop with mixed lectures and hands-on sessions on a number of selected subjects, including shared memory programming, distributed memory programming, parallel data processing, and general purpose graphics processing unit programming.

Below are descriptions and course material for the Central version, given from July 11th to July 15th, 2016, in Toronto.

The location of this version of the summer school is:

'''Wallberg Building''' 
'''Rooms 116 and 119''' 
University of Toronto 
St. George Campus 
184-200 College Street 
Toronto, Ontario, M5S 3E5 
Canada

More information can be found below and in the [[Media:handout2016.pdf|handout package]].

__TOC__

===Introduction to HPC and SciNet===

=====Abstract=====
This half-day session will provide an introduction to basic concepts of high-performance computing. It is intended to be a high level primer for those largely new to HPC, and serve as a foundation upon which to build over the coming days. Topics will include motivation for HPC, essential issues, problem characteristics as they apply to parallelism and a high level overview of parallel programming models. Strategies of running large sets of serial processes using e.g. GNU parallel, will also be presented.

=====Details=====
''Time/date''

Monday July 11, 2016, 09:30 - 12:30

''Location''

Wallberg Building, Room 116

''Instructor''

Dr. Mike Nolta (SciNet)

=====Prerequisites=====
Basic Linux skills are useful to follow along. If you do not have these, consider taking the "Intro to the Linux Shell" in the afternoon.

=====Slides and other material=====

* [https://support.scinet.utoronto.ca/~nolta/HPC_intro.pdf Slides]

----

===Programming Shared Memory Systems with OpenMP===

=====Abstract=====
In these two half-day sessions, lectures and hands-on labs are interspersed, and the students will learn the basics of shared memory programming with OpenMP. In particular, we will discuss the OpenMP's threads, memory, and performance, reductions and load balancing. We will also discuss extensions to heterogeneous systems such as offered by the OpenMP 4.0 and OpenACC standards.

=====Details=====

''Time/date''

Monday July 11, 2016, 13:30 - 16:30 
Tuesday July 12, 2016, 09:30 - 12:30

''Location''

Wallberg Building, Room 116

''Instructor''

Dr. Ramses van Zon (SciNet)

=====Prerequisites=====
C/C++ and/or Fortran scientific programming; Experience editing and compiling code in a Linux environment.

=====Slides and other material=====

* [[Media:OpenMP2016.pdf | Slides]]
* [[Media:OpenMP_SS2015_code.tar.gz|Code]]

----

===Introduction to the Linux Shell===

====Abstract====
Working with many of the HPC systems in Ontario involves using the Linux/UNIX command line. This provides a very powerful interface, but it can be quite daunting for the uninitiated. In this half-day session, you can become initiated with this course. This hands on session will cover basic commands and scripting, as well as touching on some powerful constructs like regular expressions. It could be a great boon for your productivity!

====Details====
''Time/date''

Monday July 11, 13:30 - 16:30

''Location''

Wallberg Building, Room 119

''Instructor''

Dr. Mike Nolta (SciNet)

====Prerequisites====
A laptop with a linux-like environment.

====Slides and other material====

* [[Media:intro_to_the_unix_shell.pdf | Slides ]]

----

===R for Data Science===

=====Abstract=====
This session offers a brief introduction to R, with a focus on data analysis and statistics.

=====Details=====

''Date/time''

Tuesday July 12, 2016, 09:30 - 12:30

''Location''

Wallberg Building, Room 119 

''Instructor''

Dr. Erik Spence (SciNet)

=====Prerequisites=====

Some programming experience in any language. You should have used and written functions. Bring a laptop with R or Rstudio installed.

=====Slides and other material=====

* [https://wiki.scinet.utoronto.ca/wiki/index.php/File:Intro_R.pdf Slides]

----

===Programming Clusters with MPI===

=====Abstract=====
In this 1.5-day session, through lectures interspersed with hands-on labs, the students will learn the basics of MPI programming. Examples and exercises will be based on parallelization of common scientific computing problems.

=====Details=====
''Time/date''

Tuesday July 12, 2016, 13:30 - 16:30 
Wednesday July 13, 2016, 09:30 - 12:30 
Wednesday July 13, 2016, 13:30 - 16:30 

''Location''

Wallberg Building, Room 116

''Instructor''

Dr. Scott Northrup (SciNet)

=====Prerequisites=====
C and/or Fortran scientific programming; experience editing and compiling code in a Linux environment.

=====Slides and other material=====
* [https://support.scinet.utoronto.ca/~northrup/SS2016/MPIss2016.pdf Slides]
----

===Parallel R for Data Science===

=====Abstract=====
This half-day session will cover parallel programming in R, with a focus on parallel data analysis. Topics covered will include snow, parallel, and foreach/doparallel.

=====Details=====

''Time/date''

Tuesday July 12, 2016, 13:30 - 16:30

''Location''

Wallberg Building, Room 119

''Instructor''

Dr. Erik Spence (SciNet)

=====Prerequisites=====
Experience with programming in R and a rudimentary understanding of parallel computing.

=====Slides and other material=====

----

===Python for Scientific Computing===

=====Abstract=====
In this half-day session, you will be taught how to use python for research computing purposes. Topics include: the basics of python, automation, numpy, scipy, file i/o, and visualization.

=====Details=====
''Time/date'''

Wednesday July 13, 9:30 - 12:30

''Location''

Wallberg Building, Room 119

''Instructor''

Dr. Erik Spence (SciNet)

=====Prerequisites=====
A bit of coding experience in any programming or scripting language. Command-line experience in Linux is a plus.

=====Slides and other material=====

----

===Python for High Performance Computing===

=====Abstract=====
This half-day session will cover parallel programming in python, with a focus on parallel data analysis. We will cover subprocess, multiprocessing, pypar and other parallel-enabling python packages.

=====Details=====
''Time/date''

Wednesday July 13, 2016, 13:30 - 16:30

''Location''

Wallberg Building, Room 119

''Instructor''

Dr. Ramses van Zon (SciNet)

=====Prerequisites=====
A bit of coding experience in any programming or scripting language. Command-line experience in Linux is a plus.

=====Slides and other material=====

----

===Programming GPUs with CUDA===

=====Abstract=====
This two-day session is an introductory course covering programming and computing on GPUs - graphics processing units - which are an increasingly common presence in massively parallel computing architectures. This session will cover both of the available C-like programming frameworks: NVIDIA’s CUDA C. The basics of GPU programming will be covered, and students will work through a number of hands on examples. Demonstrations of profiling and debugging applications running on the GPU will also be included. The structuring of data and computations that makes full use of the GPU will be discussed in detail. Students should be able to leave the course with the knowledge necessary to begin developing their own GPU applications.

=====Details=====
''Time/date''

Thursday July 14, 2016, 09:30 - 12:30 
Thursday July 14, 2016, 13:30 - 16:30 
Friday July 15, 2016, 09:30 - 12:30 
Friday July 15, 2016, 13:30 - 16:30

''Location''

Wallberg Building, Room 116

''Instructors''
Dr. Pawel Pomorski and Dr. Sergey Mashchenko (SHARCNET)

=====Prerequisites=====
C/C++ scientific programming, experience editing and compiling code in a Linux environment. Some experience with CUDA and/or OpenMP a plus.

=====Slides and other materials=====

----

===Visualization===

=====Abstract=====
The first part of this whole-day session will be a basic introduction to general-purpose scientific visualization tools such as gnuplot, grace(xmgr), remote tools (tunneling, VNC), python's matplotlib, and examples from other. The afternoon part will be a mix of lecture and hands-on to introduce more specialized scientific visualization software such as vmd, paraview, and visit.

=====Details=====
''Time/date''

Thursday July 14, 2016, 09:30 - 12:30 
Thursday July 14, 2016, 13:30 - 16:30

''Location''

Wallberg Building, Room 119

''Instructor''
Dr. Marcelo Ponce (SciNet)

=====Prerequisites=====
Software installed, for the morning session: gnuplot, xmgr ([http://plasma-gate.weizmann.ac.il/Grace/ Grace],[http://plasma-gate.weizmann.ac.il/Xmgr/ xmgr]), ssh (graphic client), your favorite vnc client (egs. [https://www.realvnc.com/download/viewer/ RealVNC],[http://www.tightvnc.com/download.php TightVNC]), python and matplotlib ([http://matplotlib.org/ matplotlib]); afternoon session: ParaView ([http://www.paraview.org/download/ ParaView]) and VisIt ([https://wci.llnl.gov/simulation/computer-codes/visit/executables VisIt]).

=====Slides and other material=====
======Slides======
======Datasets======
[ Morning datasets and scripts]

[https://support.scinet.utoronto.ca/~mponce/ss2016/datasets.tar.gz Afternoon dataset]

[https://support.scinet.utoronto.ca/~mponce/ss2016/aneurysm_data.tar.gz Aneurysm dataset]
----

===Debugging===

=====Abstract=====
Debugging is an important step in developing a new code, or porting an old one to a new machine. In this half-day session, we will discuss the debugging of frequently encountered bugs in serial code with gdb and the debugging of parallel (MPI and threaded) codes with DDT on live systems.

=====Details=====
''Date/time''

Friday July 15, 09:30 - 12:30

''Location''

Wallberg Building, Room 119

Note the different room number!

''Instructors''

Dr. Marcelo Ponce (SciNet)

=====Prerequisites=====
Basic familiarity with a high-level language (C/C++/Fortran) as well as compiling and running programs via the command line environment on a HPC system.

=====Slides and other material=====

----

===Bring Your Own Code===

=====Abstract=====
The final afternoon of the summer school is your chance to get help with your own code from the SciNet software analysts. Bring your code, explain your problem or concerns, and we'll give you advice and work with you to try to improve it.

=====Details=====
''Date/time''

Friday July 15, 2016, 13:30 - 16:30

''Location''

Wallberg Building, Room 119

''Instructors''

Various SciNet software analysts.

=====Prerequisites=====
None.

File:Intro R.pdf

2016-07-12T12:38:15Z

Ejspence: Slides for the "R for Data Science" class in the 2016 Summer School.

Slides for the "R for Data Science" class in the 2016 Summer School.

2016 Ontario Summer School for High Performance Computing Central

2016-07-06T15:29:20Z

Ejspence: /* Programming Shared Memory Systems with OpenMP */

===About===

The 2016 Ontario Summer School on High Performance Computing, whose second installment of the year will be hosted by SciNet, provides attendees with opportunities to learn and share knowledge and experience in high performance and technical computing. The format: a five-day workshop with mixed lectures and hands-on sessions on a number of selected subjects, including shared memory programming, distributed memory programming, parallel data processing, and general purpose graphics processing unit programming.

Below are descriptions and course material for the Central version, given from July 11th to July 15th, 2016, in Toronto.

__TOC__

===Introduction to HPC and SciNet===

=====Abstract=====
This half-day session will provide an introduction to basic concepts of high-performance computing. It is intended to be a high level primer for those largely new to HPC, and serve as a foundation upon which to build over the coming days. Topics will include motivation for HPC, essential issues, problem characteristics as they apply to parallelism and a high level overview of parallel programming models. Strategies of running large sets of serial processes using e.g. GNU parallel, will also be presented.

=====Details=====
''Time/date''

Monday July 11, 2016, 09:30 - 12:30

''Location''

MC 252 
5 King's College Road 
Toronto, ON M5S 3G8

''Instructor''

Dr. Mike Nolta (SciNet)

=====Prerequisites=====
Basic Linux skills are useful to follow along. If you do not have these, consider taking the "Intro to the Linux Shell" in the afternoon.

=====Slides and other material=====

----

===Programming Shared Memory Systems with OpenMP===

=====Abstract=====
In these two half-day sessions, lectures and hands-on labs are interspersed, and the students will learn the basics of shared memory programming with OpenMP. In particular, we will discuss the OpenMP's threads, memory, and performance, reductions and load balancing. We will also discuss extensions to heterogeneous systems such as offered by the OpenMP 4.0 and OpenACC standards.

=====Details=====

''Time/date''

Monday July 11, 2016, 13:30 - 16:30 
Tuesday July 12, 2016, 09:30 - 12:30

''Location''

MC 252 
5 King's College Road 
Toronto, ON M5S 3G8

''Instructor''

Dr. Ramses van Zon (SciNet)

=====Prerequisites=====
C/C++ and/or Fortran scientific programming; Experience editing and compiling code in a Linux environment.

=====Slides and other material=====

----

===Introduction to the Linux Shell===

====Abstract====
Working with many of the HPC systems in Ontario involves using the Linux/UNIX command line. This provides a very powerful interface, but it can be quite daunting for the uninitiated. In this half-day session, you can become initiated with this course. This hands on session will cover basic commands and scripting, as well as touching on some powerful constructs like regular expressions. It could be a great boon for your productivity!

====Details====
''Time/date''

Monday July 11, 13:30 - 16:30

''Location''

MC 254 
5 King's College Road 
Toronto, ON M5S 3G8

''Instructor''

Dr. Mike Nolta (SciNet)

====Prerequisites====
A laptop with a linux-like environment.

====Slides and other material====

----

===R for Data Science===

=====Abstract=====
This session offers a brief introduction to R, with a focus on data analysis and statistics.

=====Details=====

''Date/time''

Tuesday July 12, 2016, 09:30 - 12:30

''Location''

MC 254 
5 King's College Road 
Toronto, ON M5S 3G8

''Instructor''

Dr. Erik Spence (SciNet)

=====Prerequisites=====

Some programming experience in any language. You should have used and written functions. Bring a laptop with R or Rstudio installed.

=====Slides and other material=====

----

===Programming Clusters with MPI===

=====Abstract=====
In this 1.5-day session, through lectures interspersed with hands-on labs, the students will learn the basics of MPI programming. Examples and exercises will be based on parallelization of common scientific computing problems.

=====Details=====
''Time/date''

Tuesday July 12, 2016, 13:30 - 16:30 
Wednesday July 13, 2016, 09:30 - 12:30 
Wednesday July 13, 2016, 13:30 - 16:30 

''Location''

MC 252 
5 King's College Road 
Toronto, ON M5S 3G8

''Instructor''

Dr. Scott Northrup (SciNet)

=====Prerequisites=====
C and/or Fortran scientific programming; experience editing and compiling code in a Linux environment.

=====Slides and other material=====

----

===Parallel R for Data Science===

=====Abstract=====
This half-day session will cover parallel programming in R, with a focus on parallel data analysis. Topics covered will include snow, parallel, and foreach/doparallel.

=====Details=====

''Time/date''

Tuesday July 12, 2016, 13:30 - 16:30

''Location''

MC 254 
5 King's College Road 
Toronto, ON M5S 3G8

''Instructor''

Dr. Erik Spence (SciNet)

=====Prerequisites=====
Experience with programming in R and a rudimentary understanding of parallel computing.

=====Slides and other material=====

----

===Python for Scientific Computing===

=====Abstract=====
In this half-day session, you will be taught how to use python for research computing purposes. Topics include: the basics of python, automation, numpy, scipy, file i/o, and visualization.

=====Details=====
''Time/date'''

Wednesday July 13, 9:30 - 12:30

''Location''

MC 254 
5 King's College Road 
Toronto, ON M5S 3G8

''Instructor''

Dr. Erik Spence (SciNet)

=====Prerequisites=====
A bit of coding experience in any programming or scripting language. Command-line experience in Linux is a plus.

=====Slides and other material=====

----

===Python for High Performance Computing===

=====Abstract=====
This half-day session will cover parallel programming in python, with a focus on parallel data analysis. We will cover subprocess, multiprocessing, pypar and other parallel-enabling python packages.

=====Details=====
''Time/date''

Wednesday July 13, 2016, 13:30 - 16:30

''Location''

MC 254 
5 King's College Road 
Toronto, ON M5S 3G8

''Instructor''

Dr. Ramses von Zan (SciNet)

=====Prerequisites=====
A bit of coding experience in any programming or scripting language. Command-line experience in Linux is a plus.

=====Slides and other material=====

----

===Programming GPUs with CUDA===

=====Abstract=====
This two-day session is an introductory course covering programming and computing on GPUs - graphics processing units - which are an increasingly common presence in massively parallel computing architectures. This session will cover both of the available C-like programming frameworks: NVIDIA’s CUDA C. The basics of GPU programming will be covered, and students will work through a number of hands on examples. Demonstrations of profiling and debugging applications running on the GPU will also be included. The structuring of data and computations that makes full use of the GPU will be discussed in detail. Students should be able to leave the course with the knowledge necessary to begin developing their own GPU applications.

=====Details=====
''Time/date''

Thursday July 14, 2016, 09:30 - 12:30 
Thursday July 14, 2016, 13:30 - 16:30 
Friday July 15, 2016, 09:30 - 12:30 
Friday July 15, 2016, 13:30 - 16:30

''Location''

MC 252 
5 King's College Road 
Toronto, ON M5S 3G8

''Instructors''
Dr. Pawel Pomorski and Dr. Sergey Mashchenko (SHARCNET)

=====Prerequisites=====
C/C++ scientific programming, experience editing and compiling code in a Linux environment. Some experience with CUDA and/or OpenMP a plus.

=====Slides and other materials=====

----

===Visualization===

=====Abstract=====
The first part of this whole-day session will be a basic introduction to general-purpose scientific visualization tools such as gnuplot, grace(xmgr), remote tools (tunneling, VNC), python's matplotlib, and examples from other. The afternoon part will be a mix of lecture and hands-on to introduce more specialized scientific visualization software such as vmd, paraview, and visit.

=====Details=====
''Time/date''

Thursday July 14, 2016, 09:30 - 12:30 
Thursday July 14, 2016, 13:30 - 16:30

''Location''

MC 254 
5 King's College Road 
Toronto, ON M5S 3G8

''Instructor''
Dr. Marcelo Ponce (SciNet)

=====Prerequisites=====
Software installed, for the morning session: gnuplot, ssh (graphic client), vnc client, python and matplotlib; afternoon session: paraview.

=====Slides and other material=====

----

===Debugging===

=====Abstract=====
Debugging is an important step in developing a new code, or porting an old one to a new machine. In this half-day session, we will discuss the debugging of frequently encountered bugs in serial code with gdb and the debugging of parallel (MPI and threaded) codes with DDT on live systems.

=====Details=====
''Date/time''

Friday July 15, 09:30 - 12:30

''Location''

MC 254 
5 King's College Road 
Toronto, ON M5S 3G8

''Instructors''

Dr. Marcelo Ponce (SciNet)

=====Prerequisites=====
Basic familiarity with a high-level language (C/C++/Fortran) as well as compiling and running programs via the command line environment on a HPC system.

=====Slides and other material=====

----

===Bring Your Own Code===

=====Abstract=====
The final afternoon of the summer school is your chance to get help with your own code from the SciNet software analysts. Bring your code, explain your problem or concerns, and we'll give you advice and work with you to try to improve it.

=====Details=====
''Date/time''

Friday July 15, 2016, 13:30 - 16:30

''Location''

MC 254 
5 King's College Road 
Toronto, ON M5S 3G8

''Instructors''

Various SciNet software analysts.

=====Prerequisites=====
None.

2016 Ontario Summer School for High Performance Computing Central

2016-07-06T15:29:08Z

Ejspence: /* Introduction to the Linux Shell */

===About===

The 2016 Ontario Summer School on High Performance Computing, whose second installment of the year will be hosted by SciNet, provides attendees with opportunities to learn and share knowledge and experience in high performance and technical computing. The format: a five-day workshop with mixed lectures and hands-on sessions on a number of selected subjects, including shared memory programming, distributed memory programming, parallel data processing, and general purpose graphics processing unit programming.

Below are descriptions and course material for the Central version, given from July 11th to July 15th, 2016, in Toronto.

__TOC__

===Introduction to HPC and SciNet===

=====Abstract=====
This half-day session will provide an introduction to basic concepts of high-performance computing. It is intended to be a high level primer for those largely new to HPC, and serve as a foundation upon which to build over the coming days. Topics will include motivation for HPC, essential issues, problem characteristics as they apply to parallelism and a high level overview of parallel programming models. Strategies of running large sets of serial processes using e.g. GNU parallel, will also be presented.

=====Details=====
''Time/date''

Monday July 11, 2016, 09:30 - 12:30

''Location''

MC 252 
5 King's College Road 
Toronto, ON M5S 3G8

''Instructor''

Dr. Mike Nolta (SciNet)

=====Prerequisites=====
Basic Linux skills are useful to follow along. If you do not have these, consider taking the "Intro to the Linux Shell" in the afternoon.

=====Slides and other material=====

----

===Programming Shared Memory Systems with OpenMP===

=====Abstract=====
In these two half-day sessions, lectures and hands-on labs are interspersed, and the students will learn the basics of shared memory programming with OpenMP. In particular, we will discuss the OpenMP's threads, memory, and performance, reductions and load balancing. We will also discuss extensions to heterogeneous systems such as offered by the OpenMP 4.0 and OpenACC standards.

=====Details=====

''Time/date''

Monday July 11, 2016, 13:30 - 16:30 
Tuesday July 12, 2016, 09:30 - 12:30

''Location''

MC 252 
5 King's College Road 
Toronto, ON M5S 3G8

''Instructor''

Dr. Ramses van Zon (SciNet)

=====Prerequisites=====
C/C++ and/or Fortran scientific programming; Experience editing and compiling code in a Linux environment.

=====Slides and other material=====

----

===Introduction to the Linux Shell===

====Abstract====
Working with many of the HPC systems in Ontario involves using the Linux/UNIX command line. This provides a very powerful interface, but it can be quite daunting for the uninitiated. In this half-day session, you can become initiated with this course. This hands on session will cover basic commands and scripting, as well as touching on some powerful constructs like regular expressions. It could be a great boon for your productivity!

====Details====
''Time/date''

Monday July 11, 13:30 - 16:30

''Location''

MC 254 
5 King's College Road 
Toronto, ON M5S 3G8

''Instructor''

Dr. Mike Nolta (SciNet)

====Prerequisites====
A laptop with a linux-like environment.

====Slides and other material====

----

===R for Data Science===

=====Abstract=====
This session offers a brief introduction to R, with a focus on data analysis and statistics.

=====Details=====

''Date/time''

Tuesday July 12, 2016, 09:30 - 12:30

''Location''

MC 254 
5 King's College Road 
Toronto, ON M5S 3G8

''Instructor''

Dr. Erik Spence (SciNet)

=====Prerequisites=====

Some programming experience in any language. You should have used and written functions. Bring a laptop with R or Rstudio installed.

=====Slides and other material=====

----

===Programming Clusters with MPI===

=====Abstract=====
In this 1.5-day session, through lectures interspersed with hands-on labs, the students will learn the basics of MPI programming. Examples and exercises will be based on parallelization of common scientific computing problems.

=====Details=====
''Time/date''

Tuesday July 12, 2016, 13:30 - 16:30 
Wednesday July 13, 2016, 09:30 - 12:30 
Wednesday July 13, 2016, 13:30 - 16:30 

''Location''

MC 252 
5 King's College Road 
Toronto, ON M5S 3G8

''Instructor''

Dr. Scott Northrup (SciNet)

=====Prerequisites=====
C and/or Fortran scientific programming; experience editing and compiling code in a Linux environment.

=====Slides and other material=====

----

===Parallel R for Data Science===

=====Abstract=====
This half-day session will cover parallel programming in R, with a focus on parallel data analysis. Topics covered will include snow, parallel, and foreach/doparallel.

=====Details=====

''Time/date''

Tuesday July 12, 2016, 13:30 - 16:30

''Location''

MC 254 
5 King's College Road 
Toronto, ON M5S 3G8

''Instructor''

Dr. Erik Spence (SciNet)

=====Prerequisites=====
Experience with programming in R and a rudimentary understanding of parallel computing.

=====Slides and other material=====

----

===Python for Scientific Computing===

=====Abstract=====
In this half-day session, you will be taught how to use python for research computing purposes. Topics include: the basics of python, automation, numpy, scipy, file i/o, and visualization.

=====Details=====
''Time/date'''

Wednesday July 13, 9:30 - 12:30

''Location''

MC 254 
5 King's College Road 
Toronto, ON M5S 3G8

''Instructor''

Dr. Erik Spence (SciNet)

=====Prerequisites=====
A bit of coding experience in any programming or scripting language. Command-line experience in Linux is a plus.

=====Slides and other material=====

----

===Python for High Performance Computing===

=====Abstract=====
This half-day session will cover parallel programming in python, with a focus on parallel data analysis. We will cover subprocess, multiprocessing, pypar and other parallel-enabling python packages.

=====Details=====
''Time/date''

Wednesday July 13, 2016, 13:30 - 16:30

''Location''

MC 254 
5 King's College Road 
Toronto, ON M5S 3G8

''Instructor''

Dr. Ramses von Zan (SciNet)

=====Prerequisites=====
A bit of coding experience in any programming or scripting language. Command-line experience in Linux is a plus.

=====Slides and other material=====

----

===Programming GPUs with CUDA===

=====Abstract=====
This two-day session is an introductory course covering programming and computing on GPUs - graphics processing units - which are an increasingly common presence in massively parallel computing architectures. This session will cover both of the available C-like programming frameworks: NVIDIA’s CUDA C. The basics of GPU programming will be covered, and students will work through a number of hands on examples. Demonstrations of profiling and debugging applications running on the GPU will also be included. The structuring of data and computations that makes full use of the GPU will be discussed in detail. Students should be able to leave the course with the knowledge necessary to begin developing their own GPU applications.

=====Details=====
''Time/date''

Thursday July 14, 2016, 09:30 - 12:30 
Thursday July 14, 2016, 13:30 - 16:30 
Friday July 15, 2016, 09:30 - 12:30 
Friday July 15, 2016, 13:30 - 16:30

''Location''

MC 252 
5 King's College Road 
Toronto, ON M5S 3G8

''Instructors''
Dr. Pawel Pomorski and Dr. Sergey Mashchenko (SHARCNET)

=====Prerequisites=====
C/C++ scientific programming, experience editing and compiling code in a Linux environment. Some experience with CUDA and/or OpenMP a plus.

=====Slides and other materials=====

----

===Visualization===

=====Abstract=====
The first part of this whole-day session will be a basic introduction to general-purpose scientific visualization tools such as gnuplot, grace(xmgr), remote tools (tunneling, VNC), python's matplotlib, and examples from other. The afternoon part will be a mix of lecture and hands-on to introduce more specialized scientific visualization software such as vmd, paraview, and visit.

=====Details=====
''Time/date''

Thursday July 14, 2016, 09:30 - 12:30 
Thursday July 14, 2016, 13:30 - 16:30

''Location''

MC 254 
5 King's College Road 
Toronto, ON M5S 3G8

''Instructor''
Dr. Marcelo Ponce (SciNet)

=====Prerequisites=====
Software installed, for the morning session: gnuplot, ssh (graphic client), vnc client, python and matplotlib; afternoon session: paraview.

=====Slides and other material=====

----

===Debugging===

=====Abstract=====
Debugging is an important step in developing a new code, or porting an old one to a new machine. In this half-day session, we will discuss the debugging of frequently encountered bugs in serial code with gdb and the debugging of parallel (MPI and threaded) codes with DDT on live systems.

=====Details=====
''Date/time''

Friday July 15, 09:30 - 12:30

''Location''

MC 254 
5 King's College Road 
Toronto, ON M5S 3G8

''Instructors''

Dr. Marcelo Ponce (SciNet)

=====Prerequisites=====
Basic familiarity with a high-level language (C/C++/Fortran) as well as compiling and running programs via the command line environment on a HPC system.

=====Slides and other material=====

----

===Bring Your Own Code===

=====Abstract=====
The final afternoon of the summer school is your chance to get help with your own code from the SciNet software analysts. Bring your code, explain your problem or concerns, and we'll give you advice and work with you to try to improve it.

=====Details=====
''Date/time''

Friday July 15, 2016, 13:30 - 16:30

''Location''

MC 254 
5 King's College Road 
Toronto, ON M5S 3G8

''Instructors''

Various SciNet software analysts.

=====Prerequisites=====
None.

2016 Ontario Summer School for High Performance Computing Central

2016-07-06T15:27:14Z

Ejspence: /* Details */

===About===

The 2016 Ontario Summer School on High Performance Computing, whose second installment of the year will be hosted by SciNet, provides attendees with opportunities to learn and share knowledge and experience in high performance and technical computing. The format: a five-day workshop with mixed lectures and hands-on sessions on a number of selected subjects, including shared memory programming, distributed memory programming, parallel data processing, and general purpose graphics processing unit programming.

Below are descriptions and course material for the Central version, given from July 11th to July 15th, 2016, in Toronto.

__TOC__

===Introduction to HPC and SciNet===

=====Abstract=====
This half-day session will provide an introduction to basic concepts of high-performance computing. It is intended to be a high level primer for those largely new to HPC, and serve as a foundation upon which to build over the coming days. Topics will include motivation for HPC, essential issues, problem characteristics as they apply to parallelism and a high level overview of parallel programming models. Strategies of running large sets of serial processes using e.g. GNU parallel, will also be presented.

=====Details=====
''Time/date''

Monday July 11, 2016, 09:30 - 12:30

''Location''

MC 252 
5 King's College Road 
Toronto, ON M5S 3G8

''Instructor''

Dr. Mike Nolta (SciNet)

=====Prerequisites=====
Basic Linux skills are useful to follow along. If you do not have these, consider taking the "Intro to the Linux Shell" in the afternoon.

=====Slides and other material=====

----

===Programming Shared Memory Systems with OpenMP===

=====Abstract=====
In these two half-day sessions, lectures and hands-on labs are interspersed, and the students will learn the basics of shared memory programming with OpenMP. In particular, we will discuss the OpenMP's threads, memory, and performance, reductions and load balancing. We will also discuss extensions to heterogeneous systems such as offered by the OpenMP 4.0 and OpenACC standards.

=====Details=====

''Time/date''

Monday July 11, 2016, 13:30 - 16:30 
Tuesday July 12, 2016, 09:30 - 12:30

''Location''

MC 252 
5 King's College Road 
Toronto, ON M5S 3G8

''Instructor''

Dr. Ramses van Zon (SciNet)

=====Prerequisites=====
C/C++ and/or Fortran scientific programming; Experience editing and compiling code in a Linux environment.

=====Slides and other material=====

----

===Introduction to the Linux Shell===

====Abstract====
Working with many of the HPC systems in Ontario involves using the Linux/UNIX command line. This provides a very powerful interface, but it can be quite daunting for the uninitiated. In this half-day session, you can become initiated with this course. This hands on session will cover basic commands and scripting, as well as touching on some powerful constructs like regular expressions. It could be a great boon for your productivity!

====Details====
''Time/date''

Monday July 11, 13:30 - 16:30

''Location''

MC 254 
5 King's College Road 
Toronto, ON M5S 3G8

''Instructor''

Dr. Mike Nolta (SciNet)

====Prerequisites====
A laptop with a linux-like environment.

====Slides and other material====

----

===R for Data Science===

=====Abstract=====
This session offers a brief introduction to R, with a focus on data analysis and statistics.

=====Details=====

''Date/time''

Tuesday July 12, 2016, 09:30 - 12:30

''Location''

MC 254 
5 King's College Road 
Toronto, ON M5S 3G8

''Instructor''

Dr. Erik Spence (SciNet)

=====Prerequisites=====

Some programming experience in any language. You should have used and written functions. Bring a laptop with R or Rstudio installed.

=====Slides and other material=====

----

===Programming Clusters with MPI===

=====Abstract=====
In this 1.5-day session, through lectures interspersed with hands-on labs, the students will learn the basics of MPI programming. Examples and exercises will be based on parallelization of common scientific computing problems.

=====Details=====
''Time/date''

Tuesday July 12, 2016, 13:30 - 16:30 
Wednesday July 13, 2016, 09:30 - 12:30 
Wednesday July 13, 2016, 13:30 - 16:30 

''Location''

MC 252 
5 King's College Road 
Toronto, ON M5S 3G8

''Instructor''

Dr. Scott Northrup (SciNet)

=====Prerequisites=====
C and/or Fortran scientific programming; experience editing and compiling code in a Linux environment.

=====Slides and other material=====

----

===Parallel R for Data Science===

=====Abstract=====
This half-day session will cover parallel programming in R, with a focus on parallel data analysis. Topics covered will include snow, parallel, and foreach/doparallel.

=====Details=====

''Time/date''

Tuesday July 12, 2016, 13:30 - 16:30

''Location''

MC 254 
5 King's College Road 
Toronto, ON M5S 3G8

''Instructor''

Dr. Erik Spence (SciNet)

=====Prerequisites=====
Experience with programming in R and a rudimentary understanding of parallel computing.

=====Slides and other material=====

----

===Python for Scientific Computing===

=====Abstract=====
In this half-day session, you will be taught how to use python for research computing purposes. Topics include: the basics of python, automation, numpy, scipy, file i/o, and visualization.

=====Details=====
''Time/date'''

Wednesday July 13, 9:30 - 12:30

''Location''

MC 254 
5 King's College Road 
Toronto, ON M5S 3G8

''Instructor''

Dr. Erik Spence (SciNet)

=====Prerequisites=====
A bit of coding experience in any programming or scripting language. Command-line experience in Linux is a plus.

=====Slides and other material=====

----

===Python for High Performance Computing===

=====Abstract=====
This half-day session will cover parallel programming in python, with a focus on parallel data analysis. We will cover subprocess, multiprocessing, pypar and other parallel-enabling python packages.

=====Details=====
''Time/date''

Wednesday July 13, 2016, 13:30 - 16:30

''Location''

MC 254 
5 King's College Road 
Toronto, ON M5S 3G8

''Instructor''

Dr. Ramses von Zan (SciNet)

=====Prerequisites=====
A bit of coding experience in any programming or scripting language. Command-line experience in Linux is a plus.

=====Slides and other material=====

----

===Programming GPUs with CUDA===

=====Abstract=====
This two-day session is an introductory course covering programming and computing on GPUs - graphics processing units - which are an increasingly common presence in massively parallel computing architectures. This session will cover both of the available C-like programming frameworks: NVIDIA’s CUDA C. The basics of GPU programming will be covered, and students will work through a number of hands on examples. Demonstrations of profiling and debugging applications running on the GPU will also be included. The structuring of data and computations that makes full use of the GPU will be discussed in detail. Students should be able to leave the course with the knowledge necessary to begin developing their own GPU applications.

=====Details=====
''Time/date''

Thursday July 14, 2016, 09:30 - 12:30 
Thursday July 14, 2016, 13:30 - 16:30 
Friday July 15, 2016, 09:30 - 12:30 
Friday July 15, 2016, 13:30 - 16:30

''Location''

MC 252 
5 King's College Road 
Toronto, ON M5S 3G8

''Instructors''
Dr. Pawel Pomorski and Dr. Sergey Mashchenko (SHARCNET)

=====Prerequisites=====
C/C++ scientific programming, experience editing and compiling code in a Linux environment. Some experience with CUDA and/or OpenMP a plus.

=====Slides and other materials=====

----

===Visualization===

=====Abstract=====
The first part of this whole-day session will be a basic introduction to general-purpose scientific visualization tools such as gnuplot, grace(xmgr), remote tools (tunneling, VNC), python's matplotlib, and examples from other. The afternoon part will be a mix of lecture and hands-on to introduce more specialized scientific visualization software such as vmd, paraview, and visit.

=====Details=====
''Time/date''

Thursday July 14, 2016, 09:30 - 12:30 
Thursday July 14, 2016, 13:30 - 16:30

''Location''

MC 254 
5 King's College Road 
Toronto, ON M5S 3G8

''Instructor''
Dr. Marcelo Ponce (SciNet)

=====Prerequisites=====
Software installed, for the morning session: gnuplot, ssh (graphic client), vnc client, python and matplotlib; afternoon session: paraview.

=====Slides and other material=====

----

===Debugging===

=====Abstract=====
Debugging is an important step in developing a new code, or porting an old one to a new machine. In this half-day session, we will discuss the debugging of frequently encountered bugs in serial code with gdb and the debugging of parallel (MPI and threaded) codes with DDT on live systems.

=====Details=====
''Date/time''

Friday July 15, 09:30 - 12:30

''Location''

MC 254 
5 King's College Road 
Toronto, ON M5S 3G8

''Instructors''

Dr. Marcelo Ponce (SciNet)

=====Prerequisites=====
Basic familiarity with a high-level language (C/C++/Fortran) as well as compiling and running programs via the command line environment on a HPC system.

=====Slides and other material=====

----

===Bring Your Own Code===

=====Abstract=====
The final afternoon of the summer school is your chance to get help with your own code from the SciNet software analysts. Bring your code, explain your problem or concerns, and we'll give you advice and work with you to try to improve it.

=====Details=====
''Date/time''

Friday July 15, 2016, 13:30 - 16:30

''Location''

MC 254 
5 King's College Road 
Toronto, ON M5S 3G8

''Instructors''

Various SciNet software analysts.

=====Prerequisites=====
None.

2016 Ontario Summer School for High Performance Computing Central

2016-07-06T15:24:36Z

Ejspence: /* Visualization */

===About===

The 2016 Ontario Summer School on High Performance Computing, whose second installment of the year will be hosted by SciNet, provides attendees with opportunities to learn and share knowledge and experience in high performance and technical computing. The format: a five-day workshop with mixed lectures and hands-on sessions on a number of selected subjects, including shared memory programming, distributed memory programming, parallel data processing, and general purpose graphics processing unit programming.

Below are descriptions and course material for the Central version, given from July 11th to July 15th, 2016, in Toronto.

__TOC__

===Introduction to HPC and SciNet===

=====Abstract=====
This half-day session will provide an introduction to basic concepts of high-performance computing. It is intended to be a high level primer for those largely new to HPC, and serve as a foundation upon which to build over the coming days. Topics will include motivation for HPC, essential issues, problem characteristics as they apply to parallelism and a high level overview of parallel programming models. Strategies of running large sets of serial processes using e.g. GNU parallel, will also be presented.

=====Details=====
''Time/date''

Monday July 11, 2016, 09:30 - 12:30

''Location''

MC 252 
5 King's College Road 
Toronto, ON M5S 3G8

''Instructor''

Dr. Mike Nolta (SciNet)

=====Prerequisites=====
Basic Linux skills are useful to follow along. If you do not have these, consider taking the "Intro to the Linux Shell" in the afternoon.

=====Slides and other material=====

----

===Programming Shared Memory Systems with OpenMP===

=====Abstract=====
In these two half-day sessions, lectures and hands-on labs are interspersed, and the students will learn the basics of shared memory programming with OpenMP. In particular, we will discuss the OpenMP's threads, memory, and performance, reductions and load balancing. We will also discuss extensions to heterogeneous systems such as offered by the OpenMP 4.0 and OpenACC standards.

=====Details=====

''Time/date''

Monday July 11, 2016, 13:30 - 16:30 
Tuesday July 12, 2016, 09:30 - 12:30

''Location''

MC 252 
5 King's College Road 
Toronto, ON M5S 3G8

''Instructor''

Dr. Ramses van Zon (SciNet)

=====Prerequisites=====
C/C++ and/or Fortran scientific programming; Experience editing and compiling code in a Linux environment.

=====Slides and other material=====

----

===Introduction to the Linux Shell===

====Abstract====
Working with many of the HPC systems in Ontario involves using the Linux/UNIX command line. This provides a very powerful interface, but it can be quite daunting for the uninitiated. In this half-day session, you can become initiated with this course. This hands on session will cover basic commands and scripting, as well as touching on some powerful constructs like regular expressions. It could be a great boon for your productivity!

====Details====
''Time/date''

Monday July 11, 13:30 - 16:30

''Location''

MC 254 
5 King's College Road 
Toronto, ON M5S 3G8

''Instructor''

Dr. Mike Nolta (SciNet)

====Prerequisites====
A laptop with a linux-like environment.

====Slides and other material====

----

===R for Data Science===

=====Abstract=====
This session offers a brief introduction to R, with a focus on data analysis and statistics.

=====Details=====

''Date/time''

Tuesday July 12, 2016, 09:30 - 12:30

''Location''

MC 254 
5 King's College Road 
Toronto, ON M5S 3G8

''Instructor''

Dr. Erik Spence (SciNet)

=====Prerequisites=====

Some programming experience in any language. You should have used and written functions. Bring a laptop with R or Rstudio installed.

=====Slides and other material=====

----

===Programming Clusters with MPI===

=====Abstract=====
In this 1.5-day session, through lectures interspersed with hands-on labs, the students will learn the basics of MPI programming. Examples and exercises will be based on parallelization of common scientific computing problems.

=====Details=====
''Time/date''

Tuesday July 12, 2016, 13:30 - 16:30 
Wednesday July 16, 2016, 09:30 - 12:30 
Wednesday July 16, 2016, 13:30 - 16:30 

''Location''

MC 252 
5 King's College Road 
Toronto, ON M5S 3G8

''Instructor''

Dr. Scott Northrup (SciNet)

=====Prerequisites=====
C and/or Fortran scientific programming; experience editing and compiling code in a Linux environment.

=====Slides and other material=====

----

===Parallel R for Data Science===

=====Abstract=====
This half-day session will cover parallel programming in R, with a focus on parallel data analysis. Topics covered will include snow, parallel, and foreach/doparallel.

=====Details=====

''Time/date''

Tuesday July 12, 2016, 13:30 - 16:30

''Location''

MC 254 
5 King's College Road 
Toronto, ON M5S 3G8

''Instructor''

Dr. Erik Spence (SciNet)

=====Prerequisites=====
Experience with programming in R and a rudimentary understanding of parallel computing.

=====Slides and other material=====

----

===Python for Scientific Computing===

=====Abstract=====
In this half-day session, you will be taught how to use python for research computing purposes. Topics include: the basics of python, automation, numpy, scipy, file i/o, and visualization.

=====Details=====
''Time/date'''

Wednesday July 13, 9:30 - 12:30

''Location''

MC 254 
5 King's College Road 
Toronto, ON M5S 3G8

''Instructor''

Dr. Erik Spence (SciNet)

=====Prerequisites=====
A bit of coding experience in any programming or scripting language. Command-line experience in Linux is a plus.

=====Slides and other material=====

----

===Python for High Performance Computing===

=====Abstract=====
This half-day session will cover parallel programming in python, with a focus on parallel data analysis. We will cover subprocess, multiprocessing, pypar and other parallel-enabling python packages.

=====Details=====
''Time/date''

Wednesday July 13, 2016, 13:30 - 16:30

''Location''

MC 254 
5 King's College Road 
Toronto, ON M5S 3G8

''Instructor''

Dr. Ramses von Zan (SciNet)

=====Prerequisites=====
A bit of coding experience in any programming or scripting language. Command-line experience in Linux is a plus.

=====Slides and other material=====

----

===Programming GPUs with CUDA===

=====Abstract=====
This two-day session is an introductory course covering programming and computing on GPUs - graphics processing units - which are an increasingly common presence in massively parallel computing architectures. This session will cover both of the available C-like programming frameworks: NVIDIA’s CUDA C. The basics of GPU programming will be covered, and students will work through a number of hands on examples. Demonstrations of profiling and debugging applications running on the GPU will also be included. The structuring of data and computations that makes full use of the GPU will be discussed in detail. Students should be able to leave the course with the knowledge necessary to begin developing their own GPU applications.

=====Details=====
''Time/date''

Thursday July 14, 2016, 09:30 - 12:30 
Thursday July 14, 2016, 13:30 - 16:30 
Friday July 15, 2016, 09:30 - 12:30 
Friday July 15, 2016, 13:30 - 16:30

''Location''

MC 252 
5 King's College Road 
Toronto, ON M5S 3G8

''Instructors''
Dr. Pawel Pomorski and Dr. Sergey Mashchenko (SHARCNET)

=====Prerequisites=====
C/C++ scientific programming, experience editing and compiling code in a Linux environment. Some experience with CUDA and/or OpenMP a plus.

=====Slides and other materials=====

----

===Visualization===

=====Abstract=====
The first part of this whole-day session will be a basic introduction to general-purpose scientific visualization tools such as gnuplot, grace(xmgr), remote tools (tunneling, VNC), python's matplotlib, and examples from other. The afternoon part will be a mix of lecture and hands-on to introduce more specialized scientific visualization software such as vmd, paraview, and visit.

=====Details=====
''Time/date''

Thursday July 14, 2016, 09:30 - 12:30 
Thursday July 14, 2016, 13:30 - 16:30

''Location''

MC 254 
5 King's College Road 
Toronto, ON M5S 3G8

''Instructor''
Dr. Marcelo Ponce (SciNet)

=====Prerequisites=====
Software installed, for the morning session: gnuplot, ssh (graphic client), vnc client, python and matplotlib; afternoon session: paraview.

=====Slides and other material=====

----

===Debugging===

=====Abstract=====
Debugging is an important step in developing a new code, or porting an old one to a new machine. In this half-day session, we will discuss the debugging of frequently encountered bugs in serial code with gdb and the debugging of parallel (MPI and threaded) codes with DDT on live systems.

=====Details=====
''Date/time''

Friday July 15, 09:30 - 12:30

''Location''

MC 254 
5 King's College Road 
Toronto, ON M5S 3G8

''Instructors''

Dr. Marcelo Ponce (SciNet)

=====Prerequisites=====
Basic familiarity with a high-level language (C/C++/Fortran) as well as compiling and running programs via the command line environment on a HPC system.

=====Slides and other material=====

----

===Bring Your Own Code===

=====Abstract=====
The final afternoon of the summer school is your chance to get help with your own code from the SciNet software analysts. Bring your code, explain your problem or concerns, and we'll give you advice and work with you to try to improve it.

=====Details=====
''Date/time''

Friday July 15, 2016, 13:30 - 16:30

''Location''

MC 254 
5 King's College Road 
Toronto, ON M5S 3G8

''Instructors''

Various SciNet software analysts.

=====Prerequisites=====
None.

2016 Ontario Summer School for High Performance Computing Central

2016-07-06T15:23:05Z

Ejspence:

===About===

The 2016 Ontario Summer School on High Performance Computing, whose second installment of the year will be hosted by SciNet, provides attendees with opportunities to learn and share knowledge and experience in high performance and technical computing. The format: a five-day workshop with mixed lectures and hands-on sessions on a number of selected subjects, including shared memory programming, distributed memory programming, parallel data processing, and general purpose graphics processing unit programming.

Below are descriptions and course material for the Central version, given from July 11th to July 15th, 2016, in Toronto.

__TOC__

===Introduction to HPC and SciNet===

=====Abstract=====
This half-day session will provide an introduction to basic concepts of high-performance computing. It is intended to be a high level primer for those largely new to HPC, and serve as a foundation upon which to build over the coming days. Topics will include motivation for HPC, essential issues, problem characteristics as they apply to parallelism and a high level overview of parallel programming models. Strategies of running large sets of serial processes using e.g. GNU parallel, will also be presented.

=====Details=====
''Time/date''

Monday July 11, 2016, 09:30 - 12:30

''Location''

MC 252 
5 King's College Road 
Toronto, ON M5S 3G8

''Instructor''

Dr. Mike Nolta (SciNet)

=====Prerequisites=====
Basic Linux skills are useful to follow along. If you do not have these, consider taking the "Intro to the Linux Shell" in the afternoon.

=====Slides and other material=====

----

===Programming Shared Memory Systems with OpenMP===

=====Abstract=====
In these two half-day sessions, lectures and hands-on labs are interspersed, and the students will learn the basics of shared memory programming with OpenMP. In particular, we will discuss the OpenMP's threads, memory, and performance, reductions and load balancing. We will also discuss extensions to heterogeneous systems such as offered by the OpenMP 4.0 and OpenACC standards.

=====Details=====

''Time/date''

Monday July 11, 2016, 13:30 - 16:30 
Tuesday July 12, 2016, 09:30 - 12:30

''Location''

MC 252 
5 King's College Road 
Toronto, ON M5S 3G8

''Instructor''

Dr. Ramses van Zon (SciNet)

=====Prerequisites=====
C/C++ and/or Fortran scientific programming; Experience editing and compiling code in a Linux environment.

=====Slides and other material=====

----

===Introduction to the Linux Shell===

====Abstract====
Working with many of the HPC systems in Ontario involves using the Linux/UNIX command line. This provides a very powerful interface, but it can be quite daunting for the uninitiated. In this half-day session, you can become initiated with this course. This hands on session will cover basic commands and scripting, as well as touching on some powerful constructs like regular expressions. It could be a great boon for your productivity!

====Details====
''Time/date''

Monday July 11, 13:30 - 16:30

''Location''

MC 254 
5 King's College Road 
Toronto, ON M5S 3G8

''Instructor''

Dr. Mike Nolta (SciNet)

====Prerequisites====
A laptop with a linux-like environment.

====Slides and other material====

----

===R for Data Science===

=====Abstract=====
This session offers a brief introduction to R, with a focus on data analysis and statistics.

=====Details=====

''Date/time''

Tuesday July 12, 2016, 09:30 - 12:30

''Location''

MC 254 
5 King's College Road 
Toronto, ON M5S 3G8

''Instructor''

Dr. Erik Spence (SciNet)

=====Prerequisites=====

Some programming experience in any language. You should have used and written functions. Bring a laptop with R or Rstudio installed.

=====Slides and other material=====

----

===Programming Clusters with MPI===

=====Abstract=====
In this 1.5-day session, through lectures interspersed with hands-on labs, the students will learn the basics of MPI programming. Examples and exercises will be based on parallelization of common scientific computing problems.

=====Details=====
''Time/date''

Tuesday July 12, 2016, 13:30 - 16:30 
Wednesday July 16, 2016, 09:30 - 12:30 
Wednesday July 16, 2016, 13:30 - 16:30 

''Location''

MC 252 
5 King's College Road 
Toronto, ON M5S 3G8

''Instructor''

Dr. Scott Northrup (SciNet)

=====Prerequisites=====
C and/or Fortran scientific programming; experience editing and compiling code in a Linux environment.

=====Slides and other material=====

----

===Parallel R for Data Science===

=====Abstract=====
This half-day session will cover parallel programming in R, with a focus on parallel data analysis. Topics covered will include snow, parallel, and foreach/doparallel.

=====Details=====

''Time/date''

Tuesday July 12, 2016, 13:30 - 16:30

''Location''

MC 254 
5 King's College Road 
Toronto, ON M5S 3G8

''Instructor''

Dr. Erik Spence (SciNet)

=====Prerequisites=====
Experience with programming in R and a rudimentary understanding of parallel computing.

=====Slides and other material=====

----

===Python for Scientific Computing===

=====Abstract=====
In this half-day session, you will be taught how to use python for research computing purposes. Topics include: the basics of python, automation, numpy, scipy, file i/o, and visualization.

=====Details=====
''Time/date'''

Wednesday July 13, 9:30 - 12:30

''Location''

MC 254 
5 King's College Road 
Toronto, ON M5S 3G8

''Instructor''

Dr. Erik Spence (SciNet)

=====Prerequisites=====
A bit of coding experience in any programming or scripting language. Command-line experience in Linux is a plus.

=====Slides and other material=====

----

===Python for High Performance Computing===

=====Abstract=====
This half-day session will cover parallel programming in python, with a focus on parallel data analysis. We will cover subprocess, multiprocessing, pypar and other parallel-enabling python packages.

=====Details=====
''Time/date''

Wednesday July 13, 2016, 13:30 - 16:30

''Location''

MC 254 
5 King's College Road 
Toronto, ON M5S 3G8

''Instructor''

Dr. Ramses von Zan (SciNet)

=====Prerequisites=====
A bit of coding experience in any programming or scripting language. Command-line experience in Linux is a plus.

=====Slides and other material=====

----

===Programming GPUs with CUDA===

=====Abstract=====
This two-day session is an introductory course covering programming and computing on GPUs - graphics processing units - which are an increasingly common presence in massively parallel computing architectures. This session will cover both of the available C-like programming frameworks: NVIDIA’s CUDA C. The basics of GPU programming will be covered, and students will work through a number of hands on examples. Demonstrations of profiling and debugging applications running on the GPU will also be included. The structuring of data and computations that makes full use of the GPU will be discussed in detail. Students should be able to leave the course with the knowledge necessary to begin developing their own GPU applications.

=====Details=====
''Time/date''

Thursday July 14, 2016, 09:30 - 12:30 
Thursday July 14, 2016, 13:30 - 16:30 
Friday July 15, 2016, 09:30 - 12:30 
Friday July 15, 2016, 13:30 - 16:30

''Location''

MC 252 
5 King's College Road 
Toronto, ON M5S 3G8

''Instructors''
Dr. Pawel Pomorski and Dr. Sergey Mashchenko (SHARCNET)

=====Prerequisites=====
C/C++ scientific programming, experience editing and compiling code in a Linux environment. Some experience with CUDA and/or OpenMP a plus.

=====Slides and other materials=====

----

===Visualization===

=====Abstract=====
: The first part of this whole-day session will be a basic introduction to general-purpose scientific visualization tools such as gnuplot, grace(xmgr), remote tools (tunneling, VNC), python's matplotlib, and examples from other. The afternoon part will be a mix of lecture and hands-on to introduce more specialized scientific visualization software such as vmd, paraview, and visit.

=====Details=====
''Time/date''

Thursday July 14, 2016, 09:30 - 12:30 
Thursday July 14, 2016, 13:30 - 16:30

''Location''

MC 254 
5 King's College Road 
Toronto, ON M5S 3G8

''Instructor''
Dr. Marcelo Ponce (SciNet)

=====Prerequisites=====
Software installed, for the morning session: gnuplot, ssh (graphic client), vnc client, python and matplotlib; afternoon session: paraview.

=====Slides and other material=====

----

===Debugging===

=====Abstract=====
Debugging is an important step in developing a new code, or porting an old one to a new machine. In this half-day session, we will discuss the debugging of frequently encountered bugs in serial code with gdb and the debugging of parallel (MPI and threaded) codes with DDT on live systems.

=====Details=====
''Date/time''

Friday July 15, 09:30 - 12:30

''Location''

MC 254 
5 King's College Road 
Toronto, ON M5S 3G8

''Instructors''

Dr. Marcelo Ponce (SciNet)

=====Prerequisites=====
Basic familiarity with a high-level language (C/C++/Fortran) as well as compiling and running programs via the command line environment on a HPC system.

=====Slides and other material=====

----

===Bring Your Own Code===

=====Abstract=====
The final afternoon of the summer school is your chance to get help with your own code from the SciNet software analysts. Bring your code, explain your problem or concerns, and we'll give you advice and work with you to try to improve it.

=====Details=====
''Date/time''

Friday July 15, 2016, 13:30 - 16:30

''Location''

MC 254 
5 King's College Road 
Toronto, ON M5S 3G8

''Instructors''

Various SciNet software analysts.

=====Prerequisites=====
None.

2016 Ontario Summer School for High Performance Computing Central

2016-07-06T15:19:55Z

Ejspence:

===About===

The 2016 Ontario Summer School on High Performance Computing, whose second installment of the year will be hosted by SciNet, provides attendees with opportunities to learn and share knowledge and experience in high performance and technical computing. The format: a five-day workshop with mixed lectures and hands-on sessions on a number of selected subjects, including shared memory programming, distributed memory programming, parallel data processing, and general purpose graphics processing unit programming.

Below are descriptions and course material for the Central version, given from July 11th to July 15th, 2016, in Toronto.

__TOC__

===Introduction to HPC and SciNet===

=====Abstract=====
This half-day session will provide an introduction to basic concepts of high-performance computing. It is intended to be a high level primer for those largely new to HPC, and serve as a foundation upon which to build over the coming days. Topics will include motivation for HPC, essential issues, problem characteristics as they apply to parallelism and a high level overview of parallel programming models. Strategies of running large sets of serial processes using e.g. GNU parallel, will also be presented.

=====Details=====
''Time/date''

Monday July 11, 2016, 09:30 - 12:30

''Location''

MC 252 
5 King's College Road 
Toronto, ON M5S 1A4

''Instructor''

Dr. Mike Nolta (SciNet)

=====Prerequisites=====
Basic Linux skills are useful to follow along. If you do not have these, consider taking the "Intro to the Linux Shell" in the afternoon.

=====Slides and other material=====

----

===Programming Shared Memory Systems with OpenMP===

=====Abstract=====
In these two half-day sessions, lectures and hands-on labs are interspersed, and the students will learn the basics of shared memory programming with OpenMP. In particular, we will discuss the OpenMP's threads, memory, and performance, reductions and load balancing. We will also discuss extensions to heterogeneous systems such as offered by the OpenMP 4.0 and OpenACC standards.

=====Details=====

''Time/date''

Monday July 11, 2016, 13:30 - 16:30 
Tuesday July 12, 2016, 09:30 - 12:30

''Location''

MC 252 
5 King's College Road 
Toronto, ON M5S 1A4

''Instructor''

Dr. Ramses van Zon (SciNet)

=====Prerequisites=====
C/C++ and/or Fortran scientific programming; Experience editing and compiling code in a Linux environment.

=====Slides and other material=====

----

===Introduction to the Linux Shell===

====Abstract====
Working with many of the HPC systems in Ontario involves using the Linux/UNIX command line. This provides a very powerful interface, but it can be quite daunting for the uninitiated. In this half-day session, you can become initiated with this course. This hands on session will cover basic commands and scripting, as well as touching on some powerful constructs like regular expressions. It could be a great boon for your productivity!

====Details====
''Time/date''

Monday July 11, 13:30 - 16:30

''Location''

MC 254 
5 King's College Road 
Toronto, ON M5S 1A4

''Instructor''

Dr. Mike Nolta (SciNet)

====Prerequisites====
A laptop with a linux-like environment.

====Slides and other material====

----

===R for Data Science===

=====Abstract=====
This session offers a brief introduction to R, with a focus on data analysis and statistics.

=====Details=====

''Date/time''

Tuesday July 12, 2016, 09:30 - 12:30

''Location''

MC 254 
5 King's College Road 
Toronto, ON M5S 1A4

''Instructor''

Dr. Erik Spence (SciNet)

=====Prerequisites=====

Some programming experience in any language. You should have used and written functions. Bring a laptop with R or Rstudio installed.

=====Slides and other material=====

----

===Programming Clusters with MPI===

=====Abstract=====
In this 1.5-day session, through lectures interspersed with hands-on labs, the students will learn the basics of MPI programming. Examples and exercises will be based on parallelization of common scientific computing problems.

=====Details=====
''Time/date''

Tuesday July 12, 2016, 13:30 - 16:30 
Wednesday July 16, 2016, 09:30 - 12:30 
Wednesday July 16, 2016, 13:30 - 16:30 

''Location''

MC 252 
5 King's College Road 
Toronto, ON M5S 1A4

''Instructor''

Dr. Scott Northrup (SciNet)

=====Prerequisites=====
C and/or Fortran scientific programming; experience editing and compiling code in a Linux environment.

=====Slides and other material=====

----

===Parallel R for Data Science===

=====Abstract=====
This half-day session will cover parallel programming in R, with a focus on parallel data analysis. Topics covered will include snow, parallel, and foreach/doparallel.

=====Details=====

''Time/date''

Tuesday July 12, 2016, 13:30 - 16:30

''Location''

MC 254 
5 King's College Road 
Toronto, ON M5S 1A4

''Instructor''

Dr. Erik Spence (SciNet)

=====Prerequisites=====
Experience with programming in R and a rudimentary understanding of parallel computing.

=====Slides and other material=====

----

===Python for Scientific Computing===

=====Abstract=====
In this half-day session, you will be taught how to use python for research computing purposes. Topics include: the basics of python, automation, numpy, scipy, file i/o, and visualization.

=====Details=====
''Time/date'''

Wednesday July 13, 9:30 - 12:30

''Location''

MC 254 
5 King's College Road 
Toronto, ON M5S 1A4

''Instructor''

Dr. Erik Spence (SciNet)

=====Prerequisites=====
A bit of coding experience in any programming or scripting language. Command-line experience in Linux is a plus.

=====Slides and other material=====

----

===Python for High Performance Computing===

=====Abstract=====
This half-day session will cover parallel programming in python, with a focus on parallel data analysis. We will cover subprocess, multiprocessing, pypar and other parallel-enabling python packages.

=====Details=====
''Time/date''

Wednesday July 13, 2016, 13:30 - 16:30

''Location''

MC 254 
5 King's College Road 
Toronto, ON M5S 1A4

''Instructor''

Dr. Ramses von Zan (SciNet)

=====Prerequisites=====
A bit of coding experience in any programming or scripting language. Command-line experience in Linux is a plus.

=====Slides and other material=====

----

===Programming GPUs with CUDA===

=====Abstract=====
This two-day session is an introductory course covering programming and computing on GPUs - graphics processing units - which are an increasingly common presence in massively parallel computing architectures. This session will cover both of the available C-like programming frameworks: NVIDIA’s CUDA C. The basics of GPU programming will be covered, and students will work through a number of hands on examples. Demonstrations of profiling and debugging applications running on the GPU will also be included. The structuring of data and computations that makes full use of the GPU will be discussed in detail. Students should be able to leave the course with the knowledge necessary to begin developing their own GPU applications.

=====Details=====
''Time/date''

Thursday July 14, 2016, 09:30 - 12:30 
Thursday July 14, 2016, 13:30 - 16:30 
Friday July 15, 2016, 09:30 - 12:30 
Friday July 15, 2016, 13:30 - 16:30

''Location''

MC 252 
5 King's College Road 
Toronto, ON M5S 1A4

''Instructors''
Dr. Pawel Pomorski and Dr. Sergey Mashchenko (SHARCNET)

=====Prerequisites=====
C/C++ scientific programming, experience editing and compiling code in a Linux environment. Some experience with CUDA and/or OpenMP a plus.

=====Slides and other materials=====

----

===Visualization===

=====Abstract=====
: The first part of this whole-day session will be a basic introduction to general-purpose scientific visualization tools such as gnuplot, grace(xmgr), remote tools (tunneling, VNC), python's matplotlib, and examples from other. The afternoon part will be a mix of lecture and hands-on to introduce more specialized scientific visualization software such as vmd, paraview, and visit.

=====Details=====
''Time/date''

Thursday July 14, 2016, 09:30 - 12:30 
Thursday July 14, 2016, 13:30 - 16:30

''Location''

MC 254 
5 King's College Road 
Toronto, ON M5S 1A4

''Instructor''
Dr. Marcelo Ponce (SciNet)

=====Prerequisites=====
Software installed, for the morning session: gnuplot, ssh (graphic client), vnc client, python and matplotlib; afternoon session: paraview.

=====Slides and other material=====

----

===Debugging===

=====Abstract=====
Debugging is an important step in developing a new code, or porting an old one to a new machine. In this half-day session, we will discuss the debugging of frequently encountered bugs in serial code with gdb and the debugging of parallel (MPI and threaded) codes with DDT on live systems.

=====Details=====
''Date/time''

Friday July 15, 09:30 - 12:30

''Location''

MC 254 
5 King's College Road 
Toronto, ON M5S 1A4

''Instructors''

Dr. Marcelo Ponce (SciNet)

=====Prerequisites=====
Basic familiarity with a high-level language (C/C++/Fortran) as well as compiling and running programs via the command line environment on a HPC system.

=====Slides and other material=====

----

===Bring Your Own Code===

=====Abstract=====
The final afternoon of the summer school is your chance to get help with your own code from the SciNet software analysts. Bring your code, explain your problem or concerns, and we'll give you advice and work with you to try to improve it.

=====Details=====
''Date/time''

Friday July 15, 2016, 13:30 - 16:30

''Location''

MC 254 
5 King's College Road 
Toronto, ON M5S 1A4

''Instructors''

Various SciNet software analysts.

=====Prerequisites=====
None.

2016 Ontario Summer School for High Performance Computing Central

2016-07-06T15:17:18Z

Ejspence:

===About===

The 2016 Ontario Summer School on High Performance Computing, whose second installment of the year will be hosted by SciNet, provides attendees with opportunities to learn and share knowledge and experience in high performance and technical computing. The format: a five-day workshop with mixed lectures and hands-on sessions on a number of selected subjects, including shared memory programming, distributed memory programming, parallel data processing, and general purpose graphics processing unit programming.

Below are descriptions and course material for the Central version, given from July 11th to July 15th, 2016, in Toronto.

__TOC__

===Introduction to HPC and SciNet===

=====Abstract=====
This half-day session will provide an introduction to basic concepts of high-performance computing. It is intended to be a high level primer for those largely new to HPC, and serve as a foundation upon which to build over the coming days. Topics will include motivation for HPC, essential issues, problem characteristics as they apply to parallelism and a high level overview of parallel programming models. Strategies of running large sets of serial processes using e.g. GNU parallel, will also be presented.

=====Details=====
''Time/date''

Monday July 11, 2016, 09:30 - 12:30

''Location''

MC 252

5 King's College Road

Toronto, ON M5S 1A4

''Instructor''

Dr. Mike Nolta (SciNet)

=====Prerequisites=====
Basic Linux skills are useful to follow along. If you do not have these, consider taking the "Intro to the Linux Shell" in the afternoon.

=====Slides and other material=====

----

===Programming Shared Memory Systems with OpenMP===

=====Abstract=====
In these two half-day sessions, lectures and hands-on labs are interspersed, and the students will learn the basics of shared memory programming with OpenMP. In particular, we will discuss the OpenMP's threads, memory, and performance, reductions and load balancing. We will also discuss extensions to heterogeneous systems such as offered by the OpenMP 4.0 and OpenACC standards.

=====Details=====

''Time/date''

Monday July 11, 2016, 13:30 - 16:30 
Tuesday July 12, 2016, 09:30 - 12:30

''Location''

MC 252

5 King's College Road

Toronto, ON M5S 1A4

''Instructor''

Dr. Ramses van Zon (SciNet)

=====Prerequisites=====
C/C++ and/or Fortran scientific programming; Experience editing and compiling code in a Linux environment.

=====Slides and other material=====

----

===Introduction to the Linux Shell===

====Abstract====
Working with many of the HPC systems in Ontario involves using the Linux/UNIX command line. This provides a very powerful interface, but it can be quite daunting for the uninitiated. In this half-day session, you can become initiated with this course. This hands on session will cover basic commands and scripting, as well as touching on some powerful constructs like regular expressions. It could be a great boon for your productivity!

====Details====
''Time/date''

Monday July 11, 13:30 - 16:30

''Location''

MC 254

5 King's College Road

Toronto, ON M5S 1A4

''Instructor''

Dr. Mike Nolta (SciNet)

====Prerequisites====
A laptop with a linux-like environment.

====Slides and other material====

----

===R for Data Science===

=====Abstract=====
This session offers a brief introduction to R, with a focus on data analysis and statistics.

=====Details=====

''Date/time''

Tuesday July 12, 2016, 09:30 - 12:30

''Location''

MC 254

5 King's College Road

Toronto, ON M5S 1A4

''Instructor''

Dr. Erik Spence (SciNet)

=====Prerequisites=====

Some programming experience in any language. You should have used and written functions. Bring a laptop with R or Rstudio installed.

=====Slides and other material=====

----

===Programming Clusters with MPI===

=====Abstract=====
In this 1.5-day session, through lectures interspersed with hands-on labs, the students will learn the basics of MPI programming. Examples and exercises will be based on parallelization of common scientific computing problems.

=====Details=====
''Time/date''

Tuesday July 12, 2016, 13:30 - 16:30 
Wednesday July 16, 2016, 09:30 - 12:30 
Wednesday July 16, 2016, 13:30 - 16:30 

''Location''

MC 252

5 King's College Road

Toronto, ON M5S 1A4

''Instructor''

Dr. Scott Northrup (SciNet)

=====Prerequisites=====
C and/or Fortran scientific programming; experience editing and compiling code in a Linux environment.

=====Slides and other material=====

----

===Parallel R for Data Science===

=====Abstract=====
This half-day session will cover parallel programming in R, with a focus on parallel data analysis. Topics covered will include snow, parallel, and foreach/doparallel.

=====Details=====

''Time/date''

Tuesday July 12, 2016, 13:30 - 16:30

''Location''

MC 254

5 King's College Road

Toronto, ON M5S 1A4

''Instructor''

Dr. Erik Spence (SciNet)

=====Prerequisites=====
Experience with programming in R and a rudimentary understanding of parallel computing.

=====Slides and other material=====

----

===Python for Scientific Computing===

=====Abstract=====
In this half-day session, you will be taught how to use python for research computing purposes. Topics include: the basics of python, automation, numpy, scipy, file i/o, and visualization.

=====Details=====
''Time/date'''

Wednesday July 13, 9:30 - 12:30

''Location''

MC 254

5 King's College Road

Toronto, ON M5S 1A4

''Instructor''

Dr. Erik Spence (SciNet)

=====Prerequisites=====
A bit of coding experience in any programming or scripting language. Command-line experience in Linux is a plus.

=====Slides and other material=====

----

===Python for High Performance Computing===

=====Abstract=====
This half-day session will cover parallel programming in python, with a focus on parallel data analysis. We will cover subprocess, multiprocessing, pypar and other parallel-enabling python packages.

=====Details=====
''Time/date''

Wednesday July 13, 2016, 13:30 - 16:30

''Location''

MC 254

5 King's College Road

Toronto, ON M5S 1A4

''Instructor''

Dr. Ramses von Zan (SciNet)

=====Prerequisites=====
A bit of coding experience in any programming or scripting language. Command-line experience in Linux is a plus.

=====Slides and other material=====

----

===Programming GPUs with CUDA===

=====Abstract=====
This two-day session is an introductory course covering programming and computing on GPUs - graphics processing units - which are an increasingly common presence in massively parallel computing architectures. This session will cover both of the available C-like programming frameworks: NVIDIA’s CUDA C. The basics of GPU programming will be covered, and students will work through a number of hands on examples. Demonstrations of profiling and debugging applications running on the GPU will also be included. The structuring of data and computations that makes full use of the GPU will be discussed in detail. Students should be able to leave the course with the knowledge necessary to begin developing their own GPU applications.

=====Details=====
''Time/date''

Thursday July 14, 2016, 09:30 - 12:30 
Thursday July 14, 2016, 13:30 - 16:30 
Friday July 15, 2016, 09:30 - 12:30 
Friday July 15, 2016, 13:30 - 16:30

''Location''

MC 252 
5 King's College Road 
Toronto, ON M5S 1A4

''Instructors''
Dr. Pawel Pomorski and Dr. Sergey Mashchenko (SHARCNET)

=====Prerequisites=====
C/C++ scientific programming, experience editing and compiling code in a Linux environment. Some experience with CUDA and/or OpenMP a plus.

=====Slides and other materials=====

----

===Visualization===

=====Abstract=====
: The first part of this whole-day session will be a basic introduction to general-purpose scientific visualization tools such as gnuplot, grace(xmgr), remote tools (tunneling, VNC), python's matplotlib, and examples from other. The afternoon part will be a mix of lecture and hands-on to introduce more specialized scientific visualization software such as vmd, paraview, and visit.

=====Details=====
''Time/date''

Thursday July 14, 2016, 09:30 - 12:30 
Thursday July 14, 2016, 13:30 - 16:30

''Location''

MC 254 
5 King's College Road 
Toronto, ON M5S 1A4

''Instructor''
Dr. Marcelo Ponce (SciNet)

=====Prerequisites=====
Software installed, for the morning session: gnuplot, ssh (graphic client), vnc client, python and matplotlib; afternoon session: paraview.

=====Slides and other material=====

----

===Debugging===

=====Abstract=====
Debugging is an important step in developing a new code, or porting an old one to a new machine. In this half-day session, we will discuss the debugging of frequently encountered bugs in serial code with gdb and the debugging of parallel (MPI and threaded) codes with DDT on live systems.

=====Details=====
''Date/time''

Friday July 15, 09:30 - 12:30

''Location''

MC 254 
5 King's College Road 
Toronto, ON M5S 1A4

''Instructors''

Dr. Marcelo Ponce (SciNet)

=====Prerequisites=====
Basic familiarity with a high-level language (C/C++/Fortran) as well as compiling and running programs via the command line environment on a HPC system.

=====Slides and other material=====

----

===Bring Your Own Code===

=====Abstract=====
The final afternoon of the summer school is your chance to get help with your own code from the SciNet software analysts. Bring your code, explain your problem or concerns, and we'll give you advice and work with you to try to improve it.

=====Details=====
''Date/time''

Friday July 15, 2016, 13:30 - 16:30

''Location''

MC 254 
5 King's College Road 
Toronto, ON M5S 1A4

''Instructors''

Various SciNet software analysts.

=====Prerequisites=====
None.

2016 Ontario Summer School for High Performance Computing Central

2016-07-06T14:59:26Z

Ejspence:

===About===

The 2016 Ontario Summer School on High Performance Computing, whose second installment of the year will be hosted by SciNet, provides attendees with opportunities to learn and share knowledge and experience in high performance and technical computing. The format: a five-day workshop with mixed lectures and hands-on sessions on a number of selected subjects, including shared memory programming, distributed memory programming, parallel data processing, and general purpose graphics processing unit programming.

Below are descriptions and course material for the Central version, given from July 11th to July 15th, 2016, in Toronto.

__TOC__

===Introduction to HPC and SciNet===

=====Abstract=====
This half-day session will provide an introduction to basic concepts of high-performance computing. It is intended to be a high level primer for those largely new to HPC, and serve as a foundation upon which to build over the coming days. Topics will include motivation for HPC, essential issues, problem characteristics as they apply to parallelism and a high level overview of parallel programming models. Strategies of running large sets of serial processes using e.g. GNU parallel, will also be presented.

=====Details=====
''Time/date''

Monday July 11, 2016, 09:30 - 12:30

''Location''

MC 252

5 King's College Road

Toronto, ON M5S 1A4

''Instructor''

Dr. Mike Nolta (SciNet)

=====Prerequisites=====
Basic Linux skills are useful to follow along. If you do not have these, consider taking the "Intro to the Linux Shell" in the afternoon.

=====Slides and other material=====

----

===Programming Shared Memory Systems with OpenMP===

=====Abstract=====
In these two half-day sessions, lectures and hands-on labs are interspersed, and the students will learn the basics of shared memory programming with OpenMP. In particular, we will discuss the OpenMP's threads, memory, and performance, reductions and load balancing. We will also discuss extensions to heterogeneous systems such as offered by the OpenMP 4.0 and OpenACC standards.

=====Details=====

''Time/date''

Monday July 11, 2016, 13:30 - 16:30 
Tuesday July 12, 2016, 09:30 - 12:30

''Location''

MC 252

5 King's College Road

Toronto, ON M5S 1A4

''Instructor''

Dr. Ramses van Zon (SciNet)

=====Prerequisites=====
C/C++ and/or Fortran scientific programming; Experience editing and compiling code in a Linux environment.

=====Slides and other material=====

----

===Introduction to the Linux Shell===

====Abstract====
Working with many of the HPC systems in Ontario involves using the Linux/UNIX command line. This provides a very powerful interface, but it can be quite daunting for the uninitiated. In this half-day session, you can become initiated with this course. This hands on session will cover basic commands and scripting, as well as touching on some powerful constructs like regular expressions. It could be a great boon for your productivity!

====Details====
''Time/date''

Monday July 11, 13:30 - 16:30

''Location''

MC 254

5 King's College Road

Toronto, ON M5S 1A4

''Instructor''

Dr. Mike Nolta (SciNet)

====Prerequisites====
A laptop with a linux-like environment.

====Slides and other material====

----

===R for Data Science===

=====Abstract=====
This session offers a brief introduction to R, with a focus on data analysis and statistics.

=====Details=====

''Date/time''

Tuesday July 12, 2016, 09:30 - 12:30

''Location''

MC 254

5 King's College Road

Toronto, ON M5S 1A4

''Instructor''

Dr. Erik Spence (SciNet)

=====Prerequisites=====

Some programming experience in any language. You should have used and written functions. Bring a laptop with R or Rstudio installed.

=====Slides and other material=====

----

===Programming Clusters with MPI===

=====Abstract=====
In this 1.5-day session, through lectures interspersed with hands-on labs, the students will learn the basics of MPI programming. Examples and exercises will be based on parallelization of common scientific computing problems.

=====Details=====
''Time/date''

Tuesday July 12, 2016, 13:30 - 16:30 
Wednesday July 16, 2016, 09:30 - 12:30 
Wednesday July 16, 2016, 13:30 - 16:30 

''Location''

MC 252

5 King's College Road

Toronto, ON M5S 1A4

''Instructor''

Dr. Scott Northrup (SciNet)

=====Prerequisites=====
C and/or Fortran scientific programming; experience editing and compiling code in a Linux environment.

=====Slides and other material=====

----

===Parallel R for Data Science===

=====Abstract=====
This half-day session will cover parallel programming in R, with a focus on parallel data analysis. Topics covered will include snow, parallel, and foreach/doparallel.

=====Details=====

''Time/date''

Tuesday July 12, 2016, 13:30 - 16:30

''Location''

MC 254

5 King's College Road

Toronto, ON M5S 1A4

''Instructor''

Dr. Erik Spence (SciNet)

=====Prerequisites=====
Experience with programming in R and a rudimentary understanding of parallel computing.

=====Slides and other material=====

----

===Python for Scientific Computing===

=====Abstract=====
In this half-day session, you will be taught how to use python for research computing purposes. Topics include: the basics of python, automation, numpy, scipy, file i/o, and visualization.

=====Details=====
''Time/date'''

Wednesday July 13, 9:30 - 12:30

''Location''

MC 254

5 King's College Road

Toronto, ON M5S 1A4

''Instructor''

Dr. Erik Spence (SciNet)

=====Prerequisites=====
A bit of coding experience in any programming or scripting language. Command-line experience in Linux is a plus.

=====Slides and other material=====

----

===Python for High Performance Computing===

=====Abstract=====
This half-day session will cover parallel programming in python, with a focus on parallel data analysis. We will cover subprocess, multiprocessing, pypar and other parallel-enabling python packages.

=====Details=====
''Time/date''

Wednesday July 13, 2016, 13:30 - 16:30

''Location''

MC 254

5 King's College Road

Toronto, ON M5S 1A4

''Instructor''

Dr. Ramses von Zan (SciNet)

=====Prerequisites=====
A bit of coding experience in any programming or scripting language. Command-line experience in Linux is a plus.

=====Slides and other material=====

----

===Programming GPUs with CUDA===

=====Abstract=====
This two-day session is an introductory course covering programming and computing on GPUs - graphics processing units - which are an increasingly common presence in massively parallel computing architectures. This session will cover both of the available C-like programming frameworks: NVIDIA’s CUDA C. The basics of GPU programming will be covered, and students will work through a number of hands on examples. Demonstrations of profiling and debugging applications running on the GPU will also be included. The structuring of data and computations that makes full use of the GPU will be discussed in detail. Students should be able to leave the course with the knowledge necessary to begin developing their own GPU applications.

=====Details=====
''Time/date''

Thursday July 14, 2016, 09:30 - 12:30 
Thursday July 14, 2016, 13:30 - 16:30 
Friday July 15, 2016, 09:30 - 12:30 
Friday July 15, 2016, 13:30 - 16:30

''Location''

Galbraith Building, Room 405

35 St. George Street

Toronto, ON M5S 1A4

''Instructors''
Dr. Pawel Pomorski and Dr. Sergey Mashchenko (SHARCNET)

=====Prerequisites=====
C/C++ scientific programming, experience editing and compiling code in a Linux environment. Some experience with CUDA and/or OpenMP a plus.

=====Slides and other materials=====
''Day 1:''
* [[Media:CUDA_summer_school_2015_central.pdf|Slides]]
* [[Media:CUDA_day1.tar.gz|Code]]
See also [http://ppomorsk.sharcnet.ca http://ppomorsk.sharcnet.ca]

''Day 2:''
* [[Media:CUDA2.pdf|Slides]]

----

===Visualization===

=====Abstract=====
The first part of this session will be a basic introduction to general-purpose scientific visualization tools such as gnuplot, grace(xmgr), remote tools (tunneling, VNC), python's matplotlib, and examples from other. The second part will be a mix of lecture and hands-on to introduce more specialized scientific visualization software such as vmd, paraview, and visit.

=====Details=====
''Time/date''

Friday July 17, 9:30 - 16:30

''Location''

Galbraith Building, Room 404

35 St. George Street

Toronto, ON M5S 1A4

''Instructor''
Dr. Marcelo Ponce (SciNet)

=====Prerequisites=====
Software installed, for the morning session: gnuplot, ssh (graphic client), vnc client, python and matplotlib; afternoon session: ParaView & VisIt.

=====Slides and other material=====
Morning session: [http://www.scinethpc.ca/~mponce/ss2015/ss2015_visualization-morning.pdf Visualization I]

Afternoon session: [http://www.scinethpc.ca/~mponce/ss2015/ss2015_visualization-afternoon.pdf Visualization II]

Samples to use during the tutorial: [http://www.scinethpc.ca/~mponce/ss2015/data.tar.gz Data sets]

An example of a complete visualization session in VisIt: [http://www.scinethpc.ca/~mponce/ss2015/noise-example_VisIt.tar.gz VisIt session] (un-tar this file and load the session from the File/Menu in VisIt using "Restore session with sources...", then choose your local "noise.silo" file to be used within the session).
----

===Debugging, Optimization, and Best Practices===

=====Abstract=====
Debugging is an important step in developing a new code, or porting an old one to a new machine. We will discuss the debugging of frequently encountered bugs in serial code with gdb and the debugging of parallel (MPI and threaded) codes with DDT on live systems. Next, we will present general issues, common pitfalls and optimization strategies that are application to HPC systems. By using profiling tools available for parallel programs, we can see when the bottlenecks are. We can try to alleviate them by following common best practices, allowing one to obtain their results more quickly and at the same time minimize their burden on shared system resources. Walk-through examples and case studies will be presented during the session to illustrate the concepts.

=====Details=====
''Date/time''

Friday July 17, 9:30 - 16:30

''Location''

Galbraith Building, Room 405

35 St. George Street

Toronto, ON M5S 1A4

''Instructors''

Dr. Ramses van Zon and Dr. Mike Nolta (SciNet)

=====Prerequisites=====
Basic familiarity with a high-level language (C/C++/Fortran) as well as compiling and running programs via the command line environment on a HPC system.

=====Slides and other material=====
* [[Media:DebugSS2015.pdf | Slides for the debugging session]]
* [[Media:DebugSS2015code.tar.gz | Debuggable code ]]
* [[Media:SSHPC_OBP2013.pdf|Slides for the best practices session]]

===Group Photo===

[[File:GroupPhotoSS2015.jpg|800px]]

2016 Ontario Summer School for High Performance Computing Central

2016-07-06T14:52:44Z

Ejspence:

===About===

The 2016 Ontario Summer School on High Performance Computing, whose second installment of the year will be hosted by SciNet, provides attendees with opportunities to learn and share knowledge and experience in high performance and technical computing. The format: a five-day workshop with mixed lectures and hands-on sessions on a number of selected subjects, including shared memory programming, distributed memory programming, parallel data processing, and general purpose graphics processing unit programming.

Below are descriptions and course material for the Central version, given from July 11th to July 15th, 2016, in Toronto.

__TOC__

===Introduction to HPC and SciNet===

=====Abstract=====
This half-day session will provide an introduction to basic concepts of high-performance computing. It is intended to be a high level primer for those largely new to HPC, and serve as a foundation upon which to build over the coming days. Topics will include motivation for HPC, essential issues, problem characteristics as they apply to parallelism and a high level overview of parallel programming models. Strategies of running large sets of serial processes using e.g. GNU parallel, will also be presented.

=====Details=====
''Time/date''

Monday July 11, 2016, 09:30 - 12:30

''Location''

MC 252

5 King's College Road

Toronto, ON M5S 1A4

''Instructor''

Dr. Mike Nolta (SciNet)

=====Prerequisites=====
Basic Linux skills are useful to follow along. If you do not have these, consider taking the "Intro to the Linux Shell" in the afternoon.

=====Slides and other material=====

----

===Programming Shared Memory Systems with OpenMP===

=====Abstract=====
In these two half-day sessions, lectures and hands-on labs are interspersed, and the students will learn the basics of shared memory programming with OpenMP. In particular, we will discuss the OpenMP's threads, memory, and performance, reductions and load balancing. We will also discuss extensions to heterogeneous systems such as offered by the OpenMP 4.0 and OpenACC standards.

=====Details=====

''Time/date''

Monday July 11, 2016, 13:30 - 16:30 
Tuesday July 12, 2016, 09:30 - 12:30

''Location''

MC 252

5 King's College Road

Toronto, ON M5S 1A4

''Instructor''

Dr. Ramses van Zon (SciNet)

=====Prerequisites=====
C/C++ and/or Fortran scientific programming; Experience editing and compiling code in a Linux environment.

=====Slides and other material=====

----

===Introduction to the Linux Shell===

====Abstract====
Working with many of the HPC systems in Ontario involves using the Linux/UNIX command line. This provides a very powerful interface, but it can be quite daunting for the uninitiated. In this half-day session, you can become initiated with this course. This hands on session will cover basic commands and scripting, as well as touching on some powerful constructs like regular expressions. It could be a great boon for your productivity!

====Details====
''Time/date''

Monday July 11, 13:30 - 16:30

''Location''

MC 254

5 King's College Road

Toronto, ON M5S 1A4

''Instructor''

Dr. Mike Nolta (SciNet)

====Prerequisites====
A laptop with a linux-like environment.

====Slides and other material====

----

===R for Data Science===

=====Abstract=====
This session offers a brief introduction to R, with a focus on data analysis and statistics.

=====Details=====

''Date/time''

Tuesday July 12, 2016, 09:30 - 12:30

''Location''

MC 254

5 King's College Road

Toronto, ON M5S 1A4

''Instructor''

Dr. Erik Spence (SciNet)

=====Prerequisites=====

Some programming experience in any language. You should have used and written functions. Bring a laptop with R or Rstudio installed.

=====Slides and other material=====

----

===Programming Clusters with MPI===

=====Abstract=====
In this 1.5-day session, through lectures interspersed with hands-on labs, the students will learn the basics of MPI programming. Examples and exercises will be based on parallelization of common scientific computing problems.

=====Details=====
''Time/date''

Tuesday July 12, 2016, 13:30 - 16:30 
Wednesday July 16, 2016, 09:30 - 12:30 
Wednesday July 16, 2016, 13:30 - 16:30 

''Location''

MC 252

5 King's College Road

Toronto, ON M5S 1A4

''Instructor''

Dr. Scott Northrup (SciNet)

=====Prerequisites=====
C and/or Fortran scientific programming; experience editing and compiling code in a Linux environment.

=====Slides and other material=====

----

===Parallel R for Data Science===

=====Abstract=====
This half-day session will cover parallel programming in R, with a focus on parallel data analysis. Topics covered will include snow, parallel, and foreach/doparallel.

=====Details=====

''Time/date''

Tuesday July 12, 2016, 13:30 - 16:30

''Location''

MC 252

5 King's College Road

Toronto, ON M5S 1A4

''Instructor''

Dr. Erik Spence (SciNet)

=====Prerequisites=====
Experience with programming in R and a rudimentary understanding of parallel computing.

=====Slides and other material=====
[https://wiki.scinet.utoronto.ca/wiki/images/a/ac/Parallel-R.pdf Slides].

----

===Scientific Computing with Python===

=====Abstract=====
In this session, you will be taught how to use python for research computing purposes. Topics include: the basics of python, automation, numpy, scipy, file i/o, and visualization.

=====Details=====
''Time/date'''

Tuesday July 14, 9:30 - 12:30

''Location''

Galbraith Building, Room 405

35 St. George Street

Toronto, ON M5S 1A4

''Instructor''

Dr. Erik Spence (SciNet)

=====Prerequisites=====
A bit of coding experience in any programming or scripting language. Command-line experience in Linux is a plus.

=====Slides and other material=====
[https://wiki.scinet.utoronto.ca/wiki/images/f/f3/Intro-Python.pdf Slides].

----

===Python for High Performance Computing===

=====Abstract=====
This session will cover parallel programming in python, with a focus on parallel data analysis. We will cover subprocess, multiprocessing, pypar and other parallel-enabling python packages.

=====Details=====
''Time/date''

Tuesday July 14, 13:30 - 16:30

''Location''

Galbraith Building, Room 405

35 St. George Street

Toronto, ON M5S 1A4

''Instructor''

Dr. Erik Spence (SciNet)

=====Prerequisites=====
Python programming experience and a rudimentary understanding of parallel computing.

=====Slides and other material=====
[https://wiki.scinet.utoronto.ca/wiki/images/2/2b/Parallel-Python.pdf Slides].

----

===Programming GPUs with CUDA===

=====Abstract=====
This is an introductory course covering programming and computing on GPUs---graphics processing units---which are an increasingly common presence in massively parallel computing architectures. This session will cover both of the available C-like programming frameworks: NVIDIA’s CUDA-C. The basics of GPU programming will be covered, and students will work through a number of hands on examples. Demonstrations of profiling and debugging applications running on the GPU will also be included. The structuring of data and computations that makes full use of the GPU will be discussed in detail. Students should be able to leave the course with the knowledge necessary to begin developing their own GPU applications.

=====Details=====
''Time/date''

Wednesday July 15, 9:30 - 16:30

Thursday July 16, 9:30 - 16:30

''Location''

Galbraith Building, Room 405

35 St. George Street

Toronto, ON M5S 1A4

''Instructors''
Dr. Pawel Pomorski and Dr. Sergey Mashchenko (SHARCNET)

=====Prerequisites=====
C/C++ scientific programming, experience editing and compiling code in a Linux environment. Some experience with CUDA and/or OpenMP a plus.

=====Slides and other materials=====
''Day 1:''
* [[Media:CUDA_summer_school_2015_central.pdf|Slides]]
* [[Media:CUDA_day1.tar.gz|Code]]
See also [http://ppomorsk.sharcnet.ca http://ppomorsk.sharcnet.ca]

''Day 2:''
* [[Media:CUDA2.pdf|Slides]]

----

===Visualization===

=====Abstract=====
The first part of this session will be a basic introduction to general-purpose scientific visualization tools such as gnuplot, grace(xmgr), remote tools (tunneling, VNC), python's matplotlib, and examples from other. The second part will be a mix of lecture and hands-on to introduce more specialized scientific visualization software such as vmd, paraview, and visit.

=====Details=====
''Time/date''

Friday July 17, 9:30 - 16:30

''Location''

Galbraith Building, Room 404

35 St. George Street

Toronto, ON M5S 1A4

''Instructor''
Dr. Marcelo Ponce (SciNet)

=====Prerequisites=====
Software installed, for the morning session: gnuplot, ssh (graphic client), vnc client, python and matplotlib; afternoon session: ParaView & VisIt.

=====Slides and other material=====
Morning session: [http://www.scinethpc.ca/~mponce/ss2015/ss2015_visualization-morning.pdf Visualization I]

Afternoon session: [http://www.scinethpc.ca/~mponce/ss2015/ss2015_visualization-afternoon.pdf Visualization II]

Samples to use during the tutorial: [http://www.scinethpc.ca/~mponce/ss2015/data.tar.gz Data sets]

An example of a complete visualization session in VisIt: [http://www.scinethpc.ca/~mponce/ss2015/noise-example_VisIt.tar.gz VisIt session] (un-tar this file and load the session from the File/Menu in VisIt using "Restore session with sources...", then choose your local "noise.silo" file to be used within the session).
----

===Debugging, Optimization, and Best Practices===

=====Abstract=====
Debugging is an important step in developing a new code, or porting an old one to a new machine. We will discuss the debugging of frequently encountered bugs in serial code with gdb and the debugging of parallel (MPI and threaded) codes with DDT on live systems. Next, we will present general issues, common pitfalls and optimization strategies that are application to HPC systems. By using profiling tools available for parallel programs, we can see when the bottlenecks are. We can try to alleviate them by following common best practices, allowing one to obtain their results more quickly and at the same time minimize their burden on shared system resources. Walk-through examples and case studies will be presented during the session to illustrate the concepts.

=====Details=====
''Date/time''

Friday July 17, 9:30 - 16:30

''Location''

Galbraith Building, Room 405

35 St. George Street

Toronto, ON M5S 1A4

''Instructors''

Dr. Ramses van Zon and Dr. Mike Nolta (SciNet)

=====Prerequisites=====
Basic familiarity with a high-level language (C/C++/Fortran) as well as compiling and running programs via the command line environment on a HPC system.

=====Slides and other material=====
* [[Media:DebugSS2015.pdf | Slides for the debugging session]]
* [[Media:DebugSS2015code.tar.gz | Debuggable code ]]
* [[Media:SSHPC_OBP2013.pdf|Slides for the best practices session]]

===Group Photo===

[[File:GroupPhotoSS2015.jpg|800px]]

2016 Ontario Summer School for High Performance Computing Central

2016-07-06T14:42:42Z

Ejspence:

===About===

The 2016 Ontario Summer School on High Performance Computing, whose second installment of the year will be hosted by SciNet, provides attendees with opportunities to learn and share knowledge and experience in high performance and technical computing. The format: a five-day workshop with mixed lectures and hands-on sessions on a number of selected subjects, including shared memory programming, distributed memory programming, parallel data processing, and general purpose graphics processing unit programming.

Below are descriptions and course material for the Central version, given from July 11th to July 15th, 2016, in Toronto.

__TOC__

===Introduction to HPC and SciNet===

=====Abstract=====
This half-day session will provide an introduction to basic concepts of high-performance computing. It is intended to be a high level primer for those largely new to HPC, and serve as a foundation upon which to build over the coming days. Topics will include motivation for HPC, essential issues, problem characteristics as they apply to parallelism and a high level overview of parallel programming models. Strategies of running large sets of serial processes using e.g. GNU parallel, will also be presented.

=====Details=====
''Time/date''

Monday July 11, 2016, 09:30 - 12:30

''Location''

MC 252

5 King's College Road

Toronto, ON M5S 1A4

''Instructor''

Dr. Mike Nolta (SciNet)

=====Prerequisites=====
Basic Linux skills are useful to follow along. If you do not have these, consider taking the "Intro to the Linux Shell" in the afternoon.

=====Slides and other material=====

----

===Programming Shared Memory Systems with OpenMP===

=====Abstract=====
In these two half-day sessions, lectures and hands-on labs are interspersed, and the students will learn the basics of shared memory programming with OpenMP. In particular, we will discuss the OpenMP's threads, memory, and performance, reductions and load balancing. We will also discuss extensions to heterogeneous systems such as offered by the OpenMP 4.0 and OpenACC standards.

=====Details=====

''Time/date''

Monday July 11, 2016, 09:30 - 12:30

''Location''

MC 252

5 King's College Road

Toronto, ON M5S 1A4

''Instructor''

Dr. Ramses van Zon (SciNet)

=====Prerequisites=====
C/C++ and/or Fortran scientific programming; Experience editing and compiling code in a Linux environment.

=====Slides and other material=====

----

===Introduction to the Linux Shell===

====Abstract====

Working with many of the HPC systems in Ontario involves using the linux/unix command line. This provides a very powerful interface, but it can be quite daunting for the uninitiated. Become initiated with this course. This hands on session will cover basic commands and scripting, as well as touching on some powerful constructs like regular expressions. It could be a great boon for your productivity!

====Details====
''Time/date''

Monday July 13, 9:30 - 12:30

''Location''

Galbraith Building, Room 404

35 St. George Street

Toronto, ON M5S 1A4

''Instructor''

Dr. Mike Nolta (SciNet)

====Prerequisites====
A laptop with a linux-like environment.

====Slides and other material====
* [[Media:SS2015_Introduction_to_the_Unix_Shell.pdf|Slides]]

----

===Data Analysis with R===

=====Abstract=====
This session offers a brief introduction to R, with a focus on data analysis and statistics.

=====Details=====

''Date/time''

Monday July 13, 9:30 - 12:30

''Location''

Galbraith Building, Room 405

35 St. George Street

Toronto, ON M5S 1A4

''Instructor''

Dr. Erik Spence (SciNet)

=====Prerequisites=====

Some programming experience in any language. You should have used and written functions. Bring a laptop with a R and Rstudio installed.

=====Slides and other material=====
[https://wiki.scinet.utoronto.ca/wiki/images/7/73/Intro-R.pdf Slides].

----

===Parallel R===

=====Abstract=====
This session will cover parallel programming R, with a focus on parallel data analysis. Topics covered include snow, parallel, and foreach/doparallel.

=====Details=====

''Time/date''

Monday July 13, 13:30 - 16:30

''Location''

Galbraith Building, Room 405

35 St. George Street

Toronto, ON M5S 1A4

''Instructor''

Dr. Erik Spence (SciNet)

=====Prerequisites=====
Experience with programming in R and a rudimentary understanding of parallel computing.

=====Slides and other material=====
[https://wiki.scinet.utoronto.ca/wiki/images/a/ac/Parallel-R.pdf Slides].

----

===Scientific Computing with Python===

=====Abstract=====
In this session, you will be taught how to use python for research computing purposes. Topics include: the basics of python, automation, numpy, scipy, file i/o, and visualization.

=====Details=====
''Time/date'''

Tuesday July 14, 9:30 - 12:30

''Location''

Galbraith Building, Room 405

35 St. George Street

Toronto, ON M5S 1A4

''Instructor''

Dr. Erik Spence (SciNet)

=====Prerequisites=====
A bit of coding experience in any programming or scripting language. Command-line experience in Linux is a plus.

=====Slides and other material=====
[https://wiki.scinet.utoronto.ca/wiki/images/f/f3/Intro-Python.pdf Slides].

----

===Python for High Performance Computing===

=====Abstract=====
This session will cover parallel programming in python, with a focus on parallel data analysis. We will cover subprocess, multiprocessing, pypar and other parallel-enabling python packages.

=====Details=====
''Time/date''

Tuesday July 14, 13:30 - 16:30

''Location''

Galbraith Building, Room 405

35 St. George Street

Toronto, ON M5S 1A4

''Instructor''

Dr. Erik Spence (SciNet)

=====Prerequisites=====
Python programming experience and a rudimentary understanding of parallel computing.

=====Slides and other material=====
[https://wiki.scinet.utoronto.ca/wiki/images/2/2b/Parallel-Python.pdf Slides].

----

===Programming Clusters with MPI===

=====Abstract=====
In this two-day session, through lectures interspersed with hands-on labs, the students will learn the basics of MPI programming. Examples and exercises will be based on parallelization of common scientific computing problems.

=====Details=====
''Time/date''

Wednesday July 15, 9:30 - 16:30

Thursday July 16, 9:30 - 16:30

''Location''

Galbraith Building, Room 404

35 St. George Street

Toronto, ON M5S 1A4

''Instructor''

Dr. Scott Northrup (SciNet)

=====Prerequisites=====
Programming experience with Fortran or C/C++.

=====Slides and other material=====
The slides for this class can be found [http://support.scinet.utoronto.ca/~northrup/SS2015 here].

----

===Programming GPUs with CUDA===

=====Abstract=====
This is an introductory course covering programming and computing on GPUs---graphics processing units---which are an increasingly common presence in massively parallel computing architectures. This session will cover both of the available C-like programming frameworks: NVIDIA’s CUDA-C. The basics of GPU programming will be covered, and students will work through a number of hands on examples. Demonstrations of profiling and debugging applications running on the GPU will also be included. The structuring of data and computations that makes full use of the GPU will be discussed in detail. Students should be able to leave the course with the knowledge necessary to begin developing their own GPU applications.

=====Details=====
''Time/date''

Wednesday July 15, 9:30 - 16:30

Thursday July 16, 9:30 - 16:30

''Location''

Galbraith Building, Room 405

35 St. George Street

Toronto, ON M5S 1A4

''Instructors''
Dr. Pawel Pomorski and Dr. Sergey Mashchenko (SHARCNET)

=====Prerequisites=====
C/C++ scientific programming, experience editing and compiling code in a Linux environment. Some experience with CUDA and/or OpenMP a plus.

=====Slides and other materials=====
''Day 1:''
* [[Media:CUDA_summer_school_2015_central.pdf|Slides]]
* [[Media:CUDA_day1.tar.gz|Code]]
See also [http://ppomorsk.sharcnet.ca http://ppomorsk.sharcnet.ca]

''Day 2:''
* [[Media:CUDA2.pdf|Slides]]

----

===Visualization===

=====Abstract=====
The first part of this session will be a basic introduction to general-purpose scientific visualization tools such as gnuplot, grace(xmgr), remote tools (tunneling, VNC), python's matplotlib, and examples from other. The second part will be a mix of lecture and hands-on to introduce more specialized scientific visualization software such as vmd, paraview, and visit.

=====Details=====
''Time/date''

Friday July 17, 9:30 - 16:30

''Location''

Galbraith Building, Room 404

35 St. George Street

Toronto, ON M5S 1A4

''Instructor''
Dr. Marcelo Ponce (SciNet)

=====Prerequisites=====
Software installed, for the morning session: gnuplot, ssh (graphic client), vnc client, python and matplotlib; afternoon session: ParaView & VisIt.

=====Slides and other material=====
Morning session: [http://www.scinethpc.ca/~mponce/ss2015/ss2015_visualization-morning.pdf Visualization I]

Afternoon session: [http://www.scinethpc.ca/~mponce/ss2015/ss2015_visualization-afternoon.pdf Visualization II]

Samples to use during the tutorial: [http://www.scinethpc.ca/~mponce/ss2015/data.tar.gz Data sets]

An example of a complete visualization session in VisIt: [http://www.scinethpc.ca/~mponce/ss2015/noise-example_VisIt.tar.gz VisIt session] (un-tar this file and load the session from the File/Menu in VisIt using "Restore session with sources...", then choose your local "noise.silo" file to be used within the session).
----

===Debugging, Optimization, and Best Practices===

=====Abstract=====
Debugging is an important step in developing a new code, or porting an old one to a new machine. We will discuss the debugging of frequently encountered bugs in serial code with gdb and the debugging of parallel (MPI and threaded) codes with DDT on live systems. Next, we will present general issues, common pitfalls and optimization strategies that are application to HPC systems. By using profiling tools available for parallel programs, we can see when the bottlenecks are. We can try to alleviate them by following common best practices, allowing one to obtain their results more quickly and at the same time minimize their burden on shared system resources. Walk-through examples and case studies will be presented during the session to illustrate the concepts.

=====Details=====
''Date/time''

Friday July 17, 9:30 - 16:30

''Location''

Galbraith Building, Room 405

35 St. George Street

Toronto, ON M5S 1A4

''Instructors''

Dr. Ramses van Zon and Dr. Mike Nolta (SciNet)

=====Prerequisites=====
Basic familiarity with a high-level language (C/C++/Fortran) as well as compiling and running programs via the command line environment on a HPC system.

=====Slides and other material=====
* [[Media:DebugSS2015.pdf | Slides for the debugging session]]
* [[Media:DebugSS2015code.tar.gz | Debuggable code ]]
* [[Media:SSHPC_OBP2013.pdf|Slides for the best practices session]]

===Group Photo===

[[File:GroupPhotoSS2015.jpg|800px]]

2016 Ontario Summer School for High Performance Computing Central

2016-07-06T14:35:35Z

Ejspence:

===About===

The 2016 Ontario Summer School on High Performance Computing, whose second installment of the year will be hosted by SciNet, provides attendees with opportunities to learn and share knowledge and experience in high performance and technical computing. The format: a five-day workshop with mixed lectures and hands-on sessions on a number of selected subjects, including shared memory programming, distributed memory programming, parallel data processing, and general purpose graphics processing unit programming.

Below are descriptions and course material for the Central version, given from July 11th to July 15th, 2016, in Toronto.

__TOC__

===Introduction to the Linux Shell===

====Abstract====

Working with many of the HPC systems in Ontario involves using the linux/unix command line. This provides a very powerful interface, but it can be quite daunting for the uninitiated. Become initiated with this course. This hands on session will cover basic commands and scripting, as well as touching on some powerful constructs like regular expressions. It could be a great boon for your productivity!

====Details====
''Time/date''

Monday July 13, 9:30 - 12:30

''Location''

Galbraith Building, Room 404

35 St. George Street

Toronto, ON M5S 1A4

''Instructor''

Dr. Mike Nolta (SciNet)

====Prerequisites====
A laptop with a linux-like environment.

====Slides and other material====
* [[Media:SS2015_Introduction_to_the_Unix_Shell.pdf|Slides]]

----

===Introduction to High Performance Computing===

=====Abstract=====
This session will provide an introduction to basic concepts of high performance computing. It is intended to be a high level primer for those largely new to HPC, and serve as a foundation upon which to build over the coming days. Topics will include motivation for HPC, essential issues, problem characteristics as they apply to parallelism and a high level overview of parallel programming models. Strategies of running large sets of serial processes using e.g. GNU parallel, will also be presented.

=====Details=====
''Time/date''

Monday July 13, 13:30 - 16:30

''Location''

Galbraith Building, Room 404

35 St. George Street

Toronto, ON M5S 1A4

''Instructor''

Dr. Marcelo Ponce (SciNet)

=====Prerequisites=====
Basic Linux shell skills Registration for all summer school sessions will become available at https://www.sharcnet.ca/events/ss2015.

=====Slides and other material=====
[http://www.scinethpc.ca/~mponce/ss2015/HPC_intro-2015.pdf Slides]

----

===Data Analysis with R===

=====Abstract=====
This session offers a brief introduction to R, with a focus on data analysis and statistics.

=====Details=====

''Date/time''

Monday July 13, 9:30 - 12:30

''Location''

Galbraith Building, Room 405

35 St. George Street

Toronto, ON M5S 1A4

''Instructor''

Dr. Erik Spence (SciNet)

=====Prerequisites=====

Some programming experience in any language. You should have used and written functions. Bring a laptop with a R and Rstudio installed.

=====Slides and other material=====
[https://wiki.scinet.utoronto.ca/wiki/images/7/73/Intro-R.pdf Slides].

----

===Parallel R===

=====Abstract=====
This session will cover parallel programming R, with a focus on parallel data analysis. Topics covered include snow, parallel, and foreach/doparallel.

=====Details=====

''Time/date''

Monday July 13, 13:30 - 16:30

''Location''

Galbraith Building, Room 405

35 St. George Street

Toronto, ON M5S 1A4

''Instructor''

Dr. Erik Spence (SciNet)

=====Prerequisites=====
Experience with programming in R and a rudimentary understanding of parallel computing.

=====Slides and other material=====
[https://wiki.scinet.utoronto.ca/wiki/images/a/ac/Parallel-R.pdf Slides].

----

===Programming Shared Memory Systems with OpenMP===

=====Abstract=====
In this session, in which lectures and hands-on labs are interspersed, the students will learn the basics of shared memory programming with OpenMP. In particular, we will discuss the OpenMP's threads, memory, and performance, reductions and load balancing. We will also discuss extensions to heterogeneous systems such as offered by the OpenMP 4.0 and OpenACC standards.

=====Details=====

''Time/date''

Tuesday July 14, 9:30 - 16:30

''Location''

Galbraith Building, Room 404

35 St. George Street

Toronto, ON M5S 1A4

''Instructor''

Dr. Ramses van Zon (SciNet)

=====Prerequisites=====
C/C++ and/or Fortran scientific programming; Experience editing and compiling code in a Linux environment.

=====Slides and other material=====
* [[Media:OpenMP_SS2015.pdf|Slides]]
* [[Media:OpenMP_SS2015_code.tar.gz|Code]]

----

===Scientific Computing with Python===

=====Abstract=====
In this session, you will be taught how to use python for research computing purposes. Topics include: the basics of python, automation, numpy, scipy, file i/o, and visualization.

=====Details=====
''Time/date'''

Tuesday July 14, 9:30 - 12:30

''Location''

Galbraith Building, Room 405

35 St. George Street

Toronto, ON M5S 1A4

''Instructor''

Dr. Erik Spence (SciNet)

=====Prerequisites=====
A bit of coding experience in any programming or scripting language. Command-line experience in Linux is a plus.

=====Slides and other material=====
[https://wiki.scinet.utoronto.ca/wiki/images/f/f3/Intro-Python.pdf Slides].

----

===Python for High Performance Computing===

=====Abstract=====
This session will cover parallel programming in python, with a focus on parallel data analysis. We will cover subprocess, multiprocessing, pypar and other parallel-enabling python packages.

=====Details=====
''Time/date''

Tuesday July 14, 13:30 - 16:30

''Location''

Galbraith Building, Room 405

35 St. George Street

Toronto, ON M5S 1A4

''Instructor''

Dr. Erik Spence (SciNet)

=====Prerequisites=====
Python programming experience and a rudimentary understanding of parallel computing.

=====Slides and other material=====
[https://wiki.scinet.utoronto.ca/wiki/images/2/2b/Parallel-Python.pdf Slides].

----

===Programming Clusters with MPI===

=====Abstract=====
In this two-day session, through lectures interspersed with hands-on labs, the students will learn the basics of MPI programming. Examples and exercises will be based on parallelization of common scientific computing problems.

=====Details=====
''Time/date''

Wednesday July 15, 9:30 - 16:30

Thursday July 16, 9:30 - 16:30

''Location''

Galbraith Building, Room 404

35 St. George Street

Toronto, ON M5S 1A4

''Instructor''

Dr. Scott Northrup (SciNet)

=====Prerequisites=====
Programming experience with Fortran or C/C++.

=====Slides and other material=====
The slides for this class can be found [http://support.scinet.utoronto.ca/~northrup/SS2015 here].

----

===Programming GPUs with CUDA===

=====Abstract=====
This is an introductory course covering programming and computing on GPUs---graphics processing units---which are an increasingly common presence in massively parallel computing architectures. This session will cover both of the available C-like programming frameworks: NVIDIA’s CUDA-C. The basics of GPU programming will be covered, and students will work through a number of hands on examples. Demonstrations of profiling and debugging applications running on the GPU will also be included. The structuring of data and computations that makes full use of the GPU will be discussed in detail. Students should be able to leave the course with the knowledge necessary to begin developing their own GPU applications.

=====Details=====
''Time/date''

Wednesday July 15, 9:30 - 16:30

Thursday July 16, 9:30 - 16:30

''Location''

Galbraith Building, Room 405

35 St. George Street

Toronto, ON M5S 1A4

''Instructors''
Dr. Pawel Pomorski and Dr. Sergey Mashchenko (SHARCNET)

=====Prerequisites=====
C/C++ scientific programming, experience editing and compiling code in a Linux environment. Some experience with CUDA and/or OpenMP a plus.

=====Slides and other materials=====
''Day 1:''
* [[Media:CUDA_summer_school_2015_central.pdf|Slides]]
* [[Media:CUDA_day1.tar.gz|Code]]
See also [http://ppomorsk.sharcnet.ca http://ppomorsk.sharcnet.ca]

''Day 2:''
* [[Media:CUDA2.pdf|Slides]]

----

===Visualization===

=====Abstract=====
The first part of this session will be a basic introduction to general-purpose scientific visualization tools such as gnuplot, grace(xmgr), remote tools (tunneling, VNC), python's matplotlib, and examples from other. The second part will be a mix of lecture and hands-on to introduce more specialized scientific visualization software such as vmd, paraview, and visit.

=====Details=====
''Time/date''

Friday July 17, 9:30 - 16:30

''Location''

Galbraith Building, Room 404

35 St. George Street

Toronto, ON M5S 1A4

''Instructor''
Dr. Marcelo Ponce (SciNet)

=====Prerequisites=====
Software installed, for the morning session: gnuplot, ssh (graphic client), vnc client, python and matplotlib; afternoon session: ParaView & VisIt.

=====Slides and other material=====
Morning session: [http://www.scinethpc.ca/~mponce/ss2015/ss2015_visualization-morning.pdf Visualization I]

Afternoon session: [http://www.scinethpc.ca/~mponce/ss2015/ss2015_visualization-afternoon.pdf Visualization II]

Samples to use during the tutorial: [http://www.scinethpc.ca/~mponce/ss2015/data.tar.gz Data sets]

An example of a complete visualization session in VisIt: [http://www.scinethpc.ca/~mponce/ss2015/noise-example_VisIt.tar.gz VisIt session] (un-tar this file and load the session from the File/Menu in VisIt using "Restore session with sources...", then choose your local "noise.silo" file to be used within the session).
----

===Debugging, Optimization, and Best Practices===

=====Abstract=====
Debugging is an important step in developing a new code, or porting an old one to a new machine. We will discuss the debugging of frequently encountered bugs in serial code with gdb and the debugging of parallel (MPI and threaded) codes with DDT on live systems. Next, we will present general issues, common pitfalls and optimization strategies that are application to HPC systems. By using profiling tools available for parallel programs, we can see when the bottlenecks are. We can try to alleviate them by following common best practices, allowing one to obtain their results more quickly and at the same time minimize their burden on shared system resources. Walk-through examples and case studies will be presented during the session to illustrate the concepts.

=====Details=====
''Date/time''

Friday July 17, 9:30 - 16:30

''Location''

Galbraith Building, Room 405

35 St. George Street

Toronto, ON M5S 1A4

''Instructors''

Dr. Ramses van Zon and Dr. Mike Nolta (SciNet)

=====Prerequisites=====
Basic familiarity with a high-level language (C/C++/Fortran) as well as compiling and running programs via the command line environment on a HPC system.

=====Slides and other material=====
* [[Media:DebugSS2015.pdf | Slides for the debugging session]]
* [[Media:DebugSS2015code.tar.gz | Debuggable code ]]
* [[Media:SSHPC_OBP2013.pdf|Slides for the best practices session]]

===Group Photo===

[[File:GroupPhotoSS2015.jpg|800px]]

2016 Ontario Summer School for High Performance Computing Central

2016-07-06T14:34:02Z

Ejspence:

[[File:Summerschool_hpc_2015.gif]]

===About===

The 2016 Ontario Summer School on High Performance Computing, whose second installment of the year will be hosted by SciNet, provides attendees with opportunities to learn and share knowledge and experience in high performance and technical computing. The format: a five-day workshop with mixed lectures and hands-on sessions on a number of selected subjects, including shared memory programming, distributed memory programming, parallel data processing, and general purpose graphics processing unit programming.

Below are descriptions and course material for the Central version, given from July 11th to July 15th, 2016, in Toronto.

__TOC__

===Introduction to the Linux Shell===

====Abstract====

Working with many of the HPC systems in Ontario involves using the linux/unix command line. This provides a very powerful interface, but it can be quite daunting for the uninitiated. Become initiated with this course. This hands on session will cover basic commands and scripting, as well as touching on some powerful constructs like regular expressions. It could be a great boon for your productivity!

====Details====
''Time/date''

Monday July 13, 9:30 - 12:30

''Location''

Galbraith Building, Room 404

35 St. George Street

Toronto, ON M5S 1A4

''Instructor''

Dr. Mike Nolta (SciNet)

====Prerequisites====
A laptop with a linux-like environment.

====Slides and other material====
* [[Media:SS2015_Introduction_to_the_Unix_Shell.pdf|Slides]]

----

===Introduction to High Performance Computing===

=====Abstract=====
This session will provide an introduction to basic concepts of high performance computing. It is intended to be a high level primer for those largely new to HPC, and serve as a foundation upon which to build over the coming days. Topics will include motivation for HPC, essential issues, problem characteristics as they apply to parallelism and a high level overview of parallel programming models. Strategies of running large sets of serial processes using e.g. GNU parallel, will also be presented.

=====Details=====
''Time/date''

Monday July 13, 13:30 - 16:30

''Location''

Galbraith Building, Room 404

35 St. George Street

Toronto, ON M5S 1A4

''Instructor''

Dr. Marcelo Ponce (SciNet)

=====Prerequisites=====
Basic Linux shell skills Registration for all summer school sessions will become available at https://www.sharcnet.ca/events/ss2015.

=====Slides and other material=====
[http://www.scinethpc.ca/~mponce/ss2015/HPC_intro-2015.pdf Slides]

----

===Data Analysis with R===

=====Abstract=====
This session offers a brief introduction to R, with a focus on data analysis and statistics.

=====Details=====

''Date/time''

Monday July 13, 9:30 - 12:30

''Location''

Galbraith Building, Room 405

35 St. George Street

Toronto, ON M5S 1A4

''Instructor''

Dr. Erik Spence (SciNet)

=====Prerequisites=====

Some programming experience in any language. You should have used and written functions. Bring a laptop with a R and Rstudio installed.

=====Slides and other material=====
[https://wiki.scinet.utoronto.ca/wiki/images/7/73/Intro-R.pdf Slides].

----

===Parallel R===

=====Abstract=====
This session will cover parallel programming R, with a focus on parallel data analysis. Topics covered include snow, parallel, and foreach/doparallel.

=====Details=====

''Time/date''

Monday July 13, 13:30 - 16:30

''Location''

Galbraith Building, Room 405

35 St. George Street

Toronto, ON M5S 1A4

''Instructor''

Dr. Erik Spence (SciNet)

=====Prerequisites=====
Experience with programming in R and a rudimentary understanding of parallel computing.

=====Slides and other material=====
[https://wiki.scinet.utoronto.ca/wiki/images/a/ac/Parallel-R.pdf Slides].

----

===Programming Shared Memory Systems with OpenMP===

=====Abstract=====
In this session, in which lectures and hands-on labs are interspersed, the students will learn the basics of shared memory programming with OpenMP. In particular, we will discuss the OpenMP's threads, memory, and performance, reductions and load balancing. We will also discuss extensions to heterogeneous systems such as offered by the OpenMP 4.0 and OpenACC standards.

=====Details=====

''Time/date''

Tuesday July 14, 9:30 - 16:30

''Location''

Galbraith Building, Room 404

35 St. George Street

Toronto, ON M5S 1A4

''Instructor''

Dr. Ramses van Zon (SciNet)

=====Prerequisites=====
C/C++ and/or Fortran scientific programming; Experience editing and compiling code in a Linux environment.

=====Slides and other material=====
* [[Media:OpenMP_SS2015.pdf|Slides]]
* [[Media:OpenMP_SS2015_code.tar.gz|Code]]

----

===Scientific Computing with Python===

=====Abstract=====
In this session, you will be taught how to use python for research computing purposes. Topics include: the basics of python, automation, numpy, scipy, file i/o, and visualization.

=====Details=====
''Time/date'''

Tuesday July 14, 9:30 - 12:30

''Location''

Galbraith Building, Room 405

35 St. George Street

Toronto, ON M5S 1A4

''Instructor''

Dr. Erik Spence (SciNet)

=====Prerequisites=====
A bit of coding experience in any programming or scripting language. Command-line experience in Linux is a plus.

=====Slides and other material=====
[https://wiki.scinet.utoronto.ca/wiki/images/f/f3/Intro-Python.pdf Slides].

----

===Python for High Performance Computing===

=====Abstract=====
This session will cover parallel programming in python, with a focus on parallel data analysis. We will cover subprocess, multiprocessing, pypar and other parallel-enabling python packages.

=====Details=====
''Time/date''

Tuesday July 14, 13:30 - 16:30

''Location''

Galbraith Building, Room 405

35 St. George Street

Toronto, ON M5S 1A4

''Instructor''

Dr. Erik Spence (SciNet)

=====Prerequisites=====
Python programming experience and a rudimentary understanding of parallel computing.

=====Slides and other material=====
[https://wiki.scinet.utoronto.ca/wiki/images/2/2b/Parallel-Python.pdf Slides].

----

===Programming Clusters with MPI===

=====Abstract=====
In this two-day session, through lectures interspersed with hands-on labs, the students will learn the basics of MPI programming. Examples and exercises will be based on parallelization of common scientific computing problems.

=====Details=====
''Time/date''

Wednesday July 15, 9:30 - 16:30

Thursday July 16, 9:30 - 16:30

''Location''

Galbraith Building, Room 404

35 St. George Street

Toronto, ON M5S 1A4

''Instructor''

Dr. Scott Northrup (SciNet)

=====Prerequisites=====
Programming experience with Fortran or C/C++.

=====Slides and other material=====
The slides for this class can be found [http://support.scinet.utoronto.ca/~northrup/SS2015 here].

----

===Programming GPUs with CUDA===

=====Abstract=====
This is an introductory course covering programming and computing on GPUs---graphics processing units---which are an increasingly common presence in massively parallel computing architectures. This session will cover both of the available C-like programming frameworks: NVIDIA’s CUDA-C. The basics of GPU programming will be covered, and students will work through a number of hands on examples. Demonstrations of profiling and debugging applications running on the GPU will also be included. The structuring of data and computations that makes full use of the GPU will be discussed in detail. Students should be able to leave the course with the knowledge necessary to begin developing their own GPU applications.

=====Details=====
''Time/date''

Wednesday July 15, 9:30 - 16:30

Thursday July 16, 9:30 - 16:30

''Location''

Galbraith Building, Room 405

35 St. George Street

Toronto, ON M5S 1A4

''Instructors''
Dr. Pawel Pomorski and Dr. Sergey Mashchenko (SHARCNET)

=====Prerequisites=====
C/C++ scientific programming, experience editing and compiling code in a Linux environment. Some experience with CUDA and/or OpenMP a plus.

=====Slides and other materials=====
''Day 1:''
* [[Media:CUDA_summer_school_2015_central.pdf|Slides]]
* [[Media:CUDA_day1.tar.gz|Code]]
See also [http://ppomorsk.sharcnet.ca http://ppomorsk.sharcnet.ca]

''Day 2:''
* [[Media:CUDA2.pdf|Slides]]

----

===Visualization===

=====Abstract=====
The first part of this session will be a basic introduction to general-purpose scientific visualization tools such as gnuplot, grace(xmgr), remote tools (tunneling, VNC), python's matplotlib, and examples from other. The second part will be a mix of lecture and hands-on to introduce more specialized scientific visualization software such as vmd, paraview, and visit.

=====Details=====
''Time/date''

Friday July 17, 9:30 - 16:30

''Location''

Galbraith Building, Room 404

35 St. George Street

Toronto, ON M5S 1A4

''Instructor''
Dr. Marcelo Ponce (SciNet)

=====Prerequisites=====
Software installed, for the morning session: gnuplot, ssh (graphic client), vnc client, python and matplotlib; afternoon session: ParaView & VisIt.

=====Slides and other material=====
Morning session: [http://www.scinethpc.ca/~mponce/ss2015/ss2015_visualization-morning.pdf Visualization I]

Afternoon session: [http://www.scinethpc.ca/~mponce/ss2015/ss2015_visualization-afternoon.pdf Visualization II]

Samples to use during the tutorial: [http://www.scinethpc.ca/~mponce/ss2015/data.tar.gz Data sets]

An example of a complete visualization session in VisIt: [http://www.scinethpc.ca/~mponce/ss2015/noise-example_VisIt.tar.gz VisIt session] (un-tar this file and load the session from the File/Menu in VisIt using "Restore session with sources...", then choose your local "noise.silo" file to be used within the session).
----

===Debugging, Optimization, and Best Practices===

=====Abstract=====
Debugging is an important step in developing a new code, or porting an old one to a new machine. We will discuss the debugging of frequently encountered bugs in serial code with gdb and the debugging of parallel (MPI and threaded) codes with DDT on live systems. Next, we will present general issues, common pitfalls and optimization strategies that are application to HPC systems. By using profiling tools available for parallel programs, we can see when the bottlenecks are. We can try to alleviate them by following common best practices, allowing one to obtain their results more quickly and at the same time minimize their burden on shared system resources. Walk-through examples and case studies will be presented during the session to illustrate the concepts.

=====Details=====
''Date/time''

Friday July 17, 9:30 - 16:30

''Location''

Galbraith Building, Room 405

35 St. George Street

Toronto, ON M5S 1A4

''Instructors''

Dr. Ramses van Zon and Dr. Mike Nolta (SciNet)

=====Prerequisites=====
Basic familiarity with a high-level language (C/C++/Fortran) as well as compiling and running programs via the command line environment on a HPC system.

=====Slides and other material=====
* [[Media:DebugSS2015.pdf | Slides for the debugging session]]
* [[Media:DebugSS2015code.tar.gz | Debuggable code ]]
* [[Media:SSHPC_OBP2013.pdf|Slides for the best practices session]]

===Group Photo===

[[File:GroupPhotoSS2015.jpg|800px]]

2016 Ontario Summer School for High Performance Computing Central

2016-07-06T14:31:20Z

Ejspence: Created page with "File:Summerschool_hpc_2015.gif ===About=== The 2015 Ontario Summer School on High Performance Computing, whose second installment of the year will be hosted by SciNet, ..."

[[File:Summerschool_hpc_2015.gif]]

===About===

The 2015 Ontario Summer School on High Performance Computing, whose second installment of the year will be hosted by SciNet, provides attendees with opportunities to learn and share knowledge and experience in high performance and technical computing. The format: a five-day workshop with mixed lectures and hands-on sessions on a number of selected subjects, including shared memory programming, distributed memory programming, parallel data processing, and general purpose graphics processing unit programming.

Below are descriptions and course material for the Central version, given from July 13th to July 17th, 2015, in Toronto.

__TOC__

===Introduction to the Linux Shell===

====Abstract====

Working with many of the HPC systems in Ontario involves using the linux/unix command line. This provides a very powerful interface, but it can be quite daunting for the uninitiated. Become initiated with this course. This hands on session will cover basic commands and scripting, as well as touching on some powerful constructs like regular expressions. It could be a great boon for your productivity!

====Details====
''Time/date''

Monday July 13, 9:30 - 12:30

''Location''

Galbraith Building, Room 404

35 St. George Street

Toronto, ON M5S 1A4

''Instructor''

Dr. Mike Nolta (SciNet)

====Prerequisites====
A laptop with a linux-like environment.

====Slides and other material====
* [[Media:SS2015_Introduction_to_the_Unix_Shell.pdf|Slides]]

----

===Introduction to High Performance Computing===

=====Abstract=====
This session will provide an introduction to basic concepts of high performance computing. It is intended to be a high level primer for those largely new to HPC, and serve as a foundation upon which to build over the coming days. Topics will include motivation for HPC, essential issues, problem characteristics as they apply to parallelism and a high level overview of parallel programming models. Strategies of running large sets of serial processes using e.g. GNU parallel, will also be presented.

=====Details=====
''Time/date''

Monday July 13, 13:30 - 16:30

''Location''

Galbraith Building, Room 404

35 St. George Street

Toronto, ON M5S 1A4

''Instructor''

Dr. Marcelo Ponce (SciNet)

=====Prerequisites=====
Basic Linux shell skills Registration for all summer school sessions will become available at https://www.sharcnet.ca/events/ss2015.

=====Slides and other material=====
[http://www.scinethpc.ca/~mponce/ss2015/HPC_intro-2015.pdf Slides]

----

===Data Analysis with R===

=====Abstract=====
This session offers a brief introduction to R, with a focus on data analysis and statistics.

=====Details=====

''Date/time''

Monday July 13, 9:30 - 12:30

''Location''

Galbraith Building, Room 405

35 St. George Street

Toronto, ON M5S 1A4

''Instructor''

Dr. Erik Spence (SciNet)

=====Prerequisites=====

Some programming experience in any language. You should have used and written functions. Bring a laptop with a R and Rstudio installed.

=====Slides and other material=====
[https://wiki.scinet.utoronto.ca/wiki/images/7/73/Intro-R.pdf Slides].

----

===Parallel R===

=====Abstract=====
This session will cover parallel programming R, with a focus on parallel data analysis. Topics covered include snow, parallel, and foreach/doparallel.

=====Details=====

''Time/date''

Monday July 13, 13:30 - 16:30

''Location''

Galbraith Building, Room 405

35 St. George Street

Toronto, ON M5S 1A4

''Instructor''

Dr. Erik Spence (SciNet)

=====Prerequisites=====
Experience with programming in R and a rudimentary understanding of parallel computing.

=====Slides and other material=====
[https://wiki.scinet.utoronto.ca/wiki/images/a/ac/Parallel-R.pdf Slides].

----

===Programming Shared Memory Systems with OpenMP===

=====Abstract=====
In this session, in which lectures and hands-on labs are interspersed, the students will learn the basics of shared memory programming with OpenMP. In particular, we will discuss the OpenMP's threads, memory, and performance, reductions and load balancing. We will also discuss extensions to heterogeneous systems such as offered by the OpenMP 4.0 and OpenACC standards.

=====Details=====

''Time/date''

Tuesday July 14, 9:30 - 16:30

''Location''

Galbraith Building, Room 404

35 St. George Street

Toronto, ON M5S 1A4

''Instructor''

Dr. Ramses van Zon (SciNet)

=====Prerequisites=====
C/C++ and/or Fortran scientific programming; Experience editing and compiling code in a Linux environment.

=====Slides and other material=====
* [[Media:OpenMP_SS2015.pdf|Slides]]
* [[Media:OpenMP_SS2015_code.tar.gz|Code]]

----

===Scientific Computing with Python===

=====Abstract=====
In this session, you will be taught how to use python for research computing purposes. Topics include: the basics of python, automation, numpy, scipy, file i/o, and visualization.

=====Details=====
''Time/date'''

Tuesday July 14, 9:30 - 12:30

''Location''

Galbraith Building, Room 405

35 St. George Street

Toronto, ON M5S 1A4

''Instructor''

Dr. Erik Spence (SciNet)

=====Prerequisites=====
A bit of coding experience in any programming or scripting language. Command-line experience in Linux is a plus.

=====Slides and other material=====
[https://wiki.scinet.utoronto.ca/wiki/images/f/f3/Intro-Python.pdf Slides].

----

===Python for High Performance Computing===

=====Abstract=====
This session will cover parallel programming in python, with a focus on parallel data analysis. We will cover subprocess, multiprocessing, pypar and other parallel-enabling python packages.

=====Details=====
''Time/date''

Tuesday July 14, 13:30 - 16:30

''Location''

Galbraith Building, Room 405

35 St. George Street

Toronto, ON M5S 1A4

''Instructor''

Dr. Erik Spence (SciNet)

=====Prerequisites=====
Python programming experience and a rudimentary understanding of parallel computing.

=====Slides and other material=====
[https://wiki.scinet.utoronto.ca/wiki/images/2/2b/Parallel-Python.pdf Slides].

----

===Programming Clusters with MPI===

=====Abstract=====
In this two-day session, through lectures interspersed with hands-on labs, the students will learn the basics of MPI programming. Examples and exercises will be based on parallelization of common scientific computing problems.

=====Details=====
''Time/date''

Wednesday July 15, 9:30 - 16:30

Thursday July 16, 9:30 - 16:30

''Location''

Galbraith Building, Room 404

35 St. George Street

Toronto, ON M5S 1A4

''Instructor''

Dr. Scott Northrup (SciNet)

=====Prerequisites=====
Programming experience with Fortran or C/C++.

=====Slides and other material=====
The slides for this class can be found [http://support.scinet.utoronto.ca/~northrup/SS2015 here].

----

===Programming GPUs with CUDA===

=====Abstract=====
This is an introductory course covering programming and computing on GPUs---graphics processing units---which are an increasingly common presence in massively parallel computing architectures. This session will cover both of the available C-like programming frameworks: NVIDIA’s CUDA-C. The basics of GPU programming will be covered, and students will work through a number of hands on examples. Demonstrations of profiling and debugging applications running on the GPU will also be included. The structuring of data and computations that makes full use of the GPU will be discussed in detail. Students should be able to leave the course with the knowledge necessary to begin developing their own GPU applications.

=====Details=====
''Time/date''

Wednesday July 15, 9:30 - 16:30

Thursday July 16, 9:30 - 16:30

''Location''

Galbraith Building, Room 405

35 St. George Street

Toronto, ON M5S 1A4

''Instructors''
Dr. Pawel Pomorski and Dr. Sergey Mashchenko (SHARCNET)

=====Prerequisites=====
C/C++ scientific programming, experience editing and compiling code in a Linux environment. Some experience with CUDA and/or OpenMP a plus.

=====Slides and other materials=====
''Day 1:''
* [[Media:CUDA_summer_school_2015_central.pdf|Slides]]
* [[Media:CUDA_day1.tar.gz|Code]]
See also [http://ppomorsk.sharcnet.ca http://ppomorsk.sharcnet.ca]

''Day 2:''
* [[Media:CUDA2.pdf|Slides]]

----

===Visualization===

=====Abstract=====
The first part of this session will be a basic introduction to general-purpose scientific visualization tools such as gnuplot, grace(xmgr), remote tools (tunneling, VNC), python's matplotlib, and examples from other. The second part will be a mix of lecture and hands-on to introduce more specialized scientific visualization software such as vmd, paraview, and visit.

=====Details=====
''Time/date''

Friday July 17, 9:30 - 16:30

''Location''

Galbraith Building, Room 404

35 St. George Street

Toronto, ON M5S 1A4

''Instructor''
Dr. Marcelo Ponce (SciNet)

=====Prerequisites=====
Software installed, for the morning session: gnuplot, ssh (graphic client), vnc client, python and matplotlib; afternoon session: ParaView & VisIt.

=====Slides and other material=====
Morning session: [http://www.scinethpc.ca/~mponce/ss2015/ss2015_visualization-morning.pdf Visualization I]

Afternoon session: [http://www.scinethpc.ca/~mponce/ss2015/ss2015_visualization-afternoon.pdf Visualization II]

Samples to use during the tutorial: [http://www.scinethpc.ca/~mponce/ss2015/data.tar.gz Data sets]

An example of a complete visualization session in VisIt: [http://www.scinethpc.ca/~mponce/ss2015/noise-example_VisIt.tar.gz VisIt session] (un-tar this file and load the session from the File/Menu in VisIt using "Restore session with sources...", then choose your local "noise.silo" file to be used within the session).
----

===Debugging, Optimization, and Best Practices===

=====Abstract=====
Debugging is an important step in developing a new code, or porting an old one to a new machine. We will discuss the debugging of frequently encountered bugs in serial code with gdb and the debugging of parallel (MPI and threaded) codes with DDT on live systems. Next, we will present general issues, common pitfalls and optimization strategies that are application to HPC systems. By using profiling tools available for parallel programs, we can see when the bottlenecks are. We can try to alleviate them by following common best practices, allowing one to obtain their results more quickly and at the same time minimize their burden on shared system resources. Walk-through examples and case studies will be presented during the session to illustrate the concepts.

=====Details=====
''Date/time''

Friday July 17, 9:30 - 16:30

''Location''

Galbraith Building, Room 405

35 St. George Street

Toronto, ON M5S 1A4

''Instructors''

Dr. Ramses van Zon and Dr. Mike Nolta (SciNet)

=====Prerequisites=====
Basic familiarity with a high-level language (C/C++/Fortran) as well as compiling and running programs via the command line environment on a HPC system.

=====Slides and other material=====
* [[Media:DebugSS2015.pdf | Slides for the debugging session]]
* [[Media:DebugSS2015code.tar.gz | Debuggable code ]]
* [[Media:SSHPC_OBP2013.pdf|Slides for the best practices session]]

===Group Photo===

[[File:GroupPhotoSS2015.jpg|800px]]

MATLAB

2016-05-24T15:05:39Z

Ejspence: /* Creating a MATLAB executable */

We often get questions about running MATLAB on GPC. With a few exceptions for compilers and debuggers, SciNet does not purchase licenses for commercial software. As such, SciNet does not have a license for MATLAB, nor will it in the future. If users wish to run MATLAB they must supply their own license, or explore alternative options. This page gives information about the options for getting your MATLAB code to run, in recommended order.

== Not using MATLAB ==

Users can attempt to run MATLAB code using the open-source program Octave, accessible through the octave module. Though there are some differences between the two programs, Octave has been designed to interpret MATLAB code and can often be used in place of MATLAB. If your MATLAB code does not use some of the more-fancy MATLAB toolboxes, you may be able to get away with using Octave instead. Be sure to test your implementation in Octave thoroughly before committing to this option.

It is worth observing that, while convenient for prototyping and running on a single workstation, there are reasons to avoid using MATLAB for larger HPC/ARC projects. These include the prohibitive license cost for large-scale work, poor performance at scale, and portability issues. If you can switch to a license-free option, such as Python, it may be worth the effort.

== Creating a MATLAB executable ==

If MATLAB must be used, you may be able to compile your MATLAB code into a stand-alone executable, and run this on a GPC compute node. The version of MATLAB being used will require a compiler license, and the compilation must be done on a Linux machine. Once the compilation is done, the executable can be copied to SciNet, and run using the MATLAB Compiler Runtime (MCR), which can be accessed using the MCR module. The MCR used must be the same version of MATLAB as the compiler. If the version of MCR that you need is not listed among the MCR module versions, contact us and we will install the version which you require.

Here is an example script which uses the MCR:

<source lang="bash">
#!/bin/bash
# MOAB/Torque submission script for SciNet GPC
#
#PBS -l nodes=1:ppn=8,walltime=1:00:00
#PBS -N test_matlab
#PBS -q debug

# load modules (gcc is needed for MCR/2015a)
module load Xlibraries gcc/5.2.0 MCR/2015a

# DIRECTORY TO RUN - $PBS_O_WORKDIR is directory from which the job was submitted
cd $PBS_O_WORKDIR

# Directory for the MCR to use to write temporary files. Use whatever directory you wish.
mkdir -p $SCRATCH/temp
export MCR_CACHE_ROOT=$SCRATCH/temp

# EXECUTION COMMAND (note that the MATLAB script may require that LD_LIBRARY_PATH be added
# to the script arguments). Note that, if the calculations are serial, you must bundle 8 such
# calculations together for production runs!
./run_myscript.sh $MCR:$LD_LIBRARY_PATH
</source>

== Tunnelling to a license server ==

If you have access to a non-SciNet MATLAB license server, and have installed MATLAB in your $HOME directory, you can setup your submission script to access the external license server. The following lines should be added to the beginning of your submission script:

<pre>
PORT=XXX # port number of the license server
LICENSE_IP=AAA.BBB.CCC.DDD # IP address of the license server
ssh gpc01 -L${PORT}:${LICENSE_IP}:${PORT} -N &
</pre>

This last line will tunnel the port from the compute node back to the license server, through gpc01. The port number and IP address of the licence server must be supplied by the system administrator of the license server.

== Using a different Consortium ==

Both [https://www.sharcnet.ca/my/software/show/54 Sharcnet] and [https://www.westgrid.ca/support/software/matlab Westgrid] have purchased different types of MATLAB licenses. Users can contact those consortia if they wish to attempt to run on those systems.

MATLAB

2016-05-24T15:04:47Z

Ejspence: /* Creating a MATLAB executable */

We often get questions about running MATLAB on GPC. With a few exceptions for compilers and debuggers, SciNet does not purchase licenses for commercial software. As such, SciNet does not have a license for MATLAB, nor will it in the future. If users wish to run MATLAB they must supply their own license, or explore alternative options. This page gives information about the options for getting your MATLAB code to run, in recommended order.

== Not using MATLAB ==

Users can attempt to run MATLAB code using the open-source program Octave, accessible through the octave module. Though there are some differences between the two programs, Octave has been designed to interpret MATLAB code and can often be used in place of MATLAB. If your MATLAB code does not use some of the more-fancy MATLAB toolboxes, you may be able to get away with using Octave instead. Be sure to test your implementation in Octave thoroughly before committing to this option.

It is worth observing that, while convenient for prototyping and running on a single workstation, there are reasons to avoid using MATLAB for larger HPC/ARC projects. These include the prohibitive license cost for large-scale work, poor performance at scale, and portability issues. If you can switch to a license-free option, such as Python, it may be worth the effort.

== Creating a MATLAB executable ==

If MATLAB must be used, you may be able to compile your MATLAB code into a stand-alone executable, and run this on a GPC compute node. The version of MATLAB being used will require a compiler license, and the compilation must be done on a Linux machine. Once the compilation is done, the executable can be copied to SciNet, and run using the MATLAB Compiler Runtime (MCR), which can be accessed using the MCR module. The MCR used must be the same version of MATLAB as the compiler. If the version of MCR that you need is not listed among the MCR module versions, contact us and we will install the version which you require.

Here is an example script which uses the MCR:

<source lang="bash">
#!/bin/bash
# MOAB/Torque submission script for SciNet GPC
#
#PBS -l nodes=1:ppn=8,walltime=1:00:00
#PBS -N test_matlab

# load modules (gcc is needed for MCR/2015a)
module load Xlibraries gcc/5.2.0 MCR/2015a

# DIRECTORY TO RUN - $PBS_O_WORKDIR is directory from which the job was submitted
cd $PBS_O_WORKDIR

# Directory for the MCR to use to write temporary files. Use whatever directory you wish.
mkdir -p $SCRATCH/temp
export MCR_CACHE_ROOT=$SCRATCH/temp

# EXECUTION COMMAND (note that the MATLAB script may require that LD_LIBRARY_PATH be added
# to the script arguments). Note that, if the calculations are serial, you must bundle 8 such
# calculations together for production runs!
./run_myscript.sh $MCR:$LD_LIBRARY_PATH
</source>

== Tunnelling to a license server ==

If you have access to a non-SciNet MATLAB license server, and have installed MATLAB in your $HOME directory, you can setup your submission script to access the external license server. The following lines should be added to the beginning of your submission script:

<pre>
PORT=XXX # port number of the license server
LICENSE_IP=AAA.BBB.CCC.DDD # IP address of the license server
ssh gpc01 -L${PORT}:${LICENSE_IP}:${PORT} -N &
</pre>

This last line will tunnel the port from the compute node back to the license server, through gpc01. The port number and IP address of the licence server must be supplied by the system administrator of the license server.

== Using a different Consortium ==

Both [https://www.sharcnet.ca/my/software/show/54 Sharcnet] and [https://www.westgrid.ca/support/software/matlab Westgrid] have purchased different types of MATLAB licenses. Users can contact those consortia if they wish to attempt to run on those systems.

MATLAB

2016-05-24T14:55:22Z

Ejspence:

We often get questions about running MATLAB on GPC. With a few exceptions for compilers and debuggers, SciNet does not purchase licenses for commercial software. As such, SciNet does not have a license for MATLAB, nor will it in the future. If users wish to run MATLAB they must supply their own license, or explore alternative options. This page gives information about the options for getting your MATLAB code to run, in recommended order.

== Not using MATLAB ==

Users can attempt to run MATLAB code using the open-source program Octave, accessible through the octave module. Though there are some differences between the two programs, Octave has been designed to interpret MATLAB code and can often be used in place of MATLAB. If your MATLAB code does not use some of the more-fancy MATLAB toolboxes, you may be able to get away with using Octave instead. Be sure to test your implementation in Octave thoroughly before committing to this option.

It is worth observing that, while convenient for prototyping and running on a single workstation, there are reasons to avoid using MATLAB for larger HPC/ARC projects. These include the prohibitive license cost for large-scale work, poor performance at scale, and portability issues. If you can switch to a license-free option, such as Python, it may be worth the effort.

== Creating a MATLAB executable ==

If MATLAB must be used, you may be able to compile your MATLAB code into a stand-alone executable, and run this on a GPC compute node. The version of MATLAB being used will require a compiler license, and the compilation must be done on a Linux machine. Once the compilation is done, the executable can be copied to SciNet, and run using the MATLAB Compiler Runtime (MCR), which can be accessed using the MCR module. The MCR used must be the same version of MATLAB as the compiler. If the version of MCR that you need is not listed among the MCR module versions, contact us and we will install the version which you require.

Here is an example script which uses the MCR:

<source lang="bash">
#!/bin/bash
# MOAB/Torque submission script for SciNet GPC
#
#PBS -l nodes=1:ppn=8,walltime=1:00:00
#PBS -N test_matlab

# load modules (gcc is needed for MCR/2015a)
module load Xlibraries gcc/5.2.0 MCR/2015a

# DIRECTORY TO RUN - $PBS_O_WORKDIR is directory from which the job was submitted
cd $PBS_O_WORKDIR

# EXECUTION COMMAND (note that the MATLAB script may require that LD_LIBRARY_PATH be added
# to the script arguments). Note that, if the calculations are serial, you must bundle 8 such
# calculations together for production runs!
./run_myscript.sh $MCR:$LD_LIBRARY_PATH
</source>

== Tunnelling to a license server ==

If you have access to a non-SciNet MATLAB license server, and have installed MATLAB in your $HOME directory, you can setup your submission script to access the external license server. The following lines should be added to the beginning of your submission script:

<pre>
PORT=XXX # port number of the license server
LICENSE_IP=AAA.BBB.CCC.DDD # IP address of the license server
ssh gpc01 -L${PORT}:${LICENSE_IP}:${PORT} -N &
</pre>

This last line will tunnel the port from the compute node back to the license server, through gpc01. The port number and IP address of the licence server must be supplied by the system administrator of the license server.

== Using a different Consortium ==

Both [https://www.sharcnet.ca/my/software/show/54 Sharcnet] and [https://www.westgrid.ca/support/software/matlab Westgrid] have purchased different types of MATLAB licenses. Users can contact those consortia if they wish to attempt to run on those systems.

Scheduler

2015-11-16T15:44:49Z

Ejspence: /* Cancel a Job */

The queueing system used at SciNet is based around Cluster Resources [http://www.adaptivecomputing.com/products/hpc-products/moab-hpc-basic-edition/ Moab Workload Manager].
Moab is used on both the GPC and TCS however [http://www.adaptivecomputing.com/products/open-source/torque/ Torque] is used as the backend resource manager on the GPC and IBM's [http://www.ibm.com/software/products/en/tivoliworkloadschedulerloadleveler/ LoadLeveler] is used on the TCS.

This page outlines some of the most common Moab commands with full documentation available from Moab [http://docs.adaptivecomputing.com/mwm/7-2-9/help.htm#a.gcommandoverview.html%3FTocPath%3D4.0%2520Scheduler%2520Commands|_____0 here], the torque (pbs) commands full documentation is [http://docs.adaptivecomputing.com/torque/4-2-9/help.htm here].

Some common questions about the queuing system can be found on the [[FAQ]] as well.

=== Queues ===

==== GPC ====

===== batch =====

The batch queue is the default queue on the GPC allowing the user access to all the
resources for jobs between 15 minutes and upto 48 hours. If a specific queue is not specified, <tt>-q</tt> flag,
then a job is submitted to the batch queue. Most jobs will run in batch, and user's can use
feature flags to determine the type of nodes they require.

For example, to request two nodes anywhere on the GPC (QDR or DDR), use
<pre>
#PBS -l nodes=2:ppn=8,walltime=1:00:00
</pre>

For two nodes using DDR, use
<pre>
#PBS -l nodes=2:ddr:ppn=8,walltime=1:00:00
</pre>

To get two nodes using QDR, instead, you would use
<pre>
#PBS -l nodes=2:qdr:ppn=8,walltime=1:00:00
</pre>

===== debug =====

A debug queue has been set up primarily for code developers to quickly test
and evaluate their codes and configurations without having to wait in the batch queue. There are 10 nodes
currently reserved for the debug queue. It has quite restrictive limits to promote high turnover
and availability thus a user can only use 2 nodes (16 cores) for 2 hours, to a maximum
of 8 nodes (64 cores) for 1/2 an hour and can only have one job in the debug queue at a time. There
is no minimum time limit on this queue.

<pre>
$ qsub -l nodes=1:ppn=8,walltime=1:00:00 -q debug -I
</pre>

===== largemem =====

The largemem queue is used for accessing one of two 16 core with 128 GB memory intel Xeon (non-nehalem) nodes.

<pre>
$ qsub -l nodes=1:ppn=16,walltime=1:00:00 -q largemem -I
</pre>

==== TCS ====

The TCS currently only has one queue, or class, in use called "verylong" and its only
limitation is that jobs must be under 48 hours.

<pre>
#@ class = verylong
</pre>

=== Job Info===

To see all jobs queued on a system use
<pre>
$ showq
</pre>

Three sections are shown; running, idle, and blocked. Idle jobs are commonly referred to as queued jobs
as they meet all the requirements, however they are waiting for available resources. Blocked jobs
are either caused by improper resource requests or more commonly by exceeding a user or groups allowable
resources. For example if you are allowed to submit 10 jobs and you submit 20, the first 10
jobs will be submitted properly and either run right away or be queued, however the other 10 jobs
will be blocked and the jobs won't be submitted to the queue until one of the first 10 finishes.

=== Available Resources ===

Determining when your job will run can be tricky as it involves a combination of queue type, node type, system reservations, and job priority. The following commands are provided to help you figure out what resources are currently available, however they may not tell you exactly when your job will run for the aforementioned reasons.

==== GPC ====
To show how many qdr nodes are currently free, use the show back fill command
<pre>
$ showbf -f qdr
</pre>

To show how many ddr nodes are free, use
<pre>
$ showbf -f ddr
</pre>

==== TCS ====
To show how many TCS nodes are free, use
<pre>
$ showbf -c verylong
</pre>

For example checking for a qdr job

<pre>
$ showbf -f qdr
Partition Tasks Nodes Duration StartOffset StartDate
--------- ----- ----- ------------ ------------ --------------
ALL 14728 1839 7:36:23 00:00:00 00:23:37_09/24
ALL 256 30 INFINITY 00:00:00 00:23:37_09/24
</pre>

shows that for jobs under 7:36:23 you can use 1839 nodes, but if you submit
a job over that time only 30 will be available. In this case this is
due to a large reservation made my SciNet staff, but from a users point
of view, showbf tells you very simply what is available and at what time point.
In this case, a user may wish to set #PBS -l walltime=7:30:00 in their script, or add -l walltime=7:30:00 to their qsub command in order to ensure that the jobs backfill the reserved nodes.

'''NOTE:''' showbf shows currently available nodes, however just because nodes are available
doesn't mean that your job will start right away. Job priority, system reservations
along with dedicated nodes, such as those for the debug queue, will alter when jobs
run so even if enough nodes appear "free", it doesn't mean your job will actually run right
away.

=== Job Submission ===

==== Interactive ====

On the GPC an interactive queue session can be requested using the following
<pre>
$ qsub -l nodes=2:ppn=8,walltime=1:00:00 -I
</pre>

In case you may experience longer than usual delays when requesting an interactive session, try specifying the "debug" queue:
<pre>
$ qsub -l nodes=2:ppn=8,walltime=1:00:00 -I -q debug
</pre>

==== Non-interactive (Batch) ====

For a non-interactive job submission you require a submission script formatted for the appropriate resource manger. Examples
are provided for the [[GPC_Quickstart#Submitting_A_Batch_Job | GPC]] and [[TCS_Quickstart#Submitting_A_Job | TCS]].

=== Job Status ===
<pre>
$ qstat jobid
</pre>
To see the status of all your jobs, use
<pre>
showq -u username
</pre>
or
<pre>
qstat -u username
</pre>

If your job appears to be blocked, you can try the following command
<pre>
$ checkjob jobid
</pre>
which gives more verbose and often a bit confusing output.

=== Cancel a Job ===

<pre>
$ canceljob jobid
</pre>

To cancel all of your jobs, try this command:

<pre>
$ canceljob `showq | grep yourusername | cut -c -8`
</pre>

=== Accounting ===

For any user with an NRAC/LRAC allocation, a special account with the Resource Allocation Project (RAP) identifier (RAPI) from Compute Canada Database (CCDB) is set up in order to access the allocated resources. Please use the following instructions to run your job using your special allocation. This is necessary both for accounting purposes as well as to assign the appropriate priority to your jobs.

Each job run on the system will have a default RAP associated with it. Most users already have their default RAP properly set. However, if you have more than one allocation (different RAPs), you may need/want to change your default RAP in order to charge your jobs to a particular RAP.

==== Changing your default RAP ====

# Go to the [https://portal.scinet.utoronto.ca portal], login with your SciNet username and password.
# Click on "Change SciNet default RAP" and change your default RAP.

==== Specifying the RAP for GPC ====

Alternatively, you may want to assign a RAP for each particular job you run. There are two ways to specify an account for Moab/Torque: From the command line or inside the batch submission script.

===== Command line =====

Use the '-A RAPI' flag when you submit your job using qsub. Note that the command line option will override the submission script if an account is specified on both the submission script and the command line. "RAPI" is the RAP Identifier, e.g. abc-123-de.

===== Submission Script =====

Add a line in your submit script as follows:
<pre>
#PBS -A RAPI
</pre>

Please replace "RAPI" with your RAP Identifier.

==== Specifiying the RAP for TCS ====

Add a line in your submit script as follows:
<pre>
# @ account_no = RAPI
</pre>

Please replace "RAPI" with your RAP Identifier.

=== User Stats ===

Show current usage stats for a $USER

<pre>
$ showstats -u $USER
</pre>

=== Reservations ===
<pre>
$ showres
</pre>

Standard users can only see their reservations not other users or system ones.
To determine what is available a user can use "showbf", it shows what resources are
available and at what time level, taking into account running jobs and all the reservations. Refer to the [[Moab#Available_Resources | Available Resources]] section of this page for more details.

=== Job Dependencies ===

Sometimes you may want one job not to start until another job finishes, however
you would like to submit them both at the same time. This can be done
using job dependencies on both the GPC and TCS, however the commands
are different due to the underlying resource managers being different.

==== GPC ====

Use the -W flag with the following syntax in your submission script to have this job not start
until the job with jobid has successfully finished
<pre>
-W depend=afterok:jobid
</pre>

This functionality also allows to add dependencies on several jobs, eg.

<pre>
-W depend=afterok:jobid1:jobid2:...:jobidN
</pre>

which in this case, the job will not start until all the jobs jobid1, jobid2, ..., jobidN have successfully finished. Note that the length of the string "afterok:jobid1:jobid2...:jobidN" cannot exceed 1024 characters.

More detailed syntax, dependency options and examples can be found
[[http://docs.adaptivecomputing.com/torque/4-1-4/Content/topics/commands/qsub.htm#-W here]].

==== TCS ====

Loadleveler does job dependencies using what they call steps.
See the [[TCS_Quickstart#Steps | TCS Quickstart]] guide for an example.

=== Adjusting Job Priority ===

The ability to adjust job priorities downwards can also be of use to adjust relative priorities of jobs between users who are running jobs of the same allocation (eg, a default or NRAC allocation of the same PI). Priorities are determined by how much of the time of that allocation been currently used, and all users using that account will have identical priorities. This mechanism allows users to voluntarily reduce their priority to allow other users of the same allocation to run ahead of them.

In principle, by adjusting a jobs priority downwards, you could reduce your jobs priority to the point that someone elses job entirely could go ahead of yours. In practice, however, this is extremely unlikely. Users with NRAC allocations have priorities that are extremely large positive numbers that depend on their allocation and how much of it they have already used during the past fairshare window (2 weeks); it is very unlikely that two groups would have priorities that are within 10 or 100 or 1000 of each other.

Note that at the moment, we do not allow priorities to go negative; they are integers that can go no lower than 1. (This may change in the future) That means that users of accounts that have already used their full allocation during the current fairshare period (eg, over the past two weeks), and so whose priority would normally be negative but is capped at 1, can not lower their priority any further. Similar, users with a `default' allocation have priority 1, and cannot lower their priorities any further.

==== GPC ====

Moab allows users to adjust their jobs' priority moderately downwards, with the <tt>-p</tt> flag; that is, on a qsub line
<pre>
$ qsub ... -p -10 JOBID
</pre>
or in a script
<pre>
...
#PBS -p -10
..
</pre>

The number used (-10 in the examples above) can be any negative number down to -1024.

The ability to adjust job priorities downwards can be useful when you are running a number of jobs and want some to enter the queue at higher priorities than others. Note that if you absolutely require some jobs to start before others, you could use [[#Job Dependencies | job dependencies]] instead.

For a job that is currently queued, one can adjust its priority with
<pre>
$ qalter -p -10 JOBID
</pre>



=== Suspending a Running Job ===

Separate from, and in addition to, the ability to place a hold on a queued job, you may want to suspend a running job. For example, you may want to test the timing of events in a weakly coupled parallel environment.

==== GPC ====

To suspend a job:

<pre>
qsig -s STOP <jobid>
</pre>

and to start it again:

<pre>
qsig -s CONT <jobid>.
</pre>

Scripts are suspendable by default, so you don't need to add any signal handling for this to work.
As far as we can tell, the result is identical to using fg and ctrl-Z (or kill -STOP <PID>) in an interactive run.

More about using (and trapping) signals can be found on the [[Using Signals]] page.

=== QDR Network Switch Affinity ===

The QDR network is globally 5:1 oversubscribed, but on switch it has full 1:1 cross-section whereas the DDR is completely 1:1 non-blocking.
When a job is submitted to the GPC QDR nodes, the queuing system tries to fulfill the requirements with nodes on the same switch to improve network
performance. If not enough nodes are available on the same switch to satisfy the job request, the queue will then use any available nodes. This behavior can be changed
by using the submission flag " -l nodesetisoptional=false " which forces the queuing system to only
run this job on the same switch, thus the job will stay queued until enough nodes on one switch are available
to satisfy this request. Note that the maximum number of nodes on one switch is 30, so
a request of greater than 30 nodes with this flag will never run.
<pre>
qsub -l nodes=4:ppn=8,walltime=10:00,nodesetisoptional=false
</pre>

=== Multiple Job Submissions ===

If you are doing doing batch processing of a number of similar jobs on the GPC, torque has a feature called job arrays that can be used to simplify this process.
By using the "-t 0-N" option on the command line during job submission or putting it in the job script file, #PBS -t 0-N, torque will expand your
single job submission into N+1 jobs and sets the environment variable PBS_ARRAYID equal to that jobs specific number, ie 0-N, for each job. This
reduces the amount of calls to qsub, and can allow the user to have many less submission scripts. Job arrays also have the benefit of batching
groups of jobs allowing commands like qalter, qdel, qhold to work on all or a subset of the job array jobs with one command, instead of having
to run the command for each job.

In the following example, 10 jobs are submitted using a single command
<pre>
qsub -t 0-9 jobscript.sh
</pre>

and the submission script then modifies the job based on the PBS_ARRAYID.
<pre>
#!/bin/bash
#PBS -l nodes=1:ppn=8,walltime=10:00:00
#PBS -N array_jobs

cd ${PBS_O_WORKDIR}
mkdir job.${PBS_ARRAYID}
cd job.${PBS_ARRAYID}

echo "Running job ${PBS_ARRAYID}"
mpirun -np 8 ./mycode >& array_job.${PBS_ARRAYID}.out
</pre>

The JOBID and the job name both get the additional ARRAYID added onto them in the form of a hyphen, ie JOBID-ARRAYID. If for example you wanted
to cancel all the jobs in a job array you would use "qdel JOBID", whereas if you wanted to cancel just one of the jobs you would use
"qdel JOBID-ARRAYID".

See [http://docs.adaptivecomputing.com/torque/4-1-3/Content/topics/2-jobs/multiJobSubmission.htm here] and [http://docs.adaptivecomputing.com/torque/4-1-4/Content/topics/commands/qsub.htm here]
for full details.

=== Job Tools and Scripts ===

The following list of commands and scripts are useful at the moment of monitoring and managing submissions and jobs.
We have discussed them in a recent [https://support.scinet.utoronto.ca/education/go.php/183/file_storage/index.php/download/1/files%5B%5D/3702/ TechTalk].
More details can be found [https://wiki.scinet.utoronto.ca/wiki/index.php/SciNet_Command_Line_Utilities here].







In addition, the following scripts are available:
<pre>
diskUsage
</pre>
informs about the user and group file system usage.

<pre>
quota
</pre>
offers a shorter version for the user.

<pre>
qsum
</pre>
similar to showq, summarizes by user.

<pre>
jobperf jobID
</pre>
informs about the performance per-node of a given job.

<pre>
jobError <jobID | jobNAME>
</pre>
displays on realtime the error output of a given job.

<pre>
jobOutput <jobID | jobNAME>
</pre>
displays on realtime the standard output of a given job.

<pre>
jobcd <jobID | jobNAME>
</pre>
allows users to quickly move into the working directory of a given job.

<pre>
jobscript <jobID | jobNAME>
</pre>
displays the submission script used when submitting a given job.

<pre>
jobssh <jobID | jobNAME>
</pre>
allows users to connect to the head-node of a given job.

<pre>
jobtop <jobID | jobNAME>
</pre>
allows users to "top" on the head-node of a given job.

<pre>
jobtree [user]
</pre>
displays the jobs tree of dependencies for a given user

<pre>
jobdep <jobID>
</pre>
displays the dependencies of a given job.

<pre>
jobperf <jobID>
</pre>
displays current performance statistics of a given job (must be running).

===Monitoring Jobs===
[https://wiki.scinet.utoronto.ca/wiki/images/a/a0/TechTalkJobMonitoring.pdf Tech Talk on Monitoring Jobs]

===Checking the memory usage from jobs===

In many occasions it can be really useful to take a look at how much memory your job is using while it is running.
There a couple of ways to do so:

1) using some of the [https://wiki.scinet.utoronto.ca/wiki/index.php/SciNet_Command_Line_Utilities command line utilities] we have developed, e.g: by using the '''jobperf''' or '''jobtop''' utilities, it will allow you to check the job performance and head's node utilization respectively.

2) ''ssh'' into the nodes where your job is being run and check for memory usage and system stats right there. For instance, trying the 'top' or 'free' commands, in those nodes.

Also, it always a good a idea and strongly encouraged to inspect the standard output-log and error-log generated for your job submissions.
These files are named respectively: ''JobName.{o|e}jobIdNumber''; where ''JobName'' is the name you gave to the job (via the '-N' PBS flag) and ''JobIdNumber'' is the id number of the job.
These files are saved in the working directory after the job is finished, but they can be also accessed on real-time using the '''jobError''' and '''jobOutput''' [https://wiki.scinet.utoronto.ca/wiki/index.php/SciNet_Command_Line_Utilities command line utilities].

Other related topics to memory usage: 
[https://wiki.scinet.utoronto.ca/wiki/index.php/GPC_Quickstart#Ram_Disk Using Ram Disk]
 
[https://wiki.scinet.utoronto.ca/wiki/index.php/GPC_Quickstart#Memory_Configuration Different Memory Configuration nodes]
 
[https://wiki.scinet.utoronto.ca/wiki/index.php/FAQ#Monitoring_jobs_in_the_queue Monitoring Jobs in the Queue]

2015 Ontario Summer School for High Performance Computing Central

2015-07-15T14:16:52Z

Ejspence:

[[File:Summerschool_hpc_2015.gif]]

===About===

The 2015 Ontario Summer School on High Performance Computing, whose second installment of the year will be hosted by SciNet, provides attendees with opportunities to learn and share knowledge and experience in high performance and technical computing. The format: a five-day workshop with mixed lectures and hands-on sessions on a number of selected subjects, including shared memory programming, distributed memory programming, parallel data processing, and general purpose graphics processing unit programming.

Below are descriptions and course material for the Central version, given from July 13th to July 17th, 2015, in Toronto.



__TOC__

===Introduction to the Linux Shell===

====Abstract====

Working with many of the HPC systems in Ontario involves using the linux/unix command line. This provides a very powerful interface, but it can be quite daunting for the uninitiated. Become initiated with this course. This hands on session will cover basic commands and scripting, as well as touching on some powerful constructs like regular expressions. It could be a great boon for your productivity!

====Details====
''Time/date''

Monday July 13, 9:30 - 12:30

''Location''

Galbraith Building, Room 404

35 St. George Street

Toronto, ON M5S 1A4

''Instructor''

Dr. Mike Nolta (SciNet)

====Prerequisites====
A laptop with a linux-like environment.

====Slides and other material====
* [[Media:SS2015_Introduction_to_the_Unix_Shell.pdf|Slides]]

----

===Introduction to High Performance Computing===

=====Abstract=====
This session will provide an introduction to basic concepts of high performance computing. It is intended to be a high level primer for those largely new to HPC, and serve as a foundation upon which to build over the coming days. Topics will include motivation for HPC, essential issues, problem characteristics as they apply to parallelism and a high level overview of parallel programming models. Strategies of running large sets of serial processes using e.g. GNU parallel, will also be presented.

=====Details=====
''Time/date''

Monday July 13, 13:30 - 16:30

''Location''

Galbraith Building, Room 404

35 St. George Street

Toronto, ON M5S 1A4

''Instructor''

Dr. Marcelo Ponce (SciNet)

=====Prerequisites=====
Basic Linux shell skills Registration for all summer school sessions will become available at https://www.sharcnet.ca/events/ss2015.

=====Slides and other material=====
[http://www.scinethpc.ca/~mponce/HPC_intro-2015.pdf Slides]

----

===Data Analysis with R===

=====Abstract=====
This session offers a brief introduction to R, with a focus on data analysis and statistics.

=====Details=====

''Date/time''

Monday July 13, 9:30 - 12:30

''Location''

Galbraith Building, Room 405

35 St. George Street

Toronto, ON M5S 1A4

''Instructor''

Dr. Erik Spence (SciNet)

=====Prerequisites=====

Some programming experience in any language. You should have used and written functions. Bring a laptop with a R and Rstudio installed.

=====Slides and other material=====
[https://wiki.scinet.utoronto.ca/wiki/images/7/73/Intro-R.pdf Slides].

----

===Parallel R===

=====Abstract=====
This session will cover parallel programming R, with a focus on parallel data analysis. Topics covered include snow, parallel, and foreach/doparallel.

=====Details=====

''Time/date''

Monday July 13, 13:30 - 16:30

''Location''

Galbraith Building, Room 405

35 St. George Street

Toronto, ON M5S 1A4

''Instructor''

Dr. Erik Spence (SciNet)

=====Prerequisites=====
Experience with programming in R and a rudimentary understanding of parallel computing.

=====Slides and other material=====
[https://wiki.scinet.utoronto.ca/wiki/images/a/ac/Parallel-R.pdf Slides].

----

===Programming Shared Memory Systems with OpenMP===

=====Abstract=====
In this session, in which lectures and hands-on labs are interspersed, the students will learn the basics of shared memory programming with OpenMP. In particular, we will discuss the OpenMP's threads, memory, and performance, reductions and load balancing. We will also discuss extensions to heterogeneous systems such as offered by the OpenMP 4.0 and OpenACC standards.

=====Details=====

''Time/date''

Tuesday July 14, 9:30 - 16:30

''Location''

Galbraith Building, Room 404

35 St. George Street

Toronto, ON M5S 1A4

''Instructor''

Dr. Ramses van Zon (SciNet)

=====Prerequisites=====
C/C++ and/or Fortran scientific programming; Experience editing and compiling code in a Linux environment.

=====Slides and other material=====
[[Media:OpenMP_SS2015.pdf|Slides]]

----

===Scientific Computing with Python===

=====Abstract=====
In this session, you will be taught how to use python for research computing purposes. Topics include: the basics of python, automation, numpy, scipy, file i/o, and visualization.

=====Details=====
''Time/date'''

Tuesday July 14, 9:30 - 12:30

''Location''

Galbraith Building, Room 405

35 St. George Street

Toronto, ON M5S 1A4

''Instructor''

Dr. Erik Spence (SciNet)

=====Prerequisites=====
A bit of coding experience in any programming or scripting language. Command-line experience in Linux is a plus.

=====Slides and other material=====
[https://wiki.scinet.utoronto.ca/wiki/images/f/f3/Intro-Python.pdf Slides].

----

===Python for High Performance Computing===

=====Abstract=====
This session will cover parallel programming in python, with a focus on parallel data analysis. We will cover subprocess, multiprocessing, pypar and other parallel-enabling python packages.

=====Details=====
''Time/date''

Tuesday July 14, 13:30 - 16:30

''Location''

Galbraith Building, Room 405

35 St. George Street

Toronto, ON M5S 1A4

''Instructor''

Dr. Erik Spence (SciNet)

=====Prerequisites=====
Python programming experience and a rudimentary understanding of parallel computing.

=====Slides and other material=====
[https://wiki.scinet.utoronto.ca/wiki/images/2/2b/Parallel-Python.pdf Slides].

----

===Programming Clusters with MPI===

=====Abstract=====
In this two-day session, through lectures interspersed with hands-on labs, the students will learn the basics of MPI programming. Examples and exercises will be based on parallelization of common scientific computing problems.

=====Details=====
''Time/date''

Wednesday July 15, 9:30 - 16:30

Thursday July 16, 9:30 - 16:30

''Location''

Galbraith Building, Room 404

35 St. George Street

Toronto, ON M5S 1A4

''Instructor''

Dr. Scott Northrup (SciNet)

=====Prerequisites=====
Programming experience with Fortran or C/C++.

=====Slides and other material=====
The slides for this class can be found [http://support.scinet.utoronto.ca/~northrup/SS2015 here].

----

===Programming GPUs with CUDA===

=====Abstract=====
This is an introductory course covering programming and computing on GPUs---graphics processing units---which are an increasingly common presence in massively parallel computing architectures. This session will cover both of the available C-like programming frameworks: NVIDIA’s CUDA-C. The basics of GPU programming will be covered, and students will work through a number of hands on examples. Demonstrations of profiling and debugging applications running on the GPU will also be included. The structuring of data and computations that makes full use of the GPU will be discussed in detail. Students should be able to leave the course with the knowledge necessary to begin developing their own GPU applications.

=====Details=====
''Time/date''

Wednesday July 15, 9:30 - 16:30

Thursday July 16, 9:30 - 16:30

''Location''

Galbraith Building, Room 405

35 St. George Street

Toronto, ON M5S 1A4

''Instructors''
Dr. Pawel Pomorski and Dr. Sergey Mashchenko (SHARCNET)

=====Prerequisites=====
C/C++ scientific programming, experience editing and compiling code in a Linux environment. Some experience with CUDA and/or OpenMP a plus.

=====Slides and other materials=====
''Day 1:''
* [[Media:CUDA_summer_school_2015_central.pdf|Slides]]
* [[Media:CUDA_day1.tar.gz|Code]]
See also [http://ppomorsk.sharcnet.ca http://ppomorsk.sharcnet.ca]

''Day 2:''
* [[Media:CUDA2.pdf|Slides]]

----

===Visualization===

=====Abstract=====
The first part of this session will be a basic introduction to general-purpose scientific visualization tools such as gnuplot, grace(xmgr), remote tools (tunneling, VNC), python's matplotlib, and examples from other. The second part will be a mix of lecture and hands-on to introduce more specialized scientific visualization software such as vmd, paraview, and visit.

=====Details=====
''Time/date''

Friday July 17, 9:30 - 16:30

''Location''

Galbraith Building, Room 404

35 St. George Street

Toronto, ON M5S 1A4

''Instructor''
Dr. Marcelo Ponce (SciNet)

=====Prerequisites=====
Software installed, for the morning session: gnuplot, ssh (graphic client), vnc client, python and matplotlib; afternoon session: ParaView & VisIt.

=====Slides and other material=====
To come

----

===Debugging, Optimization, and Best Practices===

=====Abstract=====
Debugging is an important step in developing a new code, or porting an old one to a new machine. We will discuss the debugging of frequently encountered bugs in serial code with gdb and the debugging of parallel (MPI and threaded) codes with DDT on live systems. Next, we will present general issues, common pitfalls and optimization strategies that are application to HPC systems. By using profiling tools available for parallel programs, we can see when the bottlenecks are. We can try to alleviate them by following common best practices, allowing one to obtain their results more quickly and at the same time minimize their burden on shared system resources. Walk-through examples and case studies will be presented during the session to illustrate the concepts.

=====Details=====
''Date/time''

Friday July 17, 9:30 - 16:30

''Location''

Galbraith Building, Room 405

35 St. George Street

Toronto, ON M5S 1A4

''Instructors''

Dr. Ramses van Zon and Dr. Mike Nolta (SciNet)

=====Prerequisites=====
Basic familiarity with a high-level language (C/C++/Fortran) as well as compiling and running programs via the command line environment on a HPC system.

=====Slides and other material=====
To come

2015 Ontario Summer School for High Performance Computing Central

2015-07-15T14:04:50Z

Ejspence: /* Slides and other material */

[[File:Summerschool_hpc_2015.gif]]

===About===

The 2015 Ontario Summer School on High Performance Computing, whose second installment of the year will be hosted by SciNet, provides attendees with opportunities to learn and share knowledge and experience in high performance and technical computing. The format: a five-day workshop with mixed lectures and hands-on sessions on a number of selected subjects, including shared memory programming, distributed memory programming, parallel data processing, and general purpose graphics processing unit programming.

Below are descriptions and course material for the Central version, given from July 13th to July 17th, 2015, in Toronto.



__TOC__

===Introduction to the Linux Shell===

====Abstract====

Working with many of the HPC systems in Ontario involves using the linux/unix command line. This provides a very powerful interface, but it can be quite daunting for the uninitiated. Become initiated with this course. This hands on session will cover basic commands and scripting, as well as touching on some powerful constructs like regular expressions. It could be a great boon for your productivity!

====Details====
''Time/date''

Monday July 13, 9:30 - 12:30

''Location''

Galbraith Building, Room 404

35 St. George Street

Toronto, ON M5S 1A4

''Instructor''

Dr. Mike Nolta (SciNet)

====Prerequisites====
A laptop with a linux-like environment.

====Slides and other material====
* [[Media:SS2015_Introduction_to_the_Unix_Shell.pdf|Slides]]

===Introduction to High Performance Computing===

=====Abstract=====
This session will provide an introduction to basic concepts of high performance computing. It is intended to be a high level primer for those largely new to HPC, and serve as a foundation upon which to build over the coming days. Topics will include motivation for HPC, essential issues, problem characteristics as they apply to parallelism and a high level overview of parallel programming models. Strategies of running large sets of serial processes using e.g. GNU parallel, will also be presented.

=====Details=====
''Time/date''

Monday July 13, 13:30 - 16:30

''Location''

Galbraith Building, Room 404

35 St. George Street

Toronto, ON M5S 1A4

''Instructor''

Dr. Marcelo Ponce (SciNet)

=====Prerequisites=====
Basic Linux shell skills Registration for all summer school sessions will become available at https://www.sharcnet.ca/events/ss2015.

=====Slides and other material=====
[http://www.scinethpc.ca/~mponce/HPC_intro-2015.pdf Slides]

===Data Analysis with R===

=====Abstract=====
This session offers a brief introduction to R, with a focus on data analysis and statistics.

=====Details=====

''Date/time''

Monday July 13, 9:30 - 12:30

''Location''

Galbraith Building, Room 405

35 St. George Street

Toronto, ON M5S 1A4

''Instructor''

Dr. Erik Spence (SciNet)

=====Prerequisites=====

Some programming experience in any language. You should have used and written functions. Bring a laptop with a R and Rstudio installed.

=====Slides and other material=====
[https://wiki.scinet.utoronto.ca/wiki/images/7/73/Intro-R.pdf Slides].

===Parallel R===

=====Abstract=====
This session will cover parallel programming R, with a focus on parallel data analysis. Topics covered include snow, parallel, and foreach/doparallel.

=====Details=====

''Time/date''

Monday July 13, 13:30 - 16:30

''Location''

Galbraith Building, Room 405

35 St. George Street

Toronto, ON M5S 1A4

''Instructor''

Dr. Erik Spence (SciNet)

=====Prerequisites=====
Experience with programming in R and a rudimentary understanding of parallel computing.

=====Slides and other material=====
[https://wiki.scinet.utoronto.ca/wiki/images/a/ac/Parallel-R.pdf Slides].

===Programming Shared Memory Systems with OpenMP===

=====Abstract=====
In this session, in which lectures and hands-on labs are interspersed, the students will learn the basics of shared memory programming with OpenMP. In particular, we will discuss the OpenMP's threads, memory, and performance, reductions and load balancing. We will also discuss extensions to heterogeneous systems such as offered by the OpenMP 4.0 and OpenACC standards.

=====Details=====

''Time/date''

Tuesday July 14, 9:30 - 16:30

''Location''

Galbraith Building, Room 404

35 St. George Street

Toronto, ON M5S 1A4

''Instructor''

Dr. Ramses van Zon (SciNet)

=====Prerequisites=====
C/C++ and/or Fortran scientific programming; Experience editing and compiling code in a Linux environment.

=====Slides and other material=====
[[Media:OpenMP_SS2015.pdf|Slides]]

===Scientific Computing with Python===

=====Abstract=====
In this session, you will be taught how to use python for research computing purposes. Topics include: the basics of python, automation, numpy, scipy, file i/o, and visualization.

=====Details=====
''Time/date'''

Tuesday July 14, 9:30 - 12:30

''Location''

Galbraith Building, Room 405

35 St. George Street

Toronto, ON M5S 1A4

''Instructor''

Dr. Erik Spence (SciNet)

=====Prerequisites=====
A bit of coding experience in any programming or scripting language. Command-line experience in Linux is a plus.

=====Slides and other material=====
[https://wiki.scinet.utoronto.ca/wiki/images/f/f3/Intro-Python.pdf Slides].

===Python for High Performance Computing===

=====Abstract=====
This session will cover parallel programming in python, with a focus on parallel data analysis. We will cover subprocess, multiprocessing, pypar and other parallel-enabling python packages.

=====Details=====
''Time/date''

Tuesday July 14, 13:30 - 16:30

''Location''

Galbraith Building, Room 405

35 St. George Street

Toronto, ON M5S 1A4

''Instructor''

Dr. Erik Spence (SciNet)

=====Prerequisites=====
Python programming experience and a rudimentary understanding of parallel computing.

=====Slides and other material=====
[https://wiki.scinet.utoronto.ca/wiki/images/2/2b/Parallel-Python.pdf Slides].

===Programming Clusters with MPI===

=====Abstract=====
In this two-day session, through lectures interspersed with hands-on labs, the students will learn the basics of MPI programming. Examples and exercises will be based on parallelization of common scientific computing problems.

=====Details=====
''Time/date''

Wednesday July 15, 9:30 - 16:30

Thursday July 16, 9:30 - 16:30

''Location''

Galbraith Building, Room 404

35 St. George Street

Toronto, ON M5S 1A4

''Instructor''

Dr. Scott Northrup (SciNet)

=====Prerequisites=====
Programming experience with Fortran or C/C++.

=====Slides and other material=====
The slides for this class can be found [http://support.scinet.utoronto.ca/~northrup/SS2015 here].

===Programming GPUs with CUDA===

=====Abstract=====
This is an introductory course covering programming and computing on GPUs---graphics processing units---which are an increasingly common presence in massively parallel computing architectures. This session will cover both of the available C-like programming frameworks: NVIDIA’s CUDA-C. The basics of GPU programming will be covered, and students will work through a number of hands on examples. Demonstrations of profiling and debugging applications running on the GPU will also be included. The structuring of data and computations that makes full use of the GPU will be discussed in detail. Students should be able to leave the course with the knowledge necessary to begin developing their own GPU applications.

=====Details=====
''Time/date''

Wednesday July 15, 9:30 - 16:30

Thursday July 16, 9:30 - 16:30

''Location''

Galbraith Building, Room 405

35 St. George Street

Toronto, ON M5S 1A4

''Instructors''
Dr. Pawel Pomorski and Dr. Sergey Mashchenko (SHARCNET)

=====Prerequisites=====
C/C++ scientific programming, experience editing and compiling code in a Linux environment. Some experience with CUDA and/or OpenMP a plus.

=====Slides and other materials=====
''Day 1:''
* [[Media:CUDA_summer_school_2015_central.pdf|Slides]]
* [[Media:CUDA_day1.tar.gz|Code]]
See also [http://ppomorsk.sharcnet.ca http://ppomorsk.sharcnet.ca]

''Day 2:''
* [[Media:CUDA2.pdf|Slides]]

===Visualization===

=====Abstract=====
The first part of this session will be a basic introduction to general-purpose scientific visualization tools such as gnuplot, grace(xmgr), remote tools (tunneling, VNC), python's matplotlib, and examples from other. The second part will be a mix of lecture and hands-on to introduce more specialized scientific visualization software such as vmd, paraview, and visit.

=====Details=====
''Time/date''

Friday July 17, 9:30 - 16:30

''Location''

Galbraith Building, Room 404

35 St. George Street

Toronto, ON M5S 1A4

''Instructor''
Dr. Marcelo Ponce (SciNet)

=====Prerequisites=====
Software installed, for the morning session: gnuplot, ssh (graphic client), vnc client, python and matplotlib; afternoon session: ParaView & VisIt.

=====Slides and other material=====
To come

===Debugging, Optimization, and Best Practices===

=====Abstract=====
Debugging is an important step in developing a new code, or porting an old one to a new machine. We will discuss the debugging of frequently encountered bugs in serial code with gdb and the debugging of parallel (MPI and threaded) codes with DDT on live systems. Next, we will present general issues, common pitfalls and optimization strategies that are application to HPC systems. By using profiling tools available for parallel programs, we can see when the bottlenecks are. We can try to alleviate them by following common best practices, allowing one to obtain their results more quickly and at the same time minimize their burden on shared system resources. Walk-through examples and case studies will be presented during the session to illustrate the concepts.

=====Details=====
''Date/time''

Friday July 17, 9:30 - 16:30

''Location''

Galbraith Building, Room 405

35 St. George Street

Toronto, ON M5S 1A4

''Instructors''

Dr. Ramses van Zon and Dr. Mike Nolta (SciNet)

=====Prerequisites=====
Basic familiarity with a high-level language (C/C++/Fortran) as well as compiling and running programs via the command line environment on a HPC system.

=====Slides and other material=====
To come

2015 Ontario Summer School for High Performance Computing Central

2015-07-14T13:11:06Z

Ejspence: /* Slides and other material */

[[File:Summerschool_hpc_2015.gif]]

===About===

The 2015 Ontario Summer School on High Performance Computing, whose second installment of the year will be hosted by SciNet, provides attendees with opportunities to learn and share knowledge and experience in high performance and technical computing. The format: a five-day workshop with mixed lectures and hands-on sessions on a number of selected subjects, including shared memory programming, distributed memory programming, parallel data processing, and general purpose graphics processing unit programming.

Below are descriptions and course material for the Central version, given from July 13th to July 17th, 2015, in Toronto.



__TOC__

===Introduction to the Linux Shell===

====Abstract====

Working with many of the HPC systems in Ontario involves using the linux/unix command line. This provides a very powerful interface, but it can be quite daunting for the uninitiated. Become initiated with this course. This hands on session will cover basic commands and scripting, as well as touching on some powerful constructs like regular expressions. It could be a great boon for your productivity!

====Details====
''Time/date''

Monday July 13, 9:30 - 12:30

''Location''

Galbraith Building, Room 404

35 St. George Street

Toronto, ON M5S 1A4

''Instructor''

Dr. Mike Nolta (SciNet)

====Prerequisites====
A laptop with a linux-like environment.

====Slides and other material====
* [[Media:SS2015_Introduction_to_the_Unix_Shell.pdf|Slides]]
* [[Media:SS2014_IntroToShell.tgz|Files]]

===Introduction to High Performance Computing===

=====Abstract=====
This session will provide an introduction to basic concepts of high performance computing. It is intended to be a high level primer for those largely new to HPC, and serve as a foundation upon which to build over the coming days. Topics will include motivation for HPC, essential issues, problem characteristics as they apply to parallelism and a high level overview of parallel programming models. Strategies of running large sets of serial processes using e.g. GNU parallel, will also be presented.

=====Details=====
''Time/date''

Monday July 13, 13:30 - 16:30

''Location''

Galbraith Building, Room 404

35 St. George Street

Toronto, ON M5S 1A4

''Instructor''

Dr. Marcelo Ponce (SciNet)

=====Prerequisites=====
Basic Linux shell skills Registration for all summer school sessions will become available at https://www.sharcnet.ca/events/ss2015.

=====Slides and other material=====
[http://www.scinethpc.ca/~mponce/HPC_intro-2015.pdf Slides]

===Data Analysis with R===

=====Abstract=====
This session offers a brief introduction to R, with a focus on data analysis and statistics.

=====Details=====

''Date/time''

Monday July 13, 9:30 - 12:30

''Location''

Galbraith Building, Room 405

35 St. George Street

Toronto, ON M5S 1A4

''Instructor''

Dr. Erik Spence (SciNet)

=====Prerequisites=====

Some programming experience in any language. You should have used and written functions. Bring a laptop with a R and Rstudio installed.

=====Slides and other material=====
[https://wiki.scinet.utoronto.ca/wiki/images/7/73/Intro-R.pdf Slides].

===Parallel R===

=====Abstract=====
This session will cover parallel programming R, with a focus on parallel data analysis. Topics covered include snow, parallel, and foreach/doparallel.

=====Details=====

''Time/date''

Monday July 13, 13:30 - 16:30

''Location''

Galbraith Building, Room 405

35 St. George Street

Toronto, ON M5S 1A4

''Instructor''

Dr. Erik Spence (SciNet)

=====Prerequisites=====
Experience with programming in R and a rudimentary understanding of parallel computing.

=====Slides and other material=====
[https://wiki.scinet.utoronto.ca/wiki/images/a/ac/Parallel-R.pdf Slides].

===Programming Shared Memory Systems with OpenMP===

=====Abstract=====
In this session, in which lectures and hands-on labs are interspersed, the students will learn the basics of shared memory programming with OpenMP. In particular, we will discuss the OpenMP's threads, memory, and performance, reductions and load balancing. We will also discuss extensions to heterogeneous systems such as offered by the OpenMP 4.0 and OpenACC standards.

=====Details=====

''Time/date''

Tuesday July 14, 9:30 - 16:30

''Location''

Galbraith Building, Room 404

35 St. George Street

Toronto, ON M5S 1A4

''Instructor''

Dr. Ramses van Zon (SciNet)

=====Prerequisites=====
C/C++ and/or Fortran scientific programming; Experience editing and compiling code in a Linux environment.

=====Slides and other material=====
[[Media:OpenMP_SS2015.pdf|Slides]]

===Scientific Computing with Python===

=====Abstract=====
In this session, you will be taught how to use python for research computing purposes. Topics include: the basics of python, automation, numpy, scipy, file i/o, and visualization.

=====Details=====
''Time/date'''

Tuesday July 14, 9:30 - 12:30

''Location''

Galbraith Building, Room 405

35 St. George Street

Toronto, ON M5S 1A4

''Instructor''

Dr. Erik Spence (SciNet)

=====Prerequisites=====
A bit of coding experience in any programming or scripting language. Command-line experience in Linux is a plus.

=====Slides and other material=====
[https://wiki.scinet.utoronto.ca/wiki/images/f/f3/Intro-Python.pdf Slides].

===Python for High Performance Computing===

=====Abstract=====
This session will cover parallel programming in python, with a focus on parallel data analysis. We will cover subprocess, multiprocessing, pypar and other parallel-enabling python packages.

=====Details=====
''Time/date''

Tuesday July 14, 13:30 - 16:30

''Location''

Galbraith Building, Room 405

35 St. George Street

Toronto, ON M5S 1A4

''Instructor''

Dr. Erik Spence (SciNet)

=====Prerequisites=====
Python programming experience and a rudimentary understanding of parallel computing.

=====Slides and other material=====
[https://wiki.scinet.utoronto.ca/wiki/images/2/2b/Parallel-Python.pdf Slides].

===Programming Clusters with MPI===

=====Abstract=====
In this two-day session, through lectures interspersed with hands-on labs, the students will learn the basics of MPI programming. Examples and exercises will be based on parallelization of common scientific computing problems.

=====Details=====
''Time/date''

Wednesday July 15, 9:30 - 16:30

Thursday July 16, 9:30 - 16:30

''Location''

Galbraith Building, Room 404

35 St. George Street

Toronto, ON M5S 1A4

''Instructor''

Dr. Scott Northrup (SciNet)

=====Prerequisites=====
Programming experience with Fortran or C/C++.

=====Slides and other material=====
To come

===Programming GPUs with CUDA===

=====Abstract=====
This is an introductory course covering programming and computing on GPUs---graphics processing units---which are an increasingly common presence in massively parallel computing architectures. This session will cover both of the available C-like programming frameworks: NVIDIA’s CUDA-C. The basics of GPU programming will be covered, and students will work through a number of hands on examples. Demonstrations of profiling and debugging applications running on the GPU will also be included. The structuring of data and computations that makes full use of the GPU will be discussed in detail. Students should be able to leave the course with the knowledge necessary to begin developing their own GPU applications.

=====Details=====
''Time/date''

Wednesday July 15, 9:30 - 16:30

Thursday July 16, 9:30 - 16:30

''Location''

Galbraith Building, Room 405

35 St. George Street

Toronto, ON M5S 1A4

''Instructors''
Dr. Pawel Pomorski and Dr. Sergey Mashchenko (SHARCNET)

=====Prerequisites=====
C/C++ scientific programming, experience editing and compiling code in a Linux environment. Some experience with CUDA and/or OpenMP a plus.

=====Slides and other materials=====
To come

===Visualization===

=====Abstract=====
The first part of this session will be a basic introduction to general-purpose scientific visualization tools such as gnuplot, grace(xmgr), remote tools (tunneling, VNC), python's matplotlib, and examples from other. The second part will be a mix of lecture and hands-on to introduce more specialized scientific visualization software such as vmd, paraview, and visit.

=====Details=====
''Time/date''

Friday July 17, 9:30 - 16:30

''Location''

Galbraith Building, Room 404

35 St. George Street

Toronto, ON M5S 1A4

''Instructor''
Dr. Marcelo Ponce (SciNet)

=====Prerequisites=====
Software installed, for the morning session: gnuplot, ssh (graphic client), vnc client, python and matplotlib; afternoon session: paraview.

=====Slides and other material=====
To come

===Debugging, Optimization, and Best Practices===

=====Abstract=====
Debugging is an important step in developing a new code, or porting an old one to a new machine. We will discuss the debugging of frequently encountered bugs in serial code with gdb and the debugging of parallel (MPI and threaded) codes with DDT on live systems. Next, we will present general issues, common pitfalls and optimization strategies that are application to HPC systems. By using profiling tools available for parallel programs, we can see when the bottlenecks are. We can try to alleviate them by following common best practices, allowing one to obtain their results more quickly and at the same time minimize their burden on shared system resources. Walk-through examples and case studies will be presented during the session to illustrate the concepts.

=====Details=====
''Date/time''

Friday July 17, 9:30 - 16:30

''Location''

Galbraith Building, Room 405

35 St. George Street

Toronto, ON M5S 1A4

''Instructors''

Dr. Ramses van Zon and Dr. Mike Nolta (SciNet)

=====Prerequisites=====
Basic familiarity with a high-level language (C/C++/Fortran) as well as compiling and running programs via the command line environment on a HPC system.

=====Slides and other material=====
To come

File:Parallel-Python.pdf

2015-07-14T13:10:38Z

Ejspence: Slides for the Parallel Python class at the 2015 Summer School.

Slides for the Parallel Python class at the 2015 Summer School.

2015 Ontario Summer School for High Performance Computing Central

2015-07-14T12:19:54Z

Ejspence: /* Slides and other material */

[[File:Summerschool_hpc_2015.gif]]

===About===

The 2015 Ontario Summer School on High Performance Computing, whose second installment of the year will be hosted by SciNet, provides attendees with opportunities to learn and share knowledge and experience in high performance and technical computing. The format: a five-day workshop with mixed lectures and hands-on sessions on a number of selected subjects, including shared memory programming, distributed memory programming, parallel data processing, and general purpose graphics processing unit programming.

Below are descriptions and course material for the Central version, given from July 13th to July 17th, 2015, in Toronto.



__TOC__

===Introduction to the Linux Shell===

====Abstract====

Working with many of the HPC systems in Ontario involves using the linux/unix command line. This provides a very powerful interface, but it can be quite daunting for the uninitiated. Become initiated with this course. This hands on session will cover basic commands and scripting, as well as touching on some powerful constructs like regular expressions. It could be a great boon for your productivity!

====Details====
''Time/date''

Monday July 13, 9:30 - 12:30

''Location''

Galbraith Building, Room 404

35 St. George Street

Toronto, ON M5S 1A4

''Instructor''

Dr. Mike Nolta (SciNet)

====Prerequisites====
A laptop with a linux-like environment.

====Slides and other material====
* [[Media:SS2015_Introduction_to_the_Unix_Shell.pdf|Slides]]
* [[Media:SS2014_IntroToShell.tgz|Files]]

===Introduction to High Performance Computing===

=====Abstract=====
This session will provide an introduction to basic concepts of high performance computing. It is intended to be a high level primer for those largely new to HPC, and serve as a foundation upon which to build over the coming days. Topics will include motivation for HPC, essential issues, problem characteristics as they apply to parallelism and a high level overview of parallel programming models. Strategies of running large sets of serial processes using e.g. GNU parallel, will also be presented.

=====Details=====
''Time/date''

Monday July 13, 13:30 - 16:30

''Location''

Galbraith Building, Room 404

35 St. George Street

Toronto, ON M5S 1A4

''Instructor''

Dr. Marcelo Ponce (SciNet)

=====Prerequisites=====
Basic Linux shell skills Registration for all summer school sessions will become available at https://www.sharcnet.ca/events/ss2015.

=====Slides and other material=====
[http://www.scinethpc.ca/~mponce/HPC_intro-2015.pdf Slides]

===Data Analysis with R===

=====Abstract=====
This session offers a brief introduction to R, with a focus on data analysis and statistics.

=====Details=====

''Date/time''

Monday July 13, 9:30 - 12:30

''Location''

Galbraith Building, Room 405

35 St. George Street

Toronto, ON M5S 1A4

''Instructor''

Dr. Erik Spence (SciNet)

=====Prerequisites=====

Some programming experience in any language. You should have used and written functions. Bring a laptop with a R and Rstudio installed.

=====Slides and other material=====
[https://wiki.scinet.utoronto.ca/wiki/images/7/73/Intro-R.pdf Slides].

===Parallel R===

=====Abstract=====
This session will cover parallel programming R, with a focus on parallel data analysis. Topics covered include snow, parallel, and foreach/doparallel.

=====Details=====

''Time/date''

Monday July 13, 13:30 - 16:30

''Location''

Galbraith Building, Room 405

35 St. George Street

Toronto, ON M5S 1A4

''Instructor''

Dr. Erik Spence (SciNet)

=====Prerequisites=====
Experience with programming in R and a rudimentary understanding of parallel computing.

=====Slides and other material=====
[https://wiki.scinet.utoronto.ca/wiki/images/a/ac/Parallel-R.pdf Slides].

===Programming Shared Memory Systems with OpenMP===

=====Abstract=====
In this session, in which lectures and hands-on labs are interspersed, the students will learn the basics of shared memory programming with OpenMP. In particular, we will discuss the OpenMP's threads, memory, and performance, reductions and load balancing. We will also discuss extensions to heterogeneous systems such as offered by the OpenMP 4.0 and OpenACC standards.

=====Details=====

''Time/date''

Tuesday July 14, 9:30 - 16:30

''Location''

Galbraith Building, Room 404

35 St. George Street

Toronto, ON M5S 1A4

''Instructor''

Dr. Ramses van Zon (SciNet)

=====Prerequisites=====
C/C++ and/or Fortran scientific programming; Experience editing and compiling code in a Linux environment.

=====Slides and other material=====
[[Media:OpenMP_SS2015.pdf|Slides]]

===Scientific Computing with Python===

=====Abstract=====
In this session, you will be taught how to use python for research computing purposes. Topics include: the basics of python, automation, numpy, scipy, file i/o, and visualization.

=====Details=====
''Time/date'''

Tuesday July 14, 9:30 - 12:30

''Location''

Galbraith Building, Room 405

35 St. George Street

Toronto, ON M5S 1A4

''Instructor''

Dr. Erik Spence (SciNet)

=====Prerequisites=====
A bit of coding experience in any programming or scripting language. Command-line experience in Linux is a plus.

=====Slides and other material=====
[https://wiki.scinet.utoronto.ca/wiki/images/f/f3/Intro-Python.pdf Slides].

===Python for High Performance Computing===

=====Abstract=====
This session will cover parallel programming in python, with a focus on parallel data analysis. We will cover subprocess, multiprocessing, pypar and other parallel-enabling python packages.

=====Details=====
''Time/date''

Tuesday July 14, 13:30 - 16:30

''Location''

Galbraith Building, Room 405

35 St. George Street

Toronto, ON M5S 1A4

''Instructor''

Dr. Erik Spence (SciNet)

=====Prerequisites=====
Python programming experience and a rudimentary understanding of parallel computing.

=====Slides and other material=====
To come

===Programming Clusters with MPI===

=====Abstract=====
In this two-day session, through lectures interspersed with hands-on labs, the students will learn the basics of MPI programming. Examples and exercises will be based on parallelization of common scientific computing problems.

=====Details=====
''Time/date''

Wednesday July 15, 9:30 - 16:30

Thursday July 16, 9:30 - 16:30

''Location''

Galbraith Building, Room 404

35 St. George Street

Toronto, ON M5S 1A4

''Instructor''

Dr. Scott Northrup (SciNet)

=====Prerequisites=====
Programming experience with Fortran or C/C++.

=====Slides and other material=====
To come

===Programming GPUs with CUDA===

=====Abstract=====
This is an introductory course covering programming and computing on GPUs---graphics processing units---which are an increasingly common presence in massively parallel computing architectures. This session will cover both of the available C-like programming frameworks: NVIDIA’s CUDA-C. The basics of GPU programming will be covered, and students will work through a number of hands on examples. Demonstrations of profiling and debugging applications running on the GPU will also be included. The structuring of data and computations that makes full use of the GPU will be discussed in detail. Students should be able to leave the course with the knowledge necessary to begin developing their own GPU applications.

=====Details=====
''Time/date''

Wednesday July 15, 9:30 - 16:30

Thursday July 16, 9:30 - 16:30

''Location''

Galbraith Building, Room 405

35 St. George Street

Toronto, ON M5S 1A4

''Instructors''
Dr. Pawel Pomorski and Dr. Sergey Mashchenko (SHARCNET)

=====Prerequisites=====
C/C++ scientific programming, experience editing and compiling code in a Linux environment. Some experience with CUDA and/or OpenMP a plus.

=====Slides and other materials=====
To come

===Visualization===

=====Abstract=====
The first part of this session will be a basic introduction to general-purpose scientific visualization tools such as gnuplot, grace(xmgr), remote tools (tunneling, VNC), python's matplotlib, and examples from other. The second part will be a mix of lecture and hands-on to introduce more specialized scientific visualization software such as vmd, paraview, and visit.

=====Details=====
''Time/date''

Friday July 17, 9:30 - 16:30

''Location''

Galbraith Building, Room 404

35 St. George Street

Toronto, ON M5S 1A4

''Instructor''
Dr. Marcelo Ponce (SciNet)

=====Prerequisites=====
Software installed, for the morning session: gnuplot, ssh (graphic client), vnc client, python and matplotlib; afternoon session: paraview.

=====Slides and other material=====
To come

===Debugging, Optimization, and Best Practices===

=====Abstract=====
Debugging is an important step in developing a new code, or porting an old one to a new machine. We will discuss the debugging of frequently encountered bugs in serial code with gdb and the debugging of parallel (MPI and threaded) codes with DDT on live systems. Next, we will present general issues, common pitfalls and optimization strategies that are application to HPC systems. By using profiling tools available for parallel programs, we can see when the bottlenecks are. We can try to alleviate them by following common best practices, allowing one to obtain their results more quickly and at the same time minimize their burden on shared system resources. Walk-through examples and case studies will be presented during the session to illustrate the concepts.

=====Details=====
''Date/time''

Friday July 17, 9:30 - 16:30

''Location''

Galbraith Building, Room 405

35 St. George Street

Toronto, ON M5S 1A4

''Instructors''

Dr. Ramses van Zon and Dr. Mike Nolta (SciNet)

=====Prerequisites=====
Basic familiarity with a high-level language (C/C++/Fortran) as well as compiling and running programs via the command line environment on a HPC system.

=====Slides and other material=====
To come

File:Intro-Python.pdf

2015-07-14T12:17:39Z

Ejspence: Slides for the Scientific Computing with Python class for the 2015 Summer School.

Slides for the Scientific Computing with Python class for the 2015 Summer School.

2015 Ontario Summer School for High Performance Computing Central

2015-07-13T12:40:07Z

Ejspence: /* Slides and other material */

[[File:Summerschool_hpc_2015.gif]]

===About===

The 2015 Ontario Summer School on High Performance Computing, whose second installment of the year will be hosted by SciNet, provides attendees with opportunities to learn and share knowledge and experience in high performance and technical computing. The format: a five-day workshop with mixed lectures and hands-on sessions on a number of selected subjects, including shared memory programming, distributed memory programming, parallel data processing, and general purpose graphics processing unit programming.

Below are descriptions and course material for the Central version, given from July 13th to July 17th, 2015, in Toronto.



__TOC__

===Introduction to the Linux Shell===

====Abstract====

Working with many of the HPC systems in Ontario involves using the linux/unix command line. This provides a very powerful interface, but it can be quite daunting for the uninitiated. Become initiated with this course. This hands on session will cover basic commands and scripting, as well as touching on some powerful constructs like regular expressions. It could be a great boon for your productivity!

====Details====
''Time/date''

Monday July 13, 9:30 - 12:30

''Location''

Galbraith Building, Room 404

35 St. George Street

Toronto, ON M5S 1A4

''Instructor''

Dr. Mike Nolta (SciNet)

====Prerequisites====
A laptop with a linux-like environment.

====Slides and other material====
* To come
* [[Media:SS2014_IntroToShell.tgz|Files]]

===Introduction to High Performance Computing===

=====Abstract=====
This session will provide an introduction to basic concepts of high performance computing. It is intended to be a high level primer for those largely new to HPC, and serve as a foundation upon which to build over the coming days. Topics will include motivation for HPC, essential issues, problem characteristics as they apply to parallelism and a high level overview of parallel programming models. Strategies of running large sets of serial processes using e.g. GNU parallel, will also be presented.

=====Details=====
''Time/date''

Monday July 13, 13:30 - 16:30

''Location''

Galbraith Building, Room 404

35 St. George Street

Toronto, ON M5S 1A4

''Instructor''

Dr. Marcelo Ponce (SciNet)

=====Prerequisites=====
Basic Linux shell skills Registration for all summer school sessions will become available at https://www.sharcnet.ca/events/ss2015.

=====Slides and other material=====
To come

===Data Analysis with R===

=====Abstract=====
This session offers a brief introduction to R, with a focus on data analysis and statistics.

=====Details=====

''Date/time''

Monday July 13, 9:30 - 12:30

''Location''

Galbraith Building, Room 405

35 St. George Street

Toronto, ON M5S 1A4

''Instructor''

Dr. Erik Spence (SciNet)

=====Prerequisites=====

Some programming experience in any language. You should have used and written functions. Bring a laptop with a R and Rstudio installed.

=====Slides and other material=====
[https://wiki.scinet.utoronto.ca/wiki/images/7/73/Intro-R.pdf Slides].

===Parallel R===

=====Abstract=====
This session will cover parallel programming R, with a focus on parallel data analysis. Topics covered include snow, parallel, and foreach/doparallel.

=====Details=====

''Time/date''

Monday July 13, 13:30 - 16:30

''Location''

Galbraith Building, Room 405

35 St. George Street

Toronto, ON M5S 1A4

''Instructor''

Dr. Erik Spence (SciNet)

=====Prerequisites=====
Experience with programming in R and a rudimentary understanding of parallel computing.

=====Slides and other material=====
[https://wiki.scinet.utoronto.ca/wiki/images/a/ac/Parallel-R.pdf Slides].

===Programming Shared Memory Systems with OpenMP===

=====Abstract=====
In this session, in which lectures and hands-on labs are interspersed, the students will learn the basics of shared memory programming with OpenMP. In particular, we will discuss the OpenMP's threads, memory, and performance, reductions and load balancing. We will also discuss extensions to heterogeneous systems such as offered by the OpenMP 4.0 and OpenACC standards.

=====Details=====

''Time/date''

Tuesday July 14, 9:30 - 16:30

''Location''

Galbraith Building, Room 404

35 St. George Street

Toronto, ON M5S 1A4

''Instructor''

Dr. Ramses van Zon (SciNet)

=====Prerequisites=====
C/C++ and/or Fortran scientific programming; Experience editing and compiling code in a Linux environment.

=====Slides and other material=====
To come

===Scientific Computing with Python===

=====Abstract=====
In this session, you will be taught how to use python for research computing purposes. Topics include: the basics of python, automation, numpy, scipy, file i/o, and visualization.

=====Details=====
''Time/date'''

Tuesday July 14, 9:30 - 12:30

''Location''

Galbraith Building, Room 405

35 St. George Street

Toronto, ON M5S 1A4

''Instructor''

Dr. Erik Spence (SciNet)

=====Prerequisites=====
A bit of coding experience in any programming or scripting language. Command-line experience in Linux is a plus.

=====Slides and other material=====
To come

===Python for High Performance Computing===

=====Abstract=====
This session will cover parallel programming in python, with a focus on parallel data analysis. We will cover subprocess, multiprocessing, pypar and other parallel-enabling python packages.

=====Details=====
''Time/date''

Tuesday July 14, 13:30 - 16:30

''Location''

Galbraith Building, Room 405

35 St. George Street

Toronto, ON M5S 1A4

''Instructor''

Dr. Erik Spence (SciNet)

=====Prerequisites=====
Python programming experience and a rudimentary understanding of parallel computing.

=====Slides and other material=====
To come

===Programming Clusters with MPI===

=====Abstract=====
In this two-day session, through lectures interspersed with hands-on labs, the students will learn the basics of MPI programming. Examples and exercises will be based on parallelization of common scientific computing problems.

=====Details=====
''Time/date''

Wednesday July 15, 9:30 - 16:30

Thursday July 16, 9:30 - 16:30

''Location''

Galbraith Building, Room 404

35 St. George Street

Toronto, ON M5S 1A4

''Instructor''

Dr. Scott Northrup (SciNet)

=====Prerequisites=====
Programming experience with Fortran or C/C++.

=====Slides and other material=====
To come

===Programming GPUs with CUDA===

=====Abstract=====
This is an introductory course covering programming and computing on GPUs---graphics processing units---which are an increasingly common presence in massively parallel computing architectures. This session will cover both of the available C-like programming frameworks: NVIDIA’s CUDA-C. The basics of GPU programming will be covered, and students will work through a number of hands on examples. Demonstrations of profiling and debugging applications running on the GPU will also be included. The structuring of data and computations that makes full use of the GPU will be discussed in detail. Students should be able to leave the course with the knowledge necessary to begin developing their own GPU applications.

=====Details=====
''Time/date''

Wednesday July 15, 9:30 - 16:30

Thursday July 16, 9:30 - 16:30

''Location''

Galbraith Building, Room 405

35 St. George Street

Toronto, ON M5S 1A4

''Instructors''
Dr. Pawel Pomorski and Dr. Sergey Mashchenko (SHARCNET)

=====Prerequisites=====
C/C++ scientific programming, experience editing and compiling code in a Linux environment. Some experience with CUDA and/or OpenMP a plus.

=====Slides and other materials=====
To come

===Visualization===

=====Abstract=====
The first part of this session will be a basic introduction to general-purpose scientific visualization tools such as gnuplot, grace(xmgr), remote tools (tunneling, VNC), python's matplotlib, and examples from other. The second part will be a mix of lecture and hands-on to introduce more specialized scientific visualization software such as vmd, paraview, and visit.

=====Details=====
''Time/date''

Friday July 17, 9:30 - 16:30

''Location''

Galbraith Building, Room 404

35 St. George Street

Toronto, ON M5S 1A4

''Instructor''
Dr. Marcelo Ponce (SciNet)

=====Prerequisites=====
Software installed, for the morning session: gnuplot, ssh (graphic client), vnc client, python and matplotlib; afternoon session: paraview.

=====Slides and other material=====
To come

===Debugging, Optimization, and Best Practices===

=====Abstract=====
Debugging is an important step in developing a new code, or porting an old one to a new machine. We will discuss the debugging of frequently encountered bugs in serial code with gdb and the debugging of parallel (MPI and threaded) codes with DDT on live systems. Next, we will present general issues, common pitfalls and optimization strategies that are application to HPC systems. By using profiling tools available for parallel programs, we can see when the bottlenecks are. We can try to alleviate them by following common best practices, allowing one to obtain their results more quickly and at the same time minimize their burden on shared system resources. Walk-through examples and case studies will be presented during the session to illustrate the concepts.

=====Details=====
''Date/time''

Friday July 17, 9:30 - 16:30

''Location''

Galbraith Building, Room 405

35 St. George Street

Toronto, ON M5S 1A4

''Instructors''

Dr. Ramses van Zon and Dr. Mike Nolta (SciNet)

=====Prerequisites=====
Basic familiarity with a high-level language (C/C++/Fortran) as well as compiling and running programs via the command line environment on a HPC system.

=====Slides and other material=====
To come

2015 Ontario Summer School for High Performance Computing Central

2015-07-13T12:39:21Z

Ejspence: /* Slides and other material */

[[File:Summerschool_hpc_2015.gif]]

===About===

The 2015 Ontario Summer School on High Performance Computing, whose second installment of the year will be hosted by SciNet, provides attendees with opportunities to learn and share knowledge and experience in high performance and technical computing. The format: a five-day workshop with mixed lectures and hands-on sessions on a number of selected subjects, including shared memory programming, distributed memory programming, parallel data processing, and general purpose graphics processing unit programming.

Below are descriptions and course material for the Central version, given from July 13th to July 17th, 2015, in Toronto.



__TOC__

===Introduction to the Linux Shell===

====Abstract====

Working with many of the HPC systems in Ontario involves using the linux/unix command line. This provides a very powerful interface, but it can be quite daunting for the uninitiated. Become initiated with this course. This hands on session will cover basic commands and scripting, as well as touching on some powerful constructs like regular expressions. It could be a great boon for your productivity!

====Details====
''Time/date''

Monday July 13, 9:30 - 12:30

''Location''

Galbraith Building, Room 404

35 St. George Street

Toronto, ON M5S 1A4

''Instructor''

Dr. Mike Nolta (SciNet)

====Prerequisites====
A laptop with a linux-like environment.

====Slides and other material====
* To come
* [[Media:SS2014_IntroToShell.tgz|Files]]

===Introduction to High Performance Computing===

=====Abstract=====
This session will provide an introduction to basic concepts of high performance computing. It is intended to be a high level primer for those largely new to HPC, and serve as a foundation upon which to build over the coming days. Topics will include motivation for HPC, essential issues, problem characteristics as they apply to parallelism and a high level overview of parallel programming models. Strategies of running large sets of serial processes using e.g. GNU parallel, will also be presented.

=====Details=====
''Time/date''

Monday July 13, 13:30 - 16:30

''Location''

Galbraith Building, Room 404

35 St. George Street

Toronto, ON M5S 1A4

''Instructor''

Dr. Marcelo Ponce (SciNet)

=====Prerequisites=====
Basic Linux shell skills Registration for all summer school sessions will become available at https://www.sharcnet.ca/events/ss2015.

=====Slides and other material=====
To come

===Data Analysis with R===

=====Abstract=====
This session offers a brief introduction to R, with a focus on data analysis and statistics.

=====Details=====

''Date/time''

Monday July 13, 9:30 - 12:30

''Location''

Galbraith Building, Room 405

35 St. George Street

Toronto, ON M5S 1A4

''Instructor''

Dr. Erik Spence (SciNet)

=====Prerequisites=====

Some programming experience in any language. You should have used and written functions. Bring a laptop with a R and Rstudio installed.

=====Slides and other material=====
[https://wiki.scinet.utoronto.ca/wiki/images/7/73/Intro-R.pdf Slides].

===Parallel R===

=====Abstract=====
This session will cover parallel programming R, with a focus on parallel data analysis. Topics covered include snow, parallel, and foreach/doparallel.

=====Details=====

''Time/date''

Monday July 13, 13:30 - 16:30

''Location''

Galbraith Building, Room 405

35 St. George Street

Toronto, ON M5S 1A4

''Instructor''

Dr. Erik Spence (SciNet)

=====Prerequisites=====
Experience with programming in R and a rudimentary understanding of parallel computing.

=====Slides and other material=====
[https://wiki.scinet.utoronto.ca/wiki/index.php/File:Parallel-R.pdf Slides].

===Programming Shared Memory Systems with OpenMP===

=====Abstract=====
In this session, in which lectures and hands-on labs are interspersed, the students will learn the basics of shared memory programming with OpenMP. In particular, we will discuss the OpenMP's threads, memory, and performance, reductions and load balancing. We will also discuss extensions to heterogeneous systems such as offered by the OpenMP 4.0 and OpenACC standards.

=====Details=====

''Time/date''

Tuesday July 14, 9:30 - 16:30

''Location''

Galbraith Building, Room 404

35 St. George Street

Toronto, ON M5S 1A4

''Instructor''

Dr. Ramses van Zon (SciNet)

=====Prerequisites=====
C/C++ and/or Fortran scientific programming; Experience editing and compiling code in a Linux environment.

=====Slides and other material=====
To come

===Scientific Computing with Python===

=====Abstract=====
In this session, you will be taught how to use python for research computing purposes. Topics include: the basics of python, automation, numpy, scipy, file i/o, and visualization.

=====Details=====
''Time/date'''

Tuesday July 14, 9:30 - 12:30

''Location''

Galbraith Building, Room 405

35 St. George Street

Toronto, ON M5S 1A4

''Instructor''

Dr. Erik Spence (SciNet)

=====Prerequisites=====
A bit of coding experience in any programming or scripting language. Command-line experience in Linux is a plus.

=====Slides and other material=====
To come

===Python for High Performance Computing===

=====Abstract=====
This session will cover parallel programming in python, with a focus on parallel data analysis. We will cover subprocess, multiprocessing, pypar and other parallel-enabling python packages.

=====Details=====
''Time/date''

Tuesday July 14, 13:30 - 16:30

''Location''

Galbraith Building, Room 405

35 St. George Street

Toronto, ON M5S 1A4

''Instructor''

Dr. Erik Spence (SciNet)

=====Prerequisites=====
Python programming experience and a rudimentary understanding of parallel computing.

=====Slides and other material=====
To come

===Programming Clusters with MPI===

=====Abstract=====
In this two-day session, through lectures interspersed with hands-on labs, the students will learn the basics of MPI programming. Examples and exercises will be based on parallelization of common scientific computing problems.

=====Details=====
''Time/date''

Wednesday July 15, 9:30 - 16:30

Thursday July 16, 9:30 - 16:30

''Location''

Galbraith Building, Room 404

35 St. George Street

Toronto, ON M5S 1A4

''Instructor''

Dr. Scott Northrup (SciNet)

=====Prerequisites=====
Programming experience with Fortran or C/C++.

=====Slides and other material=====
To come

===Programming GPUs with CUDA===

=====Abstract=====
This is an introductory course covering programming and computing on GPUs---graphics processing units---which are an increasingly common presence in massively parallel computing architectures. This session will cover both of the available C-like programming frameworks: NVIDIA’s CUDA-C. The basics of GPU programming will be covered, and students will work through a number of hands on examples. Demonstrations of profiling and debugging applications running on the GPU will also be included. The structuring of data and computations that makes full use of the GPU will be discussed in detail. Students should be able to leave the course with the knowledge necessary to begin developing their own GPU applications.

=====Details=====
''Time/date''

Wednesday July 15, 9:30 - 16:30

Thursday July 16, 9:30 - 16:30

''Location''

Galbraith Building, Room 405

35 St. George Street

Toronto, ON M5S 1A4

''Instructors''
Dr. Pawel Pomorski and Dr. Sergey Mashchenko (SHARCNET)

=====Prerequisites=====
C/C++ scientific programming, experience editing and compiling code in a Linux environment. Some experience with CUDA and/or OpenMP a plus.

=====Slides and other materials=====
To come

===Visualization===

=====Abstract=====
The first part of this session will be a basic introduction to general-purpose scientific visualization tools such as gnuplot, grace(xmgr), remote tools (tunneling, VNC), python's matplotlib, and examples from other. The second part will be a mix of lecture and hands-on to introduce more specialized scientific visualization software such as vmd, paraview, and visit.

=====Details=====
''Time/date''

Friday July 17, 9:30 - 16:30

''Location''

Galbraith Building, Room 404

35 St. George Street

Toronto, ON M5S 1A4

''Instructor''
Dr. Marcelo Ponce (SciNet)

=====Prerequisites=====
Software installed, for the morning session: gnuplot, ssh (graphic client), vnc client, python and matplotlib; afternoon session: paraview.

=====Slides and other material=====
To come

===Debugging, Optimization, and Best Practices===

=====Abstract=====
Debugging is an important step in developing a new code, or porting an old one to a new machine. We will discuss the debugging of frequently encountered bugs in serial code with gdb and the debugging of parallel (MPI and threaded) codes with DDT on live systems. Next, we will present general issues, common pitfalls and optimization strategies that are application to HPC systems. By using profiling tools available for parallel programs, we can see when the bottlenecks are. We can try to alleviate them by following common best practices, allowing one to obtain their results more quickly and at the same time minimize their burden on shared system resources. Walk-through examples and case studies will be presented during the session to illustrate the concepts.

=====Details=====
''Date/time''

Friday July 17, 9:30 - 16:30

''Location''

Galbraith Building, Room 405

35 St. George Street

Toronto, ON M5S 1A4

''Instructors''

Dr. Ramses van Zon and Dr. Mike Nolta (SciNet)

=====Prerequisites=====
Basic familiarity with a high-level language (C/C++/Fortran) as well as compiling and running programs via the command line environment on a HPC system.

=====Slides and other material=====
To come

File:Parallel-R.pdf

2015-07-13T12:37:41Z

Ejspence: Slides for the Parallel R class in the 2015 Summer School.

Slides for the Parallel R class in the 2015 Summer School.

2015 Ontario Summer School for High Performance Computing Central

2015-07-13T12:36:54Z

Ejspence: /* Slides and other material */

[[File:Summerschool_hpc_2015.gif]]

===About===

The 2015 Ontario Summer School on High Performance Computing, whose second installment of the year will be hosted by SciNet, provides attendees with opportunities to learn and share knowledge and experience in high performance and technical computing. The format: a five-day workshop with mixed lectures and hands-on sessions on a number of selected subjects, including shared memory programming, distributed memory programming, parallel data processing, and general purpose graphics processing unit programming.

Below are descriptions and course material for the Central version, given from July 13th to July 17th, 2015, in Toronto.



__TOC__

===Introduction to the Linux Shell===

====Abstract====

Working with many of the HPC systems in Ontario involves using the linux/unix command line. This provides a very powerful interface, but it can be quite daunting for the uninitiated. Become initiated with this course. This hands on session will cover basic commands and scripting, as well as touching on some powerful constructs like regular expressions. It could be a great boon for your productivity!

====Details====
''Time/date''

Monday July 13, 9:30 - 12:30

''Location''

Galbraith Building, Room 404

35 St. George Street

Toronto, ON M5S 1A4

''Instructor''

Dr. Mike Nolta (SciNet)

====Prerequisites====
A laptop with a linux-like environment.

====Slides and other material====
* To come
* [[Media:SS2014_IntroToShell.tgz|Files]]

===Introduction to High Performance Computing===

=====Abstract=====
This session will provide an introduction to basic concepts of high performance computing. It is intended to be a high level primer for those largely new to HPC, and serve as a foundation upon which to build over the coming days. Topics will include motivation for HPC, essential issues, problem characteristics as they apply to parallelism and a high level overview of parallel programming models. Strategies of running large sets of serial processes using e.g. GNU parallel, will also be presented.

=====Details=====
''Time/date''

Monday July 13, 13:30 - 16:30

''Location''

Galbraith Building, Room 404

35 St. George Street

Toronto, ON M5S 1A4

''Instructor''

Dr. Marcelo Ponce (SciNet)

=====Prerequisites=====
Basic Linux shell skills Registration for all summer school sessions will become available at https://www.sharcnet.ca/events/ss2015.

=====Slides and other material=====
To come

===Data Analysis with R===

=====Abstract=====
This session offers a brief introduction to R, with a focus on data analysis and statistics.

=====Details=====

''Date/time''

Monday July 13, 9:30 - 12:30

''Location''

Galbraith Building, Room 405

35 St. George Street

Toronto, ON M5S 1A4

''Instructor''

Dr. Erik Spence (SciNet)

=====Prerequisites=====

Some programming experience in any language. You should have used and written functions. Bring a laptop with a R and Rstudio installed.

=====Slides and other material=====
[https://wiki.scinet.utoronto.ca/wiki/images/7/73/Intro-R.pdf Slides].

===Parallel R===

=====Abstract=====
This session will cover parallel programming R, with a focus on parallel data analysis. Topics covered include snow, parallel, and foreach/doparallel.

=====Details=====

''Time/date''

Monday July 13, 13:30 - 16:30

''Location''

Galbraith Building, Room 405

35 St. George Street

Toronto, ON M5S 1A4

''Instructor''

Dr. Erik Spence (SciNet)

=====Prerequisites=====
Experience with programming in R and a rudimentary understanding of parallel computing.

=====Slides and other material=====
[[File:file.pdf Slides]].

===Programming Shared Memory Systems with OpenMP===

=====Abstract=====
In this session, in which lectures and hands-on labs are interspersed, the students will learn the basics of shared memory programming with OpenMP. In particular, we will discuss the OpenMP's threads, memory, and performance, reductions and load balancing. We will also discuss extensions to heterogeneous systems such as offered by the OpenMP 4.0 and OpenACC standards.

=====Details=====

''Time/date''

Tuesday July 14, 9:30 - 16:30

''Location''

Galbraith Building, Room 404

35 St. George Street

Toronto, ON M5S 1A4

''Instructor''

Dr. Ramses van Zon (SciNet)

=====Prerequisites=====
C/C++ and/or Fortran scientific programming; Experience editing and compiling code in a Linux environment.

=====Slides and other material=====
To come

===Scientific Computing with Python===

=====Abstract=====
In this session, you will be taught how to use python for research computing purposes. Topics include: the basics of python, automation, numpy, scipy, file i/o, and visualization.

=====Details=====
''Time/date'''

Tuesday July 14, 9:30 - 12:30

''Location''

Galbraith Building, Room 405

35 St. George Street

Toronto, ON M5S 1A4

''Instructor''

Dr. Erik Spence (SciNet)

=====Prerequisites=====
A bit of coding experience in any programming or scripting language. Command-line experience in Linux is a plus.

=====Slides and other material=====
To come

===Python for High Performance Computing===

=====Abstract=====
This session will cover parallel programming in python, with a focus on parallel data analysis. We will cover subprocess, multiprocessing, pypar and other parallel-enabling python packages.

=====Details=====
''Time/date''

Tuesday July 14, 13:30 - 16:30

''Location''

Galbraith Building, Room 405

35 St. George Street

Toronto, ON M5S 1A4

''Instructor''

Dr. Erik Spence (SciNet)

=====Prerequisites=====
Python programming experience and a rudimentary understanding of parallel computing.

=====Slides and other material=====
To come

===Programming Clusters with MPI===

=====Abstract=====
In this two-day session, through lectures interspersed with hands-on labs, the students will learn the basics of MPI programming. Examples and exercises will be based on parallelization of common scientific computing problems.

=====Details=====
''Time/date''

Wednesday July 15, 9:30 - 16:30

Thursday July 16, 9:30 - 16:30

''Location''

Galbraith Building, Room 404

35 St. George Street

Toronto, ON M5S 1A4

''Instructor''

Dr. Scott Northrup (SciNet)

=====Prerequisites=====
Programming experience with Fortran or C/C++.

=====Slides and other material=====
To come

===Programming GPUs with CUDA===

=====Abstract=====
This is an introductory course covering programming and computing on GPUs---graphics processing units---which are an increasingly common presence in massively parallel computing architectures. This session will cover both of the available C-like programming frameworks: NVIDIA’s CUDA-C. The basics of GPU programming will be covered, and students will work through a number of hands on examples. Demonstrations of profiling and debugging applications running on the GPU will also be included. The structuring of data and computations that makes full use of the GPU will be discussed in detail. Students should be able to leave the course with the knowledge necessary to begin developing their own GPU applications.

=====Details=====
''Time/date''

Wednesday July 15, 9:30 - 16:30

Thursday July 16, 9:30 - 16:30

''Location''

Galbraith Building, Room 405

35 St. George Street

Toronto, ON M5S 1A4

''Instructors''
Dr. Pawel Pomorski and Dr. Sergey Mashchenko (SHARCNET)

=====Prerequisites=====
C/C++ scientific programming, experience editing and compiling code in a Linux environment. Some experience with CUDA and/or OpenMP a plus.

=====Slides and other materials=====
To come

===Visualization===

=====Abstract=====
The first part of this session will be a basic introduction to general-purpose scientific visualization tools such as gnuplot, grace(xmgr), remote tools (tunneling, VNC), python's matplotlib, and examples from other. The second part will be a mix of lecture and hands-on to introduce more specialized scientific visualization software such as vmd, paraview, and visit.

=====Details=====
''Time/date''

Friday July 17, 9:30 - 16:30

''Location''

Galbraith Building, Room 404

35 St. George Street

Toronto, ON M5S 1A4

''Instructor''
Dr. Marcelo Ponce (SciNet)

=====Prerequisites=====
Software installed, for the morning session: gnuplot, ssh (graphic client), vnc client, python and matplotlib; afternoon session: paraview.

=====Slides and other material=====
To come

===Debugging, Optimization, and Best Practices===

=====Abstract=====
Debugging is an important step in developing a new code, or porting an old one to a new machine. We will discuss the debugging of frequently encountered bugs in serial code with gdb and the debugging of parallel (MPI and threaded) codes with DDT on live systems. Next, we will present general issues, common pitfalls and optimization strategies that are application to HPC systems. By using profiling tools available for parallel programs, we can see when the bottlenecks are. We can try to alleviate them by following common best practices, allowing one to obtain their results more quickly and at the same time minimize their burden on shared system resources. Walk-through examples and case studies will be presented during the session to illustrate the concepts.

=====Details=====
''Date/time''

Friday July 17, 9:30 - 16:30

''Location''

Galbraith Building, Room 405

35 St. George Street

Toronto, ON M5S 1A4

''Instructors''

Dr. Ramses van Zon and Dr. Mike Nolta (SciNet)

=====Prerequisites=====
Basic familiarity with a high-level language (C/C++/Fortran) as well as compiling and running programs via the command line environment on a HPC system.

=====Slides and other material=====
To come

2015 Ontario Summer School for High Performance Computing Central

2015-07-13T12:36:24Z

Ejspence: /* Slides and other material */

[[File:Summerschool_hpc_2015.gif]]

===About===

The 2015 Ontario Summer School on High Performance Computing, whose second installment of the year will be hosted by SciNet, provides attendees with opportunities to learn and share knowledge and experience in high performance and technical computing. The format: a five-day workshop with mixed lectures and hands-on sessions on a number of selected subjects, including shared memory programming, distributed memory programming, parallel data processing, and general purpose graphics processing unit programming.

Below are descriptions and course material for the Central version, given from July 13th to July 17th, 2015, in Toronto.



__TOC__

===Introduction to the Linux Shell===

====Abstract====

Working with many of the HPC systems in Ontario involves using the linux/unix command line. This provides a very powerful interface, but it can be quite daunting for the uninitiated. Become initiated with this course. This hands on session will cover basic commands and scripting, as well as touching on some powerful constructs like regular expressions. It could be a great boon for your productivity!

====Details====
''Time/date''

Monday July 13, 9:30 - 12:30

''Location''

Galbraith Building, Room 404

35 St. George Street

Toronto, ON M5S 1A4

''Instructor''

Dr. Mike Nolta (SciNet)

====Prerequisites====
A laptop with a linux-like environment.

====Slides and other material====
* To come
* [[Media:SS2014_IntroToShell.tgz|Files]]

===Introduction to High Performance Computing===

=====Abstract=====
This session will provide an introduction to basic concepts of high performance computing. It is intended to be a high level primer for those largely new to HPC, and serve as a foundation upon which to build over the coming days. Topics will include motivation for HPC, essential issues, problem characteristics as they apply to parallelism and a high level overview of parallel programming models. Strategies of running large sets of serial processes using e.g. GNU parallel, will also be presented.

=====Details=====
''Time/date''

Monday July 13, 13:30 - 16:30

''Location''

Galbraith Building, Room 404

35 St. George Street

Toronto, ON M5S 1A4

''Instructor''

Dr. Marcelo Ponce (SciNet)

=====Prerequisites=====
Basic Linux shell skills Registration for all summer school sessions will become available at https://www.sharcnet.ca/events/ss2015.

=====Slides and other material=====
To come

===Data Analysis with R===

=====Abstract=====
This session offers a brief introduction to R, with a focus on data analysis and statistics.

=====Details=====

''Date/time''

Monday July 13, 9:30 - 12:30

''Location''

Galbraith Building, Room 405

35 St. George Street

Toronto, ON M5S 1A4

''Instructor''

Dr. Erik Spence (SciNet)

=====Prerequisites=====

Some programming experience in any language. You should have used and written functions. Bring a laptop with a R and Rstudio installed.

=====Slides and other material=====
[https://wiki.scinet.utoronto.ca/wiki/images/7/73/Intro-R.pdf Slides].

===Parallel R===

=====Abstract=====
This session will cover parallel programming R, with a focus on parallel data analysis. Topics covered include snow, parallel, and foreach/doparallel.

=====Details=====

''Time/date''

Monday July 13, 13:30 - 16:30

''Location''

Galbraith Building, Room 405

35 St. George Street

Toronto, ON M5S 1A4

''Instructor''

Dr. Erik Spence (SciNet)

=====Prerequisites=====
Experience with programming in R and a rudimentary understanding of parallel computing.

=====Slides and other material=====
[File:file.pdf Slides].

===Programming Shared Memory Systems with OpenMP===

=====Abstract=====
In this session, in which lectures and hands-on labs are interspersed, the students will learn the basics of shared memory programming with OpenMP. In particular, we will discuss the OpenMP's threads, memory, and performance, reductions and load balancing. We will also discuss extensions to heterogeneous systems such as offered by the OpenMP 4.0 and OpenACC standards.

=====Details=====

''Time/date''

Tuesday July 14, 9:30 - 16:30

''Location''

Galbraith Building, Room 404

35 St. George Street

Toronto, ON M5S 1A4

''Instructor''

Dr. Ramses van Zon (SciNet)

=====Prerequisites=====
C/C++ and/or Fortran scientific programming; Experience editing and compiling code in a Linux environment.

=====Slides and other material=====
To come

===Scientific Computing with Python===

=====Abstract=====
In this session, you will be taught how to use python for research computing purposes. Topics include: the basics of python, automation, numpy, scipy, file i/o, and visualization.

=====Details=====
''Time/date'''

Tuesday July 14, 9:30 - 12:30

''Location''

Galbraith Building, Room 405

35 St. George Street

Toronto, ON M5S 1A4

''Instructor''

Dr. Erik Spence (SciNet)

=====Prerequisites=====
A bit of coding experience in any programming or scripting language. Command-line experience in Linux is a plus.

=====Slides and other material=====
To come

===Python for High Performance Computing===

=====Abstract=====
This session will cover parallel programming in python, with a focus on parallel data analysis. We will cover subprocess, multiprocessing, pypar and other parallel-enabling python packages.

=====Details=====
''Time/date''

Tuesday July 14, 13:30 - 16:30

''Location''

Galbraith Building, Room 405

35 St. George Street

Toronto, ON M5S 1A4

''Instructor''

Dr. Erik Spence (SciNet)

=====Prerequisites=====
Python programming experience and a rudimentary understanding of parallel computing.

=====Slides and other material=====
To come

===Programming Clusters with MPI===

=====Abstract=====
In this two-day session, through lectures interspersed with hands-on labs, the students will learn the basics of MPI programming. Examples and exercises will be based on parallelization of common scientific computing problems.

=====Details=====
''Time/date''

Wednesday July 15, 9:30 - 16:30

Thursday July 16, 9:30 - 16:30

''Location''

Galbraith Building, Room 404

35 St. George Street

Toronto, ON M5S 1A4

''Instructor''

Dr. Scott Northrup (SciNet)

=====Prerequisites=====
Programming experience with Fortran or C/C++.

=====Slides and other material=====
To come

===Programming GPUs with CUDA===

=====Abstract=====
This is an introductory course covering programming and computing on GPUs---graphics processing units---which are an increasingly common presence in massively parallel computing architectures. This session will cover both of the available C-like programming frameworks: NVIDIA’s CUDA-C. The basics of GPU programming will be covered, and students will work through a number of hands on examples. Demonstrations of profiling and debugging applications running on the GPU will also be included. The structuring of data and computations that makes full use of the GPU will be discussed in detail. Students should be able to leave the course with the knowledge necessary to begin developing their own GPU applications.

=====Details=====
''Time/date''

Wednesday July 15, 9:30 - 16:30

Thursday July 16, 9:30 - 16:30

''Location''

Galbraith Building, Room 405

35 St. George Street

Toronto, ON M5S 1A4

''Instructors''
Dr. Pawel Pomorski and Dr. Sergey Mashchenko (SHARCNET)

=====Prerequisites=====
C/C++ scientific programming, experience editing and compiling code in a Linux environment. Some experience with CUDA and/or OpenMP a plus.

=====Slides and other materials=====
To come

===Visualization===

=====Abstract=====
The first part of this session will be a basic introduction to general-purpose scientific visualization tools such as gnuplot, grace(xmgr), remote tools (tunneling, VNC), python's matplotlib, and examples from other. The second part will be a mix of lecture and hands-on to introduce more specialized scientific visualization software such as vmd, paraview, and visit.

=====Details=====
''Time/date''

Friday July 17, 9:30 - 16:30

''Location''

Galbraith Building, Room 404

35 St. George Street

Toronto, ON M5S 1A4

''Instructor''
Dr. Marcelo Ponce (SciNet)

=====Prerequisites=====
Software installed, for the morning session: gnuplot, ssh (graphic client), vnc client, python and matplotlib; afternoon session: paraview.

=====Slides and other material=====
To come

===Debugging, Optimization, and Best Practices===

=====Abstract=====
Debugging is an important step in developing a new code, or porting an old one to a new machine. We will discuss the debugging of frequently encountered bugs in serial code with gdb and the debugging of parallel (MPI and threaded) codes with DDT on live systems. Next, we will present general issues, common pitfalls and optimization strategies that are application to HPC systems. By using profiling tools available for parallel programs, we can see when the bottlenecks are. We can try to alleviate them by following common best practices, allowing one to obtain their results more quickly and at the same time minimize their burden on shared system resources. Walk-through examples and case studies will be presented during the session to illustrate the concepts.

=====Details=====
''Date/time''

Friday July 17, 9:30 - 16:30

''Location''

Galbraith Building, Room 405

35 St. George Street

Toronto, ON M5S 1A4

''Instructors''

Dr. Ramses van Zon and Dr. Mike Nolta (SciNet)

=====Prerequisites=====
Basic familiarity with a high-level language (C/C++/Fortran) as well as compiling and running programs via the command line environment on a HPC system.

=====Slides and other material=====
To come

2015 Ontario Summer School for High Performance Computing Central

2015-07-13T12:35:53Z

Ejspence: /* Slides and other material */

[[File:Summerschool_hpc_2015.gif]]

===About===

The 2015 Ontario Summer School on High Performance Computing, whose second installment of the year will be hosted by SciNet, provides attendees with opportunities to learn and share knowledge and experience in high performance and technical computing. The format: a five-day workshop with mixed lectures and hands-on sessions on a number of selected subjects, including shared memory programming, distributed memory programming, parallel data processing, and general purpose graphics processing unit programming.

Below are descriptions and course material for the Central version, given from July 13th to July 17th, 2015, in Toronto.



__TOC__

===Introduction to the Linux Shell===

====Abstract====

Working with many of the HPC systems in Ontario involves using the linux/unix command line. This provides a very powerful interface, but it can be quite daunting for the uninitiated. Become initiated with this course. This hands on session will cover basic commands and scripting, as well as touching on some powerful constructs like regular expressions. It could be a great boon for your productivity!

====Details====
''Time/date''

Monday July 13, 9:30 - 12:30

''Location''

Galbraith Building, Room 404

35 St. George Street

Toronto, ON M5S 1A4

''Instructor''

Dr. Mike Nolta (SciNet)

====Prerequisites====
A laptop with a linux-like environment.

====Slides and other material====
* To come
* [[Media:SS2014_IntroToShell.tgz|Files]]

===Introduction to High Performance Computing===

=====Abstract=====
This session will provide an introduction to basic concepts of high performance computing. It is intended to be a high level primer for those largely new to HPC, and serve as a foundation upon which to build over the coming days. Topics will include motivation for HPC, essential issues, problem characteristics as they apply to parallelism and a high level overview of parallel programming models. Strategies of running large sets of serial processes using e.g. GNU parallel, will also be presented.

=====Details=====
''Time/date''

Monday July 13, 13:30 - 16:30

''Location''

Galbraith Building, Room 404

35 St. George Street

Toronto, ON M5S 1A4

''Instructor''

Dr. Marcelo Ponce (SciNet)

=====Prerequisites=====
Basic Linux shell skills Registration for all summer school sessions will become available at https://www.sharcnet.ca/events/ss2015.

=====Slides and other material=====
To come

===Data Analysis with R===

=====Abstract=====
This session offers a brief introduction to R, with a focus on data analysis and statistics.

=====Details=====

''Date/time''

Monday July 13, 9:30 - 12:30

''Location''

Galbraith Building, Room 405

35 St. George Street

Toronto, ON M5S 1A4

''Instructor''

Dr. Erik Spence (SciNet)

=====Prerequisites=====

Some programming experience in any language. You should have used and written functions. Bring a laptop with a R and Rstudio installed.

=====Slides and other material=====
[https://wiki.scinet.utoronto.ca/wiki/images/7/73/Intro-R.pdf Slides].

===Parallel R===

=====Abstract=====
This session will cover parallel programming R, with a focus on parallel data analysis. Topics covered include snow, parallel, and foreach/doparallel.

=====Details=====

''Time/date''

Monday July 13, 13:30 - 16:30

''Location''

Galbraith Building, Room 405

35 St. George Street

Toronto, ON M5S 1A4

''Instructor''

Dr. Erik Spence (SciNet)

=====Prerequisites=====
Experience with programming in R and a rudimentary understanding of parallel computing.

=====Slides and other material=====
[https://wiki.scinet.utoronto.ca/wiki/images/7/73/Parallel-R.pdf Slides].

===Programming Shared Memory Systems with OpenMP===

=====Abstract=====
In this session, in which lectures and hands-on labs are interspersed, the students will learn the basics of shared memory programming with OpenMP. In particular, we will discuss the OpenMP's threads, memory, and performance, reductions and load balancing. We will also discuss extensions to heterogeneous systems such as offered by the OpenMP 4.0 and OpenACC standards.

=====Details=====

''Time/date''

Tuesday July 14, 9:30 - 16:30

''Location''

Galbraith Building, Room 404

35 St. George Street

Toronto, ON M5S 1A4

''Instructor''

Dr. Ramses van Zon (SciNet)

=====Prerequisites=====
C/C++ and/or Fortran scientific programming; Experience editing and compiling code in a Linux environment.

=====Slides and other material=====
To come

===Scientific Computing with Python===

=====Abstract=====
In this session, you will be taught how to use python for research computing purposes. Topics include: the basics of python, automation, numpy, scipy, file i/o, and visualization.

=====Details=====
''Time/date'''

Tuesday July 14, 9:30 - 12:30

''Location''

Galbraith Building, Room 405

35 St. George Street

Toronto, ON M5S 1A4

''Instructor''

Dr. Erik Spence (SciNet)

=====Prerequisites=====
A bit of coding experience in any programming or scripting language. Command-line experience in Linux is a plus.

=====Slides and other material=====
To come

===Python for High Performance Computing===

=====Abstract=====
This session will cover parallel programming in python, with a focus on parallel data analysis. We will cover subprocess, multiprocessing, pypar and other parallel-enabling python packages.

=====Details=====
''Time/date''

Tuesday July 14, 13:30 - 16:30

''Location''

Galbraith Building, Room 405

35 St. George Street

Toronto, ON M5S 1A4

''Instructor''

Dr. Erik Spence (SciNet)

=====Prerequisites=====
Python programming experience and a rudimentary understanding of parallel computing.

=====Slides and other material=====
To come

===Programming Clusters with MPI===

=====Abstract=====
In this two-day session, through lectures interspersed with hands-on labs, the students will learn the basics of MPI programming. Examples and exercises will be based on parallelization of common scientific computing problems.

=====Details=====
''Time/date''

Wednesday July 15, 9:30 - 16:30

Thursday July 16, 9:30 - 16:30

''Location''

Galbraith Building, Room 404

35 St. George Street

Toronto, ON M5S 1A4

''Instructor''

Dr. Scott Northrup (SciNet)

=====Prerequisites=====
Programming experience with Fortran or C/C++.

=====Slides and other material=====
To come

===Programming GPUs with CUDA===

=====Abstract=====
This is an introductory course covering programming and computing on GPUs---graphics processing units---which are an increasingly common presence in massively parallel computing architectures. This session will cover both of the available C-like programming frameworks: NVIDIA’s CUDA-C. The basics of GPU programming will be covered, and students will work through a number of hands on examples. Demonstrations of profiling and debugging applications running on the GPU will also be included. The structuring of data and computations that makes full use of the GPU will be discussed in detail. Students should be able to leave the course with the knowledge necessary to begin developing their own GPU applications.

=====Details=====
''Time/date''

Wednesday July 15, 9:30 - 16:30

Thursday July 16, 9:30 - 16:30

''Location''

Galbraith Building, Room 405

35 St. George Street

Toronto, ON M5S 1A4

''Instructors''
Dr. Pawel Pomorski and Dr. Sergey Mashchenko (SHARCNET)

=====Prerequisites=====
C/C++ scientific programming, experience editing and compiling code in a Linux environment. Some experience with CUDA and/or OpenMP a plus.

=====Slides and other materials=====
To come

===Visualization===

=====Abstract=====
The first part of this session will be a basic introduction to general-purpose scientific visualization tools such as gnuplot, grace(xmgr), remote tools (tunneling, VNC), python's matplotlib, and examples from other. The second part will be a mix of lecture and hands-on to introduce more specialized scientific visualization software such as vmd, paraview, and visit.

=====Details=====
''Time/date''

Friday July 17, 9:30 - 16:30

''Location''

Galbraith Building, Room 404

35 St. George Street

Toronto, ON M5S 1A4

''Instructor''
Dr. Marcelo Ponce (SciNet)

=====Prerequisites=====
Software installed, for the morning session: gnuplot, ssh (graphic client), vnc client, python and matplotlib; afternoon session: paraview.

=====Slides and other material=====
To come

===Debugging, Optimization, and Best Practices===

=====Abstract=====
Debugging is an important step in developing a new code, or porting an old one to a new machine. We will discuss the debugging of frequently encountered bugs in serial code with gdb and the debugging of parallel (MPI and threaded) codes with DDT on live systems. Next, we will present general issues, common pitfalls and optimization strategies that are application to HPC systems. By using profiling tools available for parallel programs, we can see when the bottlenecks are. We can try to alleviate them by following common best practices, allowing one to obtain their results more quickly and at the same time minimize their burden on shared system resources. Walk-through examples and case studies will be presented during the session to illustrate the concepts.

=====Details=====
''Date/time''

Friday July 17, 9:30 - 16:30

''Location''

Galbraith Building, Room 405

35 St. George Street

Toronto, ON M5S 1A4

''Instructors''

Dr. Ramses van Zon and Dr. Mike Nolta (SciNet)

=====Prerequisites=====
Basic familiarity with a high-level language (C/C++/Fortran) as well as compiling and running programs via the command line environment on a HPC system.

=====Slides and other material=====
To come

2015 Ontario Summer School for High Performance Computing Central

2015-07-13T12:35:21Z

Ejspence: /* Slides and other material */

[[File:Summerschool_hpc_2015.gif]]

===About===

The 2015 Ontario Summer School on High Performance Computing, whose second installment of the year will be hosted by SciNet, provides attendees with opportunities to learn and share knowledge and experience in high performance and technical computing. The format: a five-day workshop with mixed lectures and hands-on sessions on a number of selected subjects, including shared memory programming, distributed memory programming, parallel data processing, and general purpose graphics processing unit programming.

Below are descriptions and course material for the Central version, given from July 13th to July 17th, 2015, in Toronto.



__TOC__

===Introduction to the Linux Shell===

====Abstract====

Working with many of the HPC systems in Ontario involves using the linux/unix command line. This provides a very powerful interface, but it can be quite daunting for the uninitiated. Become initiated with this course. This hands on session will cover basic commands and scripting, as well as touching on some powerful constructs like regular expressions. It could be a great boon for your productivity!

====Details====
''Time/date''

Monday July 13, 9:30 - 12:30

''Location''

Galbraith Building, Room 404

35 St. George Street

Toronto, ON M5S 1A4

''Instructor''

Dr. Mike Nolta (SciNet)

====Prerequisites====
A laptop with a linux-like environment.

====Slides and other material====
* To come
* [[Media:SS2014_IntroToShell.tgz|Files]]

===Introduction to High Performance Computing===

=====Abstract=====
This session will provide an introduction to basic concepts of high performance computing. It is intended to be a high level primer for those largely new to HPC, and serve as a foundation upon which to build over the coming days. Topics will include motivation for HPC, essential issues, problem characteristics as they apply to parallelism and a high level overview of parallel programming models. Strategies of running large sets of serial processes using e.g. GNU parallel, will also be presented.

=====Details=====
''Time/date''

Monday July 13, 13:30 - 16:30

''Location''

Galbraith Building, Room 404

35 St. George Street

Toronto, ON M5S 1A4

''Instructor''

Dr. Marcelo Ponce (SciNet)

=====Prerequisites=====
Basic Linux shell skills Registration for all summer school sessions will become available at https://www.sharcnet.ca/events/ss2015.

=====Slides and other material=====
To come

===Data Analysis with R===

=====Abstract=====
This session offers a brief introduction to R, with a focus on data analysis and statistics.

=====Details=====

''Date/time''

Monday July 13, 9:30 - 12:30

''Location''

Galbraith Building, Room 405

35 St. George Street

Toronto, ON M5S 1A4

''Instructor''

Dr. Erik Spence (SciNet)

=====Prerequisites=====

Some programming experience in any language. You should have used and written functions. Bring a laptop with a R and Rstudio installed.

=====Slides and other material=====
[https://wiki.scinet.utoronto.ca/wiki/images/7/73/Intro-R.pdf Slides].

===Parallel R===

=====Abstract=====
This session will cover parallel programming R, with a focus on parallel data analysis. Topics covered include snow, parallel, and foreach/doparallel.

=====Details=====

''Time/date''

Monday July 13, 13:30 - 16:30

''Location''

Galbraith Building, Room 405

35 St. George Street

Toronto, ON M5S 1A4

''Instructor''

Dr. Erik Spence (SciNet)

=====Prerequisites=====
Experience with programming in R and a rudimentary understanding of parallel computing.

=====Slides and other material=====
[file.pdf Slides].

===Programming Shared Memory Systems with OpenMP===

=====Abstract=====
In this session, in which lectures and hands-on labs are interspersed, the students will learn the basics of shared memory programming with OpenMP. In particular, we will discuss the OpenMP's threads, memory, and performance, reductions and load balancing. We will also discuss extensions to heterogeneous systems such as offered by the OpenMP 4.0 and OpenACC standards.

=====Details=====

''Time/date''

Tuesday July 14, 9:30 - 16:30

''Location''

Galbraith Building, Room 404

35 St. George Street

Toronto, ON M5S 1A4

''Instructor''

Dr. Ramses van Zon (SciNet)

=====Prerequisites=====
C/C++ and/or Fortran scientific programming; Experience editing and compiling code in a Linux environment.

=====Slides and other material=====
To come

===Scientific Computing with Python===

=====Abstract=====
In this session, you will be taught how to use python for research computing purposes. Topics include: the basics of python, automation, numpy, scipy, file i/o, and visualization.

=====Details=====
''Time/date'''

Tuesday July 14, 9:30 - 12:30

''Location''

Galbraith Building, Room 405

35 St. George Street

Toronto, ON M5S 1A4

''Instructor''

Dr. Erik Spence (SciNet)

=====Prerequisites=====
A bit of coding experience in any programming or scripting language. Command-line experience in Linux is a plus.

=====Slides and other material=====
To come

===Python for High Performance Computing===

=====Abstract=====
This session will cover parallel programming in python, with a focus on parallel data analysis. We will cover subprocess, multiprocessing, pypar and other parallel-enabling python packages.

=====Details=====
''Time/date''

Tuesday July 14, 13:30 - 16:30

''Location''

Galbraith Building, Room 405

35 St. George Street

Toronto, ON M5S 1A4

''Instructor''

Dr. Erik Spence (SciNet)

=====Prerequisites=====
Python programming experience and a rudimentary understanding of parallel computing.

=====Slides and other material=====
To come

===Programming Clusters with MPI===

=====Abstract=====
In this two-day session, through lectures interspersed with hands-on labs, the students will learn the basics of MPI programming. Examples and exercises will be based on parallelization of common scientific computing problems.

=====Details=====
''Time/date''

Wednesday July 15, 9:30 - 16:30

Thursday July 16, 9:30 - 16:30

''Location''

Galbraith Building, Room 404

35 St. George Street

Toronto, ON M5S 1A4

''Instructor''

Dr. Scott Northrup (SciNet)

=====Prerequisites=====
Programming experience with Fortran or C/C++.

=====Slides and other material=====
To come

===Programming GPUs with CUDA===

=====Abstract=====
This is an introductory course covering programming and computing on GPUs---graphics processing units---which are an increasingly common presence in massively parallel computing architectures. This session will cover both of the available C-like programming frameworks: NVIDIA’s CUDA-C. The basics of GPU programming will be covered, and students will work through a number of hands on examples. Demonstrations of profiling and debugging applications running on the GPU will also be included. The structuring of data and computations that makes full use of the GPU will be discussed in detail. Students should be able to leave the course with the knowledge necessary to begin developing their own GPU applications.

=====Details=====
''Time/date''

Wednesday July 15, 9:30 - 16:30

Thursday July 16, 9:30 - 16:30

''Location''

Galbraith Building, Room 405

35 St. George Street

Toronto, ON M5S 1A4

''Instructors''
Dr. Pawel Pomorski and Dr. Sergey Mashchenko (SHARCNET)

=====Prerequisites=====
C/C++ scientific programming, experience editing and compiling code in a Linux environment. Some experience with CUDA and/or OpenMP a plus.

=====Slides and other materials=====
To come

===Visualization===

=====Abstract=====
The first part of this session will be a basic introduction to general-purpose scientific visualization tools such as gnuplot, grace(xmgr), remote tools (tunneling, VNC), python's matplotlib, and examples from other. The second part will be a mix of lecture and hands-on to introduce more specialized scientific visualization software such as vmd, paraview, and visit.

=====Details=====
''Time/date''

Friday July 17, 9:30 - 16:30

''Location''

Galbraith Building, Room 404

35 St. George Street

Toronto, ON M5S 1A4

''Instructor''
Dr. Marcelo Ponce (SciNet)

=====Prerequisites=====
Software installed, for the morning session: gnuplot, ssh (graphic client), vnc client, python and matplotlib; afternoon session: paraview.

=====Slides and other material=====
To come

===Debugging, Optimization, and Best Practices===

=====Abstract=====
Debugging is an important step in developing a new code, or porting an old one to a new machine. We will discuss the debugging of frequently encountered bugs in serial code with gdb and the debugging of parallel (MPI and threaded) codes with DDT on live systems. Next, we will present general issues, common pitfalls and optimization strategies that are application to HPC systems. By using profiling tools available for parallel programs, we can see when the bottlenecks are. We can try to alleviate them by following common best practices, allowing one to obtain their results more quickly and at the same time minimize their burden on shared system resources. Walk-through examples and case studies will be presented during the session to illustrate the concepts.

=====Details=====
''Date/time''

Friday July 17, 9:30 - 16:30

''Location''

Galbraith Building, Room 405

35 St. George Street

Toronto, ON M5S 1A4

''Instructors''

Dr. Ramses van Zon and Dr. Mike Nolta (SciNet)

=====Prerequisites=====
Basic familiarity with a high-level language (C/C++/Fortran) as well as compiling and running programs via the command line environment on a HPC system.

=====Slides and other material=====
To come

File:Intro-R.pdf

2015-07-13T12:31:56Z

Ejspence: uploaded a new version of "File:Intro-R.pdf"

Data analysis with R slides for the 2015 Summer School.

2015 Ontario Summer School for High Performance Computing Central

2015-07-13T12:28:46Z

Ejspence: /* Slides and other material */

[[File:Summerschool_hpc_2015.gif]]

===About===

The 2015 Ontario Summer School on High Performance Computing, whose second installment of the year will be hosted by SciNet, provides attendees with opportunities to learn and share knowledge and experience in high performance and technical computing. The format: a five-day workshop with mixed lectures and hands-on sessions on a number of selected subjects, including shared memory programming, distributed memory programming, parallel data processing, and general purpose graphics processing unit programming.

Below are descriptions and course material for the Central version, given from July 13th to July 17th, 2015, in Toronto.



__TOC__

===Introduction to the Linux Shell===

====Abstract====

Working with many of the HPC systems in Ontario involves using the linux/unix command line. This provides a very powerful interface, but it can be quite daunting for the uninitiated. Become initiated with this course. This hands on session will cover basic commands and scripting, as well as touching on some powerful constructs like regular expressions. It could be a great boon for your productivity!

====Details====
''Time/date''

Monday July 13, 9:30 - 12:30

''Location''

Galbraith Building, Room 404

35 St. George Street

Toronto, ON M5S 1A4

''Instructor''

Dr. Mike Nolta (SciNet)

====Prerequisites====
A laptop with a linux-like environment.

====Slides and other material====
* To come
* [[Media:SS2014_IntroToShell.tgz|Files]]

===Introduction to High Performance Computing===

=====Abstract=====
This session will provide an introduction to basic concepts of high performance computing. It is intended to be a high level primer for those largely new to HPC, and serve as a foundation upon which to build over the coming days. Topics will include motivation for HPC, essential issues, problem characteristics as they apply to parallelism and a high level overview of parallel programming models. Strategies of running large sets of serial processes using e.g. GNU parallel, will also be presented.

=====Details=====
''Time/date''

Monday July 13, 13:30 - 16:30

''Location''

Galbraith Building, Room 404

35 St. George Street

Toronto, ON M5S 1A4

''Instructor''

Dr. Marcelo Ponce (SciNet)

=====Prerequisites=====
Basic Linux shell skills Registration for all summer school sessions will become available at https://www.sharcnet.ca/events/ss2015.

=====Slides and other material=====
To come

===Data Analysis with R===

=====Abstract=====
This session offers a brief introduction to R, with a focus on data analysis and statistics.

=====Details=====

''Date/time''

Monday July 13, 9:30 - 12:30

''Location''

Galbraith Building, Room 405

35 St. George Street

Toronto, ON M5S 1A4

''Instructor''

Dr. Erik Spence (SciNet)

=====Prerequisites=====

Some programming experience in any language. You should have used and written functions. Bring a laptop with a R and Rstudio installed.

=====Slides and other material=====
[https://wiki.scinet.utoronto.ca/wiki/images/7/73/Intro-R.pdf Slides].

===Parallel R===

=====Abstract=====
This session will cover parallel programming R, with a focus on parallel data analysis. Topics covered include snow, parallel, and foreach/doparallel.

=====Details=====

''Time/date''

Monday July 13, 13:30 - 16:30

''Location''

Galbraith Building, Room 405

35 St. George Street

Toronto, ON M5S 1A4

''Instructor''

Dr. Erik Spence (SciNet)

=====Prerequisites=====
Experience with programming in R and a rudimentary understanding of parallel computing.

=====Slides and other material=====
To come

===Programming Shared Memory Systems with OpenMP===

=====Abstract=====
In this session, in which lectures and hands-on labs are interspersed, the students will learn the basics of shared memory programming with OpenMP. In particular, we will discuss the OpenMP's threads, memory, and performance, reductions and load balancing. We will also discuss extensions to heterogeneous systems such as offered by the OpenMP 4.0 and OpenACC standards.

=====Details=====

''Time/date''

Tuesday July 14, 9:30 - 16:30

''Location''

Galbraith Building, Room 404

35 St. George Street

Toronto, ON M5S 1A4

''Instructor''

Dr. Ramses van Zon (SciNet)

=====Prerequisites=====
C/C++ and/or Fortran scientific programming; Experience editing and compiling code in a Linux environment.

=====Slides and other material=====
To come

===Scientific Computing with Python===

=====Abstract=====
In this session, you will be taught how to use python for research computing purposes. Topics include: the basics of python, automation, numpy, scipy, file i/o, and visualization.

=====Details=====
''Time/date'''

Tuesday July 14, 9:30 - 12:30

''Location''

Galbraith Building, Room 405

35 St. George Street

Toronto, ON M5S 1A4

''Instructor''

Dr. Erik Spence (SciNet)

=====Prerequisites=====
A bit of coding experience in any programming or scripting language. Command-line experience in Linux is a plus.

=====Slides and other material=====
To come

===Python for High Performance Computing===

=====Abstract=====
This session will cover parallel programming in python, with a focus on parallel data analysis. We will cover subprocess, multiprocessing, pypar and other parallel-enabling python packages.

=====Details=====
''Time/date''

Tuesday July 14, 13:30 - 16:30

''Location''

Galbraith Building, Room 405

35 St. George Street

Toronto, ON M5S 1A4

''Instructor''

Dr. Erik Spence (SciNet)

=====Prerequisites=====
Python programming experience and a rudimentary understanding of parallel computing.

=====Slides and other material=====
To come

===Programming Clusters with MPI===

=====Abstract=====
In this two-day session, through lectures interspersed with hands-on labs, the students will learn the basics of MPI programming. Examples and exercises will be based on parallelization of common scientific computing problems.

=====Details=====
''Time/date''

Wednesday July 15, 9:30 - 16:30

Thursday July 16, 9:30 - 16:30

''Location''

Galbraith Building, Room 404

35 St. George Street

Toronto, ON M5S 1A4

''Instructor''

Dr. Scott Northrup (SciNet)

=====Prerequisites=====
Programming experience with Fortran or C/C++.

=====Slides and other material=====
To come

===Programming GPUs with CUDA===

=====Abstract=====
This is an introductory course covering programming and computing on GPUs---graphics processing units---which are an increasingly common presence in massively parallel computing architectures. This session will cover both of the available C-like programming frameworks: NVIDIA’s CUDA-C. The basics of GPU programming will be covered, and students will work through a number of hands on examples. Demonstrations of profiling and debugging applications running on the GPU will also be included. The structuring of data and computations that makes full use of the GPU will be discussed in detail. Students should be able to leave the course with the knowledge necessary to begin developing their own GPU applications.

=====Details=====
''Time/date''

Wednesday July 15, 9:30 - 16:30

Thursday July 16, 9:30 - 16:30

''Location''

Galbraith Building, Room 405

35 St. George Street

Toronto, ON M5S 1A4

''Instructors''
Dr. Pawel Pomorski and Dr. Sergey Mashchenko (SHARCNET)

=====Prerequisites=====
C/C++ scientific programming, experience editing and compiling code in a Linux environment. Some experience with CUDA and/or OpenMP a plus.

=====Slides and other materials=====
To come

===Visualization===

=====Abstract=====
The first part of this session will be a basic introduction to general-purpose scientific visualization tools such as gnuplot, grace(xmgr), remote tools (tunneling, VNC), python's matplotlib, and examples from other. The second part will be a mix of lecture and hands-on to introduce more specialized scientific visualization software such as vmd, paraview, and visit.

=====Details=====
''Time/date''

Friday July 17, 9:30 - 16:30

''Location''

Galbraith Building, Room 404

35 St. George Street

Toronto, ON M5S 1A4

''Instructor''
Dr. Marcelo Ponce (SciNet)

=====Prerequisites=====
Software installed, for the morning session: gnuplot, ssh (graphic client), vnc client, python and matplotlib; afternoon session: paraview.

=====Slides and other material=====
To come

===Debugging, Optimization, and Best Practices===

=====Abstract=====
Debugging is an important step in developing a new code, or porting an old one to a new machine. We will discuss the debugging of frequently encountered bugs in serial code with gdb and the debugging of parallel (MPI and threaded) codes with DDT on live systems. Next, we will present general issues, common pitfalls and optimization strategies that are application to HPC systems. By using profiling tools available for parallel programs, we can see when the bottlenecks are. We can try to alleviate them by following common best practices, allowing one to obtain their results more quickly and at the same time minimize their burden on shared system resources. Walk-through examples and case studies will be presented during the session to illustrate the concepts.

=====Details=====
''Date/time''

Friday July 17, 9:30 - 16:30

''Location''

Galbraith Building, Room 405

35 St. George Street

Toronto, ON M5S 1A4

''Instructors''

Dr. Ramses van Zon and Dr. Mike Nolta (SciNet)

=====Prerequisites=====
Basic familiarity with a high-level language (C/C++/Fortran) as well as compiling and running programs via the command line environment on a HPC system.

=====Slides and other material=====
To come

File:Intro-R.pdf

2015-07-13T12:27:57Z

Ejspence: Data analysis with R slides for the 2015 Summer School.

Data analysis with R slides for the 2015 Summer School.

Scheduler

2015-07-03T13:29:23Z

Ejspence: /* GPC */

The queueing system used at SciNet is based around Cluster Resources [http://www.adaptivecomputing.com/products/hpc-products/moab-hpc-basic-edition/ Moab Workload Manager].
Moab is used on both the GPC and TCS however [http://www.adaptivecomputing.com/products/open-source/torque/ Torque] is used as the backend resource manager on the GPC and IBM's [http://www.ibm.com/software/products/en/tivoliworkloadschedulerloadleveler/ LoadLeveler] is used on the TCS.

This page outlines some of the most common Moab commands with full documentation available from Moab [http://docs.adaptivecomputing.com/mwm/7-2-9/help.htm#a.gcommandoverview.html%3FTocPath%3D4.0%2520Scheduler%2520Commands|_____0 here], the torque (pbs) commands full documentation is [http://docs.adaptivecomputing.com/torque/4-2-9/help.htm here].

Some common questions about the queuing system can be found on the [[FAQ]] as well.

=== Queues ===

==== GPC ====

===== batch =====

The batch queue is the default queue on the GPC allowing the user access to all the
resources for jobs between 15 minutes and upto 48 hours. If a specific queue is not specified, <tt>-q</tt> flag,
then a job is submitted to the batch queue. Most jobs will run in batch, and user's can use
feature flags to determine the type of nodes they require.

For example, to request two nodes anywhere on the GPC (QDR or DDR), use
<pre>
#PBS -l nodes=2:ppn=8,walltime=1:00:00
</pre>

For two nodes using DDR, use
<pre>
#PBS -l nodes=2:ddr:ppn=8,walltime=1:00:00
</pre>

To get two nodes using QDR, instead, you would use
<pre>
#PBS -l nodes=2:qdr:ppn=8,walltime=1:00:00
</pre>

===== debug =====

A debug queue has been set up primarily for code developers to quickly test
and evaluate their codes and configurations without having to wait in the batch queue. There are 10 nodes
currently reserved for the debug queue. It has quite restrictive limits to promote high turnover
and availability thus a user can only use 2 nodes (16 cores) for 2 hours, to a maximum
of 8 nodes (64 cores) for 1/2 an hour and can only have one job in the debug queue at a time. There
is no minimum time limit on this queue.

<pre>
$ qsub -l nodes=1:ppn=8,walltime=1:00:00 -q debug -I
</pre>

===== largemem =====

The largemem queue is used for accessing one of two 16 core with 128 GB memory intel Xeon (non-nehalem) nodes.

<pre>
$ qsub -l nodes=1:ppn=16,walltime=1:00:00 -q largemem -I
</pre>

==== TCS ====

The TCS currently only has one queue, or class, in use called "verylong" and its only
limitation is that jobs must be under 48 hours.

<pre>
#@ class = verylong
</pre>

=== Job Info===

To see all jobs queued on a system use
<pre>
$ showq
</pre>

Three sections are shown; running, idle, and blocked. Idle jobs are commonly referred to as queued jobs
as they meet all the requirements, however they are waiting for available resources. Blocked jobs
are either caused by improper resource requests or more commonly by exceeding a user or groups allowable
resources. For example if you are allowed to submit 10 jobs and you submit 20, the first 10
jobs will be submitted properly and either run right away or be queued, however the other 10 jobs
will be blocked and the jobs won't be submitted to the queue until one of the first 10 finishes.

=== Available Resources ===

Determining when your job will run can be tricky as it involves a combination of queue type, node type, system reservations, and job priority. The following commands are provided to help you figure out what resources are currently available, however they may not tell you exactly when your job will run for the aforementioned reasons.

==== GPC ====
To show how many qdr nodes are currently free, use the show back fill command
<pre>
$ showbf -f qdr
</pre>

To show how many ddr nodes are free, use
<pre>
$ showbf -f ddr
</pre>

==== TCS ====
To show how many TCS nodes are free, use
<pre>
$ showbf -c verylong
</pre>

For example checking for a qdr job

<pre>
$ showbf -f qdr
Partition Tasks Nodes Duration StartOffset StartDate
--------- ----- ----- ------------ ------------ --------------
ALL 14728 1839 7:36:23 00:00:00 00:23:37_09/24
ALL 256 30 INFINITY 00:00:00 00:23:37_09/24
</pre>

shows that for jobs under 7:36:23 you can use 1839 nodes, but if you submit
a job over that time only 30 will be available. In this case this is
due to a large reservation made my SciNet staff, but from a users point
of view, showbf tells you very simply what is available and at what time point.
In this case, a user may wish to set #PBS -l walltime=7:30:00 in their script, or add -l walltime=7:30:00 to their qsub command in order to ensure that the jobs backfill the reserved nodes.

'''NOTE:''' showbf shows currently available nodes, however just because nodes are available
doesn't mean that your job will start right away. Job priority, system reservations
along with dedicated nodes, such as those for the debug queue, will alter when jobs
run so even if enough nodes appear "free", it doesn't mean your job will actually run right
away.

=== Job Submission ===

==== Interactive ====

On the GPC an interactive queue session can be requested using the following
<pre>
$ qsub -l nodes=2:ppn=8,walltime=1:00:00 -I
</pre>

In case you may experience longer than usual delays when requesting an interactive session, try specifying the "debug" queue:
<pre>
$ qsub -l nodes=2:ppn=8,walltime=1:00:00 -I -q debug
</pre>

==== Non-interactive (Batch) ====

For a non-interactive job submission you require a submission script formatted for the appropriate resource manger. Examples
are provided for the [[GPC_Quickstart#Submitting_A_Batch_Job | GPC]] and [[TCS_Quickstart#Submitting_A_Job | TCS]].

=== Job Status ===
<pre>
$ qstat jobid
</pre>
To see the status of all your jobs, use
<pre>
showq -u username
</pre>
or
<pre>
qstat -u username
</pre>

If your job appears to be blocked, you can try the following command
<pre>
$ checkjob jobid
</pre>
which gives more verbose and often a bit confusing output.

=== Cancel a Job ===

<pre>
$ canceljob jobid
</pre>

=== Accounting ===

For any user with an NRAC/LRAC allocation, a special account with the Resource Allocation Project (RAP) identifier (RAPI) from Compute Canada Database (CCDB) is set up in order to access the allocated resources. Please use the following instructions to run your job using your special allocation. This is necessary both for accounting purposes as well as to assign the appropriate priority to your jobs.

Each job run on the system will have a default RAP associated with it. Most users already have their default RAP properly set. However, if you have more than one allocation (different RAPs), you may need/want to change your default RAP in order to charge your jobs to a particular RAP.

==== Changing your default RAP ====

# Go to the [https://portal.scinet.utoronto.ca portal], login with your SciNet username and password.
# Click on "Change SciNet default RAP" and change your default RAP.

==== Specifying the RAP for GPC ====

Alternatively, you may want to assign a RAP for each particular job you run. There are two ways to specify an account for Moab/Torque: From the command line or inside the batch submission script.

===== Command line =====

Use the '-A RAPI' flag when you submit your job using qsub. Note that the command line option will override the submission script if an account is specified on both the submission script and the command line. "RAPI" is the RAP Identifier, e.g. abc-123-de.

===== Submission Script =====

Add a line in your submit script as follows:
<pre>
#PBS -A RAPI
</pre>

Please replace "RAPI" with your RAP Identifier.

==== Specifiying the RAP for TCS ====

Add a line in your submit script as follows:
<pre>
# @ account_no = RAPI
</pre>

Please replace "RAPI" with your RAP Identifier.

=== User Stats ===

Show current usage stats for a $USER

<pre>
$ showstats -u $USER
</pre>

=== Reservations ===
<pre>
$ showres
</pre>

Standard users can only see their reservations not other users or system ones.
To determine what is available a user can use "showbf", it shows what resources are
available and at what time level, taking into account running jobs and all the reservations. Refer to the [[Moab#Available_Resources | Available Resources]] section of this page for more details.

=== Job Dependencies ===

Sometimes you may want one job not to start until another job finishes, however
you would like to submit them both at the same time. This can be done
using job dependencies on both the GPC and TCS, however the commands
are different due to the underlying resource managers being different.

==== GPC ====

Use the -W flag with the following syntax in your submission script to have this job not start
until the job with jobid has successfully finished
<pre>
-W depend=afterok:jobid
</pre>

This functionality also allows to add dependencies on several jobs, eg.

<pre>
-W depend=afterok:jobid1:jobid2:...:jobidN
</pre>

which in this case, the job will not start until all the jobs jobid1, jobid2, ..., jobidN have successfully finished. Note that the length of the string "afterok:jobid1:jobid2...:jobidN" cannot exceed 1024 characters.

More detailed syntax, dependency options and examples can be found
[[http://docs.adaptivecomputing.com/torque/4-1-4/Content/topics/commands/qsub.htm#-W here]].

==== TCS ====

Loadleveler does job dependencies using what they call steps.
See the [[TCS_Quickstart#Steps | TCS Quickstart]] guide for an example.

=== Adjusting Job Priority ===

The ability to adjust job priorities downwards can also be of use to adjust relative priorities of jobs between users who are running jobs of the same allocation (eg, a default or NRAC allocation of the same PI). Priorities are determined by how much of the time of that allocation been currently used, and all users using that account will have identical priorities. This mechanism allows users to voluntarily reduce their priority to allow other users of the same allocation to run ahead of them.

In principle, by adjusting a jobs priority downwards, you could reduce your jobs priority to the point that someone elses job entirely could go ahead of yours. In practice, however, this is extremely unlikely. Users with NRAC allocations have priorities that are extremely large positive numbers that depend on their allocation and how much of it they have already used during the past fairshare window (2 weeks); it is very unlikely that two groups would have priorities that are within 10 or 100 or 1000 of each other.

Note that at the moment, we do not allow priorities to go negative; they are integers that can go no lower than 1. (This may change in the future) That means that users of accounts that have already used their full allocation during the current fairshare period (eg, over the past two weeks), and so whose priority would normally be negative but is capped at 1, can not lower their priority any further. Similar, users with a `default' allocation have priority 1, and cannot lower their priorities any further.

==== GPC ====

Moab allows users to adjust their jobs' priority moderately downwards, with the <tt>-p</tt> flag; that is, on a qsub line
<pre>
$ qsub ... -p -10 JOBID
</pre>
or in a script
<pre>
...
#PBS -p -10
..
</pre>

The number used (-10 in the examples above) can be any negative number down to -1024.

The ability to adjust job priorities downwards can be useful when you are running a number of jobs and want some to enter the queue at higher priorities than others. Note that if you absolutely require some jobs to start before others, you could use [[#Job Dependencies | job dependencies]] instead.

For a job that is currently queued, one can adjust its priority with
<pre>
$ qalter -p -10 JOBID
</pre>



=== Suspending a Running Job ===

Separate from, and in addition to, the ability to place a hold on a queued job, you may want to suspend a running job. For example, you may want to test the timing of events in a weakly coupled parallel environment.

==== GPC ====

To suspend a job:

<pre>
qsig -s STOP <jobid>
</pre>

and to start it again:

<pre>
qsig -s CONT <jobid>.
</pre>

Scripts are suspendable by default, so you don't need to add any signal handling for this to work.
As far as we can tell, the result is identical to using fg and ctrl-Z (or kill -STOP <PID>) in an interactive run.

More about using (and trapping) signals can be found on the [[Using Signals]] page.

=== QDR Network Switch Affinity ===

The QDR network is globally 5:1 oversubscribed, but on switch it has full 1:1 cross-section whereas the DDR is completely 1:1 non-blocking.
When a job is submitted to the GPC QDR nodes, the queuing system tries to fulfill the requirements with nodes on the same switch to improve network
performance. If not enough nodes are available on the same switch to satisfy the job request, the queue will then use any available nodes. This behavior can be changed
by using the submission flag " -l nodesetisoptional=false " which forces the queuing system to only
run this job on the same switch, thus the job will stay queued until enough nodes on one switch are available
to satisfy this request. Note that the maximum number of nodes on one switch is 30, so
a request of greater than 30 nodes with this flag will never run.
<pre>
qsub -l nodes=4:ppn=8,walltime=10:00,nodesetisoptional=false
</pre>

=== Multiple Job Submissions ===

If you are doing doing batch processing of a number of similar jobs on the GPC, torque has a feature called job arrays that can be used to simplify this process.
By using the "-t 0-N" option on the command line during job submission or putting it in the job script file, #PBS -t 0-N, torque will expand your
single job submission into N+1 jobs and sets the environment variable PBS_ARRAYID equal to that jobs specific number, ie 0-N, for each job. This
reduces the amount of calls to qsub, and can allow the user to have many less submission scripts. Job arrays also have the benefit of batching
groups of jobs allowing commands like qalter, qdel, qhold to work on all or a subset of the job array jobs with one command, instead of having
to run the command for each job.

In the following example, 10 jobs are submitted using a single command
<pre>
qsub -t 0-9 jobscript.sh
</pre>

and the submission script then modifies the job based on the PBS_ARRAYID.
<pre>
#!/bin/bash
#PBS -l nodes=1:ppn=8,walltime=10:00:00
#PBS -N array_jobs

cd ${PBS_O_WORKDIR}
mkdir job.${PBS_ARRAYID}
cd job.${PBS_ARRAYID}

echo "Running job ${PBS_ARRAYID}"
mpirun -np 8 ./mycode >& array_job.${PBS_ARRAYID}.out
</pre>

The JOBID and the job name both get the additional ARRAYID added onto them in the form of a hyphen, ie JOBID-ARRAYID. If for example you wanted
to cancel all the jobs in a job array you would use "qdel JOBID", whereas if you wanted to cancel just one of the jobs you would use
"qdel JOBID-ARRAYID".

See [http://docs.adaptivecomputing.com/torque/4-1-3/Content/topics/2-jobs/multiJobSubmission.htm here] and [http://docs.adaptivecomputing.com/torque/4-1-4/Content/topics/commands/qsub.htm here]
for full details.

=== Job Tools and Scripts ===

The following list of commands and scripts are useful at the moment of monitoring and managing submissions and jobs.
We have discussed them in a recent [https://support.scinet.utoronto.ca/education/go.php/183/file_storage/index.php/download/1/files%5B%5D/3702/ TechTalk].
More details can be found [https://wiki.scinet.utoronto.ca/wiki/index.php/SciNet_Command_Line_Utilities here].







In addition by loading the '''module extras''', the following scripts are available:
<pre>
diskUsage
</pre>
informs about the user and group file system usage.

<pre>
quota
</pre>
offers a shorter version for the user.

<pre>
qsum
</pre>
similar to showq, summarizes by user.

<pre>
jobperf jobID
</pre>
informs about the performance per-node of a given job.

<pre>
jobError <jobID | jobNAME>
</pre>
displays on realtime the error output of a given job.

<pre>
jobOutput <jobID | jobNAME>
</pre>
displays on realtime the standard output of a given job.

<pre>
jobcd <jobID | jobNAME>
</pre>
allows users to quickly move into the working directory of a given job.

<pre>
jobscript <jobID | jobNAME>
</pre>
displays the submission script used when submitting a given job.

<pre>
jobssh <jobID | jobNAME>
</pre>
allows users to connect to the head-node of a given job.

<pre>
jobtop <jobID | jobNAME>
</pre>
allows users to "top" on the head-node of a given job.

<pre>
jobtree [user]
</pre>
displays the jobs tree of dependencies for a given user

<pre>
jobdep <jobID>
</pre>
displays the dependencies of a given job.

FAQ

2015-06-16T13:41:22Z

Ejspence: /* Monitoring jobs in the queue */

__TOC__

==The Basics==
===Whom do I contact for support?===

Whom do I contact if I have problems or questions about how to use the SciNet systems?

'''Answer:'''

E-mail [mailto:support@scinet.utoronto.ca <support@scinet.utoronto.ca>]

In your email, please include the following information:

* your username on SciNet
* the cluster that your question pertains to (GPC or TCS; SciNet is not a cluster!),
* any relevant error messages
* the commands you typed before the errors occured
* the path to your code (if applicable)
* the location of the job scripts (if applicable)
* the directory from which it was submitted (if applicable)
* a description of what it is supposed to do (if applicable)
* if your problem is about connecting to SciNet, the type of computer you are connecting from.

Note that your password should never, never, never be to sent to us, even if your question is about your account.

Try to avoid sending email only to specific individuals at SciNet. Your chances of a quick reply increase significantly if you email our team!

===What does ''code scaling'' mean?===

'''Answer:'''

Please see [[Introduction_To_Performance#Parallel_Speedup|A Performance Primer]]

===What do you mean by ''throughput''?===

'''Answer:'''

Please see [[Introduction_To_Performance#Throughput|A Performance Primer]].

Here is a simple example:

Suppose you need to do 10 computations. Say each of these runs for
1 day on 8 cores, but they take "only" 18 hours on 16 cores. What is the
fastest way to get all 10 computations done - as 8-core jobs or as
16-core jobs? Let us assume you have 2 nodes at your disposal.
The answer, after some simple arithmetic, is that running your 10
jobs as 8-core jobs will take 5 days, whereas if you ran them
as 16-core jobs it would take 7.5 days. Take your own conclusions...

===I changed my .bashrc/.bash_profile and now nothing works===

The default startup scripts provided by SciNet, and guidelines for them, can be found [[Important_.bashrc_guidelines|here]]. Certain things - like sourcing <tt>/etc/profile</tt>
and <tt>/etc/bashrc</tt> are ''required'' for various SciNet routines to work!

If the situation is so bad that you cannot even log in, please send email [mailto:support@scinet.utoronto.ca support].

===Could I have my login shell changed to (t)csh?===

The login shell used on our systems is bash. While the tcsh is available on the GPC and the TCS, we do not support it as the default login shell at present. So "chsh" will not work, but you can always run tcsh interactively. Also, csh scripts will be executed correctly provided that they have the correct "shebang" <tt>#!/bin/tcsh</tt> at the top.

===How can I run Matlab / IDL / Gaussian / my favourite commercial software at SciNet?===

'''Answer:'''

Because SciNet serves such a disparate group of user communities, there is just no way we can buy licenses for everyone's commercial package. The only commercial software we have purchased is that which in principle can benefit everyone -- fast compilers and math libraries (Intel's on GPC, and IBM's on TCS).

If your research group requires a commercial package that you already have or are willing to buy licenses for, contact us at [mailto:support@scinet.utoronto.ca support@scinet] and we can work together to find out if it is feasible to implement the packages licensing arrangement on the SciNet clusters, and if so, what is the the best way to do it.

Note that it is important that you contact us before installing commercially licensed software on SciNet machines, even if you have a way to do it in your own directory without requiring sysadmin intervention. It puts us in a very awkward position if someone is found to be running unlicensed or invalidly licensed software on our systems, so we need to be aware of what is being installed where.

===Do you have a recommended ssh program that will allow scinet access from Windows machines?===

'''Answer:'''

The [[Ssh#SSH_for_Windows_Users | SSH for Windows users]] programs we recommend are:

* [http://mobaxterm.mobatek.net/en/ MobaXterm] is a tabbed ssh client with some Cygwin tools, including ssh and X, all wrapped up into one executable.
* [http://www.chiark.greenend.org.uk/~sgtatham/putty/ PuTTY] - this is a terminal for windows that connects via ssh. It is a quick install and will get you up and running quickly. '''WARNING:''' Make sure you download putty from the official website, because there are "trojanized" versions of putty around that will send your login information to a site in Russia (as reported [http://blogs.cisco.com/security/trojanized-putty-software here]). To set up your passphrase protected ssh key with putty, see [http://the.earth.li/~sgtatham/putty/0.61/htmldoc/Chapter8.html#pubkey here].
* [http://www.cygwin.com/ CygWin] - this is a whole linux-like environment for windows, which also includes an X window server so that you can display remote windows on your desktop. Make sure you include the openssh and X window system in the installation for full functionality. This is recommended if you will be doing a lot of work on Linux machines, as it makes a very similar environment available on your computer. To set up your ssh keys, following the Linux instruction on the [[Ssh keys]] page.
 To set up your ssh keys, following the Linux instruction on the [[Ssh keys]] page.

===My ssh key does not work! WARNING: REMOTE HOST IDENTIFICATION HAS CHANGED! ===

'''Answer:'''

[[Ssh_keys#Testing_Your_Key | Testing Your Key]]

* If this doesn't work, you should be able to login using your password, and investigate the problem. For example, if during a login session you get an message similar to the one below, just follow the instruction and delete the offending key on line 3 (you can use vi to jump to that line with ESC plus : plus 3). That only means that you may have logged in from your home computer to SciNet in the past, and that key is obsolete.
<pre>
$ ssh USERNAME@login.scinet.utoronto.ca

@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@**@@@@@@@@@@@@@@@@@@@@@@@@@@@@@
@ WARNING: REMOTE HOST IDENTIFICATION HAS CHANGED! @
@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@**@@@@@@@@@@@@@@@@@@@@@@@@@@@@@
IT IS POSSIBLE THAT SOMEONE IS DOING SOMETHING NASTY!
Someone could be eavesdropping on you right now (man-in-the-middle
attack)!
It is also possible that the RSA host key has just been changed.
The fingerprint for the RSA key sent by the remote host is
53:f9:60:71:a8:0b:5d:74:83:52:**fe:ea:1a:9e:cc:d3.
Please contact your system administrator.
Add correct host key in /home/<user>/.ssh/known_hosts to get rid of
this message.
Offending key in /home/<user>/.ssh/known_hosts:3
RSA host key for login.scinet.utoronto.ca
<http://login.scinet.utoronto.ca <http://login.scinet.utoronto.ca>> has
changed and you have requested
</pre>

* If you get the message below you may need to logout of your gnome session and log back in since the ssh-agent needs to be
restarted with the new passphrase ssh key.
<pre>
$ ssh USERNAME@login.scinet.utoronto.ca

Agent admitted failure to sign using the key.
</pre>

===Can't forward X: "Warning: No xauth data; using fake authentication data", or "X11 connection rejected because of wrong authentication."===

I used to be able to forward X11 windows from SciNet to my home machine, but now I'm getting these messages; what's wrong?

'''Answer:'''

This very likely means that ssh/xauth can't update your ${HOME}/.Xauthority file.

The simplest pssible reason for this is that you've filled your 10GB /home quota and so can't write anything to your home directory. Use

<pre>
$ module load extras
$ diskUsage
</pre>

to check to see how close you are to your disk usage on ${HOME}.

Alternately, this could mean your .Xauthority file has become broken/corrupted/confused some how, in which case you can delete that file, and when you next log in you'll get a similar warning message involving creating .Xauthority, but things should work.

===How come I can not login to TCS?===

'''Answer:'''

A SciNet account doesn't automatically entitle you to TCS access. At a minimum, TCS jobs need to run on at least 32 cores (64 preferred because of Simultaneous Multi Threading - [[TCS_Quickstart#Node_configuration|SMT]] - on these nodes) and need the large memory (4GB/core) and bandwidth on the system. Essentially you need to be able to explain why the work can't be done on the GPC.

===How can I reset the password for my Compute Canada account?===

'''Answer:'''

You can reset your password for your Compute Canada account here:

https://ccdb.computecanada.org/security/forgot

===How can I change or reset the password for my SciNet account?===

'''Answer:'''

To reset your password at SciNet please go to [https://portal.scinet.utoronto.ca/password_resets Password reset page].

If you know your old password and want to change it, that can be done here:

https://portal.scinet.utoronto.ca/change_password

===Why am I getting the error "Permission denied (publickey,gssapi-with-mic,password)"?===

This error can pop up in a variety of situations: when trying to log in, or when after a job has finished, when the error and output files fail to be copied (there are other possible reasons for this failure as well -- see [[FAQ#My_GPC_job_died.2C_telling_me_.60Copy_Stageout_Files_Failed.27|My GPC job died, telling me:Copy Stageout Files Failed]]).
In most cases, the "Permission denioed" error is caused by incorrect permission of the (hidden) .ssh directory. Ssh is used for logging in as well as for the copying of the standard error and output files after a job.

For security reasons,
the directory .ssh should only be writable and readable to you, but yours
has read permission for everybody, and thus it fails. You can change
this by
<pre>
chmod 700 ~/.ssh
</pre>
And to be sure, also do
<pre>
chmod 600 ~/.ssh/id_rsa ~/.ssh/id_rsa.pub ~/authorized_keys
</pre>

===ERROR:102: Tcl command execution failed? when loading modules ===
Modules sometimes require other modules to be loaded first.
Module will let you know if you didn’t.
For example:
<pre>
$ module purge
$ module load python
python/2.6.2(11):ERROR:151: Module ’python/2.6.2’ depends on one of the module(s) ’gcc/4.4.0’
python/2.6.2(11):ERROR:102: Tcl command execution failed: prereq gcc/4.4.0
$ gpc-f103n084-$ module load gcc python
$
</pre>

==Compiling your Code==

===How can I get g77 to work?===

The fortran 77 compilers on the GPC are ifort and gfortran. We have dropped support for g77. This has been a conscious decision. g77 (and the associated library libg2c) were completely replaced six years ago (Apr 2005) by the gcc 4.x branch, and haven't undergone any updates at all, even bug fixes, for over five years.
If we would install g77 and libg2c, we would have to deal with the inevitable confusion caused when users accidentally link against the old, broken, wrong versions of the gcc libraries instead of the correct current versions.

If your code for some reason specifically requires five-plus-year-old libraries, availability, compatibility, and unfixed-known-bug problems are only going to get worse for you over time, and this might be as good an opportunity as any to address those issues.

''A note on porting to gfortran or ifort:''

While gfortran and ifort are rather compatible with g77, one
important difference is that by default, gfortran does not preserve
local variables between function calls, while g77 does. Preserved
local variables are for instance often used in implementations of quasi-random number
generators. Proper fortran requires to declare such variables as SAVE
but not all old code does this.
Luckily, you can change gfortran's default behavior with the flag
<tt>-fno-automatic</tt>. For ifort, the corresponding flag is <tt>-noautomatic</tt>.

===Where is libg2c.so?===

libg2c.so is part of the g77 compiler, for which we dropped support. See [[#How can I get g77 to work on the GPC?]] for our reasons.

===Autoparallelization does not work!===

I compiled my code with the <tt>-qsmp=omp,auto</tt> option, and then I specified that it should be run with 64 threads - with
export OMP_NUM_THREADS=64

However, when I check the load using <tt>llq1 -n</tt>, it shows a load on the node of 1.37. Why?

'''Answer:'''

Using the autoparallelization will only get you so far. In fact, it usually does not do too much. What is helpful is to run the compiler with the <tt>-qreport</tt> option, and then read the output listing carefully to see where the compiler thought it could parallelize, where it could not, and the reasons for this. Then you can go back to your code and carefully try to address each of the issues brought up by the compiler.
We ''emphasize'' that this is just a rough first guide, and that the compilers are still not magical! For more sophisticated approaches to parallelizing your code, email us at [mailto:support@scinet.utoronto.ca <support@scinet.utoronto.ca>] to set up an appointment with one
of our technical analysts.

===How do I link against the Intel Math Kernel Library?===

If you need to link in the Intel Math Kernel Library (MKL) libraries, you are well advised to use the Intel(R) Math Kernel Library Link Line Advisor: http://software.intel.com/en-us/articles/intel-mkl-link-line-advisor/ for help in devising the list of libraries to link with your code.

'''''Note that this give the link line for the command line. When using this in Makefiles, replace $MKLPATH by ${MKLPATH}.'''''

'''''Note too that, unless the integer arguments you will be passing to the MKL libraries are actually 64-bit integers, rather than the normal int or INTEGER types, you want to specify 32-bit integers (lp64) .'''''

===Can the compilers on the login nodes be disabled to prevent accidentally using them?===

'''Answer:'''

You can accomplish this by modifying your .bashrc to not load the compiler modules. See [[Important .bashrc guidelines]].

==="relocation truncated to fit: R_X86_64_PC32": Huh?===

What does this mean, and why can't I compile this code?

'''Answer:'''

Welcome to the joys of the x86 architecture! You're probably having trouble building arrays larger than 2GB, individually or together. Generally, you have to try to use the medium or large x86 `memory model'. For the intel compilers, this is specified with the compile options

-mcmodel=medium -shared-intel

==="feupdateenv is not implemented and will always fail"===

How do I get rid of this and what does it mean?

'''Answer:'''
First note that, as ominous as it sounds, this is really just a warning, and has to do with the intel math library. You can ignore it (unless you really are trying to manually change the exception handlers for floating point exceptions such as divide by zero), or take the safe road and get rid off it by linking with the intel math functions library:<pre>-limf</pre>See also [[#How do I link against the Intel Math Kernel Library?]]

===Cannot find rdmacm library when compiling on GPC===

I get the following error building my code on GPC: "<tt>ld: cannot find -lrdmacm</tt>". Where can I find this library?

'''Answer:'''

This library is part of the MPI libraries; if your compiler is having problems picking it up, it probably means you are mistakenly trying to compile on the login nodes (scinet01..scinet04). The login nodes aren't part of the GPC; they are for logging into the data centre only. From there you must go to the GPC or TCS development nodes to do any real work.

=== Why do I get this error when I try to compile: "icpc: error #10001: could not find directory in which /usr/bin/g++41 resides" ?===

You are trying to compile on the login nodes. As described in the wiki ( https://support.scinet.utoronto.ca/wiki/index.php/GPC_Quickstart#Login ), or in the users guide you would have received with your account, Scinet supports two main clusters, with very different architectures. Compilation must be done on the development nodes of the appropriate cluster (in this case, gpc01-04). Thus, log into gpc01, gpc02, gpc03, or gpc04, and compile from there.

==Testing your Code==

=== Can I run a something for a short time on the development nodes? ===

I am in the process of playing around with the mpi calls in my code to get it to work. I do a lot of tests and each of them takes a couple of seconds only. Can I do this on the development nodes?

'''Answer:'''

Yes, as long as it's very brief (a few minutes). People use the development nodes
for their work, and you don't want to bog it down for people, and testing a real
code can chew up a lot more resources than compiling, etc. The procedures differ
depending on what machine you're using.

==== TCS ====

On the TCS you can run small MPI jobs on the tcs02 node, which is meant for
development use. But even for this test run on one node, you'll need a host file --
a list of hosts (in this case, all tcs-f11n06, which is the `real' name of tcs02)
that the job will run on. Create a file called `hostfile' containing the following:

tcs-f11n06
tcs-f11n06
tcs-f11n06
tcs-f11n06

for a 4-task run. When you invoke "poe" or "mpirun", there are runtime
arguments that you specify pointing to this file. You can also specify it
in an environment variable MP_HOSTFILE, so, if your file is in your /scratch directory, say
${SCRATCH}/hostfile, then you would do

<pre>
export MP_HOSTFILE=${SCRATCH}/hostfile
</pre>

in your shell. You will also need to create a <tt>.rhosts</tt> file in your
home director, again listing <tt>tcs-f11n06</tt> so that <tt>poe</tt>
can start jobs. After that you can simply run your program. You can use
mpiexec:

<pre>
mpiexec -n 4 my_test_program
</pre>

adding <tt> -hostfile /path/to/my/hostfile</tt> if you did not set the environment
variable above. Alternatively, you can run it with the poe command (do a "man poe" for details), or even by
just directly running it. In this case the number of MPI processes will by default
be the number of entries in your hostfile.

==== GPC ====

On the GPC one can run short test jobs on the GPC [[GPC_Quickstart#Compile.2FDevel_Nodes | development nodes ]]<tt>gpc01</tt>..<tt>gpc04</tt>;
if they are single-node jobs (which they should be) they don't need a hostfile. Even better, though, is to request an [[ Moab#Interactive | interactive ]] job and run the tests either in regular batch queue or using a short high availability [[ Moab#debug | debug ]] queue that is reserved for this purpose.

=== How do I run a longer (but still shorter than an hour) test job quickly ? ===

'''Answer'''

On the GPC there is a high turnover short queue called [[ Moab#debug | debug ]] that is designed for
this purpose. You can use it by adding
<pre>
#PBS -q debug
</pre>
to your submission script.

==Running your jobs==

===My job can't write to /home===

My code works fine when I test on the development nodes, but when I submit a job, or even run interactively in the development queue on GPC, it fails. What's wrong?

'''Answer:'''

As [[Data_Management#Home_Disk_Space | discussed]] [https://support.scinet.utoronto.ca/wiki/images/5/54/SciNet_Tutorial.pdf elsewhere], <tt>/home</tt> is mounted read-only on the compute nodes; you can only write to <tt>/home</tt> from the login nodes and devel nodes. (The [[GPC_Quickstart#128Glargemem | largemem nodes]] on GPC, in this respect, are more like devel nodes than compute nodes). In general, to run jobs you can read from <tt>/home</tt> but you'll have to write to <tt>/scratch</tt> (or, if you were allocated space through the RAC process, on <tt>/project</tt>). More information on SciNet filesytems can be found on our [[Data_Management | Data Management]] page.

===Error Submitting My Job: qsub: Bad UID for job execution MSG=ruserok failed ===

I write up a submission script as in the examples, but when I attempt to submit the job, I get the above error. What's wrong?

'''Answer:'''

This error will occur if you try to submit a job from the login nodes. The login nodes are the gateway to all of SciNet's systems (GPC, TCS, P7, ARC), which have different hardware and queueing systems. To submit a job, you must log into a development node for the particular cluster you are submitting to and submit from there.

===OpenMP on the TCS===

How do I run an OpenMP job on the TCS?

'''Answer:'''

Please look at the [[TCS_Quickstart#Submission_Script_for_an_OpenMP_Job | TCS Quickstart ]] page.

===Can I can use hybrid codes consisting of MPI and openMP on the GPC?===

'''Answer:'''

Yes. Please look at the [[GPC_Quickstart#Hybrid_MPI.2FOpenMP_jobs | GPC Quickstart ]] page.

===How do I run serial jobs on GPC?===

'''Answer''':

So it should be said first that SciNet is a parallel computing resource,
and our priority will always be parallel jobs. Having said that, if
you can make efficient use of the resources using serial jobs and get
good science done, that's good too, and we're happy to help you.

The GPC nodes each have 8 processing cores, and making efficient use of these
nodes means using all eight cores. As a result, we'd like to have the
users take up whole nodes (eg, run multiples of 8 jobs) at a time.

It depends on the nature of your job what the best strategy is. Several approaches are presented on the [[User_Serial|serial run wiki page]].

===Why can't I request only a single cpu for my job on GPC?===

'''Answer''':

On GPC, computers are allocated by the node - that is, in chunks of 8 processors. If you want to run a job that requires only one processor, you need to bundle the jobs into groups of 8, so as to not be wasting the other 7 for 48 hours. See [[User_Serial|serial run wiki page]].

===How do I run serial jobs on TCS?===

'''Answer''': You don't.

===But in the queue I found a user who is running jobs on GPC, each of which is using only one processor, so why can't I?===

'''Answer''':

The pradat* and atlaspt* jobs, amongst others, are jobs of the ATLAS high energy physics project. That they are reported as single cpu jobs is an artifact of the moab scheduler. They are in fact being automatically bundled into 8-job bundles but have to run individually to be compatible with their international grid-based systems.

===How do I use the ramdisk on GPC?===

To use the ramdisk, create and read to / write from files in /dev/shm/.. just as one would to (eg) ${SCRATCH}. Only the amount of RAM needed to store the files will be taken up by the temporary file system; thus if you have 8 serial jobs each requiring 1 GB of RAM, and 1GB is taken up by various OS services, you would still have approximately 7GB available to use as ramdisk on a 16GB node. However, if you were to write 8 GB of data to the RAM disk, this would exceed available memory and your job would likely crash.

It is very important to delete your files from ram disk at the end of your job. If you do not do this, the next user to use that node will have less RAM available than they might expect, and this might kill their jobs.

''More details on how to setup your script to use the ramdisk can be found on the [[User_Ramdisk|Ramdisk wiki page]].''

===How can I automatically resubmit a job?===

Commonly you may have a job that you know will take longer to run than what is
permissible in the queue. As long as your program contains [[Checkpoints|checkpoint]] or
restart capability, you can have one job automatically submit the next. In
the following example it is assumed that the program finishes before
the 48 hour limit and then resubmits itself by logging into one
of the development nodes.

<source lang="bash">
#!/bin/bash
# MOAB/Torque example submission script for auto resubmission
# SciNet GPC
#
#PBS -l nodes=1:ppn=8,walltime=48:00:00
#PBS -N my_job

# DIRECTORY TO RUN - $PBS_O_WORKDIR is directory job was submitted from
cd $PBS_O_WORKDIR

# YOUR CODE HERE
./run_my_code

# RESUBMIT 10 TIMES HERE
num=$NUM
if [ $num -lt 10 ]; then
num=$(($num+1))
ssh gpc01 "cd $PBS_O_WORKDIR; qsub ./script_name.sh -v NUM=$num";
fi
</source>

<pre>
qsub script_name.sh -v
</pre>

You can alternatively use [[ Moab#Job_Dependencies | Job dependencies ]] through the queuing system which will not start one job until another job has completed.

If your job can't be made to automatically stop before the 48 hour queue window, but it does write out checkpoints, you can use the timeout command to stop the program while you still have time to resubmit; for instance

<source lang="bash">
timeout 2850m ./run_my_code argument1 argument2
</source>

will run the program for 47.5 hours (2850 minutes), and then send it SIGTERM to exit the program.

===How can I pass in arguments to my submission script?===

If you wish to make your scripts more generic you can use qsub's ability
to pass in environment variables to pass in arguments to your script.
The following example shows a case where an input and an output
file are passed in on the qsub line. Multiple variables can be
passed in using the qsub "-v" option and comma delimited.

<source lang="bash">
#!/bin/bash
# MOAB/Torque example of passing in arguments
# SciNet GPC
#
#PBS -l nodes=1:ppn=8,walltime=48:00:00
#PBS -N my_job

# DIRECTORY TO RUN - $PBS_O_WORKDIR is directory job was submitted from
cd $PBS_O_WORKDIR

# YOUR CODE HERE
./run_my_code -f $INFILE -o $OUTFILE
</source>

<pre>
qsub script_name.sh -v INFILE=input.txt,OUTFILE=outfile.txt
</pre>

=== How can I run a job longer than 48 hours? ===

'''Answer:'''

The SciNet queues have a queue limit of 48 hours. This is pretty typical for systems of its size in Canada and elsewhere, and larger systems commonly have shorter limits. The limits are there to ensure that every user gets a fair share of the system (so that no one user ties up lots of nodes for a long time), and for safety (so that if one memory board in one node fails in the middle of a very long job, you haven't lost a months' worth of work).

Since many of us have simulations that require more than that much time, most widely-used scientific applications have "checkpoint-restart" functionality, where every so often the complete state of the calculation is stored as a checkpoint file, and one can restart a simulation from one of these. In fact, these restart files tend to be quite useful for a number of purposes.

If your job will take longer, you will have to submit your job in multiple parts, restarting from a checkpoint each time. In this way, one can run a simulation much longer than the queue limit. In fact, one can even write job scripts which automatically re-submit themselves until a run is completed, using [[FAQ#How_can_I_automatically_resubmit_a_job.3F | automatic resubmission. ]]

=== Why did showstart say it would take 3 hours for my job to start before, and now it says my job will start in 10 hours? ===

'''Answer:'''

Please look at the [[FAQ#How_do_priorities_work.2Fwhy_did_that_job_jump_ahead_of_mine_in_the_queue.3F | How do priorities work/why did that job jump ahead of mine in the queue? ]] page.

===How do priorities work/why did that job jump ahead of mine in the queue?===

'''Answer:'''

The [[Moab | queueing system]] used on SciNet machines is a [http://en.wikipedia.org/wiki/Priority_queue Priority Queue]. Jobs enter the queue at the back of the queue, and slowly make their way to the front as those ahead of them are run; but a job that enters the queue with a higher priority can `cut in line'.

The main factor which determines priority is whether or not the user (or their PI) has an [http://wiki.scinethpc.ca/wiki/index.php/Application_Process RAC allocation]. These are competitively allocated grants of computer time; there is a call for proposals towards the end of every calendar year. Users with an allocation have high priorities in an attempt to make sure that they can use the amount of computer time the committees granted them. Their priority decreases as they approach their allotted usage over the current window of time; by the time that they have exhausted that allotted usage, their priority is the same as users with no allocation (unallocated, or `default' users). Unallocated users have a fixed, low, priority.

This priority system is called `fairshare'; the scheduler attempts to make sure everyone has their fair share of the machines, where the share that's fair has been determined by the allocation committee. The fairshare window is a rolling window of two weeks; that is, any time you have a job in the queue, the fairshare calculation of its priority is given by how much of your allocation of the machine has been used in the last 14 days.

A particular allocation might have some fraction of GPC - say 4% of the machine (if the PI had been allocated 10 million CPU hours on GPC). The allocations have labels; (called `Resource Allocation Proposal Identifiers', or RAPIs) they look something like

abc-123-ab

where abc-123 is the PIs CCRI, and the suffix specifies which of the allocations granted to the PI is to be used. These can be specified on a job-by-job basis. On GPC, one adds the line
#PBS -A RAPI
to your script; on TCS, one uses
# @ account_no = RAPI
If the allocation to charge isn't specified, a default is used; each user has such a default, which can be changed at the same portal where one changes one's password,

https://portal.scinet.utoronto.ca/

A jobs priority is determined primarily by the fairshare priority of the allocation it is being charged to; the previous 14 days worth of use under that allocation is calculated and compared to the allocated fraction (here, 5%) of the machine over that window (here, 14 days). The fairshare priority is a decreasing function of the allocation left; if there is no allocation left (eg, jobs running under that allocation have already used 379,038 CPU hours in the past 14 days), the priority is the same as that of a user with no granted allocation. (This last part has been the topic of some debate; as the machine gets more utilized, it will probably be the case that we allow RAC users who have greatly overused their quota to have their priorities to drop below that of unallocated users, to give the unallocated users some chance to run on our increasingly crowded system; this would have no undue effect on our allocated users as they still would be able to use the amount of resources they had been allocated by the committees.) Note that all jobs charging the same allocation get the same fairshare priority.

There are other factors that go into calculating priority, but fairshare is the most significant. Other factors include
* amount of time waiting in queue (measured in units of the requested runtime). A waiting queue job gains priority as it sits in the queue to avoid job starvation.
* User adjustment of priorities ( See below ).

The major effect of these subdominant terms is to shuffle the order of jobs running under the same allocation.

===How do we manage job priorities within our research group?===

'''Answer:'''

Obviously, managing shared resources within a large group - whether it
is conference funding or CPU time - takes some doing.

It's important to note that the fairshare periods are intentionally kept
quite short - just two weeks long. So, for example, let us say that in your resource
allocation you have about 10% of the machine. Then for someone to use
up the whole two week amount of time in 2 days, they'd have to use 70%
of the machine in those two days - which is unlikely to happen by
accident. If that does happen,
those using the same allocation as the person who used 70% of the
machine over the two days will suffer by having much lower priority for
their jobs, but only for the next 12 days - and even then, if there are
idle cpus they'll still be able to compute.

There will be online tools for seeing how the allocation is being used,
and those people who are in charge in your group will be able to use
that information to manage the users, telling them to dial it down or
up. We know that managing a large research group is hard, and we want
to make sure we provide you the information you need to do your job
effectively.

One way for users within a group to manage their priorities within the group
is with [[Moab#Adjusting_Job_Priority | user-adjusted priorities]]; this is
described in more detail on the [[Moab | Scheduling System]] page.

=== How do I charge jobs to my RAC allocation? ===

'''Answer:'''

Please see the [[Moab#Accounting|accounting section of Moab page]].

=== How does one check the amount of used CPU-hours in a project, and how does one get statistics for each user in the project? ===

'''Answer:'''

This information is available on the scinet portal,https://portal.scinet.utoronto.ca, See also [[SciNet Usage Reports]].

==Monitoring jobs in the queue==

===Why hasn't my job started?===

'''Answer:'''

Use the moab command

<pre>
checkjob -v jobid
</pre>

and the last couple of lines should explain why a job hasn't started.

Please see [[Moab| Job Scheduling System (Moab) ]] for more detailed information

===How do I figure out when my job will run?===

'''Answer:'''

Please see [[Moab#Available_Resources| Job Scheduling System (Moab) ]]



===I submit my GPC job, and I get an email saying it was rejected===
'''Answer:'''

This happens because the job you've submitted breaks one of the rules of the queues and is rejected. An email
is sent with the JOBID, JOBNAME, and the reason it was rejected. The following is an example where a job
requests more than 48 hours and was rejected.

<pre>
PBS Job Id: 3462493.gpc-sched
Job Name: STDIN
job deleted
Job deleted at request of root@gpc-sched
MOAB_INFO: job was rejected - job violates class configuration 'wclimit too high for class 'batch_ib' (345600 > 172800)'
</pre>

Jobs on the TCS or GPC may only run for 48 hours at a time; this restriction greatly increases responsiveness of the queue and queue throughput for all our users. If your computation requires longer than that, as many do, you will have to [[ Checkpoints | checkpoint ]] your job and restart it after each 48-hour queue window. You can manually re-submit jobs, or if you can have your job cleanly exit before the 48 hour window, there are ways to [[ FAQ#How_can_I_automatically_resubmit_a_job.3F | automatically resubmit jobs ]].

Other rejections return a more cryptic error saying "job violates class configuration" such as follows:
<pre>
PBS Job Id: 3462409.gpc-sched
Job Name: STDIN
job deleted
Job deleted at request of root@gpc-sched
MOAB_INFO: job was rejected - job violates class configuration 'user required by class 'batch''
</pre>

The most common problems that result in this error are:

* '''Incorrect number of processors per node''': Jobs on the GPC are scheduled per-node not per-core and since each node has 8 processor cores (ppn=8) the smallest job allowed is one node with 8 cores (nodes=1:ppn=8). For serial jobs users must bundle or batch them together in groups of 8. See [[ FAQ#How_do_I_run_serial_jobs_on_GPC.3F | How do I run serial jobs on GPC? ]]
* '''No number of nodes specified''': Jobs submitted to the main queue must request a specific number of nodes, either in the submission script (with a line like <tt>#PBS -l nodes=2:ppn=8</tt>) or on the command line (eg, <tt>qsub -l nodes=2:ppn=8,walltime=5:00:00 script.pbs</tt>). Note that for the debug queue, you can get away without specifying a number of nodes and a default of one will be assigned; for both technical and policy reasons, we do not enforce such a default for the main ("batch") queue.
* '''There is a 15 minute walltime minimum''' on all queues except debug and if you set your walltime less than this, it will be rejected.

===Running checkjob on my job gives me messages about JobFail and rejected===

Running checkjob on my job gives me messages that suggest my job has failed, as below: what did I do wrong?

<pre>
AName: test
State: Idle
Creds: user:xxxxxx group:xxxxxxxx account:xxxxxxxx class:batch_ib qos:ibqos
WallTime: 00:00:00 of 8:00:00
BecameEligible: Wed Jul 23 10:39:27
SubmitTime: Wed Jul 23 10:38:22
(Time Queued Total: 00:01:47 Eligible: 00:01:05)

Total Requested Tasks: 8

Req[0] TaskCount: 8 Partition: ALL
Opsys: centos6computeA Arch: --- Features: ---

Notification Events: JobFail

IWD: /scratch/x/xxxxxxxx/xxxxxxx/xxxxxxx
Partition List: torque,DDR
Flags: RESTARTABLE
Attr: checkpoint
StartPriority: 76
rejected for Opsys - (null)
rejected for State - (null)
rejected for Reserved - (null)
NOTE: job req cannot run in partition torque (available procs do not meet requirements : 0 of 8 procs found)
idle procs: 793 feasible procs: 0

Node Rejection Summary: [Opsys: 117][State: 2895][Reserved: 19]

NOTE: job violates constraints for partition SANDY (partition SANDY not in job partition mask)

NOTE: job violates constraints for partition GRAVITY (partition GRAVITY not in job partition mask)

rejected for State - (null)
NOTE:
</pre>

'''Answer:'''

The output from check job is a little cryptic in places, and if you are wondering why your job hasn't started yet, you might think that "rejection" and "JobFail" suggest that there's something wrong. But the above message is actually normal; you can use the <tt>showstart</tt> command on your job to get a (preliminary, subject to change) estimate as to when the job will start, and you'll find that it is in fact scheduled to start up in the near future.

In the above message:

* `Notification Events: JobFail` just means that, if notifications are enabled, you'll get a message if the job fails;
* `job req cannot run in partition torque` just means that the job cannot run just yet (that's why it's queued);
* `job req cannot run in dynamic partition DDR now (insufficient procs available: 0 < 8)` says why: there aren't processors available; and
* `job violates constraints for partition SANDY/GRAVITY` just means that the job isn't eligable to run in those paritcular (small) sections of the cluster.

that is, the above output is the normal and expected (if somewhat cryptic) explanation as to why the job is waiting - nothing to worry about.

===How can I monitor my running jobs on TCS?===

How can I monitor the load of TCS jobs?

'''Answer:'''

You can get more information with the command
/xcat/tools/tcs-scripts/LL/jobState.sh
which I alias as:
alias llq1='/xcat/tools/tcs-scripts/LL/jobState.sh'
If you run "llq1 -n" you will see a listing of jobs together with a lot of information, including the load.

===Can I run cron jobs on devel nodes to monitor my jobs?===

Can I run cron jobs on devel nodes to monitor my jobs?

'''Answer:'''

No, we do not permit cron jobs to be run by users. To monitor the status of your jobs using a cron job running on your own machine, use the command

<pre>
ssh myusername@login.scinet.utoronto.ca "qstat -u myusername"
</pre>

or some variation of this command. Of course, you will need to have SSH keys setup on the machine running the cron job, so that password entry won't be necessary.

==Errors in running jobs==



=== I couldn't find the .o output file in the .pbs_spool directory as I used to ===

On Feb 24 2011, the temporary location of standard input and output files was moved from the shared file system ${SCRATCH}/.pbs_spool to the
node-local directory /var/spool/torque/spool (which resides in ram). The final location after a job has finished is unchanged,
but to check the output/error of running jobs, users will now have to ssh into the (first) node assigned to the job and look in
/var/spool/torque/spool.

This alleviates access contention to the temporary directory, especially for those users that are running a lot of jobs, and reduces the burden on the file system in general.

Note that it is good practice to redirect output to a file rather than to count on the scheduler to do this for you.

=== My GPC job died, telling me `Copy Stageout Files Failed' ===

'''Answer:'''

When a job runs on GPC, the script's standard output and error are redirected to
<tt>$PBS_JOBID.gpc-sched.OU</tt> and <tt>$PBS_JOBID.gpc-sched.ER</tt> in
/var/spool/torque/spool on the (first) node on which your job is running. At the end of the job, those .OU and .ER files are copied to where the batch script tells them to be copied, by default <tt>$PBS_JOBNAME.o$PBS_JOBID</tt> and<tt>$PBS_JOBNAME.e$PBS_JOBID</tt>. (You can set those filenames to be something clearer with the -e and -o options in your PBS script.)

When you get errors like this:
<pre>
An error has occurred processing your job, see below.
request to copy stageout files failed on node
</pre>
it means that the copying back process has failed in some way. There could be a few reasons for this. The first thing to '''make sure that your .bashrc does not produce any output''', as the output-stageout is performed by bash and further output can cause this to fail.
But it also could have just been a random filesystem error, or it could be that your job failed spectacularly enough to shortcircuit the normal job-termination process (e.g. ran out of memory very quickly) and those files just never got copied.

Write to [mailto:support@scinet.utoronto.ca <support@scinet.utoronto.ca>] if your input/output files got lost, as we will probably be able to retrieve them for you (please supply at least the jobid, and any other information that may be relevant).

Mind you that it is good practice to redirect output to a file rather than depending on the job scheduler to do this for you.



===IB Memory Errors, eg <tt> reg_mr Cannot allocate memory </tt>===

Infiniband requires more memory than ethernet; it can use RDMA (remote direct memory access) transport for which it sets aside registered memory to transfer data.

In our current network configuration, it requires a _lot_ more memory, particularly as you go to larger process counts; unfortunately, that means you can't get around the "I need more memory" problem the usual way, by running on more nodes. Machines with different memory or
network configurations may exhibit this problem at higher or lower MPI
task counts.

Right now, the best workaround is to reduce the number and size of OpenIB queues, using XRC: with the OpenMPI, add the following options to your mpirun command:

<pre>
-mca btl_openib_receive_queues X,128,256,192,128:X,2048,256,128,32:X,12288,256,128,32 -mca btl_openib_max_send_size 12288
</pre>

With Intel MPI, you should be able to do

<pre>
module load intelmpi/4.0.3.008
mpirun -genv I_MPI_FABRICS=shm:ofa -genv I_MPI_OFA_USE_XRC=1 -genv I_MPI_OFA_DYNAMIC_QPS=1 -genv I_MPI_DEBUG=5 -np XX ./mycode
</pre>

to the same end.

For more information see [[GPC MPI Versions]].

===My compute job fails, saying <tt>libpng12.so.0: cannot open shared object file</tt> or <tt>libjpeg.so.62: cannot open shared object file</tt>===

'''Answer:'''

To maximize the amount of memory available for compute jobs, the compute nodes have a less complete system image than the development nodes. In particular, since interactive graphics libraries like matplotlib and gnuplot are usually used interactively, the libraries for their use are included in the devel nodes' image but not the compute nodes.

Many of these extra libraries are, however, available in the "extras" module. So adding a "module load extras" to your job submission script - or, for overkill, to your .bashrc - should enable these scripts to run on the compute nodes.

==Data on SciNet disks==

===How do I find out my disk usage?===

'''Answer:'''

The standard unix/linux utilities for finding the amount of disk space used by a directory are very slow, and notoriously inefficient on the GPFS filesystems that we run on the SciNet systems. There are utilities that very quickly report your disk usage:

The <tt>'''diskUsage'''</tt> command, available with the 'extras' module on the login nodes, datamovers and the GPC devel nodes, provides information in a number of ways on the home, scratch, and project file systems. For instance, how much disk space is being used by yourself and your group (with the -a option), or how much your usage has changed over a certain period ("delta information") or you may generate plots of your usage over time.
This information is only updated hourly!

More information about these filesystems is available at the [[Data_Management | Data_Management]].

===How do I transfer data to/from SciNet?===

'''Answer:'''

All incoming connections to SciNet go through relatively low-speed connections to the <tt>login.scinet</tt> gateways, so using scp to copy files the same way you ssh in is not an effective way to move lots of data. Better tools are described in our page on [[Data_Management#Data_Transfer | Data Transfer]].

===My group works with data files of size 1-2 GB. Is this too large to transfer by scp to login.scinet.utoronto.ca ?===

'''Answer:'''

Generally, occasion transfers of data less than 10GB is perfectly acceptible to so through the login nodes. See [[Data_Management#Data_Transfer | Data Transfer]].

===How can I check if I have files in /scratch that are scheduled for automatic deletion?===

'''Answer:'''

Please see [[Storage_Quickstart#Scratch_Disk_Purging_Policy | Storage At SciNet]]

===How to allow my supervisor to manage files for me using ACL-based commands?===

'''Answer:'''

Please see [[Data_Management#File.2FOwnership_Management_.28ACL.29 | File/Ownership Management]]

===Can we buy extra storage space on SciNet?===

'''Answer:'''
Yes, please see [[Data_Management#Buying_storage_space_on_GPFS_or_HPSS | Buying storage space on GPFS or HPSS ]] for more details.

===Can I transfer files between BGQ and HPSS?===

'''Answer:'''
Yes, please see [https://support.scinet.utoronto.ca/wiki/index.php/BGQ#Bridge_to_HPSS Bridge to HPSS ] for more details.

==Keep 'em Coming!==

===Next question, please===

Send your question to [mailto:support@scinet.utoronto.ca <support@scinet.utoronto.ca>]; we'll answer it asap!

GPC Quickstart

2015-06-09T11:26:56Z

Ejspence: /* Modules and Environment Variables */

{{Infobox Computer
|image=[[Image:University_of_Tor_79284gm-a.jpg|center|300px|thumb]]
|name=General Purpose Cluster (GPC)
|installed=June 2009
|operatingsystem= Linux Centos 6.4
|loginnode= gpc01..gpc04 (from <tt>login.scinet</tt>)
|nnodes=3864 (30,912 cores)
|rampernode=16 Gb
|corespernode=8 (16 threads)
|interconnect=840 nodes 1:1 DDR, 3024 nodes 5:1 QDR
|vendorcompilers=icc (C) ifort (fortran) icpc (C++)
|queuetype=[[Moab | Moab/Torque]]
}}

__TOC__

===Specifications===
The General Purpose Cluster is an extremely large cluster (ranked [http://www.top500.org/list/2009/06/100 16th] in the world at its inception, and fastest in Canada) and is where most computations are to be done at SciNet. It is an IBM iDataPlex cluster based on Intel's Nehalem architecture (one of the [http://www.hpcwire.com/features/HPC-Vendors-Jump-On-Nehalem-42360237.html first in the world] to make use of the new chips). The GPC consists of 3,780 nodes (IBM iDataPlex DX360M2) with a total of 30,912 cores (Intel Xeon E5540) at 2.53GHz, with 16GB RAM per node (2GB per core). Approximately one quarter of the cluster is interconnected with non-blocking DDR InfiniBand while the rest of the nodes are connected with 5:1 blocked QDR InfiniBand. The compute nodes are accessed through a queuing system that allows jobs with a minimum of 15minutes and maximum wall time of 48 hours .

===Login===

First login via [[Ssh | ssh]] with your SciNet account at <tt>login.scinet.utoronto.ca</tt>, and from there you can proceed to the Development nodes (<tt>gpc01</tt>, <tt>gpc02</tt>, <tt>gpc03</tt>, or <tt>gpc04</tt>) to compile/test your code.

===Compile/Devel Nodes===

From a scinet login node you can ssh to <tt>gpc01</tt>..<tt>gpc04</tt>. You may also just use <tt>/scinet/gpc/bin/gpcdev</tt> to take directly to the dev node with lowest cpu load (reassessed every 5 minutes).
<pre>
% /scinet/gpc/bin/gpcdev -h
Usage: gpcdev [usual ssh options]
Example: gpcdev -X
redirected to dev node with lowest cpu load
</pre>

These nodes have the same hardware configuration as most of the compute nodes except with more memory -- 8 Nehalem processing cores with 36GB RAM and QDR Infiniband. You can compile and test your codes on these nodes. To interactively test on more than 8 processors, you can submit an [[GPC_Quickstart#Submitting_an_Interactive_.28Debug.29_Job | interactive job request]].

Your [[Storage_Quickstart | home directory]] is in <tt>$HOME</tt> (currently <tt>/home/g/group/USER</tt> but this can change, so it's best to use the environment variable $HOME or <tt>~</tt> in scripts). You have 10GB there that is backed up. '''Your home directory cannot be written to by the compute nodes!''' Thus, to run jobs, you'll use the <tt>$SCRATCH</tt> directory (currently <tt>/scratch/g/group/USER</tt> but again, use the environment variable). Here, there is a large amount of disk space, but it is not backed up. Thus it makes sense to keep your codes in /home, compile there, and then run them in the /scratch directory.

===Modules and Environment Variables===

To use most packages on the SciNet machines - including any of the compilers - , you will have to use the `modules' command. The command <tt>module load some-package</tt> will set your environment variables (<tt>PATH</tt>, <tt>LD_LIBRARY_PATH</tt>, etc) to include the default version of that package. <tt>module load some-package/specific-version</tt> will load a specific version of that package. This makes it very easy for different users to use different versions of compilers, MPI versions, libraries etc.

Note that to use even the gcc compilers you will have to do
<pre>
$ module load gcc
</pre>
but in fact you probably should use the intel compilers installed on this system as they often produce faster executables (and occasionally, much faster.)

A list of the installed software is available in [[Software_and_Libraries | Software & Libraries]] and can
be seen on the system by typing
<pre>
$ module avail
</pre>

To load a module (for example, the default version of the intel compilers)
<pre>
$ module load intel
</pre>
To unload a module
<pre>
$ module unload intel
</pre>
To unload all modules
<pre>
$ module purge
</pre>
To list all loaded modules
<pre>
$ module list
</pre>

These commands should got in your submission scripts to make sure you
are using the correct packages. It is possible to load them in your .bashrc files as well, but this is generally not recommended (see [[Important .bashrc guidelines]]), especially if you routinely have to flip back and forth between modules.

Note that a <tt>module load</tt> command ''only'' sets the environment variables in your current shell (and any subprocesses that the shell launches). It does ''not'' affect other shell environments; in particular, a queued job that is running is unaffected by you interactively loading a module, and conversely you loading a module at the prompt and then submitting a job does not ensure that the module is loaded when the job runs. To ensure that a module is loaded when a job runs, be sure to put your <tt>module load</tt> command in your job submission script.

===Compilers===

The intel compilers are icc/icpc/ifort for C/C++/Fortran, and are available with the default module "intel". The intel compilers are recommended over the GNU compilers. Documentation about icpc is available at
http://software.intel.com/en-us/articles/intel-software-technical-documentation/. The Intel compilers accept many of the options that the GNU compilers accept, but tend to produce faster programs on our system. If, for some reason, you really need the GNU compilers, the latest version of the GNU compiler collection (currently 4.4.0) is available by loading the "gcc" module, with gcc/g++/gfortran for C/C++/Fortran. Note that f77/g77 is not supported.

To ensure that the intel compilers are in your <tt>PATH</tt> and their libraries are in your <tt>LD_LIBRARY_PATH</tt>, use the command

<pre>
$ module load intel
</pre>

Optimize your code for the GPC machine using of at least the following compiler flags:
<pre>
-O3 -xHost
</pre>
(or <tt>-O3 -march=native</tt> for the GNU compilers).

*If your program uses openmp, add <tt>-openmp</tt> (<tt>-fopenmp</tt> for GNU compilers).
*If you get the warning <tt>feupdatreenv is not implemented</tt>, add -limf to the link line.
*If you need to link in the MKL libraries, you are well advised to use the Intel(R) Math Kernel Library Link Line Advisor: http://software.intel.com/en-us/articles/intel-mkl-link-line-advisor/ for help in devising the list of libraries to link with your code. '''Note that this give the link line for the command prompt. When using this in Makefiles, replace $MKLPATH by ${MKLPATH}.'''
*More questions about compiling? See the [[FAQ#Compiling_your_Code|FAQ]].

===Debuggers===

* '''ddt''' - Allinea's graphical parallel debugger, in the <tt>ddt</tt> module. Highly recommended!
* '''gdb''' - The GNU Debugger, available in the <tt>gdb</tt> module.
* '''idbc/idb''' - The intel debuggers, part of the <tt>intel</tt> module(s).
* '''ddd''' - A graphical debuggerThe GNU Debugger, available in the <tt>ddd</tt> module.

Note that to debug code, you have to give the <pre>-g</pre> flags to the compiler. The intel compiler needs the additional option <tt>-debug parallel</tt> to debug threaded/OpenMP code.

===[[ GPC_MPI_Versions | MPI]]===

SciNet currently provides multiple MPI libraries for the GPC; [http://www.open-mpi.org/ OpenMPI], and [http://software.intel.com/en-us/intel-mpi-library/ IntelMPI]. We currently recommend OpenMPI as the default, as it quite reliably demonstrates good performance on the infiniband network (and did so too on the ethernet network). For full details and options see the complete [[ GPC_MPI_Versions | '''MPI''']] section.

The MPI libraries are compiled with both the gnu compiler suite and the intel compiler suite. To use (for instance) the intel-compiled OpenMPI libraries, which we recommend as the default (and use for most of our examples here), use

<pre>
$ module load intel openmpi
</pre>

in your job submission scripts and on the command-line before compiling. Putting these in your <tt>.bashrc</tt> is no longer recommended (since Oct 10, 2013).

Other combinations behave similarly.

The MPI libraries define the wrappers mpicc/mpicxx/mpif90/mpif77 as wrappers around the appropriate compilers, which ensure the appropriate include and library directories are used in the compilation and linking steps.

We currently recommend the Intel + OpenMPI combination. However, if you require the GNU compilers as well as MPI, you will want to find the most recent openmpi module available with `gcc' in the version name. This will enable development and runtime with gcc/g++/gfortran and OpenMPI.


For mixed OpenMP/MPI code using Intel MPI, add the compilation flag -mt_mpi for full thread-safety (no such flag is necessary for OpenMPI).

===Submitting A Batch Job===

The SciNet machines are shared systems, and jobs that are to run on them are submitted to a queue; the
[[Moab | scheduler]] then orders the jobs in order to make the best use of the machine, and has them launched
when resources become availble. The intervention of the scheduler can mean that the jobs aren't
quite run in a first-in first-out order.

The scheduler will have the job run on one or more of the compute nodes of the GPC. There are a few important differences between the devel nodes and the compute nodes:

# '''On compute nodes, your home directory is read-only'''. You have to run your jobs from the $SCRATCH directory instead. See [https://support.scinet.utoronto.ca/wiki/index.php/Data_Management Data Management ] for more details on the file systems at SciNet.

# The available memory on compute nodes is approximately 14GB. The devel nodes have 48GB, but this is shared with all the users of the node.

# Some libraries, especially those for graphics, are not installed on the compute nodes. This leaves more memory available for your job, but if you have an application that requires such libraries (notably <tt>R</tt> and <tt>octave</tt>), you will need to "<tt>module load extras</tt>" in your job script to make them work.

The maximum [[wallclock time]] for a job in the queue is 48 hours; computations that will take longer than
this must be broken into 48-hour chunks and run as several jobs. Also a minimum job length of 15 minutes is enforced and shorter jobs
should be batched together. The usual way to do this is with [[checkpoints]],
writing out the complete state of the computation every so often in such a way that a job can be restarted from
this state information and continue on from where it left off. Generating [[checkpoints]] is a good idea anyway,
as in the unlikely event of a hardware failure during your run, it allows you to restart without having lost much work.

There are limits to how many jobs you can submit. If your group has a default account, up to 32 nodes at a time for 48 hours per job on the GPC cluster are allowed to be queued. This is a total limit, e.g., you could request 64 nodes for 24 hours. Jobs of users with an LRAC or NRAC allocation will run at a higher priority than others while their resources last. Because of the group-based allocation, it is conceivable that your jobs won't run if your colleagues have already exhausted your group's limits.

Note that scheduling big jobs greatly affects the queuer and other users, so you have to talk to us first to run massively parallel jobs (> 2048 cores). We will help make sure that your jobs start and run efficiently.

If your job should run in fewer than 48 hours, specify that in your script -- your job
will start sooner. (It's easier for the [[Moab | scheduler]] to fit in a short job than a long job). On the downside, the
job will be killed automatically by the queue manager software at the end of the specified [[wallclock time]], so if you
guess wrong you might lose some work. So the standard procedure is to estimate how long your job will take and
add 10% or so.

You interact with the queuing system through the queue/resource manager, [[Moab | Moab]] and [[Moab | Torque]]. To see all the jobs in the queue use
<pre>
$ showq
</pre>

To submit your own job, you must write a script which describes the job and how it is to be run (a sample script [[GPC_Quickstart#Submission_Script | follows]]) and submit it to the queue, using the command
<pre>
$ qsub [SCRIPT-FILE-NAME]
</pre>
where you will replace <tt>[SCRIPT-FILE-NAME]</tt> with the file containing the submission script. This will return a job ID, for example 31415, which is used to identify the jobs. Information about a queued job can be found using
<pre>
$ qstat [JOB-ID]
</pre>
or, for more detailed information, using
<pre>
$ checkjob [JOB-ID]
</pre>
and jobs can be canceled with the command
<pre>
$ canceljob [JOB-ID]
</pre>

Again, these commands have many options, which can be read about on their man pages.

Much more information on the queueing system is available on our [[Moab | queue]] page.

====Batch Submission Script: MPI====

A sample submission script is shown below for an mpi job with the <tt> #PBS </tt> directives at the top and the rest being
what will be executed on the compute node.

<source lang="bash">
#!/bin/bash
# MOAB/Torque submission script for SciNet GPC
#
#PBS -l nodes=2:ppn=8,walltime=1:00:00
#PBS -N test

# load modules (must match modules used for compilation)
module load intel openmpi

# DIRECTORY TO RUN - $PBS_O_WORKDIR is directory job was submitted from
cd $PBS_O_WORKDIR

# EXECUTION COMMAND; -np = nodes*ppn
mpirun -np 16 ./a.out
</source>

The lines that begin <tt>#PBS</tt> are commands that are parsed and interpreted by qsub at submission time, and control administrative things about your job. In this example, the script above requests two nodes, using 8 processors per node, for a [[wallclock time]] of one hour. (The resources required by the job are listed on the <tt>#PBS -l</tt> line.) Other options can be given in other <tt>#PBS</tt> lines, such as <tt>#PBS -N</tt>, which sets the name of the job.

The rest of the script is run as a bash script at run time. A bash shell on the first node of the two nodes that are requested executes these commands as a normal bash script, just as if you had run this as a shell script from the terminal. The only difference is that PBS sets certain environment variables that you can use in the script. <tt>$PBS_O_WORKDIR</tt> is set to be the directory that the command was 'submitted' from - eg, <tt>$SCRATCH/SOMEDIRECTORY</tt>. The script then uses the <tt>mpirun</tt> command to launch the job.



====Submitting Collections of Serial Jobs====

You cannot run purely serial jobs on the GPC (or any of SciNet's systems), as this would mean only one core out of 8 is used. If you have serial jobs, you have to bunch them together.
SciNet-approved methods for running collections of serial jobs can be found on the [[User_Serial|serial run wiki page]].

====Batch Submission Script: OpenMP====

For running OpenMP jobs, the procedure is similar as for MPI jobs:

<source lang="bash">
#!/bin/bash
# MOAB/Torque submission script for SciNet GPC (OpenMP)
#
#PBS -l nodes=1:ppn=8,walltime=1:00:00
#PBS -N test

# load modules (must match modules used for compilation)
module load intel

# DIRECTORY TO RUN - $PBS_O_WORKDIR is directory job was submitted from
cd $PBS_O_WORKDIR

export OMP_NUM_THREADS=8
./a.out
</source>

Note that [[Introduction_To_Performance#Throughput | in some circumstances]] it can be more efficient to run (say) two jobs each running on four threads than one job running on eight threads. In that case you can use the same `ampersand-and-wait' technique outlined for serial jobs (see [[User_Serial|serial run wiki page]]) for less-than-eight-core OpenMP jobs.

====Hybrid MPI/OpenMP jobs====

'''Using Intel MPI'''

Here is how to run hybrid codes using intelmpi::

http://software.intel.com/en-us/articles/hybrid-applications-intelmpi-openmp/

Make sure you compile with the -mt_mpi option to the compilers to use the thread safe libraries.
Set the environment variable I_MPI_PIN_DOMAIN:
<pre>
$ export I_MPI_PIN_DOMAIN=omp
</pre>
This will set the process pinning domain size to be equal to OMP_NUM_THREADS (which you should set to the desired number of threads per mpi process). Therefore, each MPI process can create $OMP_NUM_THREADS number of children threads for running within the corresponding domain. If OMP_NUM_THREADS is not set, each node is treated as a separate domain (which will allow as many threads per MPI processes as there are cores).

In addition, when invoking mpirun, you should add the argument "-ppn X", where X is the number of MPI processes per node.
For example:
<pre>
$ mpirun -ppn 2 -np 8 [executable]
</pre>
would start 2 mpi processes of <tt>[executable]</tt> per node for a total of 8 processes, so mpirun will try to run mpi processes on 4 nodes
(OMP_NUM_THREADS is then probably best set at 4).
Your job script should still ask for these 4 nodes with the line
<source lang="bash">
#PBS -l nodes=4:ppn=8,walltime=....
</source>
(<tt>ppn=8</tt> is not a mistake here; the ppn parameter has a different meaning for PBS and for mpirun)

''The ppn parameter to ''mpirun'' is very important! Without it, eight mpi jobs would get bunched on the first node in this example, leaving 3 nodes unused.''

NOTE: In order to pin OpenMP threads inside the domain, use the corresponding OpenMP feature by setting the KMP_AFFINITY environment variable, see [http://software.intel.com/sites/products/documentation/hpc/compilerpro/en-us/fortran/lin/compiler_f/optaps/common/optaps_openmp_thread_affinity.htm#KMP_AFFINITY_Environment_Variable|Intel's Compiler User and Reference Guide].

The IntelMPI manual is referenced on the front page of our wiki:

http://software.intel.com/sites/products/documentation/hpc/mpi/linux/reference_manual.pdf

For the above example of a total of 8 processes on 4 nodes, you could use the following script:
<source lang="bash">
#!/bin/bash
# MOAB/Torque submission script for SciNet GPC (hybrid job)
#
#PBS -l nodes=4:ppn=8,walltime=1:00:00
#PBS -N test

# load modules (must match modules used for compilation)
module load intel intelmpi

# DIRECTORY TO RUN - $PBS_O_WORKDIR is directory job was submitted from
cd $PBS_O_WORKDIR

# SET THE NUMBER OF THREADS PER PROCESS:
export OMP_NUM_THREADS=4

# PIN THE MPI DOMAINS ACCORDING TO OMP
export I_MPI_PIN_DOMAIN=omp

# EXECUTION COMMAND; -np = nodes*ppn
mpirun -ppn 2 -np 8 ./a.out
</source>

'''Using Open MPI'''

For mixed MPI/OpenMP jobs using OpenMPI, which is the default for many users, the procedure is similar, but details differ.

* Request the number of nodes in the PBS script.
* Set OMP_NUM_THREADS to the number of threads per MPI process.
* In addition to the -np parameter for mpirun, add the argument <tt>--bynode</tt>, so that the mpi processes are not bunched up.

So for example, to start a total of 8 processes on 4 nodes, you could use the following script
<source lang="bash">
#!/bin/bash
# MOAB/Torque submission script for SciNet GPC (hybrid job)
#
#PBS -l nodes=4:ppn=8,walltime=1:00:00
#PBS -N test

# load modules (must match modules used for compilation)
module load intel openmpi

# DIRECTORY TO RUN - $PBS_O_WORKDIR is directory job was submitted from
cd $PBS_O_WORKDIR

# SET THE NUMBER OF THREADS PER PROCESS:
export OMP_NUM_THREADS=4

# EXECUTION COMMAND; -np = nodes*processes_per_nodes; --byhost forces a round robin of nodes.
mpirun -np 8 --bynode ./a.out
</source>

* More questions about running your jobs? See the [[FAQ#Running_your_jobs|FAQs Running your jobs]] and [[FAQ#Errors_in_running_jobs|Errors in running jobs]] sections.

===Automated Email Notifications from Jobs===

By default, you get an email if your job fails ('''a'''borts) when run through the scheduler. You can setup email notifications as well for when the jobs '''b'''egins or '''e'''nds. For this, you use the <tt>-m</tt> submission option, followed by any combination of '''a''' (to get emails when the job fails), '''b''' (to get emails when the job starts), and '''e''' (to get emails when the job ends), or '''n''' (for no emails).

For instance, to get email in all three cases, you could have a job script like this:
<source lang="bash">
#!/bin/bash
# MOAB/Torque submission script for SciNet GPC
#
#PBS -l nodes=2:ppn=8,walltime=1:00:00
#PBS -N test
#PBS -m abe

# load modules (must match modules used for compilation)
module load intel openmpi

# DIRECTORY TO RUN - $PBS_O_WORKDIR is directory job was submitted from
cd $PBS_O_WORKDIR

# EXECUTION COMMAND; -np = nodes*ppn
mpirun -np 16 ./a.out
</source>

Notes:

* Getting email for every job is not necessarily a good idea, as you may get a lot of email. This is why the default is to send an email only when the job fails

* The email address that the emails are sent to is the one that you gave when you created your account. You cannot have the emails send to another address (even if you there is a '''-M''' scheduler option that would allow you to specify your email address, for security, this option is disabled on the GPC).

===Submitting an Interactive (Debug) Job===

Your development work flow may require a lot of small test runs. You are allowed to do these on the development nodes, as long as it's very brief (a few minutes), and does not use all cores on the machine. For anything more you will have to use the compute nodes.

It is sometimes convenient to run a job interactively; this can be very handy for debugging purposes. In this case, you type a <tt>qsub</tt> command which submits an interactive job to the queue; when the scheduler selects this job to run, then it starts a shell running on the first node of the job, which connects to your terminal. You can then type any series of commands (for instance, the same commands listed as in the batch submission script above) to run a job interactively.

For example, to start the same sort of job as in the batch submission script above, but interactively, one would type
<pre>
$ qsub -I -l nodes=2:ppn=8,walltime=1:00:00
</pre>
If your interactive job requires graphics, add "-X" to this command.

Note that this is exactly the <tt>#PBS -l</tt> line in the batch script above (which requests all 8 processors on each of 2 nodes for one hour), but prepended with a <tt>-I</tt> for `interactive'. When this job begins, your terminal will now show you as being logged in to one of the compute nodes, and one can type in any shell command, run <tt>mpirun</tt>, etc. When you exit the shell, the job will end. Interactive jobs can be used with any of the [[ Moab#GPC | GPC queues ]] however, there is a short
high turnover queue called [[ Moab#debug | debug ]] which can be especially useful when the system is busy:
<pre>
$ qsub -I -l nodes=2:ppn=8,walltime=1:00:00 -q debug
</pre>

*More questions about running test jobs? See the [[FAQ#Testing_your_Code|FAQ]].



===QDR vs. DDR Infiniband===

The GPC Infiniband network GPC has two sections, one connecting 3,024 nodes with 5:1 blocking/oversubscribed QDR and the second connecting 840 nodes 1:1 non-blocking DDR Infiniband. By default a user's job will go to whichever network section best accommodates it, typically smaller jobs to the QDR and larger jobs to the DDR. However a user can override this by simply adding the flags "ddr" or "qdr" to the job resource request.

For example, to request two nodes anywhere on the GPC (QDR or DDR), use
<pre>
#PBS -l nodes=2:ppn=8,walltime=1:00:00
</pre>
in your job submission script.

For two nodes using DDR, use
<pre>
#PBS -l nodes=2:ddr:ppn=8,walltime=1:00:00
</pre>

To get two nodes using QDR, instead, you would use
<pre>
#PBS -l nodes=2:qdr:ppn=8,walltime=1:00:00
</pre>

The queueing system also tries its best to keep jobs within the same switch of the QDR thus avoiding the 5:1 blocking. A user can can explicitly request this behaviour if their jobs is less than 30 nodes using

<pre>
#PBS -W x=nodesetisoptional:false
</pre>

===HyperThreading===

Each GPC compute node has 8 Nehalem cores (2 sockets each with a four-core Intel Xeon E5540 @ 2.53GHz). Thus, to make full use of the computing power of a GPC node, you must be running least 8 "tasks" -- MPI processes, or OpenMP threads.

Under most circumstances, running exactly 8 tasks is the most efficient way to use these nodes. However, sometimes software design (eg, having one thread for communication and one for computation) can usefully `oversubscribe' the number of physical cores, and running (say) twice as many tasks as cores can be a useful strategy. If your code is highly memory-bandwidth bound, having one task ready to run while another waits for memory access can make more effective use of the processor.

The Nehalem processors have hardware support for such two-way overloading of processors, through "HyperThreading"; there are an extra set of registers on each core to facilitate rapid switching between two tasks, making it look to the operating system that there are in fact 16 cores per node. Depending on the nature of your code, making use of these virtual extra cores may speed up or slow down your computation; you should run small test cases before running production jobs in this manner. In most cases, the speed difference will be under 10%. Some of our users have obtained an 8% speedup by running gromacs with 16 tasks instead of 8 on a single node (mpirun -np 16 ./gromacs/mdrun -npme 4 is 108% the speed of mpirun -np 8 ./gromacs/mdrun with -npme 2 or -1).

====HyperThreading with OpenMP====

To use hyperthreading with an OpenMP job, one just runs twice as many threads as one would have previously; eg, if you were running 8 threads before (<tt>export OMP_NUM_THREADS=8</tt>) you would run with 16 (<tt>export OMP_NUM_THREADS=16</tt>). Everything else remains the same, including the job submission script; one still uses <tt>ppn=8</tt> in the submission of the job, as Torque has no way of knowing (or reason for caring) that you will be running on 16 `virtual' cores rather than 8 physical cores.

====HyperThreading with MPI====

To use hyperthreading with an MPI job, one just runs twice as many MPI processes as one would have previously; eg, if you were running on three nodes using 8 MPI tasks per node and used <tt>mpirun ... -np 24</tt>, you could run instead with <tt>-np 48</tt>. Everything else remains the same, including the job submission script; one still uses <tt>ppn=8</tt> in the submission of the job, as Torque has no way of knowing (or reason for caring) that you will be running on 16 `virtual' cores rather than 8 physical cores.

Note that if you are using OpenMPI (as is the default), there is another consideration; OpenMPI assumes that there is no oversubscription and each task very aggressively makes full use of a core when it is waiting for a message (eg, the waits are "busywaits"). If you find a significant slowdown when running multiple MPI tasks per core with OpenMPI, you may want to try adding the additional option to mpirun: <tt>--mca mpi_yield_when_idle 1</tt>. This will increase the latency of individual messages, but free up the core to do additional work while waiting.

With IntelMPI, the problem should be less pronounced, but you can still improve things by using <tt>mpirun -genv I_MPI_SPIN_COUNT 1 ...</tt>

'''Examples of hyperthreading with MPI'''

Hyperthreading using gromacs: https://support.scinet.utoronto.ca/wiki/index.php/Gromacs#Hyperthreading_with_Gromacs

====HyperThreading with Hybrid MPI/OpenMP codes====

With a hybrid code, one has extra flexibility in how to assign the "extra" cores -- you could run extra MPI tasks or extra OpenMPI threads. As with all hybrid codes, the combination which results in the best performance depends very strongly on the nature of your code, and you should experiment with different combinations. In addition, with hybrid codes processor and memory affinity issues become very important; if you're unsure as to how to tune your application for best performance, please make an appointment with the SciNet technical analysts for more help.

===Memory Configuration===

'''16G'''

There are 3756 nodes which have 16G of memory, and is the primary configuration in the GPC. These nodes will be used by default.
On these nodes, about 2 GB is taken by the operating system. So for mpi runs with 8 processes per node, this leaves about 1.75GB max per mpi process. Do not try to use more than the available memory: the node will crash and your job will either die or hang until the requested walltime has elapsed.

If you need more memory per process or per thread, you can either try to use the limit number of larger memory nodes listed below, or you can run with less mpi processes, or use a different decomposition, such that the job fits on a node.



'''32G'''

There are 84 nodes which have 32G of memory. To request these nodes use:

<pre>
$ qsub -l nodes=1:m32g:ppn=8,walltime=1:00:00
</pre>

Again, also on these nodes, about 2 GB is taken by the operating system, but this is a relatively small amount compared to the total of 32GB.

'''64G/128G/256G'''

There are 72 16-core Intel Sandybridge nodes which have 64G, 2 with 128G, and 2 with 256G of memory available as part of the contributed [[Sandy]] cluster. These nodes are requested through the <tt>sandy</tt> queue.

<pre>
$ qsub -l nodes=1:m64g:ppn=16,walltime=1:00:00 -q sandy
$ qsub -l nodes=1:m128g:ppn=16,walltime=1:00:00 -q sandy
$ qsub -l nodes=1:m256g:ppn=16,walltime=1:00:00 -q sandy
</pre>

Again, also on these nodes, about 2 GB is taken by the operating system, but this is a relatively small amount compared to the total.

'''128G'''

There are two other stand-alone large memory (128GB) nodes which are primarily to be used for data analysis of runs. They have 16 cores and are intel machines running linux, but they are of a different architecture than the GPC compute nodes, so codes may have to be compiled separately for these machines. This also means that some modules that work on the other GPC nodes, such as octave, will not work in the 128G nodes.

These nodes can be accessed using a specific <tt>largemem</tt> queue.

<pre>
$ qsub -l nodes=1:ppn=16,walltime=1:00:00 -q largemem -I
</pre>

'''Note:''' To estimate your time of access to these nodes, use

<pre>
$ showq -w class=largemem
</pre>

====How to run less MPI processes per node for memory bound applications====

If your compute job requires more memory than is available, it will crash. In some cases, this will even crash the node itself.
One way to exceed the memory is if your MPI application is too large for 8 processes to fit in memory. Oftentimes, you can solve this problem by subdividing the computation over more processes (reducing the memory per process) and ask for more nodes. Sometimes, this does not work, for instance if each MPI process requires a fixed minimum chunk of memory that is too large. In that case, you will want to run with less mpi processes than there are cpus in a nodes (i.e. less than 8 for the GPC). Running in such a mode requires a change in the mpirun command. The mpirun command now should not only specify the total number of mpi processes, but should also specify how these processes will be distributed over the nodes. The default distribution of processes is such that the first nodes get loaded with 8 processes, than the second nodes gets filled up, etc, and this will clearly lead to the same memory issue that you had before. Instead, you would want your processes to be distributed evenly over the nodes. The syntax for specifying that depends on whether you are using openmpi or intelmpi. For openmpi running on 4 nodes with 3 processes per node, for example, you would invoke mpirun as follows
<pre>
mpirun -np 12 --bynode <application>
</pre>
For intelmpi, the same setup would be invoked using:
<pre>
mpirun -np 12 -ppn 3 <application>
</pre>

===Ram Disk===

On the GPC nodes, there is a `ram disk' available - up to half of the memory on the node may be used as a temporary file system. This is particularly useful for use in the early stages of migrating destop-computing codes to a High Performance Computing platform such as the GPC. It is much faster than real disk and does not require network traffic; however, each node sees its own ramdisk and cannot see files on that of other nodes. This is a very easy way to cache writes (by writing them to fast ram disk instead of slow `real' disk); and then one would periodically copy the files to files on /scratch or /project so that they are available after the job has completed.

To use the ramdisk, create and read to or write from files in /dev/shm/... just as one would to (eg) $SCRATCH. Only the amount of RAM needed to store the files will be taken up by the temporary file system; thus if you have 8 serial jobs each requiring 1 GB of RAM, and 1GB is taken up by various OS services, you would still have approximately 7GB available to use as ramdisk on a 16GB node. However, if you were to write 8 GB of data to the RAM disk, this would exceed available memory and your job would likely crash.

NOTE: it is very important to delete your files from ram disk at the end of your job. If you do not do this, the next user to use that node will have less RAM available than they might expect, and this might kill their jobs.

More details on how to setup your script to use the ramdisk can be found on the [[User_Ramdisk|Ramdisk wiki page]].

=== Managing jobs on the Queuing system ===

Information on checking available resources, starting, viewing, managing and canceling jobs on [[Moab | Moab/Torque]]. Also check out the [[FAQ#Monitoring_jobs_in_the_queue|FAQ]].

MATLAB

2015-05-29T17:19:12Z

Ejspence:

We often get questions about running MATLAB on GPC. With a few exceptions for compilers and debuggers, SciNet does not purchase licenses for commercial software. As such, SciNet does not have a license for MATLAB, nor will it in the future. If users wish to run MATLAB they must supply their own license, or explore alternative options. This page gives information about the options for getting your MATLAB code to run, in recommended order.

== Not using MATLAB ==

Users can attempt to run MATLAB code using the open-source program Octave, accessible through the octave module. Though there are some differences between the two programs, Octave has been designed to interpret MATLAB code and can often be used in place of MATLAB. If your MATLAB code does not use some of the more-fancy MATLAB toolboxes, you may be able to get away with using Octave instead. Be sure to test your implementation in Octave thoroughly before committing to this option.

It is worth observing that, while convenient for prototyping and running on a single workstation, there are reasons to avoid using MATLAB for larger HPC/ARC projects. These include the prohibitive license cost for large-scale work, poor performance at scale, and portability issues. If you can switch to a license-free option, such as Python, it may be worth the effort.

== Creating a MATLAB executable ==

If MATLAB must be used, you may be able to compile your MATLAB code into a stand-alone executable, and run this on a GPC compute node. The version of MATLAB being used will require a compiler license, and the compilation must be done on a Linux machine. Once the compilation is done, the executable can be copied to SciNet, and run using the MATLAB Compiler Runtime (MCR), which can be accessed using the mcr module. The MCR used must be the same version of MATLAB as the compiler. If the version of MCR that you need is not listed among the mcr module versions, contact us and we will install the version which you require.

== Tunnelling to a license server ==

If you have access to a non-SciNet MATLAB license server, and have installed MATLAB in your $HOME directory, you can setup your submission script to access the external license server. The following lines should be added to the beginning of your submission script:

<pre>
PORT=XXX # port number of the license server
LICENSE_IP=AAA.BBB.CCC.DDD # IP address of the license server
ssh gpc01 -L${PORT}:${LICENSE_IP}:${PORT} -N &
</pre>

This last line will tunnel the port from the compute node back to the license server, through gpc01. The port number and IP address of the licence server must be supplied by the system administrator of the license server.

== Using a different Consortium ==

Both [https://www.sharcnet.ca/my/software/show/54 Sharcnet] and [https://www.westgrid.ca/support/software/matlab Westgrid] have purchased different types of MATLAB licenses. Users can contact those consortia if they wish to attempt to run on those systems.

MATLAB

2015-05-29T16:57:44Z

Ejspence: /* Not using MATLAB */

We often get questions about running MATLAB on GPC. This page gives information about the options for getting your MATLAB code to run, in recommended order.

== Not using MATLAB ==

Alternatively, users can attempt to run MATLAB code using the open-source program Octave, accessible through the octave module. Though there are some differences between the two programs, Octave has been designed to interpret MATLAB code and can often be used in place of MATLAB. If your MATLAB code does not use some of the more-fancy MATLAB toolboxes, you may be able to get away with using Octave instead.

== Creating a MATLAB executable ==

If MATLAB must be used, you may be able to compile your MATLAB code into a stand-alone executable, and run this on a GPC compute node. The version of MATLAB being used will require a compiler license, and the compilation must be done on a Linux machine. Once the compilation is done, the executable can be copied to SciNet, and run using the MATLAB Compiler Runtime (MCR), which can be accessed using the mcr module. The MCR used must be the same version of MATLAB as the compiler. If the version of MCR that you need is not listed among the mcr module versions, contact us and we will install the version which you require.

== Running using a license server ==

If you have access to a non-SciNet MATLAB license server, and have installed MATLAB in your $HOME directory, you can setup your submission script to access the external license server. The following lines should be added to the beginning of your submission script:

<pre>
PORT=XXX # port number of the license server
LICENSE_IP=AAA.BBB.CCC.DDD # IP address of the license server
ssh gpc01 -L${PORT}:${LICENSE_IP}:${PORT} -N &
</pre>

This last line will tunnel the port from the compute node back to the license server, through gpc01. The port number and IP address of the licence server must be supplied by the system administrator of the license server.

== Using a different Consortium ==

Both [https://www.sharcnet.ca/my/software/show/54 Sharcnet] and [https://www.westgrid.ca/support/software/matlab Westgrid] have purchased different types of MATLAB licenses. Users can contact those consortia if they wish to attempt to run on those systems.