Previous System News

The current month's changes can be found on the wiki front page.

Updates in Jun 2012:

• GPC: A versions of the Intel compilers are available as intel/12.1.5. Version 12.1.3 is still the default.
• Scratch purging: the allowed time is still three months, but now files that were modified in the last three months will not get purged, even if they were never read in that period.
• ARC: cuda/4.1 is now the default CUDA module. The module cuda/4.2 is available as well, and will work with the newer gcc 4.6.1 compiler.

Updates in May 2012:

• GPC: a newer git version 1.7.10 is now available as a module (the default is still 1.7.1).
• GPC: silo is installed as a module
• GPC: gcc 4.7.0 available as module (version 4.6.1 is still the default)
• HPSS: Jobs will now run automatically.
• ARC: cuda 4.1 and 4.2 are available as modules (Note: 4.2 is not supported by the ddt debugger).
• P7: ncl available as a module
• P7: scons available as a module

Updates in Apr 2012:

• GPC: The GPC has been upgraded to a low-latency, high-bandwidth Infiniband network throughout the cluster. The temporary mpirun settings that were recommended before for multinode ethernet runs, are no longer in effect, as all MPI traffic is now going over InfiniBand. For most cases, mpirun -np X [executable] will work.

Updates in Mar 2012:

• New Blue Gene/Q system announced.

Updates in Feb 2012:

• GPC: A new version of the Intel compiler suite has become the default module. The C/C++ and fortran compilers in this suite are at version 12.1.3, while the MKL library is at version 10.3.9.
• GPC: New versions of parallel-netcdf and mpb have been installed.

Updates in Jan 2012:

• The new Resource Allocations will take effect on January 9, for groups who were awarded an allocation.
• On January 30th, CentOS 5 was phased out.
• The "diskUsage" command has been improved and its output has been simplified.
• GPC: Due to some changes we are making to the GigE nodes, if you run multinode ethernet MPI jobs, you will need to explicitly request the ethernet interface in your mpirun: For Openmpi: mpirun --mca btl self,sm,tcp; For IntelMPI: mpirun -env I_MPI_FABRICS shm:tcp.. There is no need to do this if you run on IB, or if you run single node mpi jobs on the ethernet (GigE) nodes.

Updates in December 2011:

• GPC transition from CentOS 5 to CentOS 6 completed. A few nodes still have the old CentOS 5 for validation purposes.

Updates in Nov 2011:

• Disks added to the scratch file system and scratch now spans both of our DDN controllers. The performance of /scratch should improve as a result of more spindles and the use of a second controller while the available space increased by about 40%.
• The home, scratch, project and hpss file systems have been restructured (note: not all users have access to the latter two). As a consequence, users' files reside in different locations than before. The home and scratch file system are now group-based, and groups are furthermore clustered by the initial letter of the group name. For instance, the current home directory of user 'resu' in group 'puorg' is now /home/p/puorg/resu. The predefined variables $HOME, $SCRATCH, $PROJECT and $ARCHIVE point to the new directories.
• The High-Performance Storage System (HPSS) goes into full production with a concurrent change in /project policies. Users with storage allocations greater than 5 TB will find all their former /project files will now reside in HPSS and their /project quotas will be reduced to 5 TB.

Updates in Oct 2011:

• GPC: an OS update from CentOS 5.6 to CentOS 6 is being prepared, which will include updates to other programs (perl,gcc,python) as well. The ARC already uses the newer OS, and a few of the gpc nodes are using this as a test already, while we are in the process of porting all the modules to the new OS.

Updates in Sep 2011:

• File system: In the near future, the home, scratch, project and hpss file systems will be restructured (note: not all users have access to the latter two). To facilitate the transition, we ask the user's cooperation in making sure all their scripts and applications only use relative paths, or use the predefined variables $HOME, $SCRATCH and $PROJECT.

Updates in Aug 2011:

• GPC: an OS update from Centos 5.6 to CentOS 6 is being prepared, which will include updates to other programs (perl,gcc,python) as well. A few nodes are using this as a test already, and we are in the process of porting all the modules to the new OS. We encourage users willing to try the new environment out to contact us. Note that the ARC already uses the newer OS.
• GPC: "Climate Data Operator" versions 1.4.6 and 1.5.1 are available as modules cdo/1.4.6 and cdo/1.5.1, respectively.
• GPC: The "Climate Model Output Rewriter" is installed as module cmor/2.7.1.
• GPC: a newer version of R can now be used by explicitly loading the module R/2.13.1, while R/2.11.1 remains the default.
• GPC: ffmpeg has been added to the ImageMagick module.

Updates in Jul 2011:

• Extensive updates and tightening of security measured were performed. Users were required to change there passwords and regenerate (pass-phrase protected) ssh keys if they used these. We also updated the operating system on the gpc to close the security hole.
• GPC: nedit installed as a module.
• P7: any user that has access to the power-6 cluster tcs, can now give the power-7 cluster (p7) a try.

Updates in Jun 2011:

• HPSS, the new tape-backed storage system that expands the current storage capacity of SciNet, has entered its pilot phase. This means that the installation is complete, and select users are trying out the system. HPSS will be one of the ways in which storage allocation will be implemented.
• New IBM Power-7 cluster: The P7 cluster currently consists of 5 IBM Power 755 servers (at least 3 more servers to be added later this year). Each has four 8-core 3.3GHz Power7 CPUs and 128GB RAM, and features 4-way Simultaneous MultiThreading giving 128 threads per server. Linux is the operating system. Both the GCC and IBM compilers are installed, as well as POE and OpenMPI. LoadLeveler is used as the scheduler. Instruction on usage are on the wiki, but you will first have to ask us if you want access (support@scinet.utoronto.ca).
• GPC: The Berkeley compiler for Unified Parallel C (UPC) has been installed as the module upc. The compiler command is 'upcc'.
• GPC: Bugs in the gnuplot module were fixed.
• GPC: qhull support was added to octave.

Updates in May 2011:

• DDT, a parallel debugging program from Allinea, has been installed on the GPC, TCS, and ARC. DDT stands for "Distributed Debugging Tool" and is available as the module "ddt". It supports debugging OpenMP, MPI and CUDA applications in a graphical environment.

Updates in Apr 2011:

• GPC: Two versions of Goto Blas were installed, a single and multi-threaded one. They can be loaded as modules gotoblas/1.13-singlethreaded and gotoblas/1.13-multithreaded, respectively.
• Accelerator Research Cluster (ARC): A 8-node GPU test-cluster has been setup with a total of 64 Nehalem CPUs and 16 GPUs (NVIDIA, Cuda capability 2.0).

Updates in Mar 2011:

• TCS: The bug in the showstart command was fixed, and showstart may be used again to estimate the start time of your job.
• GPC: Issues regarding simultaneously loading the gcc/4.4.0 and the Intel compiler modules were resolved.
• GPC: A newer version of the gcc compiler suite, v4.6.0, has been installed. The default version is still 4.4.0.
• GPC: Octave version 3.2.4 has been installed on the GPC. You should for now consider this an experimental module.

Updates in Feb 2011:

• GPC: The temporary location of the standard error/output file for GPC jobs has changed.
• TCS: The showstart command has been disabled as it appears to contain a bug that puts jobs in a 'hold' state.

Updates in Jan 2011:

• Users can now request Network Switch Affinity for GPC ethernet jobs at runtime.
• For groups who were allocated compute time in this RAC allocation round, the new RAPs took effect on Jan 17th.
• File system servers were reconfigured to improve performance and stability. File access should be better, especially for writing.

Updates in Dec 2010:

• Addition of ImageMagick software packages on GPC.
• GPC: EncFS, an encrypted filesystem in user-space was installed. Works only on gpc01..04.
• GPC: Versions 12 of the Intel compilers have been installed as module 'intel/intel-v12.0.0.084'.
• GPC: The corresponding code analysis tools for these compilers are available as the module 'inteltools'.

Updates in Nov 2010:

• A number of module names have been changed on the GPC.
• GPC: A module for R was installed.
• GPC: padb was installed as a module.
• GPC: GNU parallel was installed as module 'gnu-parallel'.
• TCS: CDO (Climate Data Operators) was installed as module 'cdo/1.4.6'
• TCS: compilers xlf 13.1 and xlc 11.1 are available as modules as modules xlf/13.1 and vacpp/11.1, respectively.

Updates in Oct 2010:

• Further enhancements to diskUsage. You may also generate plots of your usage over time (with the -plot option)
• CPMD 3.13.2 installed on the GPC

Updates in Sept 2010:

• The diskUsage command has been enhanced, and now you may get information on how much your usage has changed over a certain period with the -de option.
• IntelMPI 3.x has been deprecated.
• GPFS file system was upgraded to 3.3.0.6; Stricter /scratch quotas of 10TB were implemented; check yours with /scinet/gpc/bin/diskUsage.
• GPC: The quantum chemistry software package NWChem 5.1.1 installed.
• GPC: CPMD, a Carr-Parinello molecular dynamics package, was installed.
• GPC: Gromacs 4.5.1 (single precision), a molecular simulation package was installed.

Updates in Aug 2010:

• GPC: A number of versions of PetSc 3.1 were installed.
• GPC: OpenSpeedShop v1.9.3.4 was installed.

Updates in Jul 2010:

• Started the pilot project on Hierarchical Storage Management (HSM)
• GPC: The intel module no longer automatically load the gcc module. Users which use both should have "module load gcc intel" in their .bashrc.
• GPC: The default intel compiler v11.1 was changed to Update 6 (module intel/11.1.0.72).
• TCS: OpenDX, a visualisation software package, was installed as a module.
• GPC: MEEP, a finite difference simulation software for electromagnetic systems with mpi support, was installed.
• GPC & TCS: A number of old modules has been deprecated.
• Recurring file system issues were mitigated as much as possible.

Updates in Jun 2010:

• Hyper-threading was enabled on GPC.