Transition to CentOS 6

GPC OS switch

The default operating system of the GPC will be upgraded from CentOS 5 to CentOS 6 next week. The switch will occur on Monday December 5th 2011 at noon. Jobs submitted after that time will be sent, by default, to nodes with the upgraded OS. It will be possible to specifically request the old OS for at least 2 weeks after the switchover (see below) but only a small subset of such nodes will be available.

This upgrade includes updates to other programs/modules, and will mean that you will have to recompile your code, if you have not done so already.

Transition

To prepare for this transition, 15 nodes are currently reserved for test purposes and are running with the new OS and modules. We encourage all users to try and compile their code there and to submit short test runs to the centos6 compute nodes.

As described in the Oct 16 email (and on http://www.scinethpc.ca/2011/10/gpc-os-upgrade-what-you-need-to-know), here's how you can access the CentOS 6 nodes now (until Monday at noon):

• Log in to one of the centos6 development nodes, gpc05 and gpc06.
• Make sure the right modules are loaded, as they likely will have changed in the upgrade.
• Recompile your code (don't forget a "make clean" or equivalent).
• Submit test jobs to the centos6 compute nodes using an extra flag "os=centos6computeA" in the "-l" argument, i.e.

 -l nodes=2:ppn=8,walltime=1:00:00,os=centos6computeA

And here's how you will access the CentOS 6 nodes after Monday at noon:

• The development nodes gpc01-4 will be running CentOS 6, so just login to those.
• Make sure the right modules are loaded, as they likely will have changed in the upgrade.
• Recompile your code (don't forget a "make clean" or equivalent).
• Submit jobs without any "os=" flag; the CentOS 6 operating system is now the default.

(In other words, the same procedure as now applies for CentOS 5.)

To make your .bashrc work for both versions of CentOS during the transition, check out the example .bashrc on the wiki page http://wiki.scinet.utoronto.ca/wiki/index.php/Important_.bashrc_guidelines on how to distiguish the two OSes.

Modules

Note that most modules that were available under the CentOS 5 are available with CentOS 6, but often their version numbers have changed. Use "module avail" on the centos6 devel nodes to see what module are available, or check the wiki page http://wiki.scinet.utoronto.ca/wiki/index.php/Software_and_Libraries. Let us know if there is anything that is not working as it should, or if there are modules that you need which are missing.

Procedure and time line

To make the change, nodes will start to be rebooted over the weekend, while at noon on Monday, the development nodes gpc01, gpc02, gpc03 and gpc04 will be rebooted, killing any logins to those node. However, running and queued jobs will not be killed.

After Monday at noon on, only a small number of nodes (~80) will be running CentOS 5, mainly for comparison purposes. gpc05 and gpc06 will be the corresponding CentOS 5 development nodes. To submit to these CentOS 5 nodes, you need to give an extra flag "os=centos53computeA" in the "-l" argument, i.e.

 -l nodes=2:ppn=8,walltime=1:00:00,os=centos53computeA

• Sat-Sun Dec 3-4 2011: compute nodes rebooted to prepare default OS switch (no jobs will get killed)
• Mon December 5 2011, noon: switch default operating system (gpc01-06 will be rebooted, running and scheduled jobs unaffected)
• Mid-January 2012: phase-out all CentOS 5 nodes.

CentOS 5 Phase-out

The CentOS 5 nodes will be phased out by mid-January at the latest.