Difference between revisions of "Important .bashrc guidelines"

From oldwiki.scinet.utoronto.ca
Jump to navigation Jump to search
 
(10 intermediate revisions by 2 users not shown)
Line 19: Line 19:
 
* Sourcing <tt>/etc/profile</tt> and <tt>/etc/bashrc</tt> is ''required'' for various SciNet routines to work!   
 
* Sourcing <tt>/etc/profile</tt> and <tt>/etc/bashrc</tt> is ''required'' for various SciNet routines to work!   
 
* Do not execute any commands that write to standard output, because that would breaks things like copying-out output files and <tt>scp</tt>. So for instance, no <tt>echo</tt> or <tt>module list</tt>.
 
* Do not execute any commands that write to standard output, because that would breaks things like copying-out output files and <tt>scp</tt>. So for instance, no <tt>echo</tt> or <tt>module list</tt>.
* Load only commonly used modules.  
+
* Load only commonly used modules, or preferrably, ''do not load any modules in your bashrc''.
 
* Distinguish, especially for the module command, between the gpc and tcs.
 
* Distinguish, especially for the module command, between the gpc and tcs.
* If you're trying out different implementations or versions of a module (e.g. mpi), you should not load them in the .bashrc, but explicitly on the command line before compiling and once more explicitly in your job script.
+
* If you're trying out different implementations or versions of a module (e.g. mpi), you should definitely not load them in the .bashrc, but explicitly on the command line before compiling and once more explicitly in your job script.
 
* To avoid accidentally compiling on the login nodes, you can 'shield' yourself from the compilers by wrapping the <tt>module load</tt> commands with hostname check in the .bashrc file, e.g. as follows:  
 
* To avoid accidentally compiling on the login nodes, you can 'shield' yourself from the compilers by wrapping the <tt>module load</tt> commands with hostname check in the .bashrc file, e.g. as follows:  
  
Line 31: Line 31:
 
</source>
 
</source>
  
===Default .bashrc/.bash_profile===
+
===Default .bashrc and .bash_profile===
The default startup scripts provided by SciNet are as follows.  Certain things - like sourcing <tt>/etc/profile</tt>
+
The default startup scripts provided by SciNet are as follows.  Certain things - like sourcing <tt>/etc/profile</tt> and <tt>/etc/bashrc</tt> are ''required'' for various SciNet routines to work!   
and <tt>/etc/bashrc</tt> are ''required'' for various SciNet routines to work!   
+
 
 +
Note that the "module load" command is commented out, as we highly recommend not loading any modules in the .bashrc.  The comment merely serves as an example.  This is a change from the old default, where the intel compiler and the openmpi library were loaded by default.  The new default has been in use for new users since Oct 10, 2013.
 +
 
 +
 
 +
'''.bashrc'''
 +
<source lang="bash">
 +
if [ -f /etc/bashrc ]; then
 +
      . /etc/bashrc
 +
fi
 +
 
 +
# commands which work for both GPC and TCS can go here
 +
 
 +
HOST=$(uname)
 +
 
 +
if [ "${HOST}" == "AIX" ]; then
 +
        # do things for the TCS machine
 +
        # user environment for all shells goes here
 +
        # replace colon with your own commands
 +
        :
 +
else
 +
        # do things for the GPC machine
 +
 
 +
        #module load intel openmpi
 +
 
 +
        # user environment for all shells goes here
 +
        # replace colon with your own commands
 +
        :
 +
fi
 +
</source>
  
 
'''.bash_profile'''
 
'''.bash_profile'''
Line 43: Line 71:
 
# commands which work for both GPC and TCS can go here
 
# commands which work for both GPC and TCS can go here
  
alias passwd='echo "Please use the SciNet portal to change password: https://portal.scinet.utoronto.ca/change_password"'
+
alias passwd='echo "Please use the SciNet portal to change password: https://por
 +
tal.scinet.utoronto.ca/change_password"'
 +
 
 +
HOST=$(uname)
 +
 
 +
if [ "${HOST}" == "AIX" ]
 +
then
 +
        # do things for the TCS machine
 +
        alias llq1='/xcat/tools/tcs-scripts/LL/jobState.sh'
 +
        alias llstat='/xcat/tools/tcs-scripts/LL/jobSummary.sh'
 +
 
 +
        if [ "${TERM}" = "xterm-color" ]; then
 +
                export TERM=xterm
 +
        fi
 +
 
 +
        # user environment for login shells goes here
 +
        # replace colon with your own commands
 +
        :
 +
else
 +
        # do things for the GPC machine
 +
        # user environment for login shells goes here
 +
        # replace colon with your own commands
 +
        :
 +
 
 +
fi
 +
 
 +
PS1="\h-\$ "
 +
 
 +
if [ -f ~/.bashrc ]; then
 +
      . ~/.bashrc
 +
fi
 +
</source>
 +
 
 +
Note that .bash_profile reads in the .bashrc, so that in practice one need only modify the .bashrc to set up aliases, etc...
 +
 
 +
===Distinguishing machines and operating systems in .bashrc and .bash_profile===
 +
The above startup scripts work well if you only use the GPC, or the GPC and TCS.  If you're also using the more-specialized machines like the ARC or P7 cluster, or if you're using both centos6 and centos5 nodes (e.g. related to the ATLAS project), then you can distinguish which system you are on and which operating system it is running using the following .bash_profile and .bashrc:
 +
 
 +
 
 +
'''.bashrc'''
 +
<source lang="bash">
 +
if [ -f /etc/bashrc ]; then
 +
      . /etc/bashrc
 +
fi
 +
 
 +
# commands which work for both GPC and TCS can go here
 +
 
 +
export BASH_ENV=~/.bashrc
  
 
# what machine/architecture
 
# what machine/architecture
Line 67: Line 142:
 
fi
 
fi
  
if [ "${MACHINE}" == "tcs" ]  
+
if [ "${MACHINE}" == "tcs" ]
 
then
 
then
 
         # do things for the TCS machine
 
         # do things for the TCS machine
        alias llq1='/xcat/tools/tcs-scripts/LL/jobState.sh'
+
         # user environment for all shells goes here
        alias llstat='/xcat/tools/tcs-scripts/LL/jobSummary.sh'
 
 
 
        if [ "${TERM}" = "xterm-color" ]; then
 
                export TERM=xterm
 
        fi
 
 
 
         # user environment for login shells goes here
 
 
         # replace colon with your own commands
 
         # replace colon with your own commands
 
         :
 
         :
Line 83: Line 151:
 
then
 
then
 
         # do things for the P7 machine
 
         # do things for the P7 machine
         # user environment for login shells goes here
+
 
 +
        module load pe vacpp xlf
 +
        # note that this is an example only. Generally, we do
 +
        # not recommended putting module commands in your .bashrc.
 +
 
 +
         # user environment for all shells goes here
 
         # replace colon with your own commands
 
         # replace colon with your own commands
 
         :
 
         :
Line 89: Line 162:
 
then
 
then
 
         # do things for the GPC machine
 
         # do things for the GPC machine
         # user environment for login shells goes here
+
 
 +
        module load intel openmpi
 +
        # note that this is an example only. Generally, we do
 +
        # not recommended putting module commands in your .bashrc.
 +
 
 +
         # user environment for all shells goes here
 
         # replace colon with your own commands
 
         # replace colon with your own commands
 
         :
 
         :
         # os specific commands next
+
       
 +
         # os specific commands/modules can be loaded next
 
         if [ "${OS}" == "centos5" ]
 
         if [ "${OS}" == "centos5" ]
 
         then
 
         then
             #centos 5 specific login commands
+
             #centos 5 specific modules
 
             :
 
             :
 
         elif [ "${OS}" == "centos6" ]
 
         elif [ "${OS}" == "centos6" ]
 
         then
 
         then
             #centos 6 specific login commands
+
             #centos 6 specific modules
 
             :
 
             :
 
         fi
 
         fi
else
 
  echo "Not a recognized system"
 
fi
 
  
PS1="\h-\$ "
+
else
 
+
        echo "Not a recognized system"
 
 
if [ -f ~/.bashrc ]; then
 
      . ~/.bashrc
 
 
fi
 
fi
 
</source>
 
</source>
  
'''.bashrc'''
+
'''.bash_profile'''
 
<source lang="bash">
 
<source lang="bash">
if [ -f /etc/bashrc ]; then
+
if [ -f /etc/profile ]; then
       . /etc/bashrc
+
       . /etc/profile
 
fi
 
fi
  
 
# commands which work for both GPC and TCS can go here
 
# commands which work for both GPC and TCS can go here
  
export BASH_ENV=~/.bashrc
+
alias passwd='echo "Please use the SciNet portal to change  password: https://portal.scinet.utoronto.ca/change_password"'
  
 
# what machine/architecture
 
# what machine/architecture
Line 146: Line 219:
 
fi
 
fi
  
if [ "${MACHINE}" == "tcs" ]
+
if [ "${MACHINE}" == "tcs" ]  
 
then
 
then
 
         # do things for the TCS machine
 
         # do things for the TCS machine
         # user environment for all shells goes here
+
        alias llq1='/xcat/tools/tcs-scripts/LL/jobState.sh'
 +
        alias llstat='/xcat/tools/tcs-scripts/LL/jobSummary.sh'
 +
 
 +
        if [ "${TERM}" = "xterm-color" ]; then
 +
                export TERM=xterm
 +
        fi
 +
 
 +
         # user environment for login shells goes here
 
         # replace colon with your own commands
 
         # replace colon with your own commands
 
         :
 
         :
Line 155: Line 235:
 
then
 
then
 
         # do things for the P7 machine
 
         # do things for the P7 machine
 
+
         # user environment for login shells goes here
        module load pe vacpp xlf
 
 
 
         # user environment for all shells goes here
 
 
         # replace colon with your own commands
 
         # replace colon with your own commands
 
         :
 
         :
Line 164: Line 241:
 
then
 
then
 
         # do things for the GPC machine
 
         # do things for the GPC machine
 
+
         # user environment for login shells goes here
        module load intel openmpi
 
 
 
         # user environment for all shells goes here
 
 
         # replace colon with your own commands
 
         # replace colon with your own commands
 
         :
 
         :
       
+
         # os specific commands next
         # os specific commands/modules can be loaded next
 
 
         if [ "${OS}" == "centos5" ]
 
         if [ "${OS}" == "centos5" ]
 
         then
 
         then
             #centos 5 specific modules
+
             #centos 5 specific login commands
 
             :
 
             :
 
         elif [ "${OS}" == "centos6" ]
 
         elif [ "${OS}" == "centos6" ]
 
         then
 
         then
             #centos 6 specific modules
+
             #centos 6 specific login commands
 
             :
 
             :
 
         fi
 
         fi
 +
else
 +
        echo "Not a recognized system"
 +
fi
  
else
+
PS1="\h-\$ "
        echo "Not a recognized system"
+
 
 +
if [ -f ~/.bashrc ]; then
 +
      . ~/.bashrc
 
fi
 
fi
 
</source>
 
</source>

Latest revision as of 14:34, 19 December 2013

The file .bashrc, located in your home directory, is read-in and executed whenever a bash script or bash shell is started. The exception is for login shells, in which case .bash_profile is started. The default .bash_profile calls .bashrc so that many scinet users never need to edit .bash_profile.

This means in particular that the file .bashrc is read in

  • when you log in
  • when you run compute jobs
  • when you run certain commands such as mpirun
  • at the end of a job, when copying output and error files.

And that both on the GPS and the TCS, which nonetheless often require quite a different setup.

Guidelines

To avoid difficulties, you are advised to follow these guidelines in .bashrc:

  • Sourcing /etc/profile and /etc/bashrc is required for various SciNet routines to work!
  • Do not execute any commands that write to standard output, because that would breaks things like copying-out output files and scp. So for instance, no echo or module list.
  • Load only commonly used modules, or preferrably, do not load any modules in your bashrc.
  • Distinguish, especially for the module command, between the gpc and tcs.
  • If you're trying out different implementations or versions of a module (e.g. mpi), you should definitely not load them in the .bashrc, but explicitly on the command line before compiling and once more explicitly in your job script.
  • To avoid accidentally compiling on the login nodes, you can 'shield' yourself from the compilers by wrapping the module load commands with hostname check in the .bashrc file, e.g. as follows:

<source lang="bash"> HOSTNAME=$(hostname) if [ "${HOSTNAME:0:6}" != "scinet" ]; then

   module load intel openmpi  # and any other modules

fi </source>

Default .bashrc and .bash_profile

The default startup scripts provided by SciNet are as follows. Certain things - like sourcing /etc/profile and /etc/bashrc are required for various SciNet routines to work!

Note that the "module load" command is commented out, as we highly recommend not loading any modules in the .bashrc. The comment merely serves as an example. This is a change from the old default, where the intel compiler and the openmpi library were loaded by default. The new default has been in use for new users since Oct 10, 2013.


.bashrc <source lang="bash"> if [ -f /etc/bashrc ]; then

      . /etc/bashrc

fi

  1. commands which work for both GPC and TCS can go here

HOST=$(uname)

if [ "${HOST}" == "AIX" ]; then

       # do things for the TCS machine
       # user environment for all shells goes here
       # replace colon with your own commands
       :

else

       # do things for the GPC machine
       #module load intel openmpi
       # user environment for all shells goes here
       # replace colon with your own commands
       :

fi </source>

.bash_profile <source lang="bash"> if [ -f /etc/profile ]; then

      . /etc/profile

fi

  1. commands which work for both GPC and TCS can go here

alias passwd='echo "Please use the SciNet portal to change password: https://por tal.scinet.utoronto.ca/change_password"'

HOST=$(uname)

if [ "${HOST}" == "AIX" ] then

       # do things for the TCS machine
       alias llq1='/xcat/tools/tcs-scripts/LL/jobState.sh'
       alias llstat='/xcat/tools/tcs-scripts/LL/jobSummary.sh'
       if [ "${TERM}" = "xterm-color" ]; then
               export TERM=xterm
       fi
       # user environment for login shells goes here
       # replace colon with your own commands
       :

else

       # do things for the GPC machine
       # user environment for login shells goes here
       # replace colon with your own commands
       :

fi

PS1="\h-\$ "

if [ -f ~/.bashrc ]; then

      . ~/.bashrc

fi </source>

Note that .bash_profile reads in the .bashrc, so that in practice one need only modify the .bashrc to set up aliases, etc...

Distinguishing machines and operating systems in .bashrc and .bash_profile

The above startup scripts work well if you only use the GPC, or the GPC and TCS. If you're also using the more-specialized machines like the ARC or P7 cluster, or if you're using both centos6 and centos5 nodes (e.g. related to the ATLAS project), then you can distinguish which system you are on and which operating system it is running using the following .bash_profile and .bashrc:


.bashrc <source lang="bash"> if [ -f /etc/bashrc ]; then

      . /etc/bashrc

fi

  1. commands which work for both GPC and TCS can go here

export BASH_ENV=~/.bashrc

  1. what machine/architecture

HOSTNAME=$(hostname) case $(uname -p) in

   'ppc64')    MACHINE='p7';;
   'powerpc')  MACHINE='tcs';;
   'x86_64')   MACHINE='gpc';;
   *)          MACHINE='unknown';;

esac

if uname -r | grep -q el5 then

    OS='centos5'

elif uname -r | grep -q el6 then

    OS='centos6'

elif uname -o | grep -q AIX then

    OS='aix'

else

    OS='unknown'

fi

if [ "${MACHINE}" == "tcs" ] then

       # do things for the TCS machine
       # user environment for all shells goes here
       # replace colon with your own commands
       :

elif [ "${MACHINE}" == "p7" ] then

       # do things for the P7 machine
       module load pe vacpp xlf
       # note that this is an example only. Generally, we do
       # not recommended putting module commands in your .bashrc.
       # user environment for all shells goes here
       # replace colon with your own commands
       :

elif [ "${MACHINE}" == "gpc" ] then

       # do things for the GPC machine
       module load intel openmpi
       # note that this is an example only. Generally, we do
       # not recommended putting module commands in your .bashrc.
       # user environment for all shells goes here
       # replace colon with your own commands
       :
       
       # os specific commands/modules can be loaded next
       if [ "${OS}" == "centos5" ]
       then
           #centos 5 specific modules
           :
       elif [ "${OS}" == "centos6" ]
       then
           #centos 6 specific modules
           :
       fi

else

       echo "Not a recognized system"

fi </source>

.bash_profile <source lang="bash"> if [ -f /etc/profile ]; then

      . /etc/profile

fi

  1. commands which work for both GPC and TCS can go here

alias passwd='echo "Please use the SciNet portal to change password: https://portal.scinet.utoronto.ca/change_password"'

  1. what machine/architecture

HOSTNAME=$(hostname) case $(uname -p) in

   'ppc64')    MACHINE='p7';;
   'powerpc')  MACHINE='tcs';;
   'x86_64')   MACHINE='gpc';;
   *)          MACHINE='unknown';;

esac

if uname -r | grep -q el5 then

    OS='centos5'

elif uname -r | grep -q el6 then

    OS='centos6'

elif uname -o | grep -q AIX then

    OS='aix'

else

    OS='unknown'

fi

if [ "${MACHINE}" == "tcs" ] then

       # do things for the TCS machine
       alias llq1='/xcat/tools/tcs-scripts/LL/jobState.sh'
       alias llstat='/xcat/tools/tcs-scripts/LL/jobSummary.sh'
       if [ "${TERM}" = "xterm-color" ]; then
               export TERM=xterm
       fi
       # user environment for login shells goes here
       # replace colon with your own commands
       :

elif [ "${MACHINE}" == "p7" ] then

       # do things for the P7 machine
       # user environment for login shells goes here
       # replace colon with your own commands
       :

elif [ "${MACHINE}" == "gpc" ] then

       # do things for the GPC machine
       # user environment for login shells goes here
       # replace colon with your own commands
       :
       # os specific commands next
       if [ "${OS}" == "centos5" ]
       then
           #centos 5 specific login commands
           :
       elif [ "${OS}" == "centos6" ]
       then
           #centos 6 specific login commands
           :
       fi

else

       echo "Not a recognized system"

fi

PS1="\h-\$ "

if [ -f ~/.bashrc ]; then

      . ~/.bashrc

fi </source>