Using Signals
General
In unix/linux one can send signals to a program or a script. Some common signals are:
| Signal | Meaning | Can be trapped? | ||||||
|---|---|---|---|---|---|---|---|---|
| HUP | Terminal log-out | Yes | ||||||
| INT | Interrupt signal (Ctrl-C is pressed) | Yes | ||||||
| TERM | Termination of the process (as if 'kill' was called) | Yes | ||||||
| KILL | Kill the process (as if 'kill -9' was called) | No | ||||||
| STOP | Suspend the process | No | ||||||
| CONT | Resume the process | Yes | ||||||
| USR1 | Unspecified user signal (can be given with 'kill -USR1') | Yes | ||||||
| USR2 | Unspecified user signal (can be given with 'kill -USR2') | Yes |
From the node on which a a process is running, signals can be given using
$ kill -[SIGNAL] [PID]
On the GPC, you can pass a signal to a running job using
$ qsig -s [SIGNAL] [JOBID]
So for example, you could suspend the process running under job id 314159 by qsig -s STOP 314159, and resume it using qsig -s CONT 314159 (note that the wall clock keeps ticking!).
For qsig to work for signals whose default action is to kill your job (such as HUP, INT, TERM, and, perhaps surprisingly, USR1 and USR2), your job script should either trap these explicitly (see below), or contain a line
trap : [LIST-OF-SIGNALS-TO-BE-PASSED]
(e.g. trap : USR1 USR2) before the application that traps the signal is started. Note that he spaces surrounding the colon are essential here, because the colon is actually a bash command (one that does nothing).
Signals can also be given by the system. One important instance of system signals is that on SciNet, a TERM signal is sent to a jobs whose requested time is over, after which the job has about 2 minutes to clean up.
With the techniques explained below, a program or script can be set up to listen for a particular kind of signal, except for KILL and STOP (which always kill and suspend a process, respectively). When it receive a signal of that kind, its execution is interrupted and a call is made to a function, specified earlier in the program. It is up to the program or script what this signal handling function does but it is a good idea to make the action appropriate for the event that triggers the signal. For instance, a TERM signal should be handled as a request to terminate the application. The user signals USR1 and USR2 do not have a pre-designated meaning and can be used for application-specific actions such as checkpointing.
Trapping signals in bash scripts
To trap signals in a bash script, one has to bind a specific signal to a command or function with the trap command. For example, <source lang="bash">
- !/bin/bash
trap "echo Term was trapped.; exit" TERM for ((i=0;i<60;i++)) do
echo $i sleep 1
done </source> Running this script in the background and sending it a TERM command ('kill -TERM [pid]') will cause the message 'Term was trapped.' to be printed. Notes:
- The trapped command has to use 'exit' explicitly to stop the script's execution.
- The 'sleep' command in bash cannot be interrupted, which is why the script contains a succession of 1 second 'sleep's.
Another useful example of signal trapping in a bash script is given on the wiki page about using ramdisk.
Trapping signals in C
To trap signals in a c program, one has to include the signal.h header file: <source lang="c">
- include <signal.h>
</source> and provide a signal handler function <source lang="c"> void term_trap(int sig) {
/* do something */
} </source> which is linked to the specific signal somewhere as follows: <source lang="c"> signal(SIGTERM, term_trap); </source> Note that the names of signals are prepended with "SIG" in signal.h.
A minimal example: <source lang="c">
- include <stdio.h>
- include <signal.h>
- include <unistd.h>
void term_trap(int sig) {
printf("Term was trapped.\n");
}
int main() {
signal(SIGTERM, term_trap); sleep(60);
} </source> Note that the names of the signals are prepended with SIG in C. To test this program:
- save the above code as sigex.c
- compile: icc -O3 -xHost sigex.c -o sigex
- run: sigex&
- note the pid (process idenitfier)
- you can then give the process the term signal with kill -TERM [pid].
Trapping signals in C++
The same method as for a C program works in C++, except the header file is 'csignal'. For example: <source lang="c">
- include <iostream>
- include <csignal>
- include <unistd.h>
void term_trap(int sig) {
std::cout << "Term was trapped.\n";
}
int main() {
signal(SIGTERM, term_trap); sleep(60);
} </source>
Trapping signals in Fortran
Unfortunately, signal handling in fortran is not standard, and different compilers have different ways of doing this. Below, examples are given for the three fortran compilers available on SciNet: ifort, gfortran, and xlf. These examples were designed such that even though the ways to register the handler differ, the signal handlers themselves are always the same.
ifort <source lang="fortran"> C IFORT SIGNAL TRAPPING EXAMPLE
USE IFPORT
EXTERNAL TRAP_TERM
INTEGER TRAP_TERM
INTEGER ERR
ERR = SIGNAL (SIGTERM, TRAP_TERM, -1)
CALL SLEEP (60)
END
C SIGNAL HANDLER FUNCTION
FUNCTION TRAP_TERM (SIG_NUM)
INTEGER TRAP_TERM
INTEGER SIG_NUM
PRINT *, "Term was trapped."
TRAP_TERM = 1
END
</source>
gfortran <source lang="fortran"> C GFORTRAN SIGNAL TRAPPING EXAMPLE
INTRINSIC SIGNAL
INTEGER SIGTERM
PARAMETER (SIGTERM = 15)
EXTERNAL TRAP_TERM
CALL SIGNAL (SIGTERM, TRAP_TERM)
CALL SLEEP (60)
END
C SIGNAL HANDLER FUNCTION
FUNCTION TRAP_TERM (SIG_NUM)
INTEGER TRAP_TERM
INTEGER SIG_NUM
PRINT *, "Term was trapped."
TRAP_TERM = 1
END
</source> Note: the SIGNAL function is broken in the official gfortran versions 4.6.0 and 4.6.1, but this bug has been fixed in SciNet's gcc modules.
xlf <source lang="fortran"> C XLF SIGNAL TRAPPING EXAMPLE
INCLUDE 'fexcp.h'
INTEGER SIGTERM
PARAMETER (SIGTERM = 15)
EXTERNAL TRAP_TERM
CALL SIGNAL (SIGTERM, TRAP_TERM)
CALL SLEEP (60)
END
C SIGNAL HANDLER FUNCTION
FUNCTION TRAP_TERM (SIG_NUM)
INTEGER TRAP_TERM
INTEGER SIG_NUM
PRINT *, "Term was trapped."
TRAP_TERM = 1
END
</source> --Rzon 30 June 2010