Difference between revisions of "SciNet Command Line Utilities"
Jump to navigation
Jump to search
m |
m |
||
(12 intermediate revisions by 3 users not shown) | |||
Line 1: | Line 1: | ||
+ | __TOC__ | ||
+ | |||
Below is list of handy command-line utilities written by SciNet to help manage your data and compute. | Below is list of handy command-line utilities written by SciNet to help manage your data and compute. | ||
+ | <!-- Currently, on the GPC you need to have the '''extras''' module loaded for these to work.--> | ||
− | + | Note that in the table, optional arguments are denoted with square brackets (i.e. ''''[]''''), while mandatory arguments are denoted with angular brackets (i.e. ''''<>''''). | |
− | + | = Storage = | |
− | + | {| border="1" cellpadding="5" cellspacing="0" | |
− | |||
− | {| border="1" cellpadding=" | ||
!{{Hl2}} | Command | !{{Hl2}} | Command | ||
!{{Hl2}}| Arguments | !{{Hl2}}| Arguments | ||
Line 30: | Line 31: | ||
| | | | ||
|Lists your directories that have over 1 GB of data | |Lists your directories that have over 1 GB of data | ||
− | | | + | |GPC |
+ | |- | ||
+ | |} | ||
+ | |||
+ | |||
+ | = Jobs & Queues = | ||
+ | {| border="1" cellpadding="5" cellspacing="0" | ||
+ | !{{Hl2}} | Command | ||
+ | !{{Hl2}}| Arguments | ||
+ | !{{Hl2}}| Description | ||
+ | !{{Hl2}}| Cluster | ||
|- | |- | ||
|qsum | |qsum | ||
Line 42: | Line 53: | ||
|TCS | |TCS | ||
|- | |- | ||
− | | | + | |scinet gpc priority |
| | | | ||
− | |Compute usage in the last 14 days, and how this affects your priority in the queue | + | |Compute usage in the last 14 days, and how this affects your priority in the queue. |
|GPC | |GPC | ||
|- | |- | ||
Line 101: | Line 112: | ||
|Requests a time-limited interactive session of up to 8 dedicated nodes | |Requests a time-limited interactive session of up to 8 dedicated nodes | ||
|GPC | |GPC | ||
+ | |- | ||
+ | |} | ||
+ | |||
+ | = Modules = | ||
+ | {| border="1" cellpadding="5" cellspacing="0" | ||
+ | !{{Hl2}} | Command | ||
+ | !{{Hl2}}| Arguments | ||
+ | !{{Hl2}}| Description | ||
+ | !{{Hl2}}| Cluster | ||
|- | |- | ||
|modulefind | |modulefind | ||
Line 114: | Line 134: | ||
|modulewrap | |modulewrap | ||
|''<comma-separated-modules>'' ''<command>'' | |''<comma-separated-modules>'' ''<command>'' | ||
− | | | + | |Load the modules in the ''comma-separated-modules'' list (no spaces!) only to run the given ''command''. |
|GPC | |GPC | ||
|- | |- | ||
|} | |} | ||
+ | |||
+ | |||
+ | = Related Topics = | ||
+ | [https://wiki.scinet.utoronto.ca/wiki/index.php/Moab#Checking_the_memory_usage_from_jobs Checking the memory usage from jobs] | ||
+ | <br> | ||
+ | [https://wiki.scinet.utoronto.ca/wiki/index.php/GPC_Quickstart GPC Quickstart] | ||
+ | <br> | ||
+ | [https://wiki.scinet.utoronto.ca/wiki/index.php/Moab Scheduler] | ||
+ | <br> | ||
+ | [https://wiki.scinet.utoronto.ca/wiki/index.php/FAQ#Monitoring_jobs_in_the_queue Monitoring Jobs] | ||
+ | <br> | ||
+ | [https://wiki.scinet.utoronto.ca/wiki/images/a/a0/TechTalkJobMonitoring.pdf Tech Talk on Monitoring Jobs] |
Latest revision as of 18:53, 5 September 2015
Below is list of handy command-line utilities written by SciNet to help manage your data and compute.
Note that in the table, optional arguments are denoted with square brackets (i.e. '[]'), while mandatory arguments are denoted with angular brackets (i.e. '<>').
Storage
Command | Arguments | Description | Cluster |
---|---|---|---|
quota | Short overview of a user's storage usage. | GPC | |
diskUsage | -h to see all command options | Informs about the user and group file system usage. | GPC |
topUserDirOver1000list | Lists your directories that have over 1,000 files | GPC | |
topUserDirOver1GBlist | Lists your directories that have over 1 GB of data | GPC |
Jobs & Queues
Command | Arguments | Description | Cluster |
---|---|---|---|
qsum | most showq arguments work | Lists jobs running or in the queue, grouped by user. | GPC,TCS |
llq1 | Detailed information on jobs that are actively running. | TCS | |
scinet gpc priority | Compute usage in the last 14 days, and how this affects your priority in the queue. | GPC | |
nodeperf | [userName] | Who is doing what on the current node? | GPC |
jobperf | <jobId | jobName> | Informs about the performance of all nodes of a given job. | GPC |
jobError | <jobId | jobName> | Displays realtime error output of a given job. | GPC |
jobOutput | <jobId | jobName> | Displays realtime standard output of a given job. | GPC |
jobcd | <jobId | jobName> | Allows users to quickly move into the working directory of a given job. | GPC |
jobscript | <jobId | jobName> | Displays the submission script used when submitting a given job. | GPC |
jobssh | <jobId | jobName> | Allows users to connect to the head-node of a given job. | GPC |
jobtop | <jobId | jobName> | Allows users to "top" on the head-node of a given job. | GPC |
jobdep | <jobId> | Displays dependencies of given job. | GPC |
jobtree | [userName] | Displays the jobs tree of dependencies for a given user. | GPC |
debugjob | [nodes] | Requests a time-limited interactive session of up to 8 dedicated nodes | GPC |
Modules
Command | Arguments | Description | Cluster |
---|---|---|---|
modulefind | [searchWord] | Finds modules containing searchWord and indicate their compatibility with already loaded modules. | GPC |
moduleadvice | <modulename> | Show the module commands required to load modulename, taking into account prerequisites and conflicts with already loaded modules. | GPC |
modulewrap | <comma-separated-modules> <command> | Load the modules in the comma-separated-modules list (no spaces!) only to run the given command. | GPC |
Related Topics
Checking the memory usage from jobs
GPC Quickstart
Scheduler
Monitoring Jobs
Tech Talk on Monitoring Jobs