Difference between revisions of "SciNet Command Line Utilities"
Jump to navigation
Jump to search
(Created page with "Below is list of handy command-line utilities written by SciNet to help manage your data and compute. Note that in the table, optional arguments are denoted with square brack...") |
m |
||
(28 intermediate revisions by 3 users not shown) | |||
Line 1: | Line 1: | ||
+ | __TOC__ | ||
+ | |||
Below is list of handy command-line utilities written by SciNet to help manage your data and compute. | Below is list of handy command-line utilities written by SciNet to help manage your data and compute. | ||
+ | <!-- Currently, on the GPC you need to have the '''extras''' module loaded for these to work.--> | ||
+ | |||
+ | Note that in the table, optional arguments are denoted with square brackets (i.e. ''''[]''''), while mandatory arguments are denoted with angular brackets (i.e. ''''<>''''). | ||
+ | |||
+ | = Storage = | ||
+ | {| border="1" cellpadding="5" cellspacing="0" | ||
+ | !{{Hl2}} | Command | ||
+ | !{{Hl2}}| Arguments | ||
+ | !{{Hl2}}| Description | ||
+ | !{{Hl2}}| Cluster | ||
+ | |- | ||
+ | |quota | ||
+ | | | ||
+ | |Short overview of a user's storage usage. | ||
+ | |GPC | ||
+ | |- | ||
+ | |diskUsage | ||
+ | |''-h'' to see all command options | ||
+ | |Informs about the user and group file system usage. | ||
+ | |GPC | ||
+ | |- | ||
+ | |topUserDirOver1000list | ||
+ | | | ||
+ | |Lists your directories that have over 1,000 files | ||
+ | |GPC | ||
+ | |- | ||
+ | |topUserDirOver1GBlist | ||
+ | | | ||
+ | |Lists your directories that have over 1 GB of data | ||
+ | |GPC | ||
+ | |- | ||
+ | |} | ||
− | |||
− | {| border="1" cellpadding=" | + | = Jobs & Queues = |
+ | {| border="1" cellpadding="5" cellspacing="0" | ||
!{{Hl2}} | Command | !{{Hl2}} | Command | ||
!{{Hl2}}| Arguments | !{{Hl2}}| Arguments | ||
!{{Hl2}}| Description | !{{Hl2}}| Description | ||
!{{Hl2}}| Cluster | !{{Hl2}}| Cluster | ||
+ | |- | ||
+ | |qsum | ||
+ | |most showq arguments work | ||
+ | |Lists jobs running or in the queue, grouped by user. | ||
+ | |GPC,TCS | ||
+ | |- | ||
+ | |llq1 | ||
+ | | | ||
+ | |Detailed information on jobs that are actively running. | ||
+ | |TCS | ||
+ | |- | ||
+ | |scinet gpc priority | ||
+ | | | ||
+ | |Compute usage in the last 14 days, and how this affects your priority in the queue. | ||
+ | |GPC | ||
+ | |- | ||
+ | |nodeperf | ||
+ | |[''userName''] | ||
+ | |Who is doing what on the current node? | ||
+ | |GPC | ||
+ | |- | ||
+ | |jobperf | ||
+ | | <''jobId''<nowiki> | </nowiki>''jobName''> | ||
+ | |Informs about the performance of all nodes of a given job. | ||
+ | |GPC | ||
+ | |- | ||
+ | |jobError | ||
+ | | <''jobId''<nowiki> | </nowiki>''jobName''> | ||
+ | | Displays realtime error output of a given job. | ||
+ | |GPC | ||
+ | |- | ||
+ | |jobOutput | ||
+ | | <''jobId''<nowiki> | </nowiki>''jobName''> | ||
+ | | Displays realtime standard output of a given job. | ||
+ | |GPC | ||
+ | |- | ||
+ | |jobcd | ||
+ | | <''jobId''<nowiki> | </nowiki>''jobName''> | ||
+ | | Allows users to quickly move into the working directory of a given job. | ||
+ | |GPC | ||
+ | |- | ||
+ | |jobscript | ||
+ | | <''jobId''<nowiki> | </nowiki>''jobName''> | ||
+ | | Displays the submission script used when submitting a given job. | ||
+ | |GPC | ||
+ | |- | ||
+ | |jobssh | ||
+ | | <''jobId''<nowiki> | </nowiki>''jobName''> | ||
+ | | Allows users to connect to the head-node of a given job. | ||
+ | |GPC | ||
+ | |- | ||
+ | |jobtop | ||
+ | | <''jobId''<nowiki> | </nowiki>''jobName''> | ||
+ | | Allows users to "top" on the head-node of a given job. | ||
+ | |GPC | ||
|- | |- | ||
|jobdep | |jobdep | ||
− | |< | + | |<''jobId''> |
− | |Displays dependencies of given job | + | |Displays dependencies of given job. |
|GPC | |GPC | ||
|- | |- | ||
|jobtree | |jobtree | ||
− | |[ | + | |[''userName''] |
− | |Displays the jobs tree of dependencies for a given user | + | |Displays the jobs tree of dependencies for a given user. |
+ | |GPC | ||
+ | |- | ||
+ | |debugjob | ||
+ | |[''nodes''] | ||
+ | |Requests a time-limited interactive session of up to 8 dedicated nodes | ||
|GPC | |GPC | ||
+ | |- | ||
|} | |} | ||
+ | |||
+ | = Modules = | ||
+ | {| border="1" cellpadding="5" cellspacing="0" | ||
+ | !{{Hl2}} | Command | ||
+ | !{{Hl2}}| Arguments | ||
+ | !{{Hl2}}| Description | ||
+ | !{{Hl2}}| Cluster | ||
+ | |- | ||
+ | |modulefind | ||
+ | |[''searchWord''] | ||
+ | |Finds modules containing ''searchWord'' and indicate their compatibility with already loaded modules. | ||
+ | |GPC | ||
+ | |- | ||
+ | |moduleadvice | ||
+ | |<''modulename''> | ||
+ | |Show the module commands required to load ''modulename'', taking into account prerequisites and conflicts with already loaded modules. | ||
+ | |GPC | ||
+ | |- | ||
+ | |modulewrap | ||
+ | |''<comma-separated-modules>'' ''<command>'' | ||
+ | |Load the modules in the ''comma-separated-modules'' list (no spaces!) only to run the given ''command''. | ||
+ | |GPC | ||
+ | |- | ||
+ | |} | ||
+ | |||
+ | |||
+ | = Related Topics = | ||
+ | [https://wiki.scinet.utoronto.ca/wiki/index.php/Moab#Checking_the_memory_usage_from_jobs Checking the memory usage from jobs] | ||
+ | <br> | ||
+ | [https://wiki.scinet.utoronto.ca/wiki/index.php/GPC_Quickstart GPC Quickstart] | ||
+ | <br> | ||
+ | [https://wiki.scinet.utoronto.ca/wiki/index.php/Moab Scheduler] | ||
+ | <br> | ||
+ | [https://wiki.scinet.utoronto.ca/wiki/index.php/FAQ#Monitoring_jobs_in_the_queue Monitoring Jobs] | ||
+ | <br> | ||
+ | [https://wiki.scinet.utoronto.ca/wiki/images/a/a0/TechTalkJobMonitoring.pdf Tech Talk on Monitoring Jobs] |
Latest revision as of 18:53, 5 September 2015
Below is list of handy command-line utilities written by SciNet to help manage your data and compute.
Note that in the table, optional arguments are denoted with square brackets (i.e. '[]'), while mandatory arguments are denoted with angular brackets (i.e. '<>').
Storage
Command | Arguments | Description | Cluster |
---|---|---|---|
quota | Short overview of a user's storage usage. | GPC | |
diskUsage | -h to see all command options | Informs about the user and group file system usage. | GPC |
topUserDirOver1000list | Lists your directories that have over 1,000 files | GPC | |
topUserDirOver1GBlist | Lists your directories that have over 1 GB of data | GPC |
Jobs & Queues
Command | Arguments | Description | Cluster |
---|---|---|---|
qsum | most showq arguments work | Lists jobs running or in the queue, grouped by user. | GPC,TCS |
llq1 | Detailed information on jobs that are actively running. | TCS | |
scinet gpc priority | Compute usage in the last 14 days, and how this affects your priority in the queue. | GPC | |
nodeperf | [userName] | Who is doing what on the current node? | GPC |
jobperf | <jobId | jobName> | Informs about the performance of all nodes of a given job. | GPC |
jobError | <jobId | jobName> | Displays realtime error output of a given job. | GPC |
jobOutput | <jobId | jobName> | Displays realtime standard output of a given job. | GPC |
jobcd | <jobId | jobName> | Allows users to quickly move into the working directory of a given job. | GPC |
jobscript | <jobId | jobName> | Displays the submission script used when submitting a given job. | GPC |
jobssh | <jobId | jobName> | Allows users to connect to the head-node of a given job. | GPC |
jobtop | <jobId | jobName> | Allows users to "top" on the head-node of a given job. | GPC |
jobdep | <jobId> | Displays dependencies of given job. | GPC |
jobtree | [userName] | Displays the jobs tree of dependencies for a given user. | GPC |
debugjob | [nodes] | Requests a time-limited interactive session of up to 8 dedicated nodes | GPC |
Modules
Command | Arguments | Description | Cluster |
---|---|---|---|
modulefind | [searchWord] | Finds modules containing searchWord and indicate their compatibility with already loaded modules. | GPC |
moduleadvice | <modulename> | Show the module commands required to load modulename, taking into account prerequisites and conflicts with already loaded modules. | GPC |
modulewrap | <comma-separated-modules> <command> | Load the modules in the comma-separated-modules list (no spaces!) only to run the given command. | GPC |
Related Topics
Checking the memory usage from jobs
GPC Quickstart
Scheduler
Monitoring Jobs
Tech Talk on Monitoring Jobs