Difference between revisions of "SciNet Command Line Utilities"

From oldwiki.scinet.utoronto.ca
Jump to navigation Jump to search
m
m
 
(19 intermediate revisions by 3 users not shown)
Line 1: Line 1:
 +
__TOC__
 +
 
Below is list of handy command-line utilities written by SciNet to help manage your data and compute.
 
Below is list of handy command-line utilities written by SciNet to help manage your data and compute.
 +
<!-- Currently, on the GPC you need to have the '''extras''' module loaded for these to work.-->
  
−
Currently, on the GPC you need to have the 'extras' module loaded for these to work.
+
Note that in the table, optional arguments are denoted with square brackets (i.e. ''''[]''''), while mandatory arguments are denoted with angular brackets (i.e. ''''<>'''').
  
−
Note that in the table, optional arguments are denoted with square brackets (i.e. '[]').
+
= Storage =
−
 
+
{| border="1" cellpadding="5" cellspacing="0"
−
{| border="1" cellpadding="10" cellspacing="0"
 
 
!{{Hl2}} | Command
 
!{{Hl2}} | Command
 
!{{Hl2}}| Arguments
 
!{{Hl2}}| Arguments
 
!{{Hl2}}| Description
 
!{{Hl2}}| Description
 
!{{Hl2}}| Cluster
 
!{{Hl2}}| Cluster
 +
|-
 +
|quota
 +
|
 +
|Short overview of a user's storage usage.
 +
|GPC
 
|-
 
|-
 
|diskUsage
 
|diskUsage
−
|''-h'' to see usage
+
|''-h'' to see all command options
 
|Informs about the user and group file system usage.
 
|Informs about the user and group file system usage.
 
|GPC
 
|GPC
 
|-
 
|-
−
|quota
+
|topUserDirOver1000list
 +
|
 +
|Lists your directories that have over 1,000 files
 +
|GPC
 +
|-
 +
|topUserDirOver1GBlist
 
|
 
|
−
|Short overview of a user's storage usage.
+
|Lists your directories that have over 1 GB of data
 
|GPC
 
|GPC
 +
|-
 +
|}
 +
 +
 +
= Jobs & Queues =
 +
{| border="1" cellpadding="5" cellspacing="0"
 +
!{{Hl2}} | Command
 +
!{{Hl2}}| Arguments
 +
!{{Hl2}}| Description
 +
!{{Hl2}}| Cluster
 
|-
 
|-
 
|qsum
 
|qsum
Line 30: Line 52:
 
|Detailed information on jobs that are actively running.
 
|Detailed information on jobs that are actively running.
 
|TCS
 
|TCS
 +
|-
 +
|scinet gpc priority
 +
|
 +
|Compute usage in the last 14 days, and how this affects your priority in the queue.
 +
|GPC
 +
|-
 +
|nodeperf
 +
|[''userName'']
 +
|Who is doing what on the current node?
 +
|GPC
 
|-
 
|-
 
|jobperf
 
|jobperf
Line 72: Line 104:
 
|-
 
|-
 
|jobtree
 
|jobtree
−
|[''<userName>'']
+
|[''userName'']
 
|Displays the jobs tree of dependencies for a given user.
 
|Displays the jobs tree of dependencies for a given user.
 
|GPC
 
|GPC
 
|-
 
|-
 
|debugjob
 
|debugjob
−
|[<''nodes''>]
+
|[''nodes'']
 
|Requests a time-limited interactive session of up to 8 dedicated nodes
 
|Requests a time-limited interactive session of up to 8 dedicated nodes
 
|GPC
 
|GPC
 +
|-
 +
|}
 +
 +
= Modules =
 +
{| border="1" cellpadding="5" cellspacing="0"
 +
!{{Hl2}} | Command
 +
!{{Hl2}}| Arguments
 +
!{{Hl2}}| Description
 +
!{{Hl2}}| Cluster
 
|-
 
|-
 
|modulefind
 
|modulefind
−
|[''<searchWord>'']
+
|[''searchWord'']
−
|Finds modules containing ''<searchWord>'' and indicate their compatibility with already loaded modules.
+
|Finds modules containing ''searchWord'' and indicate their compatibility with already loaded modules.
 +
|GPC
 +
|-
 +
|moduleadvice
 +
|<''modulename''>
 +
|Show the module commands required to load ''modulename'', taking into account prerequisites and conflicts with already loaded modules.
 +
|GPC
 +
|-
 +
|modulewrap
 +
|''<comma-separated-modules>'' ''<command>''
 +
|Load the modules in the ''comma-separated-modules'' list (no spaces!) only to run the given ''command''.
 
|GPC
 
|GPC
 
|-
 
|-
 
|}
 
|}
 +
 +
 +
= Related Topics =
 +
[https://wiki.scinet.utoronto.ca/wiki/index.php/Moab#Checking_the_memory_usage_from_jobs Checking the memory usage from jobs]
 +
<br>
 +
[https://wiki.scinet.utoronto.ca/wiki/index.php/GPC_Quickstart GPC Quickstart]
 +
<br>
 +
[https://wiki.scinet.utoronto.ca/wiki/index.php/Moab Scheduler]
 +
<br>
 +
[https://wiki.scinet.utoronto.ca/wiki/index.php/FAQ#Monitoring_jobs_in_the_queue Monitoring Jobs]
 +
<br>
 +
[https://wiki.scinet.utoronto.ca/wiki/images/a/a0/TechTalkJobMonitoring.pdf Tech Talk on Monitoring Jobs]

Latest revision as of 18:53, 5 September 2015

Below is list of handy command-line utilities written by SciNet to help manage your data and compute.

Note that in the table, optional arguments are denoted with square brackets (i.e. '[]'), while mandatory arguments are denoted with angular brackets (i.e. '<>').

Storage

Command Arguments Description Cluster
quota Short overview of a user's storage usage. GPC
diskUsage -h to see all command options Informs about the user and group file system usage. GPC
topUserDirOver1000list Lists your directories that have over 1,000 files GPC
topUserDirOver1GBlist Lists your directories that have over 1 GB of data GPC


Jobs & Queues

Command Arguments Description Cluster
qsum most showq arguments work Lists jobs running or in the queue, grouped by user. GPC,TCS
llq1 Detailed information on jobs that are actively running. TCS
scinet gpc priority Compute usage in the last 14 days, and how this affects your priority in the queue. GPC
nodeperf [userName] Who is doing what on the current node? GPC
jobperf <jobId | jobName> Informs about the performance of all nodes of a given job. GPC
jobError <jobId | jobName> Displays realtime error output of a given job. GPC
jobOutput <jobId | jobName> Displays realtime standard output of a given job. GPC
jobcd <jobId | jobName> Allows users to quickly move into the working directory of a given job. GPC
jobscript <jobId | jobName> Displays the submission script used when submitting a given job. GPC
jobssh <jobId | jobName> Allows users to connect to the head-node of a given job. GPC
jobtop <jobId | jobName> Allows users to "top" on the head-node of a given job. GPC
jobdep <jobId> Displays dependencies of given job. GPC
jobtree [userName] Displays the jobs tree of dependencies for a given user. GPC
debugjob [nodes] Requests a time-limited interactive session of up to 8 dedicated nodes GPC

Modules

Command Arguments Description Cluster
modulefind [searchWord] Finds modules containing searchWord and indicate their compatibility with already loaded modules. GPC
moduleadvice <modulename> Show the module commands required to load modulename, taking into account prerequisites and conflicts with already loaded modules. GPC
modulewrap <comma-separated-modules> <command> Load the modules in the comma-separated-modules list (no spaces!) only to run the given command. GPC


Related Topics

Checking the memory usage from jobs
GPC Quickstart
Scheduler
Monitoring Jobs
Tech Talk on Monitoring Jobs

Retrieved from ‘https://oldwiki.scinet.utoronto.ca/index.php?title=SciNet_Command_Line_Utilities&oldid=7980’