ISH
ish - inventory shell
Introduction
Browse directories or tarball content from local inventory files.
ish is located in /scinet/gpc/bin/ish.
Current version is 0.9 (a beta version).
Usage
From the command line
ish -h|--help show this help ish --version show version number ish [INDEX] interactive shell for file INDEX ish [INDEX] COMMAND perform single COMMAND on file INDEX
Shell Commands
Most common
ls [-lr] [DIR/[FILES]] ... list FILES from DIR in index cd DIR set current directory in index find PATTERN ... find files following PATTERN in index index DIR|TARFILE make index for folder/tarfile exit exit the shell
More
avail [-a] list (all) available index files colour 1|0 set colour usage help [COMMAND] show help on (all) commands pwd show current directory register [DIR] set new index file location settings list settings tar [-zcf] TARFILE DIR[/FILES] ... tar and make index use [INDEX] use INDEX or list available ones !COMMAND [ARGS] local commands (ls, cd, pwd)
Only in archive queue (hpss)
hindex DIR|TARFILE make index for hpss folder/tarfile htar [-cf] TARFILE DIR[/FILES] ... htar and make index
Command line examples
Local tar and gzip directory 'code': ish tar -z code.tgz code Long list of its content: ish code.tgz.igz ls -l 'code/*' Make index file for existing tar: ish index another.tar List its top level content: ish another.tar.igz ls List all of its content: ish another.tar.igz ls -r Find file 'hello.txt' in it: ish another.tar.igz find hello.txt Make index hspp.igz for hpss tree: ish hindex Create index file for an htar file: ish hindex data.tar Create htar and index file: ish htar -cpf data.tar data/
Ish commands
help - ish command to show help on ish commands
Usage:
help [COMMAND]
If no COMMAND is given, a list of all ish commands, with a brief description, is given.
ls - ish command to list directory contents
Usage:
ls [OPTION] [ PATTERN [PATTERN ... ] ]
Lists files in the index according to one or more patterns. In interactive mode, the files are displayed in colour. The list is sorted alphabetically by name.
Patterns are of the form [PATH[/]][FILES] and may contain wildcards * and ?. Without FILES, the form PATH only lists the directory name, while PATH/ lists the files in the directory.
When no patterns are given, ls lists files in the current directory in the current index file (as given on the command line or set with use).
Paths are relative to the current directory (as set with 'cd'). To specify a path from the root of the index, an initial colon (:) should be put in front of the path. If the index contains absolute paths, the root can be indicated by an initial slash (/) as well.
The optional argument can be:
-l list in long format, displaying information on file sizes, modification times, and other metadata present in the index. -r lists recursively into sub-directories.
To give both options, they have to be combined into one, e.g. -lr.
cd - ish command to set current directory in index
Usage:
cd [PATH]
Change the current directory within the index to the directory specified by PATH.
The new path is relative to the previous current directory, unless preceded by a colon (:), in which case it specifies a path from the root of the index. If the index contains absolute paths, the root can be indicated by an initial slash (/) as well.
If no PATH is given, the current directory is set to the root directory of the index file.
The current directory is always displayed in the ish prompt.
find - ish command to recursively find files in index
Usage:
find PATTERN ...
Recursively searches the directory tree in the index from the current directory (set by 'cd') for files following one or more patterns. PATTERN may be of the form PATH/FILE or just FILE, and can contain the wildcards * and ?.
Known bug: Currently, find returns more matches than is should, i.e., 'myish' is found when finding 'ish'.
exit - ish command to exit the shell
Usage:
exit
Ish will exit with the exit status of the last command that was run.
When input is redirected to be read from a file or here-document, the end-of-file also exits ish, so in that case exit is optional.
index - ish command to make an index for a directory or tarfile
Usage:
index PATH|TARFILE
Makes an index file of a directory (PATH) or tar file (TARFILE). The PATH, or the paths stored in the tar file, can be either absolute or relative (not both). The tar file can be in compressed format, provided the local tar installation supports it.
The index files are stored in the register directory (~/.ish_register by default). See 'register' for details.
For consistency, if the parent directory of any directory in PATH or the TARFILE is not in that PATH or in that TARFILE, it gets added to the index as a stub (i.e., not with its full content). For example, 'index /home/rzon' will include /home as a stub, which will seem to only contain the directory rzon. Similarly, if test.tar was created (not from within ish) with 'tar cf test.tar /home/rzon', then 'index test.tar' will add a stub for /home.
Ish automatically assigns a name to the index file, following this naming convention:
- A directory index stored in the current directory will be called PATH.igz, with any slashes replaced by underscores.
- A directory index stored in the registry will be called ABSPATH.igz, where ABSPATH is the absolute path to the directory PATH, again, with slashes replaced by underscores.
- A tar index will be called TARFILE.igz, with slashes replaced with underscores.
- When running an index command again, or running an index command that would result in the name of an existing index that would be overwritten, the existing index file NAME.igz gets renamed to NAME_DATE_TIME.igz, where date is the date of the existing index file.
The exception to this rule is that if the new index file has identical content to the old one, the old one is removed.
Once the indexing is done, the index file becomes the current one in the ish shell.
The index file for a tar file can be created when the tar is made, using the ish-command tar (see 'tar').
Currently, the index of a local directory does not contain metadata such as file sizes and modification time.
colour - ish command to set colour usage
Usage:
colour ARG
Switches the usage of colour in listings on if ARG=1 and off if ARG=0. By default, colour is on in interactive sessions, and off in single command mode.
The current colour setting can be found with the 'settings' command.
pwd - ish command to show current directory
Usage:
pwd
Shows the current directory within the index (as set by 'cd').
This command should be rarely needed, as the current directory is always displayed in the ish prompt.
register - ish command to set new index file location
Usage:
register [DIR]
Sets the location where index files are stored. This location is called the register. DIR must be an existing directory on the file system. If DIR is omitted, the register is set to the default location.
The default location of the register is the (hidden) directory ~/.ish_register, or is set by the environment variable ISHREGISTER.
Index files in the current directory will be found by 'use' as well, regardless of the register setting. But to store an index file in the current directory (using 'index' or 'hindex'), the register has to be set to '.' first.
The current setting of the register can be found with 'settings'.
settings - ish command to list settings
Usage:
settings
Lists settings such as colour and register location.
tar - ish command to tar and make index
Usage:
tar [OPTION] TARFILE [DIR][FILES] ...
Tars a directory or files list as usual, AND creates the corresponding index file. It should be equivalent to running the same tar command from the shell (i.e. not within ish), followed by "ish index TARFILE".
Common tar options:
-c create the file. This option is implied in the ish version, and
may be omitted.
-f should be the last option, and indicates that output should go to
the tarfile. Also implied and thus optional in the ish version.
-z gzip the tarball.
-j bzip2 the tarball.
To give multiple options, they have to be combined into one, e.g. -czf. If given, -f must be the last option. Other options can be found from the tar man page.
avail - ish command to list available index files
Usage:
avail [-a]
Displays a list of index files the register directory (see 'register') or the current directory. These index files can be accessed with the 'use' command.
By default, older versions of the igz files (as automatically saved by index and hindex) are not shown.
The optional argument can be:
-a When given, older saved version of the igz files are also listed.
See 'index' and 'hindex' on how to create index files, and on their naming scheme.
use - ish command to set the index file
Usage:
use INDEX
Sets the current index file to INDEX. This file should reside in the register directory (see 'register') or the current directory. INDEX should include the index file extension '.igz'.
Note that the index file can also be set when starting ish with the name of the index file as the first argument (e.g. ish data.tgz.igz).
See 'index' and 'hindex' on how to create index files, and on their naming scheme.
The current index file is always displayed in the ish prompt.
!COMMAND - ish command to local commands (ls, cd, pwd)
Usage:
!COMMAND [ARGS]
Runs a local command. This is intended to allow the local directory to be changed and listed. Any arguments will be passed to the command.
Only the commands ls, cd and pwd will be accepted to run locally.
hindex - ish command to make index for hpss folder/tarfile (hpss only)
Usage:
hindex [PATH|TARFILE]
Makes an index file of a directory (PATH) or tar file (TARFILE) on hpss. The distinction is made on whether the argument has the extension .tar.
The PATH, or the paths stored in the tar file, can be either absolute or relative (not both). The tar file cannot be in compressed format, and ideally should have been created with htar.
If neither a PATH nor a TARFILE is given, hindex will generate an index for your whole hpss directory, and store the index in hpss.igz.
The index files are stored in the register directory (~/.ish_register by default). See 'register' for details.
For consistency, if the parent directory of any directory in PATH or the TARFILE is not in that PATH or in that TARFILE, it gets added to the index as a stub (i.e., not with its full content). For example, 'hindex /archive/scinet/rzon' will include /archive and /archive/scinet as a stubs, which will seem to only contain one directory. Similarly, if test.tar was created (not from within ish) with 'htar cf test.tar /home/rzon', then 'hindex test.tar' will add a stub for /home.
Ish automatically assigns a name to the index file, following this naming convention:
- The directory index will be called PATH.igz, with any slashes replaced by underscores.
- A tar index will be called TARFILE.igz, with slashes replaced with underscores.
- The index will be called hpss.igz when hindex is called without argument.
- When running an hindex command again, or running an hindex command that would result in an existing index getting overwritten, the existing index file NAME.igz gets renamed to NAME_DATE_TIME.igz, where DATE and TIME are of the existing index file. The exception to this rule is that if the new index file has identical content to the old one, the old one is removed.
Ish calls the hsi application under the hood to get the directory listings. This means that this command has to be run on a system that has hpss with hsi. This may mean you have to submit a job for building the index. With the index file, you can then locally traverse the listings in the index and see the modification times and file sizes.
If the tar file was created with htar (outside of ish), then the index is created from the remote index file generated by a call to htar. So this command has to be run on a system that has hpss with htar.
If the hpss-resident tar file was created with tar (outside of ish), then no remote index file exists and ish will first request htar to generate one. Because this may require the tar to be reclaimed from tape, this is an expensive operation. So it is recommended to 1) use htar, or, 2) at the point of tarring, use ish's tar command, or, 3) if the tar is already created and still available locally, to use the ish command index.
Once the indexing is done, the index file becomes the current one in the ish shell.
Note that the htarring and the creation of a local index file can be done with one command, using the 'htar' ish command.
htar - ish command to htar and make index (hpss only)
Usage:
htar [OPTIONS] TARFILE [DIR][FILES] ...
Htars a directory or set of files list as usual, AND creates the corresponding local index file. It should be equivalent to running the same htar command from the shell (i.e. not from within ish), followed by "ish hindex TARFILE".
Common htar options:
-c create the file. This option is implied in the ish version, and
may be omitted.
-f should be the last option, and indicates that output should go to
the tarfile. Also implied and thus optional in the ish version.
-p Preserve time stamps etc.
To give multiple options, they have to be combined into one, e.g. -cpf. If given, -f must be the last option. Other options can be found from the htar man page.