Difference between revisions of "MARS"

From oldwiki.scinet.utoronto.ca
Jump to navigation Jump to search
m
 
(197 intermediate revisions by 2 users not shown)
Line 1: Line 1:
−
=== Massive Archive and Restore System ===
+
{| style="border-spacing: 8px; width:100%"
 +
| valign="top" style="cellpadding:1em; padding:1em; border:2px solid; background-color:#f6f674; border-radius:5px"|
 +
'''WARNING: SciNet is in the process of replacing this wiki with a new documentation site. For current information, please go to [https://docs.scinet.utoronto.ca https://docs.scinet.utoronto.ca]'''
 +
|}
  
−
(Pilot usage phase to start in May/2011 with a select group of users. Deployment and configuration are still a work in progress)
+
MARS is not more. [https://support.scinet.utoronto.ca/wiki/index.php/HPSS Follow this link]
−
 
 
−
The '''MARS''' deployment at SciNet is an effort offer a much more efficient way  to offoad/archive data from the most active file systems (scratch and project) then our current TSM-HSM solution, still without necessarily having to deal directly with the tape library or "tape commands"
 
−
 
 
−
The system is a combination of 3 software components, HPSS, HSI and HTAR, plus the underlaying hardware infrastructure and some environment customization.
 
−
 
 
−
'''HPSS''': the main component, best described as a very scalable "blackbox" engine running in the background to support the Archive and Restore operations. [http://www.hpss-collaboration.org/index.shtml High Performance Storage System - HPSS] is the result of over a decade of collaboration among five Department of Energy laboratories and IBM, with significant contributions by universities and other laboratories worldwide. For now the best way for SciNet users to [https://support.scinet.utoronto.ca/wiki/index.php/HPSS_compared_to_HSM-TSM understand HPSS] may be to compare it with our existing HSM-TSM implementation.
 
−
 
 
−
'''HSI''': it may be best understood as a supercharged ftp interface, specially designed by [http://www.mgleicher.us/GEL/hsi/ Gleicher Enterprises] to act as a front-end for HPSS, gathering some of the best features you would encounter on a shell, rsync and GridFTP (and a few more). It enables users to transfer whole directory trees from /project and /scratch, therefore freeing up space. HSI is most suitable when those directory trees do not contain too many small files to start with, or when you already have a series of tarballs.
 
−
 
 
−
'''HTAR''': similarly, htar is sort of a "super-tar" application, also specially designed by [http://www.mgleicher.us/GEL/htar/ Gleicher Enterprises] to interact with HPSS, allowing users to build and automatically transfer tarballs to HPSS on the fly. HTAR is most suitable to aggregate whole directory trees, provided that no individual files exceed 68GB. The maximum size of any htar file should not exceed 1TB either.
 

Latest revision as of 19:37, 31 August 2018

WARNING: SciNet is in the process of replacing this wiki with a new documentation site. For current information, please go to https://docs.scinet.utoronto.ca

MARS is not more. Follow this link

Retrieved from ‘https://oldwiki.scinet.utoronto.ca/index.php?title=MARS&oldid=9413’