Difference between revisions of "Oldwiki.scinet.utoronto.ca:System Alerts"
| Line 19: | Line 19: | ||
--> | --> | ||
{| | {| | ||
| − | |[[File: | + | |[[File:down.png|up|link=GPC Quickstart]][[GPC Quickstart|GPC]] |
| − | |[[File: | + | |[[File:down.png|up|link=TCS Quickstart]][[TCS Quickstart|TCS]] |
| − | |[[File: | + | |[[File:down.png|up|link=Sandy]][[Sandy]] |
| − | |[[File: | + | |[[File:down.png|up|link=Gravity]][[Gravity]] |
|[[File:up.png|up|link=BGQ]][[BGQ]] | |[[File:up.png|up|link=BGQ]][[BGQ]] | ||
|[[File:down.png|file systems unmounted]]File System | |[[File:down.png|file systems unmounted]]File System | ||
|- | |- | ||
| − | |[[File: | + | |[[File:down.png|up|link=P7 Linux Cluster]][[P7 Linux Cluster|P7]] |
| − | |[[File: | + | |[[File:down.png|up|link=P8]][[P8]] |
| − | |[[File: | + | |[[File:down.png|up|link=Knights Landing]][[Knights Landing|KNL]] |
| − | |[[File: | + | |[[File:down.png|up|link=Visualization Nodes]][[Visualization Nodes|Viz]] |
| − | |[[File: | + | |[[File:down.png|up|link=HPSS]][[HPSS]] |
|} | |} | ||
| + | |||
| + | <b>Mon Feb 6 16:01:47 EST 2017</b> Full shutdown of the systems was required, in order to bring up and orderly interconnect and filesystems. Please watch this space for updates. | ||
| + | |||
<b>Mon Feb 6 14:24:51 EST 2017</b> We're experiencing internal networking problems, which have caused filesystems to become inaccessible on some nodes. Work is underway to fix this. | <b>Mon Feb 6 14:24:51 EST 2017</b> We're experiencing internal networking problems, which have caused filesystems to become inaccessible on some nodes. Work is underway to fix this. | ||
Revision as of 17:03, 6 February 2017
System Status
 GPC
|
TCS
|
Sandy
|
Gravity
|
 BGQ
|
 File System
|
| P7
|
P8
|
KNL
|
Viz
|
HPSS
|
Mon Feb 6 16:01:47 EST 2017 Full shutdown of the systems was required, in order to bring up and orderly interconnect and filesystems. Please watch this space for updates.
Mon Feb 6 14:24:51 EST 2017 We're experiencing internal networking problems, which have caused filesystems to become inaccessible on some nodes. Work is underway to fix this.
Sat Jan 28 12:54:12 EST 2017 GPC and TCS are up, as well as the filesystems. We have reverted to the old /scratch and /project disks on the GPC, until we can ascertain what was wrong with the new appliance. In the meantime please submit your jobs as usual. Also, please help us by cleaning up unnecessary stuff on /scratch. For TCS users: the changes we implemented, where you need to use /scratchtcs, are still in effect.
Sat Jan 28 12:19:16 EST 2017 We are bringing the systems back. Expect to be ready in a couple of hours.
Sat 28 Jan 2017 8:41 EST BGQ is not affected and the system is up.
Sat 28 Jan 2017 8:15 EST Further issues found on the file system; system access to users has been closed until we can solve these issues.
Fri 27 Jan 2017 15:11:58 EST Cluster network issue is resolved and filesystem access is finally resolved after determining the root cause of the network issue.
Fri Jan 27 11:20:32 EST 2017 While we're restoring things, file systems will generally not be available for usage during this period, to facilitate our work. Sorry.
Fri Jan 27 10:02:32 EST 2017 The IB network fabric had a failure earlier today that affected the file systems. The IB fabric is back to normal, and we're working on restoring the file systems at the moment.
Fri Jan 27 7:34:00 EST 2017 Issues with the new scratch file system; we're investigating.
Thu Jan 26 21:24:14 EST 2017 Maintenance finished, systems back online and available, with the exception of the TCS, that does not accept jobs yet (but the devel nodes are accessible).
Jan 25, 2017, 18:48: BGQ is online.