Still digging out from the campus-wide power outage

We're still digging out from the campus-wide power outage.  This was as major disaster as Stanford could ever have, as the whole campus was out for a while, and chilled water was out even after power came back on.  Moreover, this hapened as everyone was sleeping (I logged off at ~11:30pm and this happened at 11:58pm), so there wasn't anything we could do to fix it until ~8 hours later.

We're pushing to get the CS's back first.  It looks like the main CS (171.65.103.100) is having hardware issues and we're working on that.  Having such a large-scale disaster has given us some ideas on how we can be more ready for this in the future, although there's already major investigations at Stanford to figure out what happened and how to prevent it (a campus-wide power outage is really not acceptable for an institution like Stanford).

Comments

[Paul]

There's a nice article in the December 8th-14th Economist that talks about FAH, along with other distributed computing projects. Here's a link to the online edition:

http://www.economist.com/printedition/displaystory.cfm?story_id=10202635

[Mark F Williams]

There was an error accessing/using the database.
Description: failed query:select count(*) from cpus where team = '198' and lasttime > subdate(now(), interval 50 day)
Please send this error message and the conditions under which it was caused to the F@H researchers.
Thank you.

[Mark F Williams]

There was an error accessing/using the database.
Description: failed query:select count(*) from cpus where team = '198' and lasttime > subdate(now(), interval 50 day)
Please send this error message and the conditions under which it was caused to the F@H researchers.
Thank you.

[Chris Benamati]

It seems the entire team tracking stats system is down. When you try to access it, it gives you a message such as:

There was an error accessing/using the database.
Description: failed query:select count(*) from cpus where team = '54345' and lasttime > subdate(now(), interval 50 day)
Please send this error message and the conditions under which it was caused to the F@H researchers.
Thank you.

This is probably being looked into at Stanford. So, if anybody is getting the same message, don't worry, it's not just you.

[Stilzchen]

Since many days i get the following error message:

[12:45:16] + Attempting to get work packet
[12:45:16] - Connecting to assignment server
[12:45:17] - Successful: assigned to (171.64.65.111).
[12:45:17] + News From Folding@Home: Welcome to Folding@Home
[12:45:17] Loaded queue successfully.
[12:45:20] - Error: Attempt #71 to get work failed, and no other work to do.
Waiting before retry.

Is it normal and a problem of the power outage?