Squarespace is the Blog hosting service I use for the Green Data Center Blog.
Squarespace is in Peer1 facility which is without utility power, and a flooded basement where the diesel fuel is stored.
Yesterday I received this e-mail.
Dear Customer,
I have some unfortunate news to share. Our primary data center, Peer1, in Lower Manhattan lost power yesterday at about 4:30PM local time. At that time, we smoothly made the transition to generator power and took comfort over the fact that we had enough fuel to last three to four days. (Peer1 stayed online during the last 3 major natural disasters in the area, including a blackout that lasted for days.)
At 8:30PM yesterday, we received reports that the lobby in the data center's building was beginning to take on water. By 10:30PM, as is sadly the case in most of Lower Manhattan, Peer1's basement had experienced serious flooding. At 5AM, we learned our data center's fuel pumps and fuel tanks were completely flooded and unable to deliver any more fuel. At 8AM, they reported that the generators would be able to run for a maximum of four more hours.
Luckily, Peer 1 did not go down as expected. What did they do? They are hand carrying the fuel to the diesel generator on the 17th floor. Here is the complete thread with updates.
Update [12:40pm ET]
Good news and bad news.
We were able to give the crew 90 minutes off because the tank is full. That’s about 4 hours of power given daytime usage. We’ll start back up at 1:30. Our awesome teammates are hoofing lunch over the Brooklyn Bridge for us. As we’ve said before, this situation is untenable. We can’t keep manpower going 24/7 for days.
They are trying to pump the fuel, but are finding out how hard a pump has to work to pump up to the 17th floor.
The building’s first attempt at an alternative method for pumping fuel to the 18th floor has failed, as the fuel pump wasn’t powerful enough. They believe they have sourced an alternate pump, but given the situation in New York City right now, we’re in a wait-and-see posture. Fuel- and water-pumps are in short supply.
Here is the flooded basement where the fuel was supposed to be pumped from.
Here are more photos they have shared.