News of Facebook's Raining inside its Data Center leaks 2 years after the event

When I was at the first Open Compute Summit I was sitting with a data center executive and when he heard about the air misters used in Facebook's 1st data center he said they were going to run into problems.  We don't hear about misters in Facebook's data center any more and news has leaked out what may have helped convince putting humidity into the data center has issues.

The Register broke the news in an interview with Facebook VP Jay Parikh.

Facebook's first data center ran into problems of a distinctly ironic nature when a literal cloud formed in the IT room and started to rain on servers.

Though Facebook has previously hinted at this via references to a "humidity event" within its first data center in Prineville, Oregon, the social network's infrastructure king Jay Parikh told The Reg on Thursday that, for a few minutes in Summer, 2011, Facebook's data center contained two clouds: one powered the social network, the other poured water on it.

"I got a call, 'Jay, there's a cloud in the data center'," Parikh says. "'What do you mean, outside?'. 'No, inside'."

There was panic.

"It was raining in the datacenter," he explains.

Facebook has made some changes to weatherproof components in the data center.

Some servers broke entirely because they had front-facing power supplies and these shorted out. For a few minutes, Parikh says, you could stand in Facebook's data center and hear the pop and fizzle of Facebook's ultra-lean servers obeying the ultra-uncompromising laws of physics.

Facebook learned from the mistakes, and now designs its servers with a seal around their power supply, or as Parikh calls it, "a rubber raincoat."