News of Facebook's Raining inside its Data Center leaks 2 years after the event

When I was at the first Open Compute Summit I was sitting with a data center executive and when he heard about the air misters used in Facebook's 1st data center he said they were going to run into problems.  We don't hear about misters in Facebook's data center any more and news has leaked out what may have helped convince putting humidity into the data center has issues.

The Register broke the news in an interview with Facebook VP Jay Parikh.

Facebook's first data center ran into problems of a distinctly ironic nature when a literal cloud formed in the IT room and started to rain on servers.

Though Facebook has previously hinted at this via references to a "humidity event" within its first data center in Prineville, Oregon, the social network's infrastructure king Jay Parikh told The Reg on Thursday that, for a few minutes in Summer, 2011, Facebook's data center contained two clouds: one powered the social network, the other poured water on it.

"I got a call, 'Jay, there's a cloud in the data center'," Parikh says. "'What do you mean, outside?'. 'No, inside'."

There was panic.

"It was raining in the datacenter," he explains.

Facebook has made some changes to weatherproof components in the data center.

Some servers broke entirely because they had front-facing power supplies and these shorted out. For a few minutes, Parikh says, you could stand in Facebook's data center and hear the pop and fizzle of Facebook's ultra-lean servers obeying the ultra-uncompromising laws of physics.

Facebook learned from the mistakes, and now designs its servers with a seal around their power supply, or as Parikh calls it, "a rubber raincoat."

As Faceboook Lulea powers up, makes sure its neighbors don't put infrastructure at risk

Part of what a big data center company like Google and Microsoft have learned to expect is that neighbors show up after they are the first to build.  Facebook going to Lulea was one of the first data center operators in the area.  And now that there are others looking at Lulea and Facebook is going through commissioning, there are concerns there is enough infrastructure.

NyTeknik reports in Swedish on the local event.

Soon it's time for the commissioning of the first of the giant data centers like Facebook is based in Luleå. Availability of green electricity from hydropower and a stable electricity grid were two of the factors that got Facebook to choose Luleå.

While trying municipality to attract more IT companies and a large computer giant is about to establish itself around Facebook in Porsön in Luleå.

But now reacts Facebook and through its agents Pinnacle Sweden, asked to Luleå further investigate the availability of energy, water and sanitation, the newspaper reported Norrländska Socialdemokraten.

Here is a picture from the same publication.

Facebook's Iowa Data Center has three phases over 6 years

Des Moines Register has a review of the 18 month process the City of Altoona went through.  

What seems interesting is a tax issue based on a three limit needed to get changed to six.

Also in March, a roadblock suddenly emerges: VanderZanden emails Durham that state rules prohibit the company from fully benefiting from Altoona’s proposed 20-year tax abatement.

The rules specify that all phases must be built within a three-year project completion window. Facebook plans three phases over six years.

“Iowa should remove as many hurdles as it can to our building the third building in Altoona,” VanderZanden writes in an email.

The issues of renewable energy shows up in the article too.

Durham says Facebook discusses a “wind farm totally dedicated to them” vs. investing in a portfolio of renewable energy.

It’s tricky territory for her. A wind farm would compete with energy Facebook gets from Mid­American, a close partner with the state in encouraging economic development in Iowa.

Eventually the plan is for there to be a 100 employees.

In the parking lot after the announcement, Durham signs the state’s finalized contract with Facebook. The state agrees to provide Facebook with $18 million in tax credits, based on job creation and capital investment.

Within days, Facebook begins filling key positions, says Kirkland, Facebook’s spokesman. State leaders believe permanent jobs will grow to about 100 when all three phases are built.

Open Compute Project History described in a Taxonomy Map

In Frank Frankovsky's keynote at Interop he showed the progression from red to green of the open compute project from data center, rack, server, storage and now network.

Frank started with the open source software used.

Then Facebook shared its data center practices.

Next came racks in the co-lo

Eventually building to the complete taxonomy below.

What's next?  The Open Compute Project comment is the one who asks for new things as well as Facebook deciding to contribute in an open source manner.

Open Compute Project's Frank Frankovsky launches Network Initiative

The folks at Open Compute Project are on a roll, driving open source ideas into servers, data centers, racks, etc.

And now their latest move is networking.

With that in mind, we are today announcing a new project within OCP that will focus on developing a specification and a reference box for an open, OS-agnostic top-of-rack switch. Najam Ahmad, who runs the network engineering team at Facebook, has volunteered to lead the project, and a wide variety of organizations — including Big Switch Networks, Broadcom, Cumulus Networks, Facebook, Intel, Netronome, OpenDaylight, the Open Networking Foundation, and VMware — are already planning to participate.

Facebook is a big sponsor of Open Compute Project which is non-profit organization, and a driving force.

Who competes against Open Compute Project?  The visibility for Open Compute Project has increased and is referenced by almost all the tech media point with an article at some point.  There isn't really much competition.

What Open Compute Project does threaten is the event conference vendors.  When you have a non-profit focused on end user benefits and changing the industry, the for profit event staff who are driven to maximize revenue will maneuver to out market their event as the premier event in the industry.  Note: watch for those words when someone claims they are the premier event.  

BTW isn't it ironic that Interop has Frank keynote what a message that Open Compute Project is revolutionizing the industry.

Location: Mandalay Bay H
Wednesday, May 8, 2013, 8:30 AM-10:15 AM

Billions of people and their many devices will be coming online in the next decade, and those who are already online are living ever-more connected lives. The industry is building out a huge physical infrastructure to support this growth, but we are doing so in a largely closed fashion, inhibiting the pace of innovation and preventing us from achieving the kinds of efficiencies that might otherwise be possible.

In this keynote, Frank Frankovsky will provide an overview of the Open Compute Project, a thriving consumer-led community dedicated to addressing this issue, promoting more openness and a greater focus on scale, efficiency, and sustainability in the development of infrastructure technologies. Frank will delve into the brief history of the project and describe its vision for the future.

Seems like Interop just gave Frank a chance to market to the audience they need to go to the next Open Compute Summit and attend for free.  Or be a sponsor where there is more transparency in how their sponsorship is used.

We'll see what kind of networking guys show up to the next Open Compute Summit.  In some data center IT environments the network gear can be 50% of the IT budget.  Networks are important, but it is hard for most to accept a 50% spend.