Facebook shares it data analysis, one way to get fired is to look where you aren't supposed to

Techcrunch has a post on Facebook's data analysis.

How Big Is Facebook’s Data? 2.5 Billion Pieces Of Content And 500+ Terabytes Ingested Every Day

JOSH CONSTINE

posted 2 hours ago
7 Comments
Facebook Big Data Numbers

Facebook revealed some big, big stats on big data to a few reporters at its HQ today, including that its system processes 2.5 billion pieces of content and 500+ terabytes of data each day. It’s pulling in 2.7 billion Like actions and 300 million photos per day, and it scans roughly 105 terabytes of data each half hour. Plus it gave the first details on its new “Project Prism”.

VP of Engineering Jay Parikh explained why this is so important to Facebook: “Big data really is about having insights and making an impact on your business. If you aren’t taking advantage of the data you’re collecting, then you just have a pile of data, you don’t have big data.” By processing data within minutes, Facebook can rollout out new products, understand user reactions, and modify designs in near real-time.

Another stat Facebook revealed was that over 100 petebytes  of data are stored in a single Hadoop disk cluster, and Parikh noted “We think we operate the single largest Hadoop system in the world.” In a hilarious moment, when asked “Is your Hadoop cluster bigger than Yahoo’s?”, Parikh proudly stated “Yes” with a wink.

If you concerned about who looks at the data, consider one way to get fired is to look where you are not supposed to.

Users might be a little bit uneasy about the idea that Facebook employees could look so deep into their activity, but Facebook assured me there are numerous protections against abuse. All data access is logged so Facebook can track which workers are looking at what. Only those working on building products that require data access get it, and there’s an intensive training process around acceptable use. And if an employee pries where they’re not supposed to, they’re fired. Parikh stated strongly “We have a zero-tolerance policy.”

Amazon makes Cold Storage sexy, Glacier has a bunch of chatter

I saw the Amazon Glacier announcement and decided to watch how things progress.

NewImage

The news is pretty high.

Amazon Glacier Offers Low-Cost Data Archiving

InformationWeek - ‎2 hours ago‎
Amazon Web Services expanded its cloud services portfolio Tuesday with the launch of Amazon Glacier, a low-cost archiving option. Befitting its chilly namesake, the new service aims to be the data equivalent of cryogenic storage--that is, it's designed for ...
 

Amazon Launches Glacier Data Archiving and Backup Service

TalkinCloud (blog) - ‎2 hours ago‎
Amazon isn't pulling any punches as it launches Glacier either. According to the cloud provider, “companies typically overpay for data archiving” due to required upfront payments and their inability to accurately guess what their capacity requirements will be.
 

Amazon launches Glacier cloud storage, hopes enterprise will go cold on tape use

ZDNet - ‎Aug 21, 2012‎
"Using Amazon Glacier... unlimited archival storage is available to [AWS customers] with a familiar pay-as-you-go model," Werner Vogels, the company's chief technology officer, wrote in a blog post on Tuesday. "The service redundantly stores data in multiple ...
 

Amazon Glacier: Low-Cost Cloud Archiving, But Not If Data Needs Occasional ...

CRN - ‎15 hours ago‎
Amazon Glacier is slated to be a highly reliable service. AWS claims average annual durability of 99.999999999 percent, or 11 nines of availability, which is the same durability as that of its Amazon S3 standard storage. Amazon estimated that a durability of ...
 

Amazon Glacier: Back Up All Your Data for Pennies a Month

Wired News - ‎21 hours ago‎
Amazon Glacier, as the new service is known, stores a copy of your data in archived format for less than a penny per gigabyte per month. The only catch is that getting the data back out can take some time since Glacier is primarily intended for backup and ...
 

Is There a Landmine Hidden in Amazon's Glacier?

Wired News - ‎19 hours ago‎
On Tuesday, Amazon unveiled a new online storage service known as Glacier. It's called Glacier because it deals in “cold storage” — i.e., the long-term storage of things like medical records or financial documents that you may need to archive for regulatory ...

But what I am finding even more interesting is the twitter traffic. "amazon glacier" is getting a fair amount of tweets per minute.

Amazon Glacier punts cloud-based backup at enterprise

 
Expand
 
12m

アマゾン、低価格アーカイブサービス Amazon Glacier を開始。GBあたり月1円以下さんから 氷河だけに凍りそうなアクセス時間…

 
Expand
 
13m

Llega Amazon Glacier, un servicio cloud low cost

 
Expand
 
13m

Is There a Landmine Hidden in Amazon’s Glacier?

 
Expand
 
13m

[Digital Inspiration] Amazon Glacier – The Most Affordable Online Backup Service Ever!  via 

Three Rules for a Successful Company in Silicon Valley - not noticed, proven with $5mil, and survive onslaught of incumbents

TechCrunch and others covered David O Sacks, CEO of Yammer's post on three things that make it hard to create a successful company in silicon valley.

David O Sacks
Saturday via mobile
I think silicon valley as we know it may be coming to an end. In order to create a successful new company, you have to find an idea that (1) has escaped the attention of the major Internet companies, which are better run than ever before; (2) is capable of being launched and proven out for ~$5M, the typical seed plus series A investment; and (3) is protectable from the onslaught of those big companies once they figure out what you're onto. How many ideas like that are left?

You can see the original Facebook post which is public.

David's main point is not that there are a shortage of ideas, but how hard it is create a company.

David O Sacks Human creativity has not changed, and there will always be new ideas and opportunities. But the question is, how many of those opportunities will be captured by startups versus incumbents? It seems like a statistical fact that when you go from virtually no incumbents to multiple well-run incumbents, an increasing percentage of opportunities will be captured by the latter. That's the point I'm making about Silicon Valley -- we may not be running out of ideas, but we might be running out of big new companies.
Saturday at 11:52am via mobile ·  · 5

One of the problems is if your company is in Silicon Valley you are surrounded by your competitors.  Staying under the radar, yet proving you have a sound revenue model, and the big guys won't crush you is really really hard. I grew up and spent my first 12 years working for HP and Apple in Cupertino and you learned to keep your mouth shut and not talk shop when you were outside of work.  

I do agree with David and I wouldn't build a successful new company in silicon valley.  The easiest way to achieve success is not to tell the VC community what the new company is.  But, then it is hard to get the $5mil of funding. 

Finding Patterns in Places, leads to insight

A group of Carnegie Melon Researchers published a Siggraph paper on "What makes Paris look like Paris?"  When I read this paper and watched the video it reminded me of many of the conversations that some of the top data center people have.  Comparing elements of what they see and do with others, discussing what others try and whether it would work in their environment.

Last year a bunch of us went to visit some top financial data center executives and toured their facilities.  We could find elements of the power, mechanical and other systems that told us who designed the building and how it compared to their building.

NewImage

One of the insights from the paper is that the famous landmarks are not what make the look and feel of the city, but the stylistic elements.

6 Conclusion
So, what makes Paris look like Paris? We argued that the “look
and feel” of a city rests not so much on the few famous landmarks
(e.g. the Eiffel Tower), but largely on a set of stylistic elements,
the visual minutiae of daily urban life. We proposed a method
that can automatically find a subset of such visual elements from
a large dataset offered by Google Street View, and demonstrated
some promising applications. 

Do you work best during a sunny or cloud day? Study says bad weather increases performance

HBR has a post that is good for those of you who live in bad weather environments.  

Morning Advantage: Busy Day Ahead? Pray For Rain

by Kevin Evers  |   5:00 AM August 17, 2012

Common sense tells us that bad weather makes us blue and therefore less productive at our jobs, but it turns out the opposite is true. As detailed in this executive brief at HBS Working Knowledge, researchers Jooa Julie Lee, Francesca Gino, and Bradley R. Staats found that performance at a mid-sized Japanese bank peaked on the days with the worst weather, as it did in a series of lab experiments they conducted too. Here’s their theory: When the weather’s bad, our minds wander less, and we’re able to focus more because we really can’t do anything else.

Now in data centers, you don't usually get to see the sun that much.  There aren't a lot of skylights or windows, so the weather may not affect you.  Or the weather may affect you because you know what is like outside.

Some suggestions from their research: Save drudge work for the wettest days and creative endeavors for sun-soaked days. Other ideas include using weather reports to determine the best staff size on a given day and moving service centers to cities with poor (but not entirely depressing) weather. Here’s to you, Seattle.