Google's Gmail self heals from shooting itself in the foot with config updates, after 25 minutes systems restored, Outage ends

There is tons of press on Gmail outage.  I was on the phone during the time the outage occurred so gmail being down didn’t bother me, but it did bother many others.

Gmail goes down briefly and everybody flips out

Atlanta Journal Constitution - ‎5 hours ago‎
If you're watching this, congratulations! You've survived the Great Google Outage of Jan. 24, 2014. At about 2:15 p.m. eastern time Friday, Gmail users across the world began seeing Temporary Error (500) error message while trying to access their email ...
 

Google's Gmail outage leaves many in the dark [San Jose Mercury News :: ]

Businessweek - ‎14 hours ago‎
Jan. 24--MOUNTAIN VIEW -- An unexplained outage affected countless users of Google's (GOOG) popular Gmail service for more than an hour Friday, while also disrupting the Google Plus social network and some of the company's other Web services, ...
 

Google's reliability team was prepping for a reddit AMA when Gmail went down

Washington Post (blog) - ‎50 minutes ago‎
While most of Twitter panicked over (and Yahoo celebrated) a Gmail outage, Google's Site Reliability Engineering was preparing to do an "Ask Me Anything" reddit thread. Depending on how paranoid you are, that may seem either incredibly ironic or like ...
 

Google services go down as Reliability team takes questions on Reddit

Fox News - ‎10 hours ago‎
Many of Google's services hiccupped briefly on Friday, as an unexplained outage knocked offline such popular services as Gmail, Calendar, Talk, Docs, Drive and more. As of 3:23 p.m. EST, the service was back up and running smoothly, according to the ...
 

Here's what caused that massive Gmail outage

Washington Post (blog) - ‎50 minutes ago‎
The outage, Traynor continued, essentially fixed itself when the system responsible for the malfunction automatically generated the correct configuration and began propagating that throughout Google's live services. Google offered an apology for the mishap ...

Here is Blog Post from Google VP Engineering Ben Traynor.  The brief summary of the problem, and how it self repaired is here.

At 10:55 a.m. PST this morning, an internal system that generates configurations—essentially, information that tells other systems how to behave—encountered a software bug and generated an incorrect configuration. The incorrect configuration was sent to live services over the next 15 minutes, caused users’ requests for their data to be ignored, and those services, in turn, generated errors. Users began seeing these errors on affected services at 11:02 a.m., and at that time our internal monitoring alerted Google’s Site Reliability Team. Engineers were still debugging 12 minutes later when the same system, having automatically cleared the original error, generated a new correct configuration at 11:14 a.m. and began sending it; errors subsided rapidly starting at this time. By 11:30 a.m. the correct configuration was live everywhere and almost all users’ service was restored.

Naive users are comparing Yahoo’s email outage to Google’s gmail.  Did Yahoo self heal?  No.  

Google knows it can win the e-mail battle with better availability.  Things happen, but if you can quickly recover and find the cause the overall site reliability should improve.

With services once again working normally, our work is now focused on (a) removing the source of failure that caused today’s outage, and (b) speeding up recovery when a problem does occur. We'll be taking the following steps in the next few days:
1. Correcting the bug in the configuration generator to prevent recurrence, and auditing all other critical configuration generation systems to ensure they do not contain a similar bug.
2. Adding additional input validation checks for configurations, so that a bad configuration generated in the future will not result in service disruption.
3. Adding additional targeted monitoring to more quickly detect and diagnose the cause of service failure.

Facebook predicts possibility Princeton will disappear

Sometimes the best response to a silly idea is to make fun of it in response.  Facebook posts its response to Princeton’s paper that Facebook will have 80% users.

Debunking Princeton

January 23, 2014 at 2:57pm

Like many of you, we were intrigued by a recent article by Princeton researchers predicting the imminent demise of Facebook. Of particular interest was the innovative use of Google search data to predict engagement trends, instead of studying the actual engagement trends. Using the same robust methodology featured in the paper, we attempted to find out more about this "Princeton University" - and you won't believe what we found!

Facebook pokes fun that simply because there is a correlation graph it can explained by causation.

In keeping with the scientific principle "correlation equals causation," our research unequivocally demonstrated that Princeton may be in danger of disappearing entirely. Looking at page likes on Facebook, we find the following alarming trend:

 

To bring up data outside Facebook, Google Trends is used.

 

Sadly, this spells bad news for this Princeton entity, whose Google Trends search scores have been declining for the last several years:

 

 

This trend suggests that Princeton will have only half its current enrollment by 2018, and by 2021 it will have no students at all, agreeing with the previous graph of scholarly scholarliness. Based on our robust scientific analysis, future generations will only be able to imagine this now-rubble institution that once walked this earth.

IBM sells x86 Server business, and doubles capacity of Softlayer Cloud - Sounds like a Good Swap

There is all kinds of news on IBM selling its x86 Server business to Lenovo.

 

Lenovo to Buy IBM Low-End Server Business for $2.3 Billion

Chinese Computer Maker Aims to Expand Corporate-Client Business Beyond Office PCs

By CONNECT
Updated Jan. 23, 2014 8:13 a.m. ET
 
The press release is here.  And there is not a single mention of Softlayer who in theory would be using IBM x86 Servers.
 
On Jan 17, 2014 IBM announced it would spend $1.2 bil to double its Softlayer Cloud capacity and word server does not show up.

IBM Commits $1.2 Billion to Expand Global Cloud Footprint

Builds a Massive Network of Local Cloud Hubs for Businesses Worldwide with 40 Data Centers Across Five Continents

Select a topic or year

ARMONK, N.Y. - 17 Jan 2014: IBM (NYSE: IBM) today announced plans to commit  over $1.2 billion to significantly expand its global cloud footprint. This investment includes a network of cloud centers designed to bring clients greater flexibility, transparency and control over how they manage their data, run their business and deploy their IT operations locally in the cloud. 

I once asked a Softlayer person if they have started running IBM servers in their Cloud environment.  His answer surprised me.  No.  We can’t run the IBM Servers.  We have fine tuned our automation to work with a particular BIOS we control and make sure is on all servers.
Check the this picture of IBM Softlayer CEO Lance Crosby.  Notice how he is back of the server rack and not in front of the servers.  I know the server vendor, but it is not appropriate to share and who knows if they have changed since I talked to the Softlayer person last summer.
NewImage
Wonder if the IBM folks got a wake-up call when they realized they couldn’t use their own servers in their cloud environment without making a lot of changes to accommodate the Softlayer BIOS and probably break a huge amount of other management tools.
 
 

Can Facebook develop the antibodies to protect its Business? 80% user loss assumes no

There is a huge amount of news on Facebook losing up to 80% of its user base by 2017.

Study: Facebook to Lose 80 Percent of Users, Become the Next MySpace

PC Magazine - ‎3 hours ago‎
Just like an infectious disease, social networks can spread rapidly, gaining millions of users in a short amount of time, and then abruptly die off. It happened to MySpace, and Facebook could be next, according to a new study from Princeton University.
 

Facebook Is About to Lose 80% of Its Users, Study Says

TIME - ‎Jan 21, 2014‎
Basically, Facebook users will lose interest in Facebook over time as their peers lose interest — if the model is correct. ”Ideas, like diseases, have been shown to spread infectiously between people before eventually dying out, and have been successfully ...
 

Facebook like an infectious disease, will lose 80 percent of users, says Princeton ...

NBCNews.com - ‎7 hours ago‎
Don't worry, like a viral outbreak, Facebook use will explode before plummeting down to Myspace levels of obscurity, says a new study from Princeton University. Using epidemiological models used to study the rapid spread of disease, the researchers found ...
 

Is Facebook like a spreading disease that's about to fade away? Princeton study ...

MarketWatch (blog) - ‎4 hours ago‎
Facebook could end up shedding a big chunk of its 2 billion users in three years, says a Princeton University study that cited 'disease-like dynamics' in the social network's upcoming doom. The study, by two Princeton PhD candidates Joshua Spechler and ...
 

Facebook Doomsday: Princeton researchers predict 80 percent users will quit by ...

Tech Times - ‎12 minutes ago‎
(Photo : Tech Times) The latest study from Princeton using epidemiological models reveal that the popularity of Facebook has begun to wane, and may see a dramatic 80 percent drop in peak user base between 2015 and 2017. These models were validated ...

There are a couple of points I want to make.  I think most people didn’t go through the whole document.  If they did they would have seen some nice graphics

The Facebook user projections.

NewImage

Current Facebook traffic vs. Myspace.

NewImage

When you look at decline linear progression of MySpace decline it seems like Facebook should follow the same slow decline vs. the bell curve shape.

This is all theoretical.  I would assume Facebook figures out the antibodies to protect its business and keep users.  The size of Facebook is much bigger than MySpace so it kind of like saying that the behavior of rabbits  can represent the behavior of bears.  Don’t think so.

Honestly, a Startup is surprised at its failure because it couldn't crack the business model?

I read this GigaOm post on a startup shutting its doors.

Chris “moot” Poole pens an honest goodbye to his failed startup, DrawQuest

by 

 

2 HOURS AGO

No Comments

ChrisPoole20121
SUMMARY:

Chris Poole’s big post-4chan startup, DrawQuest, is shutting down, and his fair, graceful blog post should be required reading for all startup CEOs.

After four years and one big strategy shift, Christopher “moot” Poole’s big venture after 4Chan, DrawQuest, is shuttering its doors. Poole honestly detailed the experience in a blog post on the matter, which discusses the failure that came despite a relatively healthy community that the “draw a day” app created:

What was funny is the founder made this point.

“It may seem surprising that a seemingly successful product could fail, but it happens all the time,” Pool said. “Although we arguably found product/market fit, we couldn’t quite crack the business side of things.”

Here is a post on advice to startups that focuses on the business model with a nice explanation of the equation to solve.  If you don’t have a business model that works then you will no survive and your product is not successful in the eyes of the business owners.

The Essence of a Business Model

As outlined in the Business Models introduction, a simple way to focus on what matters in your business model is look at these two questions:

  • Can you find a scalable way to acquire customers
  • Can you then monetize those customers at a significantly higher level than your cost of acquisition

Thinking about things in such simple terms can be very helpful. I have also developed two “rules” around the business model, which are less hard and fast “rules, but more guidelines. These are outlined below:

Luckily I figured out the business model problem 5 years ago and have told my business partners that the #1 thing we will do is be innovative in developing business models.  We will do business different than others and then fit the technology.