US CTO Candidates, GOOG, AMZN, OR MSFT -Purpose Develop Environmental Friendly Tech

October 22, 2008 Dave Ohara

BusinessWeek writes on the idea of Obama’s idea for a US CTO.

The Short List for U.S. Chief Technology Officer

Barack Obama has pledged to name a cabinet-level CTO to oversee a job-creating national broadband buildout if he's elected. Big names abound

By Tom Lowry

Barack Obama says that the U.S. is not doing nearly enough to create jobs through technology. Shortly after he launched his campaign, the Illinois Senator promised that if elected, he would create the first-ever Cabinet-level post of chief technology officer. The economic crisis has since made it certain that a White House CTO would become one of Obama's most important advisers, should he triumph in November.

Candidates for the job are Google, Amazon, and Microsoft.

Among the candidates who would be considered for the job, say Washington insiders, are Vint Cerf, Google's (GOOG) "chief internet evangelist," who is often cited as one of the fathers of the Internet; Microsoft (MSFT) chief executive officer Steve Ballmer; Amazon (AMZN) CEO Jeffrey Bezos; and Ed Felten, a prominent professor of computer science and public affairs at Princeton University.

Broadband is one focus as well as a $50 billion VC fund for environmental friendly technology.

A White House CTO would be expected to help create incentive programs to expand broadband's reach, particularly tax credits for smaller carriers. But the tech czar would almost certainly be deeply involved in overseeing a federally-backed $50 billion venture capital fund that Obama has proposed to develop more environmentally friendly technology.

It is interesting to imagine a US CTO with one of these high tech executives. Tech company politics are taken to a new level.

An Application Architect who Cares about Energy Efficiency

July 16, 2008 Dave Ohara

Pat Helland put up his post about a week at Foo Camp.

What is Foo Camp? Pat gives an entertaining summary.

A Weekend at Foo Camp

Well... I was lucky enough to get invited to Foo Camp (which was last weekend) and I figure "What the heck! Let's do it!".
Foo Camp is held at the O'Reilly headquarters in Sebastopol, CA which is 1 to 2 hours (or more depending on traffic) north of San Francisco. It is an invitation only event whose name stands for "Friends Of O'Reilly" and involves about 300 diverse and interesting individuals from different walks of the computer industry (and related industries). Tim O'Reilly host it at their headquarters and supplies very nice buffet food, showers, rest rooms, and meeting rooms. I was informed that the best way to enjoy the event is to camp there which means either pitching a tent on their lawn or finding an available space in a meeting room or hallway to throw a sleeping bag. It was an option to get a hotel room in town. Now... I haven't camped in about 20 years so this required some thinking...
I concluded I had two options:

Stay in a hotel room and ensure I remained sober enough to drive at the end of the evening, OR
Buy a tent at REI in Santa Rosa and then hit the high-end liquor store in Santa Rosa to buy enough whiskey to lubricate a serious subset of the 300 attendees.

Naturally, I chose the second option and bought six bottles (some of my favorites and others I hadn't tried).

It turns out James Hamilton and Pat Helland are friends and are putting their heads together on Energy Efficiency software. James is left, Pat is center, and Jesse Robbins is right.

But, back to the Energy Efficiency software. Here are Pat's comments.

Sometimes, it was hard to select between the various cool sessions.   I remember the following ones:

Data center power -- James Hamilton (my friend from Microsoft) and Jeff Hammerbacher who leads the Facebook data team (but will be leaving soon). Both James and Jeff were filled with information about running large and dynamic data centers.   The power issues for data centers have been on my mind the last few years and I find that James is a wonderful font of knowledge.   I most definitely love that he is at Microsoft and my friend... I plan to come pepper him with additional questions in the months to com.   Jeff, also, has tons of knowledge from supporting the data needs of Facebook as it has undergone its explosive growth. This was a fun and invigorating discussion in which I met an attendee, Roger Magoulas who is a research director at O'Reilly.   I have a feeling that there will be opportunities for me to work with Roger, too.
Parallel Programming -- Kerry Hammil of Microsoft Research.   We had a fun discussion of the difficulties of getting applications (and, indeed, their libraries and OSes) to be parallel. There were about 25 great and interesting people participating in this group and, not surprisingly, I participated, too.     This was such a lively discussion for me that it ended up in the hallway and we skipped the next session.

Here is an application architect who cares about data center power and parallel (multicore) programming. I hope we see more people like Pat show up.

Jesse Robbins in the above picture has an interesting background.

Technical Program Manager at Amazon.com
Manager - IT Operations at Amazon.com
Task Force Leader at World Shelters Task Force 1

Systems Engineer at Amazon.com
Firefighter/EMT (Intern) at Palo Alto Fire Department

Giving Cloud Storage a Try

July 15, 2008 Dave Ohara

If you have been thinking about giving Cloud Storage a try here is a blog post on Amazon.com’s Simple Storage Service.

Amazon S3 (Simple Storage Service) is an easy and inexpensive Internet hard-drive from Amazon Web Services (AWS) with absolutely no limits.

Now some people assume that Amazon S3 is a storage service meant primarily for web start-ups who store data in-the-cloud but that’s not correct because just about anyone (home users included) can benefit from S3.

For instance, you may backup your large music collection or even your entire computer hard-drive on S3. Similarly, bloggers can use Amazon S3 to store web images without worrying too much about their bandwidth bills.

If you never had a chance to explore Amazon S3 before, read the following guide that makes S3 simple even for non-geeks. It has all the information and tools you would need to quickly get started with Amazon S3.

The Basic Requirements:

To use Amazon S3 service, you’ll need an Amazon account (yes, it’s free; you pay only if you buy something from them) and an Amazon AWS S3 account.

Now go to AWS Access Key Identifiers and generate a unique Access Key ID + Secret Access Key pair that is required for using Amazon S3.

The post continues with a variety of tools you can use.

Questions for Amazon Web Services Evangelist

June 12, 2008 Dave Ohara

If you are interested in Amazon Web Services, here is video of a presentation by Mike Culver.

I have a meeting with Mike in a week to discuss Amazon Web Services. Mike and I used to work together, and we have been running into each other at the Pro Club for the past 2 years. We both left Microsoft 2 years ago, and Mike went straight to Amazon to be an evangelist for AWS.

I’ll write a blog post after my meeting with Mike. If you have any questions you would like me to ask, feel free to comment on this post, and I’ll see what I can do to get answers. Keep in mind, Mike is not in the data center operations group, but in the evangelism of web services.

One question I have for Mike are there any other tools besides Service Health Dashboard for AWS operations?

Two Different opinions, Why Amazon Went Down

June 7, 2008 Dave Ohara

You all know amazon.com suffered a 2 hour outage on Friday, suffering a 4.59% stock drop, lost sales, and damage to its reputation. I found 2 different opinions which give you some ideas on what happened.

GigaOm has a factual analysis looking for cause and effect.

So what happened? Let’s look at the facts.

Traffic to https://www.amazon.com was getting there. So DNS was configured properly to send traffic to Amazon’s data centers. Global server load balancing (GSLB) is the first line of defense when a data center goes off the air. Either GSLB didn’t detect that the main data center was down, or there was no spare to which it could send visitors.

When traffic hit the data center, the load balancer wasn’t redirecting it. This is the second line of defense, designed to catch visitors who weren’t sent elsewhere by GSLB.

If some of the servers died, the load balancer should have taken them out of rotation. Either it didn’t detect the error, or all the servers were out. This is the third line of defense.

Most companies have an “apology page” that the load balancer serves when all servers are down. This is the fourth line of defense, and it didn’t work either.

The HTTP 1.1 message users saw shows something that “speaks” HTTP was on the other end. So this probably wasn’t a router or firewall.

This sort of thing is usually caused by a misconfigured HTTP service on the load balancer. But that would happen late at night, be detected, and rolled back. It could also happen from a content delivery network (CDN) not retrieving the home page properly.

So my money’s on an AFE or CDN problem. But as Berman notes, Amazon’s store is a complex application and much of their infrastructure doesn’t follow “normal” data center design. So only time (and hopefully Amazon) will tell.

Site operators can learn from this: Look into GSLB, and make sure you have geographically distributed data centers (possibly through AWS Availability Zones.) It’s another sign we can’t take operations for granted, even in the cloud.

and WSJ blogger has a more entertaining version.

“The Amazon retail site was down for approximately 2 hours earlier today (beginning around 10:25) - and we’re bringing the site back up.

Amazon’s systems are very complex and on rare occasions, despite our best efforts, they may experience problems. We work to minimize any disruption and to get the site back as quickly as possible.

Amazon’s web services were not affected nor were our international sites.”

The statement doesn’t explain what went wrong, however. Here are some possibilities the Business Technology Blog has come up with and — in honor of tomorrow’s Belmont Stakes — our carefully-calculated odds that it’s what caused the problem.

* An explosion, fire or some other mishap at one of Amazon’s data centers: 5 to 1
* A faulty software upgrade: 7 to 1
* A so-called denial-of-service attack that tries to overwhelm the site with traffic: 10 to 1
* The site was broken into the by same guys who “RoXed Comcast”: 100 to 1
* A rush on Amazon’s Kindle e-book reader: 1,000 to 1
* Sharks with laser beams on their heads: 1,000,000 to 1

Most people will focus on what needs to happen to prevent this from happening again. Or you can accept the fact that no matter what you do, outages will happen, and it is part of an online business. So, the challenge is how you handle the customer relations of an outage. The smart companies address both issues with a good analysis of the long term effects. This coincidentally aligns with companies who think about green/sustainability action in the long term. The last thing you want to do in going green is be reactive.