Google and Yahoo set pace of Greener Data Center discussion, Om Malik moderates #greennet session

At Green:Net 2011 Om Malik hosted a discussion with Yahoo's Chris Page and Google's Bill Weihl on "Greener Data Center"

image

It was great to see that Om Malik chose the Green Data Center topic for GreenNet.  His choice for interviewing were Google and Yahoo.  Not Facebook.  Not Amazon.  Not Microsoft, who chose to speak about on how software, the web and cell phone are creating more efficient systems, but Microsoft was not on Om's panel.

Here are a few questions that I found were interesting to share with you.

Om Malik asks with Google's 100MW wind power purchase is this an indicator of Google's future site selection, making renewable energy as part of data center site?  Bill Weidl did  a good job of avoiding the questioning by making the point that he hopes more users  take into account energy composition in site selection.  The 100 MW wind power contract Google announced is 180 miles from Google's data center so the power doesn't go directly to a Google data center, but the wind power is on the same power grid as its data center.

Om says he has seen the media discuss the merits of building a big or small data center.  Is it better to build big or small?  Chris Page answers that Yahoo's approach is to build modular.  Bill answers Google builds what is most efficient.

One question I think Om asked for the audience is to have Chris and Bill decode PUE.  I hope you know the answer to this one.

Here is a live coverage page that you can go to see more.  http://gigaom.com/cleantech/greennet-2011-live-coverage/

oops is Amazon Web Services running too lean? Capacity issues are slowing down AWS recovery

News is spreading on AWS outage.

image

Major Amazon Outage Ripples Across Web

Amazon cloud outage derails Reddit, Quora

You can see the status on AWS http://status.aws.amazon.com/

Problems happen in every data center and systems are designed to recover.  But, it looks like AWS had a capacity shortage that is causing the problems to be slow in remedying.

A networking event early this morning triggered a large amount of re-mirroring of EBS volumes in US-EAST-1. This re-mirroring created a shortage of capacity in one of the US-EAST-1 Availability Zones, which impacted new EBS volume creation as well as the pace with which we could re-mirror and recover affected EBS volumes. Additionally, one of our internal control planes for EBS has become inundated such that it's difficult to create new EBS volumes and EBS backed instances. We are working as quickly as possible to add capacity to that one Availability Zone to speed up the re-mirroring, and working to restore the control plane issue. We're starting to see progress on these efforts, but are not there yet. We will continue to provide updates when we have them.

The thread started at 1:41a PT.

image

1:41 AM PDT We are currently investigating latency and error rates with EBS volumes and connectivity issues reaching EC2 instances in the US-EAST-1 region.

2:18 AM PDT We can confirm connectivity errors impacting EC2 instances and increased latencies impacting EBS volumes in multiple availability zones in the US-EAST-1 region. Increased error rates are affecting EBS CreateVolume API calls. We continue to work towards resolution.

2:49 AM PDT We are continuing to see connectivity errors impacting EC2 instances, increased latencies impacting EBS volumes in multiple availability zones in the US-EAST-1 region, and increased error rates affecting EBS CreateVolume API calls. We are also experiencing delayed launches for EBS backed EC2 instances in affected availability zones in the US-EAST-1 region. We continue to work towards resolution.

3:20 AM PDT Delayed EC2 instance launches and EBS API error rates are recovering. We're continuing to work towards full resolution.

...

 

Absorbing information at Green:Net 2011 - Greenpeace and Google make announcements

I walked in the door at 7:40a to Green:NET 2011.  The room was sparse, but within 2 minutes I saw a good friend who I didn't expect  to see. The last time we saw each other was Olivier Sanche's memorial service in Los Gatos.  We quickly had a great conversation on Greenpeace's announcement.

Greenpeace Unveils Who’s Behind the Internet’s Dirty Power

By Katie Fehrenbacher Apr. 21, 2011, 6:30am PT 1 Comment

A group of the Internet’s most recognized brands — from Facebook to Apple to Twitter to Amazon — have received failing grades when it comes to using clean power for their web services, according to a new report unveiled by Greenpeace on Thursday at our Green:Net 2011 event. Greenpeace found that while a few companies like Akamai, Google, Yahoo and IBM have been taking important steps towards clean power, overall, many web companies “are perpetuating our addiction to dirty energy technologies.”

The day kicked off with Om Malik and Katie Fehrenbacher

image

Google  Energy announced their 100MW Wind Power purchase.

Google to Buy Another 100MW of Wind Power via Google Energy

By Katie Fehrenbacher Apr. 21, 2011, 8:00am PT No Comments

Google isn’t just continuing to investhundreds of millions of dollars into clean energy projects, but it’s also continuing to commit to buying up the clean power itself. On Thursday morning Google announced that it has made its second deal via its subsidiary Google Energy, and the search engine giant plans to buy 100 MW of power from a wind farm that’s under construction in Oklahoma.

Is working at Apple the end of the career for a data center executive?

Sometimes I wonder if Olivier Sanche would still be alive if he had stayed at eBay instead of going to Apple.  If Olivier was still at eBay he would most likely be the VP that replaces Mazen Raswashdeh vacant position as eBay's VP of Technical Operations.  But, Olivier had a passion for Apple products including the dream to work there.  I spent 7 years working at Apple from 1985 - 1992 during the good years of early Macintosh Development, yet I never think should I have stayed at Apple.  I have friends who are still there with 25+ years, and some have left and gone back

What would have happened if I had stayed at Apple all these years.  Well, I would have hit 26 years at Apple this year and still be living in the SJ area.  I was raised in Saratoga, CA, went to schools in Cupertino and went to  UC Berkeley.  But, what kind of person would I be if I had spent 26 years at Apple?

Apple is a 35 year old company, but the number of data center executives who came from Apple is unknown.  Apple hired one of top data center executives with Olivier Sanche's arrival.  His replacement is not known publicly and is not Kevin Timmons.  Thanks to my blog post on Kevin leaving Microsoft for Apple., there is huge speculation Apple has big data center plans and cloud computing.

But, let me tell you another side of what happens after I posted.  Some of my data center friends cautioned me posting on Kevin leaving Microsoft going to Apple as "your post will make Kevin famous and known in the industry."  True.  But, I'll tell you one thing that happened immediately after the news spread.  Apple PR called Kevin to make sure that he doesn't talk to any media.  Not about Apple,  Not about Microsoft.  Not about anything data center related.  How do I know?  Did I talk to Kevin?  No.  But, this is exactly what happened when I blogged about Olivier Sanche leaving eBay going to Apple.  Olivier and I had no idea the news would be so big, but one of the first calls was from Apple's PR department to figure out who this data center guy was that media guys were calling about.  Apple PR is probably trying to figure out who the hell is this Dave Ohara and his green data center blog, telling the world about the data center executives we hire.

With Apple PR clamping down on data center executives like Kevin Timmons's, high visibility virtually disappears over time as there no other words shared.  With Amazon's Cloud Drive, new media services from Google around the corner, and the media watching for Apple's next step it is quite possible the Apple PR team is clamping down even harder on any data center news being shared.

Which gets back to my title question.

Is working at Apple the end of the career for a data center executive?

Working at Apple was the end of Olivier Sanche's career.

Can Apple's data center solution be put on course with the hiring of one data center executive, Kevin Timmons?

What is not totally clear is whether Apple can operate data centers to compete against Google, Microsoft,  and Amazon.

The media has their opinion.  But, would you run a data center based on what mass media thinks?  Think about that one for a while.

I trust my data center advice from the insiders who run data centers 7x24 365 days a year with decades of experience.

5 points amongst many others to think about if you buy from an ODM server vs. OEM

With Facebook sharing its server design and use of Quanta for its server supply.

Join Us

We worked with Alfa Tech, AMD, Delta, Intel,Power-One, Quanta and Synnex to develop of the first generation of technologies.

There are now bunches of IT purchasing departments looking to save money by buying direct and cutting out the middle man.  I’ve been laughing with my friends who have managed outsourced manufacturing projects on how naïve people are to how complex and difficult it is to buy direct from a manufacturer.  I got most of my hardware sourcing experience taking dozens and dozens trips to Japan, Korea, Taiwan, and China while I worked for Apple Computer, managing various peripheral projects for the Macintosh.  There is a huge reason why many companies have dual suppliers to manage price and quality, keeping the supplier constantly competing and knowing they will lose business if they can’t keep up.

To give you an idea of what few will hear about the Quanta Servers going to Facebook, check out this rumor of Quanta getting 20% margin on servers when their typical OEM server margin is 4%.

Order by the Facebook server Quanta: gross margin over 20%

April 18, 2011

Beijing the morning of April 18, according to informed sources, Taiwan’s contract manufacturers Quanta Facebook server to obtain orders, gross profit margin as high as 20% or more.

Reportedly, these servers will start shipping this fall, the order is Quanta gross margin is 5 times the daily orders, up to 20%.

Quanta declined to comment on.

Facebook server orders usually get HP and Dell. The Quanta favor, marking the influence of Taiwan’s contract manufacturers is growing.

At Apple we had teams who knew component cost, quality and performance issues.  They managed Bill of Materials (BOM) had HW engineering, Quality engineering, and purchasing expertise.  This is a big part of Apple’s IP that rarely gets out, and they’ve been working on outsourced manufacturing for decades.  Who would leave Apple’s HW team to go work on Amazon Kindle, HP, Barnes & Noble, etc.   I worked in the Apple purchasing group 25 years ago, and it was a blast in the Mac days. Smile

Even the Wal-mart purchasing staff don’t know how to do things at Apple’s level.  Have you ever seen a retail hit coming with a Wal-mart brand that was designed and developed by Wal-mart? 

Dell has attempted to educate some people with 5 points to consider in when choosing a server vendor.

Whitepaper: 5 points to consider when choosing a Server Vendor for Hyperscale Data Centers

Rate This

A whitepaper came out a little while ago from the management consulting firm, PRTM, that gives a perspective on the server industry.  The paper, which Dell was one of the contributors to, specifically focuses on something near and dear to our hearts, hyperscale data centers.

The paper, entitled Hyperscale Data Centers: Value of a Server Brand, talks about what organizations who are looking to build out these ginormous data centers should consider when selecting a system vendor.

But, there will be many who will dismiss this advice coming from a server OEM.

Want to know if I am right?  Watch the Server ODMs profit margins increase as many more people buy direct.

What most people don’t know is the ODMs are very clever in making it appear like they are supplying high quality just as good as the OEMs.  Cut out the middle man drop the price a bit.  But, there are many ways to cut costs which gives the ODM a 20% margin vs. 4% going through HP and Dell.  So, you think you have the game figured out and want a lower price.  And, the ODM looks for more ways to cut costs. 

Can you outsmart a huge team of Chinese engineers and business people with your one, two, maybe three people spec’ing a server?

Do you Trust your server supplier?

We’ve all seen disastrous product recalls on technology products.  Imagine 100s or 1,000s of servers being recalled.

For example, the unsexy capacitor has caused millions of dollars of damage.

Faulty capacitors have been discovered in motherboards as old as Socket 7 and have affected equipment manufactured up to at least 2007. The motherboard companies assembled and sold boards with faulty caps sourced from other manufacturers (see below). Major vendors such as Intel, Dell and HP were affected.[2] Circa 2005 Dell spent some US $150 million replacing motherboards entirely and another $150 million on the logistics of determining whether a system is in need of replacement. HP reportedly purged its product line in 2004. The motherboards and power supplies in Apple iMac G5s[3] and some eMacs[4] were also affected.

Welcome to the world of managing a ODM.  Late night regular phone calls to Asia, last minute war room meetings to address quality or supply issues.  Making the decision after weeks of phone calls, faxe’s, and e-mails, you need to get on a plane to China to resolve the issues.

Oh one other thing, at Apple it was worthwhile to do all this because the volumes were in 100,000s per month.  Only the server OEMs buy in this quantity.  As soon as you drop to 10,000 per month or less, you lose a lot of leverage with the ODM and you don’t have the benefit of large volumes to spread out your support costs.