Google Ads

Enter your email address:

Delivered by FeedBurner

This form does not yet contain any fields.

    Architecture of Internet Datacenters

    How many of you would like to attend a course on Architecture of Internet Data Centers?  This course is part of RAD Lab who wrote the Adove the Clouds paper.


    Well in Fall of 2007 UC Berkeley (my alma mater) had the following course for graduate students.

    CS 294-14: Architecture of Internet Datacenters (RADLab Research Seminar 2.0)

    Instructor: Randy H. Katz
    Time: MW 2:30-4:00 PM
    Place: 310 Soda
    Units: 3 (2-4, but you had better sign up for 3!)

    Course Description

    Internet Datacenters have recently emerged as a significant new computing platform, designed to provide high capacity processing for large numbers of web clients. Major web properties like Google have designed their own building-scale computer facilities, integrating processing, storage, internal and external networking, along with integral power and cooling infrastructures. The resulting datacenters typically deploy 100,000 to 1,000,000 computers within a single facility.

    In this research seminar, we will read and discuss the very recent literature on the design and implementation of processor clusters, virtual machines, virtual storage, and datacenter networking organization. Architectural approaches to deal with failures, effective sharing of processing/storage/network resources, and efficient management of power across the systems stack will be considered. Some class meetings will be dedicated to meeting with and discussing issues with industrial leaders from Google, IBM, Cisco, and Network Appliances.

    Here are the first two weeks.

    Week 1: Course Organization, Overview, and Technology Trends

    • Monday, August 27
      1. [Randy] Randy H. Katz, “Internet-scale Computing: The Berkeley RADLab Perspective,” IWQoS 2007, Evanston, IL, (June 2007). [pdf]
      2. [Randy] Stephen Alan Herrod, VMWare, “The Future of Virtualization Technology,” ISCA 2006. [pdf]
    • Wednesday, August 29
      1. [Randy] Raj Yavatkar, Intel, “Platforms Design Challenges with Many Cores,” HPCA-12, 2006. [pdf]
      2. [Randy] Renato Recio, IBM, “System IO Network Evolution: Closing the Requirement Gaps,” HPCA-12, 2006. [pdf]
      3. [Randy] Steve Kleiman, NetApp, “Trends in Managing Data at the Petabyte Scale,” FAST 2007, San Jose, CA, (February 2007). [pdf]
    Week 2: Applications Software Infrastructure
    • Monday, September 3: Labor Day Holiday
    • Wednesday, September 5
      1. [Matei] S. Ghemawat, H. Gobioff, S.-T. Leung, “The Google File System,” Proc. SOSP’03, 2003. [pdf] [Notes].
      2. [Kuang] J. Dean, S. Ghemawat, “Mapreduce: Simplified Data Processing on Large Clusters,” Proc. OSDI’04, pages 137 – 150, (December 2004). [pdf] [Notes].
      3. [Michael] F. Chang, J. Dean, S. Ghemawat, W. C. Hsieh, D. A. Wallach, M. Burrows, T. Chandra, A. Fikes, R. E. Gruber, “Bigtable: A Distributed Storage System for Structured Data,” Proc. OSDI'06, 2006. [pdf]

      1. [Randy] Intel and Sun White Papers on Multicore Architectures [Notes]
        • Intel, "Intel Multi-Core Processors: Making the Move to Quad-Core and Beyond." [pdf]
        • Intel, "Inside Intel Core Microarchitecture: Setting New Standards for Energy-Efficient Performance." pdf
        • Intel, "Preparing for Peta-scale." [pdf]
        • Harlan McGhan, "Niagara 2 Opens the Flood Gates," Microprocessor, 11/6/2006. [pdf]
      2. [Ari] L. A. Barroso, J. Dean, U. Holzle, “Web Search for a Planet: The Google Cluster Architecture,” IEEE Micro, 23(2):22–28, March/April 2003. [pdf] [Notes]
      3. [Henry] L. A. Barroso, “The Price of Performance: An Economic Case for Chip Multiprocessing," ACM Queue, 3(7), September 2005. [html] [pdf].

    What I found interesting was the student project proposals.

    Student Project Proposal Presentations

    Click to read more ...


    Site Selection Data Center Report for Columbia, MO

    I found this site selection report, Data Center Analysis & Evaluation for Columbia Missouri By Angelou Economics from Oct 2007.  This report is a reverse site analysis comparing Columbia region to other areas in the US.


    As part of the analysis they used Quincy, WA as a case study for the economics development.


    For competitive analysis, they compared, Quincy, WA – Lenoir, NC – Pryor, OK – Goose Creek, SC.


    The results of the evaluation were.


    The following issues raised in 2007 were raised 2 years ago.


    Giving timing for the Columbia Regional Economic Development group to respond.

    A local development group has acquired over 200 acres of land located adjacent to the convergence of multiple transmission lines, electrical substation, and a gas peaking plant. The Ewing Property offers redundancy of electric and broadband, along with looped water supply. The development group submitted the site as the first application to the Missouri Department of Economic Development’s Certified Site Program, and the site has met all of the requirements for certification. Access information on Missouri's First Certified Site, Columbia's Ewing Industrial Park.

    Columbia also has other positive attributes for data centers. The University of Missouri College of Engineering Computer Science and Electrical and Computer Engineering can ensure a qualified workforce. The National Security Administration recently named the University of Missouri as a Center for Academic Excellence in Information Assurance Education. The County of Boone has also developed an incentive package for qualifying data center projects based on the Chapter 100 Revenue Bond Program to encourage new data center investment in Columbia.

    Click to read more ...


    Sewage treatment plants using methane for fuel cell power generation

    Note:  One lesson from the later part of this post is plan on a hybrid of natural gas and digester gas to run the fuel cell.  Don’t think you will run only on the sewage produced methane.  And the manufacturer, fuel cell says this too.

    In many applications digester gas production volume is variable. In such applications, the plant can be designed to operate with automatic blending with natural gas.  Over 30% of FuelCell Energy's Direct FuelCell® (DFC®) installations operate on renewable biogas. has article on sewage (waste) treatment plants as a source of energy.

    Renewable energy

    The seat of power

    Dec 30th 2009
    From The Economist print edition

    Better sewage treatment is the latest thing in clean energy

    Illustration by David Simonds

    WHERE there’s muck, there’s brass—or so the old saying has it. The cynical may suggest this refers to the question of who gets what, but thoughtful readers may be forgiven for wondering, while they are recovering from the excesses of Christmas in the smallest room in the house, what exactly happens when they flush the toilet.

    The answer is encouraging. Less and less waste, these days, is actually allowed to go to waste. Instead, it is used to generate biogas, a methane-rich mixture that can be employed for heating and for the generation of electricity. Moreover, in an age concerned with the efficient use of energy, technological improvements are squeezing human fecal matter to release every last drop of the stuff. Making biogas means doing artificially to faeces what would happen to them naturally if they were simply dumped into the environment or allowed to degrade in the open air at a traditional sewage farm—namely, arranging for them to be chewed up by bacteria. Capturing the resulting methane has a double benefit. As well as yielding energy, it also prevents what is a potent greenhouse gas from being released into the atmosphere. had an article in 2004 on fuel cells and methane for power generation.

    Poop power? Sewage turned into electricity

    Fuel cells and waste sludge mix to power treatment plant

    Miguel Llanos


    updated 9:31 a.m. PT, Mon., July 19, 2004

    RENTON, Wash. - It's not as neat as spinning straw into gold, but what Greg Bush gets to do in the world of sewage treatment is pretty magical: making electricity from what's flushed down the sewer. And he does it using fuel cells, technology that's cleaner and more efficient than traditional power generation.

    How it works
    The largest project of its type in the world, the process goes like this: Biodegradable solid waste is sent to large tanks, called digesters, that provide a home for three to four weeks. There bacteria eat away at the waste, releasing methane gas and further reducing the amount of solid waste.


    James Cheng /

    Four large digester tanks sit behind the fuel cell power plant at the wastewater treatment plant in Renton, Wash.




    The fuel cells mentioned by MSBBC in King County are no longer in use.  But, here is the executive summary of the results, published in Apr 2009.

    Final Report, King County Fuel Cell Demonstration Project

    Issued April 2009

    Final Report CoverExecutive Summary

    Increasing energy costs, more stringent air emission regulations, and an interest in exploring emerging energy technologies prompted King County, Washington, to search for new and innovative ways to provide electricity for its wastewater treatment plants. In June 2004, the county began a two-year demonstration of a fuel cell power plant to be fueled by gas produced through anaerobic digestion of solids produced at its South Treatment Plant. The project was the first application in the country to use digester gas to fuel a molten carbonate fuel cell.

    King County’s fuel cell power plant was sized to produce 1 megawatt (MW) of electricity and was designed to capture waste heat from the fuel cell exhaust and return it to the treatment plant. Two project goals were established:

    • Demonstrate that the molten carbonate fuel cell technology can be adapted to use anaerobic digester gas as a fuel source.
    • Achieve a nominal plant power output target of 1 MW using either digester gas or natural gas.

    Both goals were achieved during the two‑year demonstration period. A number of secondary objectives (performance goals) were also met.

    The final pdf is here.

    Click to read more ...


    Make Sense, Be Positive, Help Out – social reporters has a post on his 2010 resolution. I liked his points as it remind what I am trying to do with

    Make Sense by using social media to capture content at events and elsewhere; listen out for the conversations taking place; highlight the stories that you hear; interpret for different interests; comment to add your own ideas, and aggregate to make it easier for people to follow what’s happening in many different places.
    Be positive so there’s more chance of good things emerging from your reporting: make friends, applaud other people’s successes, celebrate together, and spot opportunities (while not ignoring the problems).
    Help out and promote collaboration (rather than highlighting conflict) by encouraging, supporting, and signposting people to other resources.

    Two other good things to think about are referenced.

    Jeff Jarvis, suggesting journalists must see themselves as more than storytellers, could be talking about social reporters:

    When we open ourselves up, we can think of journalists as enablers, as community organizers (not just of information but of a community’s ability to organize its own information), as teachers, as curators (how could I get through this without using the word at least once?), as filters, as tool makers, as algorithm writers.

    For those in social media, Scott Gould also says that this year we must make sense or die:

    There’s too much content, both online and offline, for everyone to cohabit – meaning those that lack clarity will, by the end of 2010, die. Furthermore those who aren’t making sense probably don’t have much money left to continue not making any sense, so unless they start making sense, they too will die.

    Click to read more ...


    2010 a year of change?

    Happy New Year!

    As we start the New Year there are many who look at 2009 as a year to forget.  StorageMojo has his top 2009 stories post, but what I like best is his last points.

    The StorageMojo take
    Like a termite-riddled barn after a heavy snow, the Great Recession is seeing old models collapse. We can’t afford to keep doing what we’ve been doing.

    As the new models emerge, competition will grow in the hot areas, leading to even more innovation in the next 3 years than we’ve seen in the last 5. More on that in a future post.

    I agree 2010 will have even more innovation.

    So far most of the green data center efforts have held on to old models and tried to make them greener.  Now it is time to cast off the old models and look for new models to be green when resource consumption (carbon impact and water) is a top priority.

    As the year develops I’ll be making announcements of changes GreenM3 is making in what I write about.  2010 is going to be an exciting year.

    Click to read more ...