How to build your own supercomputer, HP-style

Daily Newsletters

Sign up to ZDNet UK's daily newsletter.

NEWS
How to build your own supercomputer: take a few off-the-shelf, stripped-down PCs, add some network switches, a maze of Ethernet cabling and some home-grown Linux software, and you'll be well on your way. Hewlett-Packard, together with a national laboratory in France, tried this recipe out and, to the great surprise of many scientists, it worked. What they ended up with is the "I-Cluster", a Mandrake Linux-powered cluster of 225 e-pcs that has benchmarked its way into the list of the top 500 most powerful computers in the world. At a technical session last summer, scientists from HP's own labs in Grenoble started talking to experts at the local INRIA Rhone-Alps (France's National Institute for Research in Computer Science) about the possibility of doing "something a little unusual": building a supercomputer out of standard hardware components like those that might be found in the typical enterprise. They started with 100 of Hewlett-Packard's e-pcs -- a simplified PC with reduced expandability -- and finally worked up to the present configuration of 225 nodes, which is near the cluster's physical limit. HP and INRIA showed the system to journalists for the first time on Wednesday. The version of the e-pc used for I-Cluster is sealed, meaning no hardware tweaks could be made, and the experiment uses standard networking equipment as well. This means that, unlike with other clustered supercomputing projects, an organisation like a business could theoretically use the I-Cluster method to draw on idle computing power from around the company network to carry out computing-intensive tasks. "These are really standard machines, we didn't even open the box," said Bruno Richard, program manager with HP Labs Grenoble. Other clusters, like the US' ASCI Red at Sandia National Laboratories, are comprised of heavily modified parts. There were formidable obstacles to face in getting the cluster running as if it were one device, Richard said, such as distributing functions like storage and network caching to general-purpose devices, and managing and programming the cluster. "Our previous cluster was 12 machines," he said. "When you have 200 you have to rethink everything." For example, even making simple software changes became a difficult task with so many machines to be altered. In the end, however, the technicians devised tools capable of reinstalling every machine in the cluster from scratch in about 12 minutes, according to Richard. The researchers plan to release the tools they developed as open-source software for anyone who might feel the urge to build a supercomputer themselves. The whole project, minus network cabling, cost about 1.5m francs (£140,000). The individual machines that made up the I-Cluster are now out of date, each running on 733MHz Pentium III processors with 256MB of RAM and a 15GB hard drive. HP introduced a faster version at the beginning of this month and will launch a Pentium 4 e-pc by the end of the year. They are linked by fast Ethernet (100Gbit/s). Features like super-quiet cooling and low power consumption, originally designed for the corporate buyer, proved useful in the supercomputing environment too -- the cluster runs surprisingly quietly and doesn't require anything more than standard air conditioning to keep it cool. As ranked by standard benchmarks, I-Cluster is ranked 385th worldwide and 15th in France for supercomputing. Richard said the experiment showed that there is a linear relationship between the number of nodes and performance, meaning that it's relatively simple to add or remove computing power depending on the task. About 60 research teams worldwide are working on the system, with half running typical supercomputing tasks and the other half exploring how I-Cluster works. The project shows that standard computing power -- like the unused processing power on an office network -- can be harnessed for serious computing work. In the business world, CAD designers and chemists are among those who need intensive computing power, Richard said. "You could gather the latent power from office PCs using this technique," he said. "We eventually want to scale it higher, to thousands of PCs." Currently the hard limit for such a cluster is about 256 nodes, because of switching capacity, but that could be surpassed by linking several clusters that are physically near each other. A more daunting task might be taking the model to a consumer environment, which, Richard pointed out, is full of often dormant processors like those in printers and DVD players. HP imagines "clouds" of devices, or "virtual entities", which discover and use the resources around a user. Richard said that supercomputing power could come in handy for certain tasks, like converting large video files from one format to another, that currently take a good amount of patience. Other scientists predict that the practical difficulties of such a home network will prove difficult to solve. Brigitte Plateau, head of INRIA's APACHE parellel computing project, says that consumer need for such power probably wouldn't make it worth the effort that such a system would require. "It is more likely that you would see an external service," she said. HP's Richard said that the use of Linux -- Mandrake 7.0 in this case -- was important because low-level changes could be easily made to the software, and then the alterations could be shared freely with other scientists, something that would have required a special agreement with Microsoft if Windows had been used. Plateau, whose APACHE project encompasses I-Cluster, said the lab is also working with Microsoft to port parallel computing applications to Windows. "We had to face heterogeneity, by spreading it over Linux and Windows too," she said. "It's not scientific, but technically it's good experience." See the Linux Lounge for the latest GNU/Linux and open source headlines. See the Hardware News Section for full coverage. Have your say instantly, and see what others have said. Click on the TalkBack button and go to the Linux lounge forum Let the editors know what you think in the Mailroom. And read other letters.

Post your comment

In order to post a comment you need to be registered and logged in.

You can also log in with Facebook. Log in or create your ZDNet UK account below

  • Login

Will not be displayed with your comment

By signing up for this service, you indicate that you agree to our Terms and Conditions and have read and understood our Privacy Policy. Questions about membership? Find the answers in the Community FAQ

Get ZDNet UK's daily newsletter

Enter your email address to sign up

ZDNet UK Live

GrueMaster

Nice review and very informative. One thing I'd like to add (in reply to whs001's 1st question), the main reason to have the same interface from...

1 hour ago by GrueMaster on A tale of two distros: Ubuntu and Linux Mint
Frederick Wrigley

I'be been using Mint 12 since the RC came out, and I am far more happy with the Cinnamon, the Mate, and, yes (with extensions), theGnome 3...

2 hours ago by Frederick Wrigley via Facebook on A tale of two distros: Ubuntu and Linux Mint
bdantas

Excellent article. One small correction, though--although a fresh installation of Linux Mint 12 will, indeed, provide the user with a version of...

3 hours ago by bdantas on A tale of two distros: Ubuntu and Linux Mint
Alan Ralph

In related news, the ISPs club together to get the members of the Home Affairs Select Committee (ya goofed on that part, ZDNet UK) copies of "The...

3 hours ago by Alan Ralph via Facebook on MPs urge ISPs to take down terrorist material
Alan Ralph

In related news, the ISPs club together to get the members of the Home Affairs Select Committee (ya goofed on that part, ZDNet UK) copies of "The...

3 hours ago by Alan Ralph via Facebook on MPs urge ISPs to take down terrorist material
Moley

For Gnome 2 die-hards, it is possible to add icons to the bottom panel (or top top panel, if you prefer) which provide the exact Gnome 2...

4 hours ago by Moley on A tale of two distros: Ubuntu and Linux Mint
ramwellian

Your comments would seem pretty naive and immature. Your 'solution' appears to be, "gee, let's all just give in to the hackers and give them...

4 hours ago by ramwellian on Cloud computing security: no more oxymoron?
BugStalker

"Interesting thought ... If you installed Win7 as a dual boot on a machine that previously only had Linux, and it wrecked your Linux installation,...

5 hours ago by BugStalker on Windows 7 Declares War on GRUB
whs001

This is an excellent summary of Ubuntu and Mint and the interface differences between them. Most such articles take a very partisan position for...

5 hours ago by whs001 on A tale of two distros: Ubuntu and Linux Mint
Moley

@ewallace. Not so clear. Anyone can obtain the text, for example from here http://www.ustr.gov/webfm_send/2379. I support ACTA so long as it and...

5 hours ago by Moley on ACTA: Facts, misconceptions and questions
45283

I think WinRT is fantastic. I just wish it was an option for people that didn't want to go through Microsoft's App Store with its attendant...

8 hours ago by 45283 on Why Windows 8 needs architectural hygiene for WOA
Burn-IT

Nine people? £30m? Who's back pocket is that lot going in? And IF they say it is for new buildings, what about all the ones the government has...

9 hours ago by Burn-IT on Police set to launch three £30m e-crime hubs
ewallace

Just to be clear, nobody knows what is in the text of ACTA, here is a photograph of the text of ACTA http://twitpic.com/8h9iju as submitted to the...

9 hours ago by ewallace on ACTA: Facts, misconceptions and questions
fgvrg56

Unfortunately main issue is that ASUS is refusing to accept that they make some mistake on this version of asus Transformer prime. 1 - GPS sensor...

10 hours ago by fgvrg56 on Asus Eee Pad Transformer Prime Wi-Fi & GPS problems?
Ben Woods

@Marcus A fair question. Just talked with Archos which said it was working on an announcement for next week....

11 hours ago by Ben Woods on Archos confirms G9 Ice Cream Sandwich update schedule
Marcus Karlsson

Any update on this, considering the claimed "first week of February"?

13 hours ago by Marcus Karlsson via Facebook on Archos confirms G9 Ice Cream Sandwich update schedule
apexwm

Bill Goodrich : Just as al_langevin pointed out, with Windows Server 2008 there is no Services for Macintosh anymore. It's gone, not available....

21 hours ago by apexwm on Windows Server 2008 drops the ball for Mac compatibility
txtrainguy

Replying to an old topic that I'm currently facing with my CEO (who is on a Mac). Our servers are primarily Windows Servers, office is about...

1 day ago by txtrainguy on Windows Server 2008 drops the ball for Mac compatibility
k0tcs3

Sure, that makes perfect sense. Pay wrong-doers money and thank them for breaching your security and pointing out your flaws, that would surely...

1 day ago by k0tcs3 on US indicts Romanian over NASA climate change hack
Random_Error

I think he's referring specifically to Android apps, as Apple do regulate their App Store, but Google seem to let any old crap onto the Android store!

1 day ago by Random_Error on RIM: BlackBerry will keep 'garbage' apps out of store

Latest in Application Development