ZDNet UK / News and Analysis / Infrastructure / Servers

Linux clusters scale better and cost less

By Grant Gross, TechRepublic, 4 November, 2002 12:43

Daily Newsletters

Sign up to ZDNet UK's daily newsletter.

Topics

Clustering, Linux, Beowulf, Supercomputer

ANALYSIS
When Jehangir Athwal and his colleagues wanted a system to do the heavy lifting at their bioscience software company, they were interested in a supercomputer system. But sticker price shock turned the IT professionals in a different direction -- to Linux clustering. The bill for the traditional supercomputer would have hit seven figures, while a Linux Beowulf cluster would cost about $80,000. That's the primary reason Athwal's company, Lion Bioscience, is running a 44-node Beowulf cluster, powered by AMD Athlon Thunderbird processors, set up by Scyld Computing Corp. Athwal, a computational scientist at the German company's San Diego division, said a supercomputer offered more processing power, and its high-profile vendor promised a generous support plan. But those benefits still couldn't get the Lion team beyond the price tag. Besides, Lion didn't need more processing power than the 44 nodes. If Lion needs more parallel processing power to run bioscience software models and predictions, adding nodes is easy with a cluster setup; it's just a matter of a few more commodity PCs. With the supercomputer solution, the company would have to buy another supercomputer. According to Linux vendors and industry analysts, cost-effectiveness is just one of several reasons prompting more enterprises to choose Linux systems. Ease of scalability factor The ease of scalability is another reason pushing Linux clustering beyond pure research institutions and into the business sector, especially in the "scalable performance cluster" category (as opposed to high-availability clustering and remote-access clustering). In addition to companies like Lion, petroleum exploration companies are embracing Linux clusters. Financial institutions are also using Linux clusters as small as eight nodes to run complex portfolio simulations and risk analysis, according to hardware and software vendors that include Dell and Sistina (a provider of file system and volume management software). "Organisations having custom, computationally intensive applications are increasingly considering Linux configurations," explained Dan Kusnetzky, VP of system software for tech analyst firm IDC. "We're also seeing quite a bit of interest in low-cost Linux configurations for point-of-sale or point-of-solution systems that are replicated in many distributed sites." Of course, Linux isn't the only player in the scalable performance clustering space, but Scyld CTO Donald Becker believes Linux makes good sense, especially when it comes to large clusters. Linux lets enterprises buy (or download) once and install everywhere, due to the open source aspect, instead of buying 2,000 licenses of Windows or proprietary UNIX. Becker, a cofounder of the Beowulf project at NASA in 1994, said Beowulf clusters have "much better price performance than traditional supercomputers" and they offer "as-needed" scalability. Advantages in addition to price include:
  • Linux clusters run on commodity platforms.
  • They offer long-term viability because there's no hardware or software vendor lock-in.
  • They're tied to PC architecture, which has the fastest performance growth rate of any computing architecture.
Ongoing improvements will strengthen choice Becker said Linux clustering initially scared potential users -- they feared installing an operating system on dozens or hundreds of machines, and maintaining a huge number of PCs. But system management tools now available, featuring GUI or Web-based interfaces, allow users to "install once, execute everywhere" and monitor the whole system from one machine. Kusnetzky noted a lack of a Windows and various Unix versions. His definition of "single system image" includes:
  • Tools allowing administrators to see the whole multisystem configuration as a single domain.
  • Tools allowing developers to create applications that work in a clustered environment without rewriting code.
  • Tools allowing nonparallel applications to make some use of the multisystem configuration.
  • Tools allowing users to queue batch and print jobs without having to know where the printing or computing happens.
Although Linux is catching up, Kusnetzky also explained that some needed administration, operations, and help desk tools, and some packaged applications, aren't yet available. "IDC expects all of these issues to be addressed by suppliers and by the open source community over the next couple of years," he said. The analyst also noted that IBM, HP, Compaq, and Dell have all announced sales of large, Linux-clustered configurations to compete with the supercomputing market. This will also help spur the needed tools currently missing. While the price tag is likely to be the big factor organisations consider in choosing Linux clusters, Kusnetzky warned that the lack of tools is just as important. "Linux is perceived to cost less and run on less costly hardware configurations. But organisations need to understand that the true cost of ownership includes costs of administration, operations, help desk, development, and other categories where staff-related costs are a factor," he said, adding that those costs are why enterprises choose the more mature administration and operations software available on Unix and Windows. While Lion's Athwal is managing the new cluster on his own, he agrees somewhat with the analyst's statements. He recommends getting the best available cluster management tools instead of scrimping in that area. "Definitely go with the bells and whistles," Athwal said. "If a node goes down, you don't want to spend all day looking for it." Athwal also suggested that companies consider having a vendor handle the installation. "When we received the cluster from the vendor, I turned it on, and it started working."
Have your say instantly, in the Tech Update forum. For a weekly round-up of the enterprise IT news, sign up for the Tech Update newsletter. Find out what's where in the new Tech Update with our Guided Tour. Tell us what you think in the Mailroom.

Related stories

Watch for top Linux/Unix threats

Improve Linux security

SeaMicro swaps out Atoms for Xeons in microserver

Post your comment

In order to post a comment you need to be registered and logged in.

You can also log in with Facebook. Log in or create your ZDNet UK account below

  • Login

Will not be displayed with your comment

By signing up for this service, you indicate that you agree to our Terms and Conditions and have read and understood our Privacy Policy. Questions about membership? Find the answers in the Community FAQ

Get ZDNet UK's daily newsletter

Enter your email address to sign up

ZDNet UK Live

Moley

@GrueMaster. I prefer horses for courses rather than one size fits all. I, and I suspect most other computer users, do not really wish to have...

30 minutes ago by Moley on A tale of two distros: Ubuntu and Linux Mint
greycynic

The product that scares me every time I have to use it is the Office 2007 version of Excel. The first bug that I found was applying the median...

33 minutes ago by greycynic on Ten flawed products that derail productivity
GrueMaster

Nice review and very informative. One thing I'd like to add (in reply to whs001's 1st question), the main reason to have the same interface from...

2 hours ago by GrueMaster on A tale of two distros: Ubuntu and Linux Mint
Frederick Wrigley

I'be been using Mint 12 since the RC came out, and I am far more happy with the Cinnamon, the Mate, and, yes (with extensions), theGnome 3...

3 hours ago by Frederick Wrigley via Facebook on A tale of two distros: Ubuntu and Linux Mint
bdantas

Excellent article. One small correction, though--although a fresh installation of Linux Mint 12 will, indeed, provide the user with a version of...

3 hours ago by bdantas on A tale of two distros: Ubuntu and Linux Mint
Alan Ralph

In related news, the ISPs club together to get the members of the Home Affairs Select Committee (ya goofed on that part, ZDNet UK) copies of "The...

4 hours ago by Alan Ralph via Facebook on MPs urge ISPs to take down terrorist material
Alan Ralph

In related news, the ISPs club together to get the members of the Home Affairs Select Committee (ya goofed on that part, ZDNet UK) copies of "The...

4 hours ago by Alan Ralph via Facebook on MPs urge ISPs to take down terrorist material
Moley

For Gnome 2 die-hards, it is possible to add icons to the bottom panel (or top top panel, if you prefer) which provide the exact Gnome 2...

5 hours ago by Moley on A tale of two distros: Ubuntu and Linux Mint
ramwellian

Your comments would seem pretty naive and immature. Your 'solution' appears to be, "gee, let's all just give in to the hackers and give them...

5 hours ago by ramwellian on Cloud computing security: no more oxymoron?
BugStalker

"Interesting thought ... If you installed Win7 as a dual boot on a machine that previously only had Linux, and it wrecked your Linux installation,...

5 hours ago by BugStalker on Windows 7 Declares War on GRUB
whs001

This is an excellent summary of Ubuntu and Mint and the interface differences between them. Most such articles take a very partisan position for...

5 hours ago by whs001 on A tale of two distros: Ubuntu and Linux Mint
Moley

@ewallace. Not so clear. Anyone can obtain the text, for example from here http://www.ustr.gov/webfm_send/2379. I support ACTA so long as it and...

6 hours ago by Moley on ACTA: Facts, misconceptions and questions
45283

I think WinRT is fantastic. I just wish it was an option for people that didn't want to go through Microsoft's App Store with its attendant...

9 hours ago by 45283 on Why Windows 8 needs architectural hygiene for WOA
Burn-IT

Nine people? £30m? Who's back pocket is that lot going in? And IF they say it is for new buildings, what about all the ones the government has...

10 hours ago by Burn-IT on Police set to launch three £30m e-crime hubs
ewallace

Just to be clear, nobody knows what is in the text of ACTA, here is a photograph of the text of ACTA http://twitpic.com/8h9iju as submitted to the...

10 hours ago by ewallace on ACTA: Facts, misconceptions and questions
fgvrg56

Unfortunately main issue is that ASUS is refusing to accept that they make some mistake on this version of asus Transformer prime. 1 - GPS sensor...

11 hours ago by fgvrg56 on Asus Eee Pad Transformer Prime Wi-Fi & GPS problems?
Ben Woods

@Marcus A fair question. Just talked with Archos which said it was working on an announcement for next week....

12 hours ago by Ben Woods on Archos confirms G9 Ice Cream Sandwich update schedule
Marcus Karlsson

Any update on this, considering the claimed "first week of February"?

13 hours ago by Marcus Karlsson via Facebook on Archos confirms G9 Ice Cream Sandwich update schedule
apexwm

Bill Goodrich : Just as al_langevin pointed out, with Windows Server 2008 there is no Services for Macintosh anymore. It's gone, not available....

22 hours ago by apexwm on Windows Server 2008 drops the ball for Mac compatibility
txtrainguy

Replying to an old topic that I'm currently facing with my CEO (who is on a Mac). Our servers are primarily Windows Servers, office is about...

1 day ago by txtrainguy on Windows Server 2008 drops the ball for Mac compatibility

Latest in Servers

SeaMicro swaps out Atoms for Xeons in microserver

Featured white papers

How cloud adds strategy to IT outsourcing

SAVVIS

How cloud adds strategy to IT outsourcing

Thanks to cloud computing, IT outsourcing is back, but this time it can help both in cutting costs and helping the business... Read more

How enterprises can achieve cloud computing benefits

Citrix

How enterprises can achieve cloud computing benefits

Is your load balancer cloud-ready? A key to the cloud's success is having an IT infrastructure that can deliver a robust set of cloud-oriented capabilities and can support all cloud computing use... Read more

I/O holds up the traffic in virtual systems

Intel

I/O holds up the traffic in virtual systems

We can now host 512 virtual machines on a single physical server. That's a lot of virtual machines trying to squeeze... Read more

Inside ZDNet UK

A tale of two distros: Ubuntu and Linux Mint

A tale of two distros: Ubuntu and Linux Mint

2012 Predictions: VCE FastPath, VI SAN Probe & Hadoop

2012 Predictions: VCE FastPath, VI SAN Probe & Hadoop

BlackBerry Bold 9790

BlackBerry Bold 9790

Case study: How cloud enables growth

Case study: How cloud enables growth

Ten factors that make Ubuntu 11.10 a hit

Ten factors that make Ubuntu 11.10 a hit

Toshiba Portégé Z830-10P Review

Toshiba Portégé Z830-10P Review

BT's archive: Pictures from the vault

BT's archive: Pictures from the vault