Linux clusters scale better and cost less

ANALYSIS

When Jehangir Athwal and his colleagues wanted a system to do the heavy lifting at their bioscience software company, they were interested in a supercomputer system. But sticker price shock turned the IT professionals in a different direction -- to Linux clustering. The bill for the traditional supercomputer would have hit seven figures, while a Linux Beowulf cluster would cost about $80,000. That's the primary reason Athwal's company, Lion Bioscience, is running a 44-node Beowulf cluster, powered by AMD Athlon Thunderbird processors, set up by Scyld Computing Corp. Athwal, a computational scientist at the German company's San Diego division, said a supercomputer offered more processing power, and its high-profile vendor promised a generous support plan. But those benefits still couldn't get the Lion team beyond the price tag. Besides, Lion didn't need more processing power than the 44 nodes. If Lion needs more parallel processing power to run bioscience software models and predictions, adding nodes is easy with a cluster setup; it's just a matter of a few more commodity PCs. With the supercomputer solution, the company would have to buy another supercomputer. According to Linux vendors and industry analysts, cost-effectiveness is just one of several reasons prompting more enterprises to choose Linux systems. Ease of scalability factor The ease of scalability is another reason pushing Linux clustering beyond pure research institutions and into the business sector, especially in the "scalable performance cluster" category (as opposed to high-availability clustering and remote-access clustering). In addition to companies like Lion, petroleum exploration companies are embracing Linux clusters. Financial institutions are also using Linux clusters as small as eight nodes to run complex portfolio simulations and risk analysis, according to hardware and software vendors that include Dell and Sistina (a provider of file system and volume management software). "Organisations having custom, computationally intensive applications are increasingly considering Linux configurations," explained Dan Kusnetzky, VP of system software for tech analyst firm IDC. "We're also seeing quite a bit of interest in low-cost Linux configurations for point-of-sale or point-of-solution systems that are replicated in many distributed sites." Of course, Linux isn't the only player in the scalable performance clustering space, but Scyld CTO Donald Becker believes Linux makes good sense, especially when it comes to large clusters. Linux lets enterprises buy (or download) once and install everywhere, due to the open source aspect, instead of buying 2,000 licenses of Windows or proprietary UNIX. Becker, a cofounder of the Beowulf project at NASA in 1994, said Beowulf clusters have "much better price performance than traditional supercomputers" and they offer "as-needed" scalability. Advantages in addition to price include:

Linux clusters run on commodity platforms.
They offer long-term viability because there's no hardware or software vendor lock-in.
They're tied to PC architecture, which has the fastest performance growth rate of any computing architecture.

Ongoing improvements will strengthen choice Becker said Linux clustering initially scared potential users -- they feared installing an operating system on dozens or hundreds of machines, and maintaining a huge number of PCs. But system management tools now available, featuring GUI or Web-based interfaces, allow users to "install once, execute everywhere" and monitor the whole system from one machine. Kusnetzky noted a lack of a Windows and various Unix versions. His definition of "single system image" includes:

Tools allowing administrators to see the whole multisystem configuration as a single domain.
Tools allowing developers to create applications that work in a clustered environment without rewriting code.
Tools allowing nonparallel applications to make some use of the multisystem configuration.
Tools allowing users to queue batch and print jobs without having to know where the printing or computing happens.

Although Linux is catching up, Kusnetzky also explained that some needed administration, operations, and help desk tools, and some packaged applications, aren't yet available. "IDC expects all of these issues to be addressed by suppliers and by the open source community over the next couple of years," he said. The analyst also noted that IBM, HP, Compaq, and Dell have all announced sales of large, Linux-clustered configurations to compete with the supercomputing market. This will also help spur the needed tools currently missing. While the price tag is likely to be the big factor organisations consider in choosing Linux clusters, Kusnetzky warned that the lack of tools is just as important. "Linux is perceived to cost less and run on less costly hardware configurations. But organisations need to understand that the true cost of ownership includes costs of administration, operations, help desk, development, and other categories where staff-related costs are a factor," he said, adding that those costs are why enterprises choose the more mature administration and operations software available on Unix and Windows. While Lion's Athwal is managing the new cluster on his own, he agrees somewhat with the analyst's statements. He recommends getting the best available cluster management tools instead of scrimping in that area. "Definitely go with the bells and whistles," Athwal said. "If a node goes down, you don't want to spend all day looking for it." Athwal also suggested that companies consider having a vendor handle the installation. "When we received the cluster from the vendor, I turned it on, and it started working."

Have your say instantly, in the Tech Update forum. For a weekly round-up of the enterprise IT news, sign up for the Tech Update newsletter. Find out what's where in the new Tech Update with our Guided Tour. Tell us what you think in the Mailroom.

Post your comment

In order to post a comment you need to be registered and logged in.

You can also log in with Facebook. Log in or create your ZDNet UK account below

By signing up for this service, you indicate that you agree to our Terms and Conditions and have read and understood our Privacy Policy. Questions about membership? Find the answers in the Community FAQ

ZDNet UK Live

'Cut Price' Data Roaming? The price has been cut, but it is certainly not 'cut price' in the sense of the phrase, and nowhere near local EU data...

22 minutes ago by SoapyTablet on Cut-price data roaming gets all-clear for July

BrownieBoy: "Such crashes are normally down to the OS and/or a rogue application, which could be fixed by re-imaging. Everybody knows how Windows...

1 hour ago by apexwm on Using Windows XP is a waste of money, says IDC

I've just started using it too, and like you I feel that Fedora is a fine GNOME 3 showcase distribution. I am torn, though, because I checked out...

1 hour ago by Thomas Gellhaus via Facebook on Fedora 17 - The "Beefy Miracle" Arrives

Why is it that Newzealand has the guts to stand up to the USA and ask to see the evidence and we don't!

1 hour ago by pjc158 on Judge orders US to share MegaUpload evidence

What a farce! Hopefully the European court will see where this is leading.

3 hours ago by Dean Talboys via Facebook on Assange loses extradition battle in Supreme Court

Wouldn't surprise me if Samsung actually really had problems producing the white model (as Apple did - it would make more sense) and this non-story...

4 hours ago by SoapyTablet on Samsung Galaxy 'S3' delayed by special paint

those conformation letters are hard to figure out what is which letters it is a pain in the back side. Please make it more Ledge-able being better...

8 hours ago by Lonnie on Screenshots: Photoshop CS6 Beta

"cites" even. Ouch!

15 hours ago by BrownieBoy on Using Windows XP is a waste of money, says IDC

...... and PDP11s too

15 hours ago by Horace Ontalhold on Fusion-io lays minefield with a billion IOPS

I had a quick skim through the PDF. It seems to be that many of these so-called cost savings would be down to a hardware refresh. Although...

16 hours ago by BrownieBoy on Using Windows XP is a waste of money, says IDC

496,999 BT Fon Hotspots lovingly situated in your next door neighbours garden, no matter how you dress that up its still a pup... Not where I need...

18 hours ago by bobandroid on London Olympics: BT needs 25,000 more Wi-Fi hotspots

Jack : I was hoping you could provide us a summary since you are familiar with this report. I am not yet sure how much of my time I'd like to...