Microsoft wants Windows on supercomputers

As first reported by ZDNet UK sister site CNET News.com, the Windows Server 2003 HPC Edition will include features for running windows on clusters of machines interconnected by a high-speed network to form a single computing resource, Microsoft said in a statement on Wednesday.

In the statement, Microsoft said it has enlisted support for the new version from several major companies, including IBM, Dell, Hewlett-Packard, Advanced Micro Devices and Intel. Other partners include Verari Systems, one of whose specialties is cluster computing, and the Cornell Theory Centre, which has explored Windows for supercomputing for years.

Clusters are hitting their stride as a mainstream element in supercomputing, but they're not well-adapted for some tasks, such as decryption. On the newest list of the top 500 supercomputers released this week, 291 were identified as clusters, the list organisers said.

And Linux is popular on clusters, too. No. 2 system "Thunder," California Digital's 4,096-processor machine built for the Lawrence Livermore National Laboratory, runs Linux, as do the two new Blue Gene/L prototypes from IBM that are in the top 10 today.

Aiming for the mainstream
As the market expands to mainstream users, Microsoft believes that it can get a foothold from technical experts with the ability and interest in the nitty-gritty details of clusters, said Dennis Oldroyd, director of the Windows Server Product Management Group.

"We see the market transitioning out of academic and government (areas) and into the enterprise," Oldroyd said. "As that move happens, you'll find that people need to have a familiar interface. They're not interested in tuning it and tweaking it. They want to get their work done."

But Microsoft has been "hot and cold, hot and cold, hot and cold" about high-performance computing and faces challenges in the market, said Nan Boden, executive vice president and co-founder of Myricom, which sells high-end networking gear widely used to build supercomputing clusters.

"From what we've seen in performance of Windows, they just haven't been competitive with Linux. There's too much overhead," Boden said. She said Myricom has products to serve customers who want to run Windows on a high-performance cluster. "But we really haven't seen a tremendous demand for it."

Microsoft knows what it's up against. "Certainly, the technology has grown up around Linux, but we see demand from customers for Windows," Oldroyd said. "We think we will have something unique to offer in the marketplace."

There will be room for Microsoft in the high-performance computing market, even with a product scheduled to arrive more than a year from now, but the clock is ticking, Summit Strategies analyst Dwight Davis said. "Some people would argue it's too late, even if it came out tomorrow. Linux has, in pretty short order, come to dominate this sector of the computing spectrum," he said. "The longer it takes to get this to market, the fewer the opportunities there will be."

Market researcher IDC expects the high-performance computing market to expand from $6.1bn (£3.4bn) in 2004 to $7.6bn in 2008.

Ironically, Microsoft's popularity in the overall computing market helped pave the way for Linux. Windows spread hand in hand with the same processors from Intel that became the most widely used chip for running Linux. And in the most recent Top500 list, 45 percent of machines used Intel's Xeon and 12 percent used Intel's Itanium 2.

Intel will help Microsoft test the new Windows version and make sure that it takes advantage of specific chip features, spokesman Scott McLaughlin said. "We work with Microsoft to make sure their operating systems run best on Intel-based platforms," he said.

Microsoft hopes that its new Windows version will make it easy to manage clusters and to develop software for them. It also promises "a secure... platform with high performance and low total cost of ownership."

Microsoft plans to release a preview version of its software in the fourth quarter, Oldroyd said. He didn't comment on whether that would include a software developer kit, but David Lifka, chief technology officer for the Cornell Theory Centre, said he expects one in coming months.

Standardising cluster technology
Computers within a cluster use software called Message Passing Interface, or MPI, to communicate. Microsoft will provide a version of the interface.

According to Microsoft's job postings, it's also working on a version of MPI to run on its .Net system, which would theoretically make it easier to move an application that uses MPI from one computer to another. The .Net technology employs the C# programming language and a package called the Common Language Runtime to run programs.

Microsoft is examining MPI options, but Oldroyd declined to comment on specific plans beyond the company's intent to make it easier to use the technology by giving a consistent interface for higher-level software.

"One real advantage of... Windows HPC Edition is, it's one platform you need to think about. You don't need to think about different MPIs, different schedulers, compile with this, compile with that," Oldroyd said. "It makes the market much more addressable for (independent software vendors). They've got a platform they can count on being there."

Another idea Microsoft is evaluating is "cycle harvesting," which is a method to harness the otherwise unused processing power of PCs, Oldroyd said. The idea has been used on a large scale by the SETI@home effort to sift through radio-telescope signals searching for extraterrestrial communications.