Techies take the rap for site blackouts

Microsoft blamed its own technicians for a crucial error that crippled the software giant's connection to the Internet, almost completely blocking access to its major Web sites for nearly 24 hours.

In a statement issued late Wednesday, Microsoft explained that a "router configuration error" had caused requests for access to the company's Web sites to go unanswered. Routers are critical pieces of the Internet that direct data between a company's network and the Internet.

After replacing the misconfigured files at about 5pm PST Wednesday, traffic to and from the affected Web sites returned to normal, Microsoft spokesman Adam Sohn said.

"This was an operational error and not the result of any issue with Microsoft or third-party products, nor with the security of our networks," he said.

Microsoft's woes began Tuesday night and effectively blocked access to some of the Internet's most popular Web sites, including Microsoft.com, MSN.com, WindowsMedia.com, Encarta.com, Carpoint.com and Expedia.com, Sohn said. Hotmail.com also was largely inaccessible, although the company could not confirm that it was affected by the same problem that severed access to its other sites.

Around 1pm PST Wednesday, some of Microsoft's Web sites had again become accessible from some parts of the country, but access was inconsistent. Four hours later the company declared that full access had been restored.

Earlier in the day, Microsoft could not be sure that a network attack wasn't responsible. "We are not ruling anything out," Sohn said. "If we know for sure if it's a denial-of-service (DoS) attack, we will tell everyone."

Microsoft acknowledged some European customers were having problems accessing its sites. In addition, a reader in Hokkaido, Japan, sent an email reporting that from "MSN's mobile service on my cell phone to Hotmail, nothing Microsoft works".

Microsoft's network of Web properties ranks as the third most-visited destination on the Internet. According to Net research company Jupiter Media Metrix, Microsoft Web sites drew 54 million unique visitors in December, trailing only America Online's 61 million and Yahoo's 55 million.

The timing and duration of the embarrassing outage came as Microsoft is trying to bolster its reputation among corporate customers. The company launched a $200m advertising campaign Monday touting its business software in competition with Oracle, IBM and Sun Microsystems. The theme for the ads is "software for the agile business".

"Businesses today need to be fast, nimble and responsive to compete and respond to customer needs, yet a lot of the technology for business is big, slow and expensive," Microsoft chief executive Steve Ballmer said in a statement about the campaign.

While Microsoft worked to track down the cause, security experts took the opportunity to examine, and criticise, the configuration of Microsoft's DNS servers

"Weld Pond", manager of research and development for network security firm @Stake who prefers to use his hacker handle, said Microsoft may have made a poor choice in setting up its domain name servers. According to his research, Weld Pond said all four of Microsoft's domain name servers appear to be located on the same network -- sort of like putting all the company's executives on the same airplane.

"I don't know why they did that," Weld Pond said. "It's a DoS attack waiting to happen." Under such an attack, a server is inundated with so many requests for irrelevent data that it slows to a crawl.

According to the Whois directory of Internet domains, other major networks, including America Online and Yahoo, have backup DNS servers on separate networks.

Microsoft would not comment on its network configurations.

The outage came just a few months after a hacker broke into the company's corporate network.

It also comes after several other Net outages caused primarily by server errors. Earlier this month, online auctioneer eBay suffered a day of lengthy outages. During the outages, eBay visitors could access the company's home page and its category listings but weren't able to view individual auctions, place bids or list items. The company said the interruption resulted from a series of failures that affected its primary and backup systems.

Last month, some Hotmail members were locked out of their email accounts for several days because of system upgrades to the free service. That was not the first time that Hotmail had experienced such problems -- a similar outage occurred in 1999, when the company failed to pay a $35 registration fee for the domain name Passport.com.

Have your say instantly, and see what others have said. Click on the TalkBack button and go to the ZDNet News forum.

Let the editors know what you think in the Mailroom. And read what others have said.

Copyright © 1995-2008 CNET Networks, Inc. All rights reserved
ZDNET is a registered service mark of CNET Networks, Inc. ZDNET Logo is a service mark of CNET Networks, Inc.