The flaws affect software and hardware that support the real-time multimedia communications and processing standard, known as the International Telecommunications Union (ITU) H.323 standard.
The security problems can cause a product that supports H.323 to crash. For example, in Cisco telecommunications products running its IOS operating system, the vulnerability could be used to cause the devices to freeze or reboot. However, on Microsoft's Internet Security and Acceleration Server 2000, which is included with Small Business Server 2000 and 2003 editions, the vulnerability could allow an attacker to take control of the system.
Ironically, in Microsoft's case, the Internet Security and Acceleration Server is designed to help protect companies' networks from online attacks. Specifically, a filter used in the server that secures VoIP communications is vulnerable to the flaw.
"It is kind of the same situation that we have seen -- a certain level of human error is going to be present and that is true even for security software," said Stephen Toulouse, security program manager for Microsoft.
Microsoft released a patch for its Internet Security and Acceleration Server on Tuesday and published ways to disable the affected service for customers who want to take time to test the software.
Also on Tuesday, Cisco Systems published an extensive advisory outlining which of its products are affected and giving instructions on how to patch them. Among the vulnerable products are CallManager version 3.0 through 3.3, Conference Connection, Internet Service Node and several VoIP switches.
Cisco would not comment on the issue except to refer people to the advisory.
Several other companies also produce products that may be affected but, as of midday Tuesday, only Cisco and Microsoft had issued advisories and patches.
Avaya, Fujitsu, Hewlett-Packard, Lucent and Nortel are investigating the issue. Apple, Hitachi, NetBSD, Red Hat and Symantec have determined that their products aren't affected by the flaws.
The flaws were found by the UK's Internet security watchdog, the National Infrastructure Security Coordination Centre. The group had been testing a variety of products used in the UK's critical communications infrastructure and discovered the problem.
The program used to test the products is an ongoing project at the University of Oulu in Finland. The university's Secure Programming Group has developed tools for finding flaws in network communications standards. Two years ago, the group's work discovered a major flaw in a basic standard used throughout the Internet and other telecommunications networks. Last year, the group discovered flaws in the Session Initiation Protocol (SIP), another technology used by VoIP networks.
The Computer Emergency Response Team (CERT) Coordination Centre in the United States released an advisory on Tuesday based on the information from NISCC.
While a malicious attacker could use the flaws to disrupt VoIP networks, companies using Microsoft's Small Business Server 2000 and 2003 are at particular risk. An attacker can gain a beachhead into a company's network using the flawed H.323 filter, said Microsoft's Toulouse.
"This sort of illuminates to me the value of security researchers where they can test all the situations in which our customers use the product," he said. "H.323 is a very specific protocol. I would hazard a guess that (most people) had not heard about it before today."
