Red Hat disputes CERT vulnerability figures

Open source experts have hit back at a study published by the United States Computer Emergency Readiness Team (US-CERT) that said more vulnerabilities were found in Linux/Unix than in Windows in 2005, labelling the report misleading and confusing.

The report — Cyber Security Bulletin 2005 — was published last week and claimed that out of 5,198 reported vulnerabilities, 812 were found in Windows operating system, 2,328 were found in Unix/Linux systems. The rest were declared to be multiple operating system vulnerabilities.

The report has attracted criticism from the open source community. Linux vendor Red Hat said the vulnerabilities had been miscategorised, and so could not be used to compare the relative security of Windows and Linux/Unix platforms.

"The study is confusing and misleading. When you look at the list, the vulnerabilities are miscategorised," Mark Cox, consulting software engineer at Red Hat, told ZDNet UK.

"For example, Firefox is categorised as a Unix/Linux operating system flaw, but it runs just as well on a Windows platform. Apache and PHP also run just as well on both platforms. There are methodological flaws in the statistics," Cox claimed.

Security experts have said that the statistics could not be used as a basis for comparison of Windows and Linux/Unix.

Common Vulnerabilities and Exposures (CVE), an organisation that maintains a common vulnerability database, said that the statistics were no basis for comparison of the relative security of the platforms, because they had been collected from different sources with different vulnerability collection criteria.

"In my opinion, refined vulnerability information sources (CVE, Bugtraq, etc) are still a year or two away from being able to produce comparable statistics," said Steven Christey, CVE editor, in an open letter online.

Security company Secunia agreed with Christey that the various vulnerability collection sources made comparison of Windows and Linux/Unix hard.

"I think Steve has got some good points on why comparing vulnerability numbers is difficult," said Thomas Kristensen, chief technical officer at Secunia.

CERT itself pointed out that the information in its bulletin "should not be considered the result of US-CERT analysis", as it included information from outside sources.

Secunia thought that the nature of the reported vulnerabilities also made it difficult to compare security on the platforms, as Linux/Unix researchers concentrate on vulnerabilities in local privilege separation, while Windows researchers look at possible remote vulnerabilities.

"Generally, many of the vulnerabilities in Linux/Unix based products are classified as local vulnerabilities, including privilege escalation, local denial of service and local exposure of sensitive data. These kind of vulnerabilities are not regarded as particularly critical, but Linux/Unix researchers tend to focus quite a lot on this category, probably because of Unix's long history of proper privilege separation. This has only recently become more relevant in Windows (NT, 2000, and XP), but many Windows researchers still focus more on remote issues."

The US-CERT study has also caused online debate within the open-source community. Newsforge, the Linux and open-source online publication, claimed the statistics were unrepresentative.

"The two figures are not representative of today's two major operating system platforms. One figure represents the vulnerabilities found in Windows operating systems: XP, NT, 98, and so on. The other represents a total figure not just for Solaris, AIX, HP-UX, the BSDs, and Linux, but for a hundred different versions of Linux."

Open source vendor Red Hat claimed Linux operating systems were more secure for businesses than Windows platforms, as fewer vulnerabilities were critical and patches were brought out more quickly.

"You should look at the number of critical vulnerabilities. It's a better comparison to look at the critical vulnerabilities that affect customers due to the platform they use. There are fewer critical vulnerabilities, and they are fixed faster in Red Hat Linux," said Cox.

"There is also the issue of timing. With Linux products, critical updates are available within a day. If you look at Red Hat Enterprise Linux 3, the average patch time is under a day. With the recent critical WMF (Windows Meta File) vulnerability, it took Microsoft seven days."

Microsoft was not available for comment at the time of writing.