Patch management: Filling security holes

Nobody knows when the first patch was issued, but it was almost certainly shortly after the first release of the first software package. No matter how much testing is done in-house, the real world and real users always exercise applications in ways the writers never foresaw or tested: even the largest software company can't afford the time or resources to exhaustively check complex products -- which is not to say they can't do better. Patches will be with us always.

At its simplest, where a software company sends out a small piece of code that must be transplanted into the body of the ailing software, patching takes a few moments. This doesn't scale well. Ask anyone responsible of late for collections of Windows machines, as soon as there are multiple machines with multiple configurations, even deploying and checking a single patch can take a long time. When there are multiple patches, patch management threatens to become a full-time job. And multiple patches are becoming the norm: the CERT security advisory team reported over 4,000 vulnerabilities in 2002, fuelling a veritable industry of patching that is variously estimated to cost businesses and governments up to $1.5bn worldwide.

The nature of patches
The problem is complicated by the nature of patches. They are often produced under great time pressure, especially when the vulnerability they seek to fix has the potential for quick exploitation over the Internet. There is rarely the time for a full quality assurance check by the patch writers, who see themselves as in competition with exploitative hackers who neither know nor care about QA.

Patch management must be owned by a person or a group with responsibility for receiving advisories, checking them in the context of the company and triaging them into must-do, might-do and reject. The patching process has to be integrated with standard system administration techniques if it itself is to be properly managed. It is not in itself risk-free, therefore it should be assessed as part of risk management and treated as any other network-wide change. Not every patch is critical, and not every patch is appropriate for all systems -- if the fix is for a service that isn't deployed, then the benefits from applying it will be outweighed by the time to apply it and the risk that the patch itself may have knock-on effects.

However, this approach isn't useful unless you have a full inventory of your system, including each computer's operating system and applications complement, the versions of each package and the services that are actually running. You must also know what patches have already been applied -- or not -- and who's responsible for each computer. In many companies, this discipline is imperfectly observed with an attitude that as long as every computer's got regularly updated antivirus (AV) protection and there's a decent corporate firewall then nothing too bad can happen. Such approaches are dangerously outdated: threats such as the Blaster worm will bypass AV protection by attacking vulnerabilities directly in the operating system, and just one infected computer can compromise everything behind the firewall. If you don't know where it is, who to call to turn it off or what's missing from its system, it can take far too long to root out the culprit. Taking a full inventory must be the first priority.