Testing and fixing 400 million transistor chips is as hard as it sounds. Although you can design and check a new circuit in simulation, there comes a time when you have to press the button and create the first silicon wafer. Experience says that each new major design has thirty to forty show-stopping bugs that have to be tracked down, repaired and re-checked before you get something you can ship.
In the old days, testing a circuit built out of discrete parts was a matter of slapping on probes, looking at voltages, soldering in new parts and rewiring the design until it worked. In the age of sub-micron, half-billion part single chips, the techniques are exactly the same -- but instead of probes, you use laser beams and infra-red microscopes that can see though silicon, and soldering in new parts involves carving micron-wide holes through the chips and beaming new connections through vacuum chambers using particle accelerators.
The first problem when your chip is down is finding the miscreant part. The hard part isn't working out where to look: the real fun is that these days, chips are mounted upside-down in their packaging. All the working connections are covered by the network of pins bringing signals to and from the circuit to the outside world: the only part available to the testers is the backside of the chip, a featureless expanse of silicon with no parts exposed.
Here, physics gives the testers their first break: silicon is transparent to infra-red (IR) light. If you shave off as much of the backside as possible and hook up an IR microscope, you can see the individual devices as they operate. Moreover, if you focus a very low power IR laser on a transistor and monitor the reflection, you can detect tiny changes in the beam as the transistor switches between on and off. This is the exact equivalent of sticking a voltmeter on a particular component in an old television set.
Another quirk of transistor design is that every ten thousand or so times a transistor switches, it gives off a solitary photon of IR light. Stick your chip in front of a camera and watch for those photons, and you can trace complex events as they percolate across your chip. You have to count individual photons over quite a long time, and do probabilistic analysis to screen out noise, but suddenly quite subtle interactions on your circuit become visible.
