06 Oct 2008 15:05
The computer centre at the European Centre for Nuclear Research (Cern) in Geneva is used to store and process data from physics research, notably for the Large Hadron Collider (LHC) experiment. The computer centre is also used to provide mail, web, database and desktop support, campus networking and administrative services for the Cern labs.
The main cluster of 20,000 disks in the computer centre runs on a modified version of Red Hat Linux. There are 8,000 servers in the room, with 16,000 processors and almost 40,000 cores. These are linked using HP ProCurve switches that handle 4.8 Tbps (terabits of data per second).
The computer centre will be used to process data from the LHC, a machine built to try to mimic conditions just after the Big Bang. The machines will produce head-on collisions between two beams of particles of the same kind, either protons or lead ions. The resulting data will then be studied, to try to gain understanding of questions such as the nature of dark matter and the origins of mass.
The LHC will produce around 15PB (petabytes) of useable data per year. The computer centre will only be able to process approximately 20 percent of the data produced, according to Cern's head of network services, Jean-Michel Jouanigot (pictured centre). The rest of the data will be processed by the LHC computing grid, a conglomerate of systems in 140 institutions in 33 countries.
The grid institutions use middleware created by Cern scientists to process LHC data. This diagram shows Cern as the initial, 'Tier 0' processing centre. The data that Cern has no capacity to process is farmed out to 11 'Tier 1' centres for storage, from where the data is processed by the remaining 'Tier 2' institutions. "The users are physicists located in the Tier 2 centres," Jouanigot said.
The Cern internet exchange is one of the oldest in the world. The Cern Internet Exchange Point (CIXP), located in the computer centre, was the point through which the first pan-European internet backbone was established in 1989. It is also part of Geant, a multi-gigabit pan-European data-communications network, used specifically for research and education purposes. CIXP will act as the conduit for the grid-computing project data.
Many of the server racks in the datacentre are currently empty. According to Jouanigot, the main problem in expanding the current datacentre is cooling it.
Cern has plans to take part of its computing infrastructure and move it to the French side of the site, to give more space between the racks. The new centre will be built in three years, and be up and running within four. "The new centre will scale by a factor of five or 10, and add more capacity," said Jouanigot.
In the basement of the computer centre are tape-storage machines, which record the LHC data before it is processed. The data is then partly processed by the blade servers upstairs, and partly through the grid. "The tape is not backup", said Jouanigot, "it's a source of data".
Five-petabyte storage disks act as a buffer for the data while it is copied from tape onto disk. From there it is passed to processors to start data analysis. Four robots, each containing 20,000 tapes, initially store the LHC data.
Story URL: http://news.zdnet.co.uk/emergingtech/0,1000000183,39498920,00.htmCopyright © 1995-2009 CBS Interactive Limited. All rights reserved
ZDNET is a registered service mark of CBS Interactive Limited. ZDNET Logo is a service mark of CBS Interactive Limited.