After 14 years measuring the world's most powerful servers, the TPC-C speed test is being supplanted by a new benchmark.
The Transaction Processing Performance Council has approved the new standard, called TPC-E, which is designed to be easier and less expensive to run and to be more representative of what databases are used for these days. The first results of the test should begin arriving this summer, once auditors are trained to make sure computer makers know and adhere to test guidelines.
TPC-E simulates traders and brokers interacting with a stock exchange. Information is stored in 33 separate database tables, with actions in one area often triggering actions in another. In comparison, TPC-C dealt with simpler data and operations stored in nine tables that simulated inventory management at a warehouse.
There's no such thing as benchmarks that measure all aspects of a computer's performance, but they still can be a useful way to compare different companies' products. TPC-C was widely used for this task — but servers and databases have changed a lot since its debut in July 1992.
"TPC-C is at the end of its life. It's time for a new benchmark," said Andreas Hotea, who leads the effort to publicise TPC-E. The council, whose members include all the major server and database companies, spent five years creating the benchmark, said TPC administrator Michael Majdalany.
One problem with TPC-C is that high scores require an inordinate amount of storage — not just raw capacity, but numerous hard drives. IBM's and Hewlett-Packard's top TPC-C scores each used more than 7,000 hard drives, Majdalany said. TPC-E will probably need only a tenth of that capacity, he said.
It's typical for a TPC-C test to cost $4m (£2m) to run, Hotea said, mostly through hardware costs. An IBM machine tested in 2006 and currently No. 2 in the TPC-C rankings with a score of 4.03 million transactions per minute cost $12m. HP topped that score with 4.09 million transactions per minute with another $12m machine.
With TPC-C, Majdalany said, "only rich companies can do it".
The new test will be cheaper to run partly because hardware demands are more reasonable, but also because TPC will supply source code for the software instead of requiring testers to write their own, Hotea said.
