Data compression solution or impossible dream?

Daily Newsletters

Sign up to ZDNet UK's daily newsletter.

NEWS
In a tiny office in West Palm Beach, Florida, a handful of clunky computers are crunching through a software problem that many consider unsolvable. These are the offices of ZeoSync, a start-up that made a name for itself in January by claiming to have discovered a way to shrink virtually any digital file to a hundredth of its size -- and then restore the file to its original size without error. If true, it would signify a huge leap forward in computer science, comparable to the invention of a water-powered engine for automakers or cold fusion for power companies. With such a technology, a full-length feature film could be sent easily over a dial-up modem or stored on a floppy disk, for example. The discovery would ultimately transform data storage, networking and virtually any computing task. But as ZeoSync hunkers down to test its claims, few believe they'll emerge successful. Eric Scheirer, a former Forrester Research analyst who holds several patents in the MPEG-4 standard for compressing audio and video, doesn't mince words when talking about ZeoSync's compression challenge. "It's impossible," Scheirer said. "This comes up over and over again...The right analogy is a perpetual motion machine." As with any problem where the stakes are high enough, ZeoSync's claims are drawing scrutiny. A small group of passionate online researchers have dedicated themselves to debunking ZeoSync's theory, as has been done with previous claims of near-perfect compression. ZeoSync now says it's close to testing the technology with a handful of big-name chipmakers, including Intel, although a representative for Intel Labs could not confirm this. Some of the company's public statements have given critics more fuel for their fire, however. An early press release offered a list of technical advisors, some of whom have admitted publicly that they had no connection with the company, or could not verify its claims. ZeoSync's chief executive, Peter St. George, admits to some mis-statements in past releases. But he vigorously defends the basic validity of his findings, and promises to soon provide independent verification. Critics have missed the point, he said. St. George isn't refuting the mathematical laws that prove perfect compression using traditional techniques is impossible. Rather -- having dedicated 12 years to the project -- he says he has hit on a unique method that successfully solves the problem. "All the sceptics are absolutely right," St. George said in a telephone interview with CNET News.com. "But they never anticipated that we would sidestep (the problem)...This is the natural course of evolution in science." The incredible shrinking machine
Data compression is the science of shrinking. Compression makes the data that creates images, sounds, video and even text small enough to be sent through telephone wires -- the network of the Internet. Without data compression, the Web as we know it simply wouldn't exist. In "lossless" compression, used by programs such as WinZip, it's possible to shrink files to a certain size and then recover them without any degradation, or data loss. To compress, the software will pull out repeating bits of data. For example: In the sentence, "The crook took a long look at the book," every "ook" could be replaced by a "1." Basic mathematical theory, however, limits such a technique. Once you take out every repeating chunk, you wind up with a random string of data. For the past 50 years, most mathematicians have come to agree that completely random data can't be further compressed. Many people have claimed to come up with ideas that could shrink any file -- including a random string of digits -- down to a fraction of its size. In every case they've been proven wrong. The sceptic community is now after ZeoSync and St. George, calling his 12-year quest for the perfect data-shrinking machine a digital version of the Don Quixote story. Robert Bristow-Johnson, an engineer at digital audio company Wave Mechanics, is one of those critics. He recently sent a letter to the United States Patent and Trademark Office warning them about ZeoSync's claims, saying that giving the company a patent would be as foolhardy as patenting a perpetual motion machine. "Even I know better," said Bristow-Johnson. "There should be hundreds of thousands of people that know better. There should be dozens of patent examiners who should know better." St. George might be considered an unlikely candidate to discover a mathematical breakthrough. First working as a vice president for technology services at a financial services company, he shifted gears to become an entrepreneur. He served as chief executive of three separate but now-defunct companies before starting ZeoSync in 1999. On his resume, St. George also says he's served as a "bandwidth delivery solutions consultant" to second- and third-world countries. Despite his pedigree, the entrepreneur said the main problem is critics don't truly understand what he's working on. The chief executive says he's simply discovered a different way of compressing data, calling it "compacting" or "filtering." Where other methods chop off a string of binary digits representing a picture or song or text, he looks at the entire string as one huge number. This number can then be represented as a separate mathematical expression, he said. Since typical digital files are represented by hundreds of thousands of zeros and ones, St. George said that crunching numbers using his method requires substantial computing power. For now, ZeoSync's shoestring budget has meant testing on computers using old Pentium II chips -- although partners may help pitch in for more powerful computers down the line. At the time of the company's first press release, it took more than a day to squash a random 128-bit file -- about 16 letters in ASCII, the alphabet most computers use -- into just 100 bits. It's not a product yet, St. George insists. But the fact that the "compacting" can be done at all means ZeoSync has made a theoretical breakthrough, he said. Question marks
Sceptics simply want proof of the breakthrough. St. George said the company is close to making deals with Intel and other chipmakers for testing on state-of-the-art machines. Intel would not independently confirm its participation. In the absence of technical proof, critics have taken a hard look at the company, which is in the process of trying to raise $40m in a private stock offering. Some non-technical questions about the company have emerged. Early on, the company provided a long list of distinguished scientists whom it said had served as technical advisors or consultants on the project. A few weeks later, this list was chopped to a third of its size. Several of the professors on the original list contacted by telephone or email now say they had little or nothing to do with the company. "I had never even heard of them before they made the announcement," said Professor Richard Stanley of the University of Washington, one of the people dropped from ZeoSync's list of advisors. "I'd never talked to them. I just assumed it was some kind of hoax or scam or something." St. George said the original list included clerical errors, and some names of people with whom the company was in contract negotiations. He is issuing a personal apology for the mistakes, he said. According to the company's financial prospectus, ZeoSync also is involved in a lawsuit with an early investor, a company called First Frontier Capital. ZeoSync has said it will return more than $2m in early funding. In addition, the company said it plans to file suit against the venture capital firm and an Indian software company over trade secret issues. Skceptics expect that St. George and his company will fade into history along with other seekers of the impossible. He'd like to prove them wrong. "I don't blame people," St. George said. "The day the Wright Brothers flew, there was a group of scientists nearby saying in concert that man couldn't fly. It's human nature to describe the world in terms of lack, in terms of what we can't do."
ZDNet UK's Developer News Section delivers the latest headlines together with the best UK jobs, right to your browser. Have your say on all developer topics. From j2ee, to C++, from Visual Basic to Javascript plus much more. Share your experience with others on the Developers Forum. Let the editors know what you think in the Mailroom.

Post your comment

In order to post a comment you need to be registered and logged in.

You can also log in with Facebook. Log in or create your ZDNet UK account below

  • Login

Will not be displayed with your comment

By signing up for this service, you indicate that you agree to our Terms and Conditions and have read and understood our Privacy Policy. Questions about membership? Find the answers in the Community FAQ

Get ZDNet UK's daily newsletter

Enter your email address to sign up

ZDNet UK Live

JCB33

How dare film makers, artists or anybody that invests in creativity stop us pirating their works for free. I want to be able to walk into my local...

4 hours ago by JCB33 on ACTA stumbles in Germany
Moley

@GrueMaster. I prefer horses for courses rather than one size fits all. I, and I suspect most other computer users, do not really wish to have...

6 hours ago by Moley on A tale of two distros: Ubuntu and Linux Mint
greycynic

The product that scares me every time I have to use it is the Office 2007 version of Excel. The first bug that I found was applying the median...

6 hours ago by greycynic on Ten flawed products that derail productivity
GrueMaster

Nice review and very informative. One thing I'd like to add (in reply to whs001's 1st question), the main reason to have the same interface from...

8 hours ago by GrueMaster on A tale of two distros: Ubuntu and Linux Mint
Frederick Wrigley

I'be been using Mint 12 since the RC came out, and I am far more happy with the Cinnamon, the Mate, and, yes (with extensions), theGnome 3...

9 hours ago by Frederick Wrigley via Facebook on A tale of two distros: Ubuntu and Linux Mint
bdantas

Excellent article. One small correction, though--although a fresh installation of Linux Mint 12 will, indeed, provide the user with a version of...

9 hours ago by bdantas on A tale of two distros: Ubuntu and Linux Mint
Alan Ralph

In related news, the ISPs club together to get the members of the Home Affairs Select Committee (ya goofed on that part, ZDNet UK) copies of "The...

10 hours ago by Alan Ralph via Facebook on MPs urge ISPs to take down terrorist material
Alan Ralph

In related news, the ISPs club together to get the members of the Home Affairs Select Committee (ya goofed on that part, ZDNet UK) copies of "The...

10 hours ago by Alan Ralph via Facebook on MPs urge ISPs to take down terrorist material
Moley

For Gnome 2 die-hards, it is possible to add icons to the bottom panel (or top top panel, if you prefer) which provide the exact Gnome 2...

11 hours ago by Moley on A tale of two distros: Ubuntu and Linux Mint
ramwellian

Your comments would seem pretty naive and immature. Your 'solution' appears to be, "gee, let's all just give in to the hackers and give them...

11 hours ago by ramwellian on Cloud computing security: no more oxymoron?
BugStalker

"Interesting thought ... If you installed Win7 as a dual boot on a machine that previously only had Linux, and it wrecked your Linux installation,...

11 hours ago by BugStalker on Windows 7 Declares War on GRUB
whs001

This is an excellent summary of Ubuntu and Mint and the interface differences between them. Most such articles take a very partisan position for...

11 hours ago by whs001 on A tale of two distros: Ubuntu and Linux Mint
Moley

@ewallace. Not so clear. Anyone can obtain the text, for example from here http://www.ustr.gov/webfm_send/2379. I support ACTA so long as it and...

12 hours ago by Moley on ACTA: Facts, misconceptions and questions
45283

I think WinRT is fantastic. I just wish it was an option for people that didn't want to go through Microsoft's App Store with its attendant...

15 hours ago by 45283 on Why Windows 8 needs architectural hygiene for WOA
Burn-IT

Nine people? £30m? Who's back pocket is that lot going in? And IF they say it is for new buildings, what about all the ones the government has...

16 hours ago by Burn-IT on Police set to launch three £30m e-crime hubs
ewallace

Just to be clear, nobody knows what is in the text of ACTA, here is a photograph of the text of ACTA http://twitpic.com/8h9iju as submitted to the...

16 hours ago by ewallace on ACTA: Facts, misconceptions and questions
fgvrg56

Unfortunately main issue is that ASUS is refusing to accept that they make some mistake on this version of asus Transformer prime. 1 - GPS sensor...

17 hours ago by fgvrg56 on Asus Eee Pad Transformer Prime Wi-Fi & GPS problems?
Ben Woods

@Marcus A fair question. Just talked with Archos which said it was working on an announcement for next week....

18 hours ago by Ben Woods on Archos confirms G9 Ice Cream Sandwich update schedule
Marcus Karlsson

Any update on this, considering the claimed "first week of February"?

19 hours ago by Marcus Karlsson via Facebook on Archos confirms G9 Ice Cream Sandwich update schedule
apexwm

Bill Goodrich : Just as al_langevin pointed out, with Windows Server 2008 there is no Services for Macintosh anymore. It's gone, not available....

1 day ago by apexwm on Windows Server 2008 drops the ball for Mac compatibility

Latest in Application Development