Sybase's IQ data analytics gets parallel smarts

Sybase has changed the structure of its Sybase IQ column-based database software to allow queries to be run in parallel, increasing the efficiency with which large datasets can be mined, queried and processed.

With version 15.3 of IQ, released on Wednesday, the business intelligence specialist has broadened the data-analytics platform to allow database queries to be parallelised, which means they are distributed across multiple separate processors. This allows companies to glean answers from dataset queries more quickly, according to Sybase.

"Basically, when you fire a query at one of the servers, it becomes a leader and does a quick calculation to see whether that query makes sense to parallelise," Andrew de Rozairo, business development manager for Sybase, told ZDNet UK.

Besides parallel queries, the incremental update to the IQ 15 platform adds in an architecture for inter-node communication; allows the grouping together of server resources into virtual computing groups; supports access via web services; and provides API support for popular developer programming language Ruby on Rails.

"The main driver for the 15.3 release was to improve the utilisation of servers. [This addresses] the challenge we see in the marketplace, which is not just the management of big data, but the management of big user bases," De Rozairo explained.

With Sybase IQ, the SAP subsidiary competes with a variety of makers of data-analytics packages. The software goes up against products from Teradata, Netezza and Oracle — which has its Exadata hardware and software appliance — as well as against upstart open-source data-analytics framework Hadoop.

Hadoop gets Sybase's dander up

Sybase is aware of Hadoop's rise in major web enterprises such as Facebook, Yahoo and Twitter, according to De Rozario. In response, it is planning to develop future versions of Sybase IQ to take on the open-source software.

"Are we doing anything about [Hadoop]? Yes we are. The next release is very much aiming at that," he said.

In some ways Hadoop is free, and in other ways it is not — it's quite expensive in terms of scaling up.

– Andrew de Rozairo, Sybase

To that end, Sybase hopes to implement MapReduce, which underpins Hadoop, in the next release of IQ — due by the end of 2011. In addition, the company is "actively evaluating support for HDFS [Hadoop Distributed File System]", which powers the platform.

"Hadoop is a fantastic indexing engine and great for a long tail of queries that you will never run again," De Rozario said. "The IQ environment is much more for a mixture of queries that are run multiple times, so [it's] for more consistent queries, standard queries and ad hoc queries."

De Rozairo believes Hadoop, though it is free to use, carries follow-on costs that mean IQ can be less expensive for certain tasks. "In some ways it is free, and in other ways it is not — it's quite expensive in terms of scaling up," he said, noting Hadoop requires a large amount of redundancy.

As for software-based analytics in general, De Rozario believes Oracle and others that yoke their software products to hardware face an increasingly large hurdle to surmount.

"More and more customers are saying they don't want their analytics choice to be determined by what hardware they buy... it needs to become more fluid," he said.