Developers dig into Google's toolbox

Daily Newsletters

Sign up to ZDNet UK's daily newsletter.

NEWS
Imagine retrieving up-to-the-second results for all of your Web searches, or personalising a high-powered navigation system for your desktop. Sound far-fetched? Not to some software developers who've found inspiration in a new service from Google, the search site everybody loves to love. Late last week, Google launched a new tool for Web developers called Web APIs (application programming interfaces). Simply put, the service lets developers automatically query its database of two billion documents on a limited basis. Then they can publish results as they choose, as long as it's for noncommercial purposes. To be sure, the experiment could turn out to be just so much hype. It is less than a week old, so it has yet to deliver anything astounding. Early creations of the APIs are cropping up as a "Google box," a display of search results that takes the pulse of any desired term 1,000 times a day. Google is promoting the APIs as a means to create an online game or use its spell-checking technology. Still, the announcement has the research and software development community dreaming up applications that could be spawned from access to Google's massive database of Web pages, documents, images, news and discussion-group archives. "It's an extremely important and forward-thinking move by Google," said Rob Sanderson, an England-based researcher working on the University of California at Berkeley's Cheshire project, a library catalogue and full-text information retrieval system. "The beta release is just a teaser for what could be done with full access to Google's engine." Google's APIs come as the company is trying to find its financial footing amid fierce competition in the market for Internet search services. Widely thought to be preparing for an initial public offering, the company has been slowly cementing its position as the No. 1 search provider on the Web, with more than three billion documents in its database. In the last couple of years, Google's technology scored it a high-profile contract with Yahoo!, taking over Inktomi's position. It has also replaced paid-listing company Overture Services on EarthLink's site. Although the API service is currently free, many software developers say it could be a catalyst to get creative juices flowing before the company puts a price tag on it. Such a move would fall into line with Google's recent efforts to boost its revenue, which for now relies on advertising. Earlier this year, it introduced an enterprise search device and unveiled new tools for advertisers to bid for better exposure on the site. The service also comes as Google seeks to stop some parties from performing automated searches of its database for commercial gain -- a forbidden practice that typically drains bandwidth resources. By requiring Web developers to register with the site and by capping the number of searches, Google can save resources while endorsing, and potentially charging for, legitimate uses of automated queries. Chris Sherman, associate editor for search publication SearchEngineWatch.com, said the move helps alleviate some of the problems Google faces with automated robots querying its database. "This is the warning shot that things are going to change here. And the nice thing is that before shutting off access, Google has opened up this gateway for the Web community to have access to vital information," Sherman said. Gaga over Google
Google's API test service uses WSDL (Web Services Description Language) and SOAP (Simple Object Access Protocol) standards, so developers can link their applications regardless of the computer language -- such as Java or Visual Studio .Net -- used to program them. But since the service launched five days ago, several people in the software development community have adapted code so that APIs can be developed in alternate programming languages, including Perl and Ruby. Web developers can only retrieve 10 "hits" at a time, performing 1,000 searches a day only on the Web. But developers are encouraged by the prospect of having access to Google's vast collection of images, directory links, news archives and newsgroups. One potential application could be an automatic query of the day's hot topic using Google News, a database of newspaper headlines and excerpts in test phase at the site. Another offshoot could be based on a query of Web documents related to a single day in history. If Google's APIs take off, the benefits could extend far beyond the company to the Web at large, according to Web developers, who said the move could provide a big boost for ballyhooed, but still mostly theoretical, Web services. By allowing Web sites and applications to sync up or share data to build new systems or sites, Google's APIs could provide the clearest example yet of how Web services will actually work. And that, proponents say, could inspire valuable developments in the Web community, including forcing other software developers such as Microsoft to follow suit. "Believe me, we're going to get tired of the Google box," said Dave Winer, publisher of a Web log and head of software company UserLand. "It's not at the core of what they released, but it says maybe they'll expose some functionality that is useful. It opens up the conversation." Even Winer acknowledges that the experiment faces some hurdles, especially if the APIs mature into robust applications that Google can tap for revenues. Because it uses SOAP, the Cheshire project's Sanderson said Google could hit a snag because of a hotly debated proposal to extend patent rights to Web standards, a possibility that he describes as a "dark cloud looming" over the experiment. Already Winer and others predict that Google will charge for various applications of the service. But Sanderson predicted that API users might not only have to pay Google for better access to its database -- they also might have to pay companies such as Microsoft and IBM to use the mechanism for performing such searches. He added that the patent owners have yet to state any sort of terms that they would consider a "reasonable and nondiscriminatory license." Nelson Minar, lead software engineer on Google's API project, said that while it's too early to tell what kinds of applications will evolve out of the experiment, interest is high. He said that after only four days, 10,000 developers signed up to use the API service. In addition, he said that 15 or 20 libraries for various programming languages have been created. "The possibilities are limitless," Minar said. "Our idea is to provide the raw material for the developer community, and (they) will come to us with creative applications that use our Web API." Real-time searches?
Winer, for one, is enthralled by Google's willingness to let developers collaborate on ideas. In the act of introducing APIs, Google has changed the status of some people from users to developers, and that could open a dialog to allow for real-time "crawling" of the Web. This means that eventually Web publishers such as CNN or Web loggers could notify Google's search engine every time data on their Web page is updated. Then Google could automatically verify the command and index the page on the fly. "They could open themselves to that kind of notification and verification so they would provide up-to-the-minute search capability," said Winer, who predicts that one day Google or another search provider will offer a service for the home computer. Google chief executive Eric Schmidt has already said real-time search results are a priority for the company, which launched in 1998. In the last year, the company improved its search technology to begin indexing sites for news organisations such as CNN and The New York Times that change more frequently. Previously, it crawled the Web every 30 days or so. Like other developers, the Cheshire project's Sanderson believes the test is valuable for the future of interoperable applications, allowing for seamless embedding of Google's resources into other programs through multiple platforms and standards. For example, Sanderson said the Google APIs could allow his project to create applications that search both traditional library catalogues as well as the entire Web from a single command. "Now not only can you find the books available on the topic, but also Web sites which might offer relevant information, and in a uniform interface rather than having to search the library, open a new browser window, and then search Google," he said. "Students in particular will find this useful when researching papers," he added. Others, while interested, are more sceptical. Aaron Straup Cope, one developer who wrote a script for APIs in Perl, said he sees the move as validation for distributed computing or Web services, but he added, "I doubt that (APIs) will make the Earth move." "Being able to 'plug' Google-ness in to your Web site will, if nothing else, provide an example of 'distributed computing' that is not as abstract as those that have come before it," Cope said. He also cited the development of cross-publishing functionality in Web logs. Cope said Google's APIs give further credence to the concept of an "Internet operating system," where people can pull in or manipulate content via a remote function as a page is being published. "That is, there is a growing interconnectedness among pages, sites (and) applications." Still, Cope predicted many of the problems that plague Web sites such as bandwidth limitations will hamper broader adoption. "Not everyone has a thousand servers like Google does," he said.
ZDNet UK's Developer News Section delivers the latest headlines together with the best UK jobs, right to your browser. Have your say on all developer topics. From j2ee, to C++, from Visual Basic to Javascript plus much more. Share your experience with others on the Developers Forum. Let the editors know what you think in the Mailroom.

Post your comment

In order to post a comment you need to be registered and logged in.

You can also log in with Facebook. Log in or create your ZDNet UK account below

  • Login

Will not be displayed with your comment

By signing up for this service, you indicate that you agree to our Terms and Conditions and have read and understood our Privacy Policy. Questions about membership? Find the answers in the Community FAQ

Get ZDNet UK's daily newsletter

Enter your email address to sign up

ZDNet UK Live

ramwellian

Your comments would seem pretty naive and immature. Your 'solution' appears to be, "gee, let's all just give in to the hackers and give them...

8 minutes ago by ramwellian on Cloud computing security: no more oxymoron?
BugStalker

"Interesting thought ... If you installed Win7 as a dual boot on a machine that previously only had Linux, and it wrecked your Linux installation,...

27 minutes ago by BugStalker on Windows 7 Declares War on GRUB
whs001

This is an excellent summary of Ubuntu and Mint and the interface differences between them. Most such articles take a very partisan position for...

31 minutes ago by whs001 on A tale of two distros: Ubuntu and Linux Mint
Moley

@ewallace. Not so clear. Anyone can obtain the text, for example from here http://www.ustr.gov/webfm_send/2379. I support ACTA so long as it and...

48 minutes ago by Moley on ACTA: Facts, misconceptions and questions
45283

I think WinRT is fantastic. I just wish it was an option for people that didn't want to go through Microsoft's App Store with its attendant...

4 hours ago by 45283 on Why Windows 8 needs architectural hygiene for WOA
Burn-IT

Nine people? £30m? Who's back pocket is that lot going in? And IF they say it is for new buildings, what about all the ones the government has...

5 hours ago by Burn-IT on Police set to launch three £30m e-crime hubs
ewallace

Just to be clear, nobody knows what is in the text of ACTA, here is a photograph of the text of ACTA http://twitpic.com/8h9iju as submitted to the...

5 hours ago by ewallace on ACTA: Facts, misconceptions and questions
fgvrg56

Unfortunately main issue is that ASUS is refusing to accept that they make some mistake on this version of asus Transformer prime. 1 - GPS sensor...

6 hours ago by fgvrg56 on Asus Eee Pad Transformer Prime Wi-Fi & GPS problems?
Ben Woods

@Marcus A fair question. Just talked with Archos which said it was working on an announcement for next week....

7 hours ago by Ben Woods on Archos confirms G9 Ice Cream Sandwich update schedule
Marcus Karlsson

Any update on this, considering the claimed "first week of February"?

9 hours ago by Marcus Karlsson via Facebook on Archos confirms G9 Ice Cream Sandwich update schedule
apexwm

Bill Goodrich : Just as al_langevin pointed out, with Windows Server 2008 there is no Services for Macintosh anymore. It's gone, not available....

17 hours ago by apexwm on Windows Server 2008 drops the ball for Mac compatibility
txtrainguy

Replying to an old topic that I'm currently facing with my CEO (who is on a Mac). Our servers are primarily Windows Servers, office is about...

23 hours ago by txtrainguy on Windows Server 2008 drops the ball for Mac compatibility
k0tcs3

Sure, that makes perfect sense. Pay wrong-doers money and thank them for breaching your security and pointing out your flaws, that would surely...

24 hours ago by k0tcs3 on US indicts Romanian over NASA climate change hack
Random_Error

I think he's referring specifically to Android apps, as Apple do regulate their App Store, but Google seem to let any old crap onto the Android store!

24 hours ago by Random_Error on RIM: BlackBerry will keep 'garbage' apps out of store
Paul Fezziwig

Keep the crap apps out?! How will they compete with Android and Apple's claim to fame of having so many life changing apps? I wonder if the media...

1 day ago by Paul Fezziwig via Facebook on RIM: BlackBerry will keep 'garbage' apps out of store
Aigars Mahinovs

It has been shown time after time that if there is an author store that sells the songs at even 1$ per song and gives you a high-quality digital...

1 day ago by Aigars Mahinovs via Facebook on Copyright isn't working, says European Commission
awbMaven

""As a result of Butyka's alleged conduct, researchers were unable to use the computers for more than two months while NASA removed the malicious...

1 day ago by awbMaven on US indicts Romanian over NASA climate change hack
subhorup

It simultaneously worries me and uplifts me that a self-proclaimed group of internet activists name themselves after Indian mythical figures....

2 days ago by subhorup on Anonymous activists release PCAnywhere source code
naviathan

It's actually far easier to work anonymously on the internet than you think. With tools like Tor bouncing your traffic around the world before...

2 days ago by naviathan on Anonymous activists release PCAnywhere source code
Agnostic_OS

1000272134 and bluedalmatian with you both there but then I'm still in 10.04 land (and happy with it)

2 days ago by Agnostic_OS on Ten factors that make Ubuntu 11.10 a hit

Latest in Application Development