Inktomi gives Web sites control -- for a price

Daily Newsletters

Sign up to ZDNet UK's daily newsletter.

NEWS
Search technology provider, Inktomi, accustomed to taking money from portals that licence its technology, is now collecting revenue from content sites. The introduction on Wednesday of Inktomi's Index Connect lets large Web sites pay to make sure they're included in the company's search index. The service gives sites control over how often their sites are indexed and what specific pages are included. "In the past, search engines always have had a relationship with people who are searching, but the relationship on the content side has been pretty random," said Troy Toman, vice president and general manager of Inktomi's search solutions division. "It's a hit-and-miss proposition when you're spidering the Web on a random basis." Most search engines work by crawling the Web with spiders, or software programs that continually surf the Web, indexing pages and words. The problem Index Connect proposes to solve is that many Web pages undergo frequent and regular change. "We refresh our index every 30 days," Toman said. "We can now say, instead of that, why not set up your relationship so that you can send us a data feed every time you update a page, and we can add that, as short as every 15 minutes to an hour?" Index Connect brings the search pioneer a step closer to some of its rivals whose "pay-for-placement" business models are based on taking money from content providers in return for prime placement in the search results. A search for "sports car" on such a site, for example, would return sports car dealers in an order determined by how much money they paid for their placement. Critics fault such schemes for corrupting the integrity of search results. Search purists prefer that results be weighted according to relevance, rather than a company's willingness to write the search engine a cheque. Inktomi acknowledged that its new service would weight its index toward paying content providers, but was quick to distance itself from the pay-for-placement schemes. Index Connect "does potentially have an impact on the index, with more content coming from paying clients," Toman said. "But our results and relevance algorithms are exactly the same. This is paid inclusion, giving sites the opportunity to be considered for a search. But we are still very democratic when we select search results." Inktomi, last year, introduced Search/Submit, which lets sites pay to be searched on a regular basis. But that service -- which Toman described as "hugely successful" -- was aimed at smaller sites, or those submitting fewer than 1,000 Web page addresses. Index Connect is designed to offer a similar service to the large sites. Index Connect gives sites more say in the number of pages that are indexed and the frequency with which they are updated. It also allows some sites to submit pages directly, rather than have Inktomi's powerful spiders come in and crawl around -- which can have the effect of slowing a site down. San Francisco news site "KQED had prohibited our search bots because it was slowing performance, so now they actually feed us the information from their site," Toman said. "Now you can have our spider on a timer or send us a data feed with all the pages on it that you want included." QED, The Hunger Site and other non-profit sites can use Index Connect free of charge. Search engine analyst, Danny Sullivan, editor of SearchEngineWatch.com, said paid submissions are not necessarily bad. But he warned that search engines would have to work to ensure equal access and representation in the index. "The key thing is to ensure that important content that should be in the index is also included, regardless of payment," said Sullivan. "Otherwise, the search engine index isn't as comprehensive or useful to searchers. Given this, Inktomi's statement, that it is to begin offering deeper inclusion to some nonprofit sites for free, is a good sign. It means that perhaps the commercial portion of the Web can to some degree subsidise the important nonprofit portion." For-profit sites pay a set-up fee, then pay on a cost-per-click basis that ranges from a nickel to 50 cents per click. For-profit customers, so far, include eBay, Epinions, LookSmart, RollingStone.com and Virage. For complete business coverage, see ZDNet UK's Enterprise Channel. Have your say instantly, and see what others have said. Click on the TalkBack button and go to the ZDNet News forum. Let the editors know what you think in the Mailroom. And read other letters.

Post your comment

In order to post a comment you need to be registered and logged in.

You can also log in with Facebook. Log in or create your ZDNet UK account below

  • Login

Will not be displayed with your comment

By signing up for this service, you indicate that you agree to our Terms and Conditions and have read and understood our Privacy Policy. Questions about membership? Find the answers in the Community FAQ

Get ZDNet UK's daily newsletter

Enter your email address to sign up

ZDNet UK Live

bootlegger

Make that 13 people now - I got refused today at Manchester airport. I thought I was up to date on this legislation - I knew of the EU ruling from...

2 hours ago by bootlegger on UK airport body scans will not be opt out
tinycg

Don't forget to check out apps like GoodReader or SlideShark either, they're indispensible for people on the go in presentation situations. Best...

5 hours ago by tinycg on Four top iPad apps for people on the move
TerryRK

Well it seems there is something a number of us agree on. Why is the Ubuntu Unity launcher so ugly? I thought perhaps it was something to do with...

9 hours ago by TerryRK on A tale of two distros: Ubuntu and Linux Mint
Freebies202

Duplicate comments are not made intentionally. Its very good to know that now you are keeping check on this problem because sometimes a commenter...

19 hours ago by Freebies202 on Microsoft fixes blog comments, speeds up blogs with open source
kevinmchapman

"the very significant number of users" and "many (most) of us" - you have no evidence for these statements. It is a fact that most users are saying...

1 day ago by kevinmchapman on A tale of two distros: Ubuntu and Linux Mint
Marg Menzies Harrison

Another grammar faux pas is the improper use of "you". When sitting down down in a restaurant, for example, I get cringe when the waitress...

1 day ago by Marg Menzies Harrison via Facebook on 10 flagrant grammar mistakes that make you look stupid
zdnetukuser

And NOW, folks, for Canonical's next trick... Kubuntu is late. Here's a pencil. Draw your own conclusions. cf.:...

1 day ago by zdnetukuser on Linux Minterface
Moley

@kevinmchapman. The discussion here reflects the very significant number of users who really do like the traditional menu system and who wish to...

1 day ago by Moley on A tale of two distros: Ubuntu and Linux Mint
kevinmchapman

Er, no... It is an efficient means of finding the application/file/setting you need in one place. The icons are a simply a fallback for when you...

1 day ago by kevinmchapman on A tale of two distros: Ubuntu and Linux Mint
TerryRK

Isn't the provision of a text based search an admission by the developers that the mass of icons approach does not work? I don't need to use a...

1 day ago by TerryRK on A tale of two distros: Ubuntu and Linux Mint
kevinmchapman

"Unity and GNOME 3 both abandon the old text-based cascading menus in favour of a graphical icon-driven system." Point truly missed. Both use a...

1 day ago by kevinmchapman on A tale of two distros: Ubuntu and Linux Mint
TerryRK

whs001 - Thank you, I'm glad you liked the article. I absolutely agree with you on your first point. I should perhaps have made it clearer that...

1 day ago by TerryRK on A tale of two distros: Ubuntu and Linux Mint
Dennis Nilsson

If we allow corporate interest to dictate the way our government circumvents due process against foreign entities then we should accept the same...

1 day ago by Dennis Nilsson via Facebook on ACTA stumbles in Germany
GHar123

I totally dislike pirating of works, I fear that artists will be deterred from creating works if they think that they are going to get ripped off....

2 days ago by GHar123 on ACTA stumbles in Germany
JCB33

How dare film makers, artists or anybody that invests in creativity stop us pirating their works for free. I want to be able to walk into my local...

2 days ago by JCB33 on ACTA stumbles in Germany
Moley

@GrueMaster. I prefer horses for courses rather than one size fits all. I, and I suspect most other computer users, do not really wish to have...

2 days ago by Moley on A tale of two distros: Ubuntu and Linux Mint
greycynic

The product that scares me every time I have to use it is the Office 2007 version of Excel. The first bug that I found was applying the median...

2 days ago by greycynic on Ten flawed products that derail productivity
GrueMaster

Nice review and very informative. One thing I'd like to add (in reply to whs001's 1st question), the main reason to have the same interface from...

2 days ago by GrueMaster on A tale of two distros: Ubuntu and Linux Mint
Frederick Wrigley

I'be been using Mint 12 since the RC came out, and I am far more happy with the Cinnamon, the Mate, and, yes (with extensions), theGnome 3...

2 days ago by Frederick Wrigley via Facebook on A tale of two distros: Ubuntu and Linux Mint
bdantas

Excellent article. One small correction, though--although a fresh installation of Linux Mint 12 will, indeed, provide the user with a version of...

2 days ago by bdantas on A tale of two distros: Ubuntu and Linux Mint