Clearly, it pays to be in the paid search business. Yahoo!'s deal with Overture has helped it achieve three consecutive quarters of profitability and has allowed management to boost financial expectations. Yet the success in partnering with commercial search providers raises the inevitable question: why pay dozens or hundreds of people to search the Web when another company wants to pay you to do the same? Yahoo! keeps the operations of its search editors close to its chest. Company executives will not comment on how many editors it employs. A Yahoo! representative said "on average" the company employs "a building full" of directory editors. At least one search engine marketer has said that Yahoo! has scaled back on its directory editors slowly over recent months, giving people new duties or emphasising paid search listings. But Srinija Srinivasan, Yahoo!'s vice president and editor in chief, denied the company has recently laid off or redeployed members of its editorial staff. "The state of search technology thankfully has improved," Srinivasan said in an interview. "That said, we firmly believe there continues to be a gap between the best technology and what we can provide incrementally with the human experience." Conceived by co-founders Jerry Yang and David Filo in a Stanford trailer in 1994, much of Yahoo!'s popularity was built on the directory's ability to give order and organisation to the unruly Web. As legend has it, Yahoo! was developed by Yang and Filo as a way to categorise their favorite sumo wrestling Web sites. Even the company name -- originally the acronym "Yet Another Hierarchical Officious Oracle" -- highlighted its directory roots. Unlike the other search competitors that emerged in the mid-1990s, such as Excite, Lycos, Infoseek and AltaVista, Yahoo! did not develop its technology to crawl through millions of Web sites. Instead, it hired humans to manually search the Web to find, organise and review sites about thousands of topics. Yahoo!'s editorial team became an emblem of the Internet's rise where legions of college graduates would do the heavy lifting to help Web newbies find what they want. Yahoo! did not rely exclusively on its directory, signing partnerships over the years with algorithmic search engines such as AltaVista, Inktomi and Google to provide backup results. But up until October, these third parties were never the centrepiece of Yahoo!'s search results. In late 1999, Yahoo! began to tinker with its coveted directory service. Under then Yahoo! chief executive Tim Koogle, the company launched a new fee plan, requiring sites to pay $299 a year -- $600 for adult sites -- in order to be considered for inclusion in its directory listings. If accepted, companies would be required to pay the fee annually to retain their listing. Yahoo! does not release results for this business, but said it has no plans to discontinue it. While many analysts called the move to paid inclusion overdue at the time, others believe the decision hurt the directory's credibility. "One has to wonder how the economic interests of search is messing with the altruistic agenda of the directory," said Lance Loveday, president of Web marketing consultancy Closed Loop Marketing. Altruism vs cash
Yahoo! isn't the only directory facing criticism these days. Search engine marketers also point to the Open Directory Project (ODP) as an example of how far directories have fallen behind algorithmic search providers, both in terms of the reach and quality of the results they provide. AOL Time Warner-owned Netscape runs the ODP, which launched in June 1998 under the name NewHoo. It uses about 210,000 volunteer editors to catalogue the Web, many of whom are search engine marketers. Though Google and AOL both draw on this directory for specialised searches, the service is thought to be plagued with troubles. Hardware failures over the winter holidays caused the directory to be out of commission for several months, for example. Elisabeth Osmeloski, a search engine marketer and a volunteer editor with ODP, said that more than 50 percent of the sites submitted for review are spam links, causing a major sap to volunteers' time. "The ODP has a huge backlog from bad submissions; there are sites waiting for two years to be reviewed. It would be better if Netscape got [more] behind it," she said. Bob Keating, editor in chief of the ODP, said that it's fighting the good fight against the encroachment of the profit motive into what is rightfully an editorial process. "We're trying to combat the commercialisation of search," said Keating, whose ODP has a catalogue of 3.8 million sites, compared to some four billion for Google. "A lot of Web directories have gone the other direction. As search gets even more commercialised, the Open Directory is the only one that's left that's really grounded in the original concept of the Net -- that it's an information source and not a money-making vehicle." He added that the ODP has checks and balances to stop spammers from controlling the directory. LookSmart, which launched in October 1996 and was originally backed by Reader's Digest, started as a directory of the top sites in any category on the Web. It employed hundreds of editors and writers to handpick sites. Now, it employs about 100 editors, but they largely review commercial sites that have bought into the directory, which is licensed to Microsoft's MSN. It also runs a noncommercial directory called Zeal.com that is staffed with about two or three editors and a team of volunteers. Many small sites say they still see value in a listing in directories like Yahoo!'s, LookSmart's and the ODP because such links are given weight by Google's PageRank, a system for evaluating the popularity of a Web page. Google rides on the back of human-screening of Web sites, and many people see directory links as an easy step on the road to popularity in major search engines. Tim Mayer, vice president of Fast Web Search, which was recently acquired by Overture, said that directories are largely treated like any other link on the Web, and some may be thought of as more authoritative than others. But he said the influence of the directories has faded as they have become more commercial. "The more authoritative the site or directory is that is linking to your site, the more weight given in the link popularity," said Mayer, adding that link popularity is only one feature of many in the relevance algorithm. Still, he said, it's less important "in many search engines in the past years as directories have moved from purely editorial to pay-for-play."
For everything Internet-related, from the latest legal and policy-related news, to domain name updates, see ZDNet UK's Internet News Section.
Let the editors know what you think in the Mailroom.
