The Harvard report comes as Google is becoming increasingly embroiled in international political disputes over copyright and censorship. China blocked access to Google last month. Google was criticised in March for bowing to a demand from the Church of Scientology to delete critical sites from its index. In a response that won praise, Google replied by pledging to report future legal threats to the ChillingEffects.org site run by law school clinics. As Google has become the way more and more people find information on the Internet, it has become an increasingly visible target for copyright complaints about cached information and allegedly infringing links. ChillingEffect.org's Google section lists 16 requests or legal threats the company in the past three months. One Google competitor and critic even suggested that the wildly popular search engine be transformed into a government-controlled "public utility". Edelman, who created the program that tested URLs against Google's index, said he was investigating a tip about Google's German-language version. "One concern that I've had for some time vis-a-vis filtering is that filtering is almost always secretive," Edelman said. "In the (library filtering) case, that meant you can't look at the list of blocked sites. In the Chinese government case, you can't see what the sites being blocked." Edelman, who is a first-year law student, testified as an expert witness for the American Civil Liberties Union (ACLU) in a court challenge to a law requiring libraries to install filtering software if they accept federal funds. He is also a plaintiff in a second lawsuit filed in June to eviscerate key portions of the Digital Millennium Copyright Act. Google's response
Google refused to reply to a list of questions sent via email, including which sites have been delisted, how many sites have been delisted, what standards are used, and what other Google-operated sites have less-than-complete listings. In an emailed response, Google's Tyler said: "As a matter of company policy we do not provide specific details about why or when we removed any one particular site from our index. We occasionally receive notices from partners, users, government agencies and the like about sites in our index. We carefully consider any credible complaint on a case by case basis and take necessary action when needed. This is not preemptive -- we only react to requests that come to us... to avoid legal liability, we remove sites from Google search results pages that may conflict with local laws." Tyler said an internal team involving lawyers, management and engineers makes the final decision on what to remove. "At Google we take these types of decisions very seriously," he said. "The objective is to limit legal exposure while continuing to deliver high quality search results that enable our users to find the information they need quickly and easily." Tyler pointed to Google's terms of service agreement, which says Google will "consider on a case-by-case basis requests" to remove links from its index. A moving target
Because Google has to keep track of a constantly moving target -- new sites arguably illegal under French or German law appear every day -- it's encountering the same problems of overinclusiveness that traditional filtering software has experienced. According to the Harvard report, some sites that Google does not list include 1488.com, a "Chinese legal consultation network", and 14words.com, a discount Web-hosting service and some conservative, anti-abortion religious sites. Those sites do not appear to violate either German or French laws. Banned from Google.de and Google.fr listings is Stormfront.org, one of the Internet's most popular "white pride" sites. Stormfront features discussion areas, a library of white nationalist articles, and essays by David Duke, a former Ku Klux Klan leader. "We've been dealing with this for quite a few years," said Don Black, who runs the site. "The German police agencies seem obsessed with Stormfront even though we're not focussed on any German language material." Black, who learned a few months ago that Google.de delisted Stormfront, says he doesn't hold it against the Mountain View, California-based company. "Google is trying to conform to their outrageous laws," Black said. "So there's really nothing we can do about it. It's really a French and German issue rather than a Google issue." The First Amendment
Because Google is a company and not a government agency, it has the right in general to delete listings from its service or alter the way they appear. (On Tuesday, however, ZDNet UK reported that an Oklahoma advertising firm has sued Google over its position in search results.) "Google may not only have the legal right to (delete listings), they may have the legal obligation to do it," said Barry Steinhardt, director of the ACLU's technology and liberty program and a co-founder of the Global Internet Liberty Campaign. "Over the long term, this will become a significant issue on the Net," Steinhardt said. "There's a wide variety of laws around the world prohibiting different forms of speech. You can imagine what the Chinese government prohibits versus what the French government prohibits versus what the US government prohibits." Edelman, of Harvard's Berkman Center, suggests that Google find a way to alert users that information is missing from their search results. "If Google is prohibited from linking to Stormfront, they could include a listing but no link," Edelman said. "And if they can't even include a listing for Stormfront, they could at least report the fact that they've hidden results from the user. The core idea here is that there's no need to be secretive."
For everything Internet-related, from the latest legal and policy-related news, to domain name updates, see ZDNet UK's Internet News Section.
Have your say instantly, and see what others have said. Go to the ZDNet news forum.
Let the editors know what you think in the Mailroom.
Talkback
Excluding hate sites is an excellent idea. However consumer complaint site against corporations should never be excluded.