Google uses a method called split A/B testing to measure exactly what changes it should make to its main search website — both to its famously Spartan search box and to the results it produces.
With the approach, Google shows different versions of the pages to users and measures how they respond, said Marissa Mayer, vice president of search products and user experience, in a speech at the Google I/O conference in San Francisco on Thursday.
For example, Mayer said, the company wanted to find out how many search results to show users: the customary 10, or 20, 25, or 30? When asked directly, users said they'd like more results on a page, but testing showed otherwise.
Specifically, Google found that when the results increased to 30 per page, people searched 20 percent less overall, Mayer said. After much analysis of server logs, the company found it was because it took about twice as long to display the longer results list for the user, and speed matters.
"As Google gets faster, people search more, and as it gets slower, people search less," she said.
The same effect happened with Google Maps. When the company trimmed the 120KB page size down by about 30 percent, the company started getting about 30 percent more map requests. "It was almost proportional. If you make a product faster, you get that back in terms of increased usage," she said.
Split A/B testing also led Google to refine exactly how much white space to pad around its logo and other elements on the search results page. And it changed from the industry practice of a pale blue background behind ads to a pale yellow background. People not only clicked on ads more, they also searched more in general, she said.
The subject clearly is close to Mayer's heart. She's an engineer who also has an interest in the more aesthetic realm of design.
"On the web in general, [creating sites] is much more a design than an art," she said. "You can find small differences and mathematically learn which is right."
A history of Google's search page
Google's search page, with its abundance of empty white space and its almost boastful "I'm feeling lucky" button, looks downright ordinary today. But it wasn't always the case.
Mayer said that back when Google was a relatively unknown 80-person start-up, the company tested Stanford students on how well they could use Google to find which country won the most gold medals in the 1994 Olympics. The result: students would sit in front of the Google screen for 15 seconds, 30 seconds, 45 seconds, a minute... Google was perplexed.
Read this
Analysis: Google's search for business customers
A tie-up with Saleforce.com sees Google pushing even further into Microsoft's business apps territory
So Mayer would eventually intervene and ask what was holding up the searchers. "I'm waiting for the rest of it," they'd say. Clearly they expected more of the flashy ads and busy text of other search pages of the 1990s.
"The very first home page was that misunderstood. People didn't resonate with it," Mayer said. One woman even thought the website was a fake construction that was part of a psychology experiment.
As a result, the company put a copyright notice at the bottom of the page. "It's not there for legal reasons," Mayer said. "It's there as punctuation. That's it. [It tells the searcher] 'Nothing else is coming; please start searching now'."
Mayer oversaw much of Google's design, but the sparse start page wasn't her doing...
