How Google Became the #1 Search Engine
No other tech company is more responsible for shaping the modern internet and modern life as we know it than Google. The project that started as a new search engine now holds more than 1 billion users in more than 8 products. Google has become such a pivotal part of the human experience in the recent decades that it’s a verb now, used not only to search the everything known to human knowledge, but also to communicate, perform, work, use media, and drive through the vast internet in 2018. In over 20 years, it’s become one of the most staggeringly successful corporations in the world. How did it become so?
Where It All Began
As mentioned in our previous post about Google’s origins, we mentioned it was actually known as BackRub initially. It began as a research project of Larry Page, a graduate in computer science program of Stanford’s class of 1995. It was where he befriended a fellow student, Sergey Brin and together, they started delving into the World Wide Web. Inspired by the academic practice of citing original works, Page combined it with mathematic algorithm and created a system that would crawl the internet to decide how the pages were linking to each other, positing that it could possibly result in a novel kind of search engine.
Congruent with Brin’s mathematics expertise, the duo created the PageRank algorithm to rank the pages in search results based on linking behaviour. This formed the base for the world’s most powerful search engine, first launched on Stanford’s private network in August 1996.
Motivated by the sheer and endless number of links between pages, and knowing how their search engine would dynamically keep growing with the web and becoming more accurate, Page and Brin renamed their project and company after the mathematical term googol - a one followed by 100 zeroes). They relocated to the garage of Susan Wojcicki, who later became the CEO of YouTube. The company became Google as that was the domain available and with a $100,000 investment from Sun Microsystems, they launched.
Understanding the Context
In his book The Innovators: How a Group of Hackers, Geniuses, and Geeks Created the Digital Revolution, Walter Isaacson recounts how Google's co-founders turned a school project into a company. The following is direct excerpt from his book:
The desire to turn their dissertation topic into a business made Page and Brin reluctant to publish or give formal presentations on what they had done. But their academic advisors kept pushing them to publish something, so in the spring of 1998 they produced a twenty-page paper that managed to explain the academic theories behind PageRank and Google without opening their kimono so wide that it revealed too many secrets to competitors. Titled "The Anatomy of a Large-Scale Hypertextual Web Search Engine," it was delivered at a conference in Australia in April 1998.
"In this paper, we present Google, a prototype of a large-scale search engine which makes heavy use of the structure present in hypertext," they began. By mapping more than a half billion of the Web's 3 billion links, they were able to calculate a PageRank for at least 25 million Web pages, which "corresponds well with people's subjective idea of importance." They detailed the "simple iterative algorithm" that produced PageRanks for every page. "Academic citation literature has been applied to the web, largely by counting citations or backlinks to a given page. This gives some approximation of a page's importance or quality. PageRank extends this idea by not counting links from all pages equally."
The paper included many technical details about ranking, crawling, indexing, and iterating the algorithms. There were also a few paragraphs about useful directions for future research. But by the end, it was clear this was not an academic exercise or purely scholarly pursuit. They were engaged in what would clearly become a commercial enterprise. "Google is designed to be a scalable search engine," they declared in conclusion. "The primary goal is to provide high quality search results."
This may have been a problem at universities where research was supposed to be pursued primarily for scholarly purposes, not commercial applications. But Stanford not only permitted students to work on commercial endeavors, it encouraged and facilitated it. There was even an office to assist with the patenting process and licensing arrangements. "We have an environment at Stanford that promotes entrepreneurship and risk-taking research," President John Hennessy declared. "People really understand here that sometimes the biggest way to deliver an effect to the world is not by writing a paper but by taking technology you believe in and making something of it."
What Made Google Better?
In all our research and analysis of Google and its history, we’ve narrowed down a handful of reasons why Google surpassed all the other search engines. It wasn’t just one singular reason though but rather a collective effect created by multiple factors.
2. Deeper Index: Since it was built on the academic thesis model, Google crawled deeper and broader than its competitors which lead to better and richer results. This was another thing they made transparent to the users on top of the page (e.g. ‘About 12,560,000 results’) to let the users know how much deeper their searches were.
3. Relevant Results: Due to the extensive indexing and crawling, Google’s results were much more relevant than their competitors. Their unique PageRank algorithm was one of the most important factors that decided this, along with other factors that went into their ranking algorithms.
4. Simple Interface: One of the most unique things about Google was how clean and minimalistic it kept its look during a time when competitors were cluttering and stuffing their UIs with banners, ads, and much more. It removed the distracting yellow box on top of the SERP and kept it simple. This ensured that the search was the only point of focus for the user, which they appreciated.
5. Specific Sections: It’s the little things that count and Google did one of those little things by being the first search engine that showed small sections or snippets on results to let the user easily know which part was relevant to their search. For users, this was tremendously useful and time-saving as it ensured they knew right away which results were relevant and which were not just by glancing at the results page.
6. Site Directory: As mentioned previously, the vast directory of pages and links have made Google the most extensive library for information in the world. It has over 25 million sites.
7. Focused Search: This is one of the most important reasons, if not the most. Google has been known for its search engine and for many years, that was its only product. This kind of singular focus on only one area led to wondrous results in product development. Even after it became the dominant search engine, Google didn’t lay back and bask in its laurels. It worked harder to improve and optimise the search engine and doubled the efforts. By investing heavily in things like building a smooth testing framework, funding in data tech, working out a monetization strategy that would not compromise the user experience, and much more, Google changed the game completely for everyone and set a new standard that became almost impossible to meet.
Beating the Competition
If we look from a historical standpoint, Yahoo was actually the first search engine on the internet when Jerry Yang compiled his favourite websites together on a platform, making it into a portal. However, Yang's list needed to be maintained manually and comprised of only broad topics, rendering it useless for users searching for anything new or specific. Yahoo did add a search bar in 1995 and implemented an algorithm but it still had multiple issues and made results irrelevant to the search.
As Google began to gain popularity, it started to provide Yahoo’s search engine as well in 2000. This led to Yahoo trying to acquire Google by the dawn of 2002 for $3 billion. Yet, Google turned down for, it is speculated, that it felt it was worth more, reportedly $5 billion.
Talking about finances, do you know how Page and Brin celebrated the funding they got initially? They went to Burger King. As Isaacson notes, they said, "We thought we should get something that tasted really good, though it was really unhealthy. And it was cheap. It seemed like the right combination of ways to celebrate the funding."
Puts success into perspective, doesn’t it?
Next: How Google Search Algorithm Works