Search engines have changed the way we all search for, find, and make use of information on any topic. As the explosion of the internet gave way to the original search engines, the complementary field of search engine optimization or SEO was born.
SEO today is a complicated beast, but this wasn’t always the case. In this series, we will explore how SEO developed from humble beginnings to the serious industry it is today.
Websites Go Live
Before there was SEO, there were websites. The first website went live in 1991 and the number of websites grew to more than 1 million in 1997. For comparison’s sake, the number of websites in existence hit the 1 billion mark as of September 2014.
With the growth of websites came the need for some way to organize and make use of web data, and search engines were created. Originally, search engines were edited by humans, who were tasked with clarifying user search requests and returning relevant results, much like the reference librarian at your local library.
The First Web Crawler
In 1994, the original web crawler (named Webcrawler) was created to index entire web pages without human intervention. This also gave the world its first full text search engine. From an SEO perspective, this meant that web content could now be machine-read and referenced in search results.
This shift in technology led to an explosion of search engines, with dozens of contenders vying for use. In the 1990’s, searchers could choose from Yahoo, Excite, Lycos, AltaVista, Webcrawler, Ask Jeeves and many more.
Even though crawlers were available, some search engines (like Yahoo) were still edited by humans and only returned crawler results if there were no human-powered results for a search query. In 1997, Search Engine Watch first launched and gave industry pros and fans a place to discuss the art of search – and of search strategy, or what would become search engine optimization.
The Beginnings of SEO
Webmasters quickly realized that they could help their website get found by optimizing their sites for web crawlers, and thus SEO was born – even if it was not known as such until later.
Initially, search engine optimization was as simple as submitting the website URL to a search engine so that the information could get found. Humans (and subsequently crawlers) would then visit the website, review the information, and then gather relevant information about the site’s content and information. The crawlers relied on such information as web copy and meta html tags that purported to tell the crawlers what the website was about.
While humans could make informed decisions on context, the early crawlers were less intelligent. Thus, many webmasters simply repeated the same word or phrase on the page several times in an effort to seem more relevant to web crawlers. When it was so easy for website owners to stuff keywords onto a webpage to rank highly – and then receive business from a prime search result placement – why not do it?
For searchers, these tactics often obscured the very act of information gathering, as websites that were not very relevant or even very authoritative could still rank very highly in search results. Since there were so many search engines, early web users could also have very different results by using different search engines. While this made the process of information retrieval even more challenging for the user, it also led to many “happy accidents” where users stumbled across unique websites.
Ultimately, as the web grew, searchers demanded a better search engine and improved search results. These came together as SEO formalized, bad practices were identified and penalized, and best practices were developed; a process that continues today. These and other changes took place rapidly when Google launched in 1997 and made link building part of their search algorithm, a move that would have a dramatic impact on SEO strategy and on the process of search.
Learn more about how Google’s launch impacted SEO history in part two of our series.