History of Search Engines
You'll have a better feel for today's search engines if you know a little of the history behind them. Of course, there's not much history to cover, since the Internet is still a youngster! Let's take a look at how it all started.
We'll begin by clearing up a common misunderstanding. Did you ever wonder what the difference is between the Internet and the World Wide Web? Many people believe that the two terms can be used interchangeably. In actuality, nothing could be further from the truth.
The Internet is a worldwide network of computer networks. It was dreamed up in 1969 by a U.S. government agency called ARPA, or the Advanced Research Projects Agency. (In fact, the Internet's original name was ARPANET.) Nowadays, you can access the Internet via telephone lines, cable, fiber optics, and other communication mediums.
The Internet: A worldwide network of computer networks
The World Wide Web is younger still. It came into being around 1990, when researchers realized that they could use the Internet to connect a web of stored hypertext pages and make them accessible to people around the world. That web of pages grew from a handful at the outset to billions today.
The World Wide Web is, by far, the most commonly used application on the Internet. (E-mail is another application of the Internet that you're familiar with.)
All users of the Web utilize HTTP, or the hypertext transfer protocol, in order to navigate from one Web site to another. Look at the address bar near the top of your Web browser. You'll find that the www is preceded by http://. This tells the Web site that you're on the World Wide Web and that you're using the hypertext transfer protocol.
It's important to understand the difference between the Internet and the Web so you can fully grasp how search engines work. While search engines use the Internet, they don't search the entire Internet. Typically, search engines only search for Web sites on the World Wide Web.
The Search Engine Power Players, Yesterday and Today
For a young industry, the search engine field already has quite a past! Here's a quick summary of important points on the search engine timeline. If you're a real history buff, you'll find more details in the FAQs for this lesson.
1993: The first widely acclaimed search engine, the World Wide Web Wanderer, appears. Created to measure the growth of the Web, it performs its job through 1997. The statistics compiled by this search engine are still available on the Web today.
1994: WebCrawler comes on the scene. The original WebCrawler database contains just 6,000 Web sites. (I think I have more Web sites than that listed in my favorites!) AOL-now there's a name you probably recognize-purchases WebCrawler in 1995, but sells it just two years later to Excite. Infospace, its current owner, buys WebCrawler when Excite declares bankruptcy.
1994: Another powerhouse, the Lycos search engine, launches with 54,000 indexed documents. The Lycos search engine is still a player today, but it's changed hands several times. Currently, it's a subsidiary of the Korean-based Daum Communications Corporation.
1995: AltaVista explodes onto the scene. AltaVista (or AV) is the first search engine to include multilingual search capabilities. After changing hands several times, AltaVista becomes the property of Overture Services. (Overture is owned by Yahoo!, which we'll discuss a bit later.) AltaVista maintains its status as the search king until the launch of Google.
1998: Larry Page and Sergey Brin introduce the world to Google, which quickly shoots to the top of the search engine rankings. (The name comes from the word googol, which is the name for the number 1 followed by 100 zeros.) Google's database now includes trillions of Web pages, and most experts agree that Google enjoys greater than 50% of total search engine traffic. This means that a top Google ranking will yield more traffic to your site than a top ranking with any other search engine...period.
That's where things stand now, with Google by far the most used search engine on the Web.
However, things change rapidly, with companies constantly buying, selling, and creating search engines. As a result, you'll want to keep tabs on which search engines are gaining or losing popularity.
Search Engines versus Directories
Are you wondering why we didn't talk about Yahoo! in the last section? That's because Yahoo! began as a directory, not a search engine.
While search engines' indexes are compiled by computers, directories are categorical lists of Web sites compiled by humans. Before it's accepted, each Web site listed in a directory is carefully scrutinized and deemed acceptable for placement in one specific category.
Some people argue that because directories are compiled by humans, they're limited because they don't offer as many search results as search engines. However, I think you'll find that the search results displayed by directories can often be much more useful than the search results displayed by search engines.
The most popular directory on the Web is Yahoo!, founded in 1994 by David Filo and Jerry Yang. They started Yahoo! on a couple of computers in a campus trailer at Stanford, initially using it to track their own interests. It surprised them by taking off quickly, and they incorporated it in 1995 with an initial investment of almost $2 million. By the way, Yahoo! Is an acronym for Yet Another Hierarchical Officious Oracle-but I promise not to test you on that!
In an effort to diversify, Yahoo decided to use Google's engine to supply users with primary search results. However, in 2004, Yahoo! unveiled its own brand-new search engine.
You can still tap into Yahoo!'s directory, and I think you'll find it extremely useful at times. If you perform a search at http://www.yahoo.com, the results you'll get are derived from the Yahoo Search Engine. However, you can obtain Yahoo Directory search results by initiating a search at http://dir.yahoo.com.
The Open Directory Project is perhaps the second most popular directory on the Web. It's compiled by more than 50,000 volunteers who've indexed nearly 4 million Web sites. The directory contains just under 500,000 categories, all of which can be searched by keyword or category.
Due to the limited resources of the Open Directory, searching for results through their Web site can be time-consuming and cumbersome. However, some search engines include Open Directory listings within their secondary search results.