Geospatial Semantic Web Blog

Search engines are essential to the success of the Web. Without search engines, our Internet experience would be crippled. Recently Ian Hendry wrote a blog post that speculates the usefulness of the search engines in the near future. He thinks that search engines are facing extinction.

Paradigm Shift

Ian’s premise is an interesting one — the emergence of specialized web sites will drive away many search queries from the general-purpose search engines like Google and Yahoo!. For example, if you want to lookup the education background of your colleague, you would go to LinkedIn and Facebook, and if you want to find out facts about John Locke, you would go to Wikipedia and Freebase.

People can often bypass the use of search engines because they know exactly which web sites to go to for finding information that they are after. The use of search engine plugins in Firefox is a good example. If a person uses the IMDB search engine plugin to lookup the actors in a movie, then the person bypasses the use of Google and Yahoo!.

What Causes the Paradigm Shift

Coming back to Ian’s theory, search engines are facing extinction in the future. I think the word “extinction” is a bit too extreme. It’s unlikely that search engines will disappear from the Web. However, they will no longer be the one-stop location for people to find information.

In the past, without specialized web sites, people have no choice but rely on search engines to find information. Today, the Web is gradually becoming a collection of independent islands of information (YouTube of videos, Facebook of people, Wikipedia of facts, etc.). People have choices in deciding where to send their search queries. “Not all search query are belong to Google”.

What’s Next for the Search Engine Companies

If my analysis is correct, then search engine companies that rely on ad revenues to operate will ask one question, how can we drive more traffic to our search engines? There are few different solutions to this problem. One, the company that can try to monopolize the general-purpose search engine space. Second, change the way people use search engines.

The first approach is easy to understand. Let’s think a bit about the second approach.

We use search engines because we want to find information. A typical flow of the process is the following: (1) We send a query. (2) A list of results is displayed. (3) Go through the first new pages on the top of the list and try to find information that we are after.

I think we spend most of our time in Step 3. If we can’t find the information we want in the first few pages, we repeat the process all over again.

Smarter Search Results

A solution to this problem is to make the result list “smarter”. For example, based on the user’s query intention, display the most relevant information on the top of the result page. When you search for “John Locke” in Google, the first few links point to books by or written about John Locke. This is a good start, but we can do better.

SearchMonkey is better solution to this problem. Third-party developers can introduce new ways to format search results and influence the search experience of the users. My favorite SearchMonkey use case is the LinkedIn public profile.

Concluding Remarks

I think Ian is right about the emergence of a paradigm shift in how we use search engines on the Web. As we become more familiar with and develop trust in specialized web sites, we can bypass the general purpose search engines to find information that we are after. In order for search engines to keep up with this paradigm shift, they must reinvent themselves. Google’s mashup search results and Yahoo!’s SearchMonkey are on the cutting edge of this new movement.