Search questions

Virtually attending LeWeb08, I got to listen to a presentation by Marissa Meyer on the future of search [http://www.techcrunch.com/2008/12/10/marissa-mayer-at-le-web-the-almost-complete-interview/].
Of course, I know very little about search, but somehow, I find the idea that personalization will be core to the future of search controversial, in a good way.

First, I doubt the personalization of search results will be based on what general search engine knows about the searcher. The reason is that, contrary to many other activities you would do on the Web, searching is fully horizontal. The motivations for searching are different at different points in time, and the same keywords may be used to look for information for very different purposes, resulting in very different documents being relevant.
How is that context captured in an horizontal search? I find it very difficult to imagine that the context can be built up from an analysis of the interactions the user has with the general search engine. Constructing the context will require many other sources, and it's of course possible that Google (Merissa Meyer's employer) is thinking about levering all their assets - including mail, blogs, instant messaging, etc.
Part of the missing context is related to the fact that a lot of the interactions the user has with the environment takes place outside those assets themselves. Reading the resulting documents, following links provided by those documents. It's frequent that in my long web sessions I only spend small time with the generic search and lots of time following paths maybe originating with the search results but mostly happening outside search. Yes, I know the search engines know about those links, but they do not know what paths the users follow through them.

But second, and this is key, generic searches do not have access to the most important contextual information: why is the user making a given search at any given point in time, what is his/her the motivation? This is much more important than we may originally think. Generic search engines, lacking that information, will provide at best the 'most popular' (by some measure) interpretation of the search - and that may or not correspond to what the user is looking for.

Why is this important? Alain Rappaport who founded Medstory [http://www.medstory.com/] and sold it to Microsoft once explained to me his view - which I find compelling - that there is a sweet spot of high value searches that correspond to 'motivated searches', searches that are made for a specific purpose, which, if identified, allows to organize the search and the results in significantly more relevant ways. For example, using Medstory, users can look for information related to medical issues in a system that establishes semantic links to connected issues and provides you with highly relevant information, even though it may not occur to the user that the connections existed. The user types "Heart attack" and gets, in addition to a Google-style list of results, a top bar that provides semantically related search refinement capabilities, etc. The site no longer provides ads (it has not been updated otherwise since the acquisition a couple of years ago), but you can imagine how targeted the ads could be.
I believe in this. I believe that in the short term we will see the emergence of verticalized search engines that will follow Google's model but will set up a context for the search and will be able to provide much more relevant results than generic search engines.

The next step may be semantic search and semantic analysis ad management (such as what Peer39 is trying to do - [http://peer39.com/]). That may be, but I continue thinking that extracting the search context will be too complicated and error-prone. It is probably much more useful (and achievable) to provide explicit verticalized searches and make them as easy to use and relevant as possible.

Maybe that's what Marissa Meyer meant, and Google's (and Microsoft's) forays into the health world may indicate so.