Home > Samples > Update > January 2005
  MSN Launches Search Tools    
   

[bio]

The following is the full text of an article published by Directions on Microsoft, an independent research firm focused exclusively on Microsoft strategy & technology. Each month we make one or more key articles available to non-subscribers.

Kicking its competition with Google into high gear, MSN has released a tool that can search hard drives, networked drives, and e-mail, and has launched a wide beta of its Internet search engine based on Microsoft (rather than third-party) technology. The desktop search tool leapfrogs Google's similar offering and shows that MSN can leverage technologies developed elsewhere within the company to respond quickly to threats. Although the desktop search tool is not suitable for corporate use, Microsoft promises similar offerings for businesses in the future. However, MSN still has work ahead before it can equal Google's strength in Web search.

The Threat from Google

The new search tools, released in public betas in Dec. 2004, are the first results from a new MSN Search team formed in spring 2003 under the leadership of Vice President Christopher Payne. Prior to that initiative, Microsoft did not view search as a high priority for MSN and instead let partners provide MSN's underlying search technology.

The main reason for the shift at MSN was fear that Web search leader Google was drawing consumers away from MSN sites and services—a major consideration, given MSN's declining dial-up user base and increasing reliance on advertising revenues. In addition, a Google desktop search tool released in Oct. 2004 gives the company a way to display keyword-targeted advertisements directly on users' desktops, which could draw advertisers away from MSN.

Beyond the direct threats to MSN, Microsoft saw some possible long-term threats from Google, such as the following:

Enterprise competition. Although its main business is selling advertising on its Web sites and through its affiliate channel (which accounted for more than 97% of its revenue in the most recent quarter), Google sells its search technology to enterprises as well. In particular, the company sells a Google Search Appliance, a combined hardware and software solution that enterprises can use to enable intranet or public Web site searches. This product competes with Microsoft's SharePoint Portal Server (SPS) and could give Google a channel to sell future products into the enterprise, such as tools for searching file servers or databases.

Hosted services threat. In less than five years, Google has emerged from start-up status to build one of the most useful, scalable, and well-publicized Web-based services. Given this track record, Microsoft had reason to believe that Google might build other successful hosted services—a suspicion confirmed when the company launched its Gmail Web-based e-mail service in Apr. 2004 with an unprecedented 1GB of storage per account. Left unchecked, Google could develop other services, such as calendaring, contact management, and storage for digital media files, that could cut into sales of Microsoft software.

Longhorn too late. Microsoft is aware of shortcomings in the search features built into Windows XP, and several times in 2003 it demonstrated its intention to fix these problems in its next OS, code-named Longhorn. However, Longhorn is not due until 2006, and in the meantime, Google was known to be working on its own desktop search tools. Therefore, Microsoft had to find a quicker way to deliver new search features, or risk ceding this important function to Google.

To slow Google's momentum, over the last 18 months MSN has engaged in a feature-for-feature competition, introducing everything from a news aggregation service (MSN Newsbot, which competes with Google News) to a personal Web page service (MSN Spaces, which competes with Google's Blogger service). In late 2004, MSN upped the competition in Google's core area—search—by releasing two beta products: a new version of the MSN Toolbar that lets users search information on local and networked drives, and a new Web search engine based on Microsoft technology rather than algorithms licensed from Inktomi (owned by Yahoo).

Toolbar Suite Offers Desktop Search

The MSN Toolbar Suite was introduced in Dec. 2004 as a successor to the MSN Toolbar, an Internet Explorer (IE) add-on that offered Web-based search and other Internet-related functions, such as pop-up blocking.

The new Toolbar Suite has all the features of the original Toolbar but adds a major new feature: MSN Desktop Search, which lets users search resources on their hard drives and networked drives. Once the new suite is installed, Desktop Search is available not only from IE but also from the Windows Taskbar and some versions of Outlook and Outlook Express.

MSN Desktop Search is based on the same technology used in SPS 2003 and some recent Office applications. Although MSN acquired Lookout—a small e-mail search company—in July 2004, and the Lookout cofounders joined the MSN Search team, MSN Desktop Search uses Microsoft-developed technologies to search Outlook, not Lookout’s Java-based open-source Lucene search engine. (Microsoft continues to offer Lookout for download, but is not supporting it.)

Desktop Search: Indexing

When installed, MSN Desktop Search automatically begins scanning the user's My Documents folder and e-mail store (the user must have Outlook or Outlook Express open) and indexes the data. Users with administrative privileges can manually change their Desktop Search options to index other local and networked sources and to ignore e-mail attachments. When users run a search, Desktop Search then looks for matches in the index.

Specifically, Desktop Search indexes the following types of information:

  • File information about all files (such as author, file type, and date modified)
  • Metadata associated with media files (such as song title and artist) and graphics files (such as a description of the file content)
  • Full text within Office 2000 or later files (Word, Excel, PowerPoint, and OneNote)
  • Full text within Adobe PDF files, although users must download and install a free add-on from Adobe called the Adobe PDF IFilter to enable this feature
  • Full text within all locally stored Outlook information (including e-mail messages, attachments, calendar entries, contacts, notes, and tasks)
  • Full text within all locally stored Outlook Express e-mail messages
  • Full text within MSN Messenger conversation logs
  • The IE Favorites menu (only the names of Favorites pages are indexed, not the contents of those pages).

Once the initial index is established, Desktop Search periodically checks all previously indexed folders for new and modified files and updates the index with this new information. The tool attempts to conduct these updates while the user's PC is idle, so as not to interfere with other tasks. However, Microsoft does not reveal exactly how often Desktop Search looks for new information, nor is this user-configurable.

Search Interface

Before conducting a search, users must choose the resources they want to search. This varies by search pane—for instance, the pane in the Windows Taskbar searches locally indexed resources by default, whereas the pane in Outlook searches only Outlook information by default. It's currently impossible to conduct a simultaneous search of both Web-based and locally indexed information with the MSN Toolbar, but this is likely to be addressed in future releases.

Desktop Search results appear in a new Windows Explorer window arranged by relevance (using the same relevance engine as SPS). The results can also be arranged by file type, date, author, and other criteria. Results can be filtered so that only specific types of information—such as e-mail messages, presentations, documents, or music files—show up.

For Web-based searches, the Toolbar opens a new Web window in the user's default browser (non-IE browsers are supported) and conducts a search on the new beta MSN Search site (described below).

Other Toolbar Features Include Form-Fill

In addition to Desktop Search, the MSN Toolbar Suite can automatically fill in Web-based forms with information preset by the user, such as name, address, e-mail address, credit card number, and password (users must set a password for the last two functions).

Like the original Toolbar, the Toolbar Suite continues to offer pop-up window blocking and the ability to highlight search terms within a particular page.

Not for Corporate Use

Microsoft does not recommend use of the MSN Toolbar Suite in corporate settings for several reasons.

First, to index files on networked sources, MSN Desktop Search must open and read files across the network, which could degrade network performance as many employees index the same resources simultaneously. Second, a separate index resides on each user's machine—there's no centralized index on a server—and takes up approximately 10% as much disk space as the material it's indexing. A naïve user who attempted to index a networked drive containing thousands of files could exceed their PC’s storage capacity. Third, the Toolbar Suite does not support Group Policy, and administrators cannot prevent users from indexing networked drives.

Therefore, in organizations where users are permitted to install their own software, IT should communicate clearly that MSN Desktop Search should not be used to search networked files, or forbid the installation of the MSN Toolbar Suite altogether. Organizations with Systems Management Server (SMS) might also want to use it to monitor whether employees have installed the Toolbar Suite.

For organizations interested in the kind of functions offered by MSN Desktop Search, Microsoft proposes SPS as a solution. The company also says it might integrate similar—but more IT-appropriate—search tools into future versions of Windows SharePoint Services (which comes with Windows Server) and Office applications.

Web Search Enters Second Beta

In Nov. 2004, Microsoft launched the second beta of its new Web search engine, which is built on technology that Microsoft developed rather than on technology licensed from partner Inktomi (Yahoo). Unlike the first beta, which ran for only a few weeks in summer 2004 and was mainly used to collect user feedback on the relevance of the results, this beta is more or less feature-complete and will be active until it replaces the current MSN Search site (expected in Jan. 2005).

To provide algorithmic results (that is, results arranged by relevance), the new search engine consults an index of more than 5 billion Web pages, collected and indexed since summer 2003 by a Microsoft technology called MSNBot and evaluated by a Microsoft-developed relevance engine. However, sponsored or paid results (in which advertisers bid for placement) are still provided by partner Overture (also owned by Yahoo), with MSN and Overture splitting the revenue. Although paid search is a lucrative and fast-growing business, Microsoft currently has no plans to bring it in-house, and in Dec. 2004, it extended its contract with Overture an extra year, through June 2006.

In addition to providing Web-based search results based on Microsoft technology, the new MSN Search site offers the following features:

Plain-language queries. The new site draws on the online version of Microsoft's Encarta encyclopedia to answer plain-language queries, such as, "What's the capital of Bolivia?"

Math. The site can perform many mathematical calculations, from addition and subtraction all the way up to trigonometric functions and algebraic equations. This feature supports both text-based and symbol-based queries (e.g., "three times twenty-four" and "3*24").

Search Near Me. This feature provides Web-based results near a user's location. When building its index, the MSNBot added geocodes to its indexed pages by looking for clues in the HTML (such as an address or phone number). When users conduct a search, MSN Search site approximates their location by their IP address, then delivers results based on that location. To better tailor location-based results, users can click the location link on top of the results page and change their location via a Web-based form. (MSN Search uses a cookie to record this information for the next time the user visits.)

Search Builder. This feature lets users narrow their searches by limiting results to a particular Web domain, country or region, or language, for instance. It also offers a result-rankings screen that lets users adjust three sliders to favor results based on certain parameters, such as how recently a page has been updated. (For an illustration, see "Search Builder Tailors Results".)

Can Microsoft Win?

Based on what it has released so far, it's clear that MSN plans to use the company's strengths, including technology that's already been developed for other products and its experience in interface design, in the quest to beat Google.

This shows most clearly in the two companies' desktop search tools: MSN Desktop Search offers more user control over the scope and timing of indexing, indexes more kinds of resources (including PDF files—made possible because the SPS team already developed the necessary APIs for the Adobe add-on), and gives users more granular results. In the long run, Microsoft's experience at building applications and its ability to integrate search technologies and interface features into the OS give it a clear advantage on the desktop.

Similarly, although MSN Desktop Search is not suitable for corporate use, Microsoft has more experience designing and selling corporate applications than Google. If Microsoft can rationalize and clearly explain its different search technologies to businesses, and provide clear roadmaps as to how these technologies are evolving, it should be able to slow Google’s progress in the enterprise.

However, Google still has the edge in Web-based search: on the day that the MSN Search beta was released, Google announced that it had upgraded its index from 4.3 billion to more than 8 billion pages. More important, casual tests (and nearly all independent reviews) suggest that the new relevance engine developed for MSN Search is not yet on par with Google's; it will have to be noticeably better than Google's before users will switch to MSN Search as their first choice. Google's desktop search tool is also better integrated with the Web than MSN's equivalent—the Google tool allows users to search their Web history (recently visited Web pages) and displays the results along with other local information.

In summary, although the first products from MSN's renewed search effort may slow Google's progress into traditional Microsoft business areas, they will have little effect on Google's popularity on the Web.

Resources

Microsoft's page covering the new MSN Toolbar Suite, including more information and the download, is at beta.toolbar.msn.com.

To read about the original MSN Toolbar, see "MSN Toolbar in Beta" on page 22 of the Mar. 2004 Update; this Toolbar is downloadable from toolbar.msn.com.

The Adobe PDF IFilter, which is necessary before MSN Desktop Search can index PDF files, can be downloaded from Adobe at www.adobe.com/support/downloads/detail.jsp?ftpID=2611.

The beta MSN Search site is at beta.search.msn.com. In Jan. 2005, it will replace the current MSN Search site at search.msn.com.

Information about MSNBot, specifically geared toward Webmasters, is at search.msn.com/webmasters/default.aspx.

Google Desktop Search can be downloaded from desktop.google.com.

Lucene is part of the Apache Jakarta project, which maintains open-source Java-based applications and offers them for no charge. For more information, see jakarta.apache.org/lucene/docs.

Lookout information and the download are available from www.lookoutsoft.com/Lookout.