By Peter Pawlak [bio]
Posted: Feb. 7, 2005

• Illustration: Exchange and Outlook Overview

The following is the full text of an article published by Directions on Microsoft, an independent research firm focused exclusively on Microsoft strategy & technology. Each month we make one or more key articles available to non-subscribers.

Although Outlook will also get a new update at the same time as Exchange, the Office team has not yet announced its plans for the product.

(For a graphical view of the Exchange and Outlook roadmap timeline, see the chart "Exchange and Outlook Overview".)

Service Pack 2, Tools in 2005

No major releases are planned for Exchange in 2005. However, a number of new features will be delivered in 2005 as part of Exchange Server 2003 SP2:

Intelligent Message Filter (IMF) update. First introduced in 2004, SP2 will update this server-side spam filtering technology to adjust its filtering methods (heuristics) to better detect the most recent patterns of spam. It will also include provisions for allowing administrators to block or allow messages from various domain names.

SenderID. SP2 will deliver Microsoft’s new open protocol for authenticating mail servers, which in turn enables companies to set policies that limit communications with unauthenticated mail servers. However, even though Microsoft continues to drive SenderID through the standards process, widespread industry adoption is by no means assured, which could ultimately limit its usefulness. (See "Antispam Standard Stumbles on Patent Issues" on page 35 of the Nov. 2004 Update.)

Improvements for mobile devices. SP2 will also include improvements for managing Exchange access by mobile devices. In addition, SP2 will include enhancements to Exchange and the ActiveSync protocol for synchronizing Exchange data with mobile devices running Pocket Outlook. However, to get these benefits, users may be required to purchase new devices based on the next version of Windows Mobile, code-named Magneto, unless their current vendors offer it as an upgrade.

Microsoft also plans to offer one or more Exchange management tools as free downloads during 2005. For example, the Exchange Best Practices Analyzer tool, a reporting tool that helps administrators identify flaws in their security configurations, will be updated, and a new version of the JetStress tool for evaluating the hardware configurations on mailbox servers will be released.

However, Exchange Edge Services, a separate server product originally planned for 2005, has been cancelled. Instead, that product’s planned functions—scanning and filtering messages flowing between the Internet and internal Exchange servers—will be subsumed in the next major release, Exchange 12. This move also eliminates a lot of potential complexities surrounding licensing and Software Assurance had Microsoft offered a new Exchange product.

Exchange 12 in 2006

Exchange 12 is currently planned for the second half of 2006. Major priorities include improvements for messaging security, support for new message types such as voice mail and fax, and simpler and scriptable management through a more modular architecture and new administrative interfaces. Exchange 12 will also introduce new, simplified programming interfaces that fit better with the company's popular developer tools and languages.

Security, Unified Messaging Major Priorities

A top priority for Exchange 12 will be improved messaging security. It will include integrated virus scanning and removal, based on an engine currently in development at Microsoft, as well as additional improvements to the IMF antispam technology. Exchange 12 will also include new APIs to support the enforcement of message archiving and retention policies, which are important in corporations to comply with disclosure regulations and legal discovery processes. Microsoft does not intend to provide archiving and retention functions out-of-the-box, but will try to line up software vendors to develop add-ons for these functions based on the new APIs. However, the inclusion of built-in antivirus scanning will be a blow to partners such as McAfee, Symantec, and Trend Micro, who currently offer add-on antivirus scanners for Exchange.

Exchange 12 will support server-based encryption and decryption of messages, enabling companies to protect mail from eavesdropping without configuring every client for Secure MIME (S/MIME) and distributing the necessary keys to users, which is required with current versions of Exchange and Outlook.

The product will also add functions for "unified messaging"—combining traditional e-mail with functions for sending and receiving voice messages and faxes to and from users’ Outlook clients—that were heretofore only available through add-on third-party products. To support unified messaging, the Exchange group is developing gateways to the most widely used types of Private Branch Exchange (PBX) telephone systems and enlisting third parties to support the less common types of equipment.

Modular Architecture, Scripting Aid Administration

Beginning with Exchange 2000, organizations could scale Exchange by configuring servers in a two-tiered architecture: front-end servers that service both Outlook Web Access (OWA) Web clients and remote clients using the Post Office Protocol (POP) or the newer Internet Message Access Protocol (IMAP); and back-end servers that host mailbox and public folder storage. Exchange 12 will continue the trend toward modular design by allowing organizations to designate Exchange servers in additional roles, and to install and configure only those components required to play the chosen roles. The new roles include the following:

• Edge servers that inspect content
• Bridgehead servers that route messages
• Unified message servers that interface with PBX switches
• Mailbox servers that store individual data
• Public folder servers that store shared data
• Client access servers that provide access points to mail clients and programmatic access to other applications that integrate with Exchange.

This modularity has several benefits for customers: it can speed deployment by reducing the complexity of the setup process, and it can make hacker attacks more difficult because only necessary components are installed and running on a given server, reducing the attack surface. Greater modularity can also lead to better performance by eliminating unneeded components from each server, and strengthen scalability, as companies can add additional servers for the server roles most in demand.

To help automate management of Exchange installations, Exchange 12 will introduce a completely new scripting interface based on Microsoft’s upcoming next-generation shell scripting engine (code-named Monad), enabling administrators to write scripts for routine operations that today are impossible or very clumsy to do. This includes tasks such as adding and removing users, moving mailboxes, and recovering lost data from backups. A new administrator's console for Exchange will use this same scripting interface, ensuring that any operation that can be done at the console can also be automated by a script.

Many other improvements are planned for Exchange 12 that could reduce downtime and improve scalability and management. Among the most important: Exchange 12 will support a new log file shipping mechanism to enable frequent periodic backup of live servers (in the realm of every five minutes, rather than the typical nightly backup performed on previous versions), speeding recovery time and reducing data loss in the event of server failure. Exchange 12 will also be able to take advantage of x86-64 processors (originally developed by AMD and now coming from Intel as well), enabling them to exploit large amounts of memory to speed response times and handle the large messages required for unified messaging.

Web Services, Managed APIs for Developers

Development on Exchange can take two different forms: independent server applications (such as customer relationship management systems) that can read or write Exchange e-mail, calendar, contact, and task data; or extensions that enhance the feature set of Exchange (such as message archiving systems).

Exchange 12 will introduce a new Web-service-based API that will make it easier for application developers to access Exchange data and that exploits Visual Studio’s many useful tools for working with Web services. The Web service API will eventually replace the confusing array of Windows APIs for accessing these data , which includes Messaging Application Program Interface (MAPI), Web Distributed Authoring and Versioning (WebDAV), and Collaboration Data Objects—Exchange (CDOEX), among others. CDOEX in particular was designed specifically to give application developers complete access to Exchange data, but it was never popular because it required the application to run on the same server as Exchange, which created scalability, stability, and management problems. The Web service model removes this requirement, and even allows the calling application to run on non-Microsoft platforms, such as IBM's WebSphere application server.

For developers writing Exchange extensions, Exchange will also expose a new managed code API that will allow developers to use .NET programming languages such as C# and Visual Basic .NET and will be much easier to work with than the current Exchange APIs.

Both new APIs are for server-side applications only; MAPI remains the primary API for communication with Outlook clients, and Exchange will continue to support POP3 and IMAP clients as well.

Exchange 12 Implications for the Future

Exchange 12 marks two important shifts in direction that will influence future versions of Exchange, as well as other products. First, Exchange will focus more tightly on e-mail-based collaboration and communication, relegating other forms of collaboration to other Microsoft products. Second, the product’s architecture is likely to remain technically stable at least through 2008, which could simplify future migrations.

Focusing on Messaging

Historically, Exchange public folders, which provide storage of group mailboxes, discussions, calendars, contact lists, and the like, were an important way for groups to collaborate on shared data; for example, managing the preparation of legal documents. Now, Microsoft recommends SharePoint technologies for this type of work. Exchange’s public folders feature is not going to be developed any further, and Exchange 12 will no longer have any dependencies on public folders for core Exchange functions, such as free/busy lists. This means that installing public folders will be optional, and the new Exchange 12 APIs will probably not support access to them.

In general, Exchange is being repositioned as Microsoft's platform for "store-and-forward communication," such as e-mail, voice mail, and fax, where users do not receive immediate acknowledgement of the communication. Synchronous communications, where users communicate with one another in near real-time, such as instant messaging and telephone calls, are the realm of Live Communications Server and Live Meeting. Finally, group work with shared documents and data will be the province of Windows SharePoint Services and SharePoint Portal Server.

Exchange Technology Stable

Exchange 12 illustrates another important point: the product’s basic technology is remaining stable. For example, MAPI/remote procedure call (RPC) remains the main protocol for communications between Outlook and Exchange, even though Exchange 12 supports new access methods such MAPI/RPC-over-HTTP and the Web service APIs. Similarly, the Exchange mailbox store is remaining on its current database technology (called the Extensible Storage Engine, also known as JET), although it will be ported to the x86-64 64-bit processor architecture. (Porting JET to 64-bit will likely need to be done anyway to support Active Directory natively on x86-64 systems; Active Directory is also built on JET.) This contrasts with the company’s plan circa 2002, which was to move the mailbox store to a completely new database technology based on SQL Server and gain 64-bit support as part of that move.

However, although not a full move to SQL-based technology, Exchange 12 will contain some SQL Server technology for the first time. In particular, it will use an embedded version of SQL Server 2005 to host data queues on servers configured in dedicated roles as edge servers or bridgehead servers, and it will replace the current mailbox/public folder indexing and search engine with SQL Server 2005’s full text indexing engine, which Microsoft claims will greatly improve search performance.

By keeping Exchange technically stable, Microsoft is simplifying future migrations. For example, it’s likely that customers will be able to move to new Exchange versions independently of Outlook versions, just as companies can do today. Similarly, customers should be able to move from Exchange 2003 to Exchange 12 without migrating data, avoiding a problem that has cropped up with other products (such as SharePoint Portal Server) that have made fundamental changes to their storage technologies.

Long-term, Exchange will probably evolve to take advantage of new Windows technologies such as the WinFS advanced file system, the Indigo messaging system for Web services, and the WinFX APIs. However, it is unlikely to do so until at least one version after Exchange 12, which would arrive in 2008 at the earliest.

Outlook Roadmap Still a Mystery

Compared to Exchange, Outlook’s roadmap remains vague. Outlook 12, the next version, will likely appear with Office 12 in 2006 at about the same time as Exchange 12. It will support Exchange 12’s unified messaging feature, and its provisioning and configuration will get easier because of new Active Directory extensions and Group Policy objects that will eliminate the need to do any client-side configuration. Calendaring will also get a major makeover, requiring changes to both Outlook and Exchange, but to date Microsoft has disclosed few details on how it will be improved.

As noted earlier, it will probably be possible to migrate to Exchange 12 without migrating to Outlook 12 and vice versa, but some new features (such as unified messaging) will probably require migrating to both before users get the benefits.

Little New for Hosters

Interestingly, there appears to be little on the Exchange roadmap aimed at meeting the special needs of providers of hosted Exchange messaging services. These hosters need to serve large numbers of typically small customer organizations without dedicating servers, even virtual servers, to each one. At the same time, the service still needs to provide the following:

• Directory partitioning, so that each customer organization can have its own Active Directory, secure from other customers, that provides a central address book and user authentication and authorization services
• Storage partitioning, so that each customer organization’s message database is completely secure and independent from those of other customers, thereby allowing mailbox restoration without affecting other customers
• Delegated administration and provisioning, so that administrators from customer organizations can perform normal Exchange administration activities for their own organization (such as managing user and group accounts) without requiring assistance from the hoster, while being blocked from making configuration changes that could affect other customers.

Microsoft has supported Exchange hosters for many years. It made changes in Exchange 2003 to better meet their needs and created a solution offering that gives them additional prescriptive guidance. However, Exchange 2003’s underlying technology limits the degree to which Microsoft can meet the conflicting goals of hardware (and license) optimization while maintaining service isolation and low support overhead.

Exchange 12 does little to resolve these issues, and Microsoft no longer considers tailoring Exchange for hosters to be a major priority, as this need is met by other products: individual consumers can get the benefits of Outlook, including support for group calendaring and contacts, with Microsoft Office Outlook Live and MSN Hotmail Plus services, and Microsoft believes that most small business will find Small Business Server, which includes Exchange and Outlook, more attractive than hosted Exchange.

Resources

For detailed information on Exchange 2003 and Outlook 2003, see the July 2003 Research Report, "Exchange Server 2003, Outlook 2003 Enhance Mobility, Scalability, Security."

Information on hosted Exchange 2003 is available at www.microsoft.com/serviceproviders/hostedmessaging/hosted_exchange.asp.

More information on Microsoft Office Outlook Live and MSN Hotmail Plus is available in "Hotmail Subscription Includes Outlook" on page 28 of the Feb. 2005 Update.