• Sidebar: Spam Requires Multidimensional Solution
• Sidebar: Patent Policy Tested
• Illustration: Spam Growth Since 2002

A committee working on a standard for authenticating senders of e-mail, an effort that could reduce unsolicited e-mail, commonly called "spam," has disbanded largely because it was unable to resolve problems posed by Microsoft intellectual property licensing policies. The dispute illustrates the tension between Microsoft’s desire to protect its intellectual property and the need for wide adoption of Internet standards. As a result of the committee failure, only weaker technical measures against spam remain, and Microsoft may eventually need to relax its intellectual property licensing requirements in order to create critical mass for antispam technology.

Fighting Spam

The Internet’s most used service, e-mail, has also proven to be the least expensive way to put a commercial message in front of millions of people. Conservative estimates say that more than 50% of all e-mail messages are spam, and AOL alone blocks more than 2 billion such messages each day. (For a graph showing the growth of spam, see the illustration "Spam Growth Since 2002".)

Not all is mere advertising: phishing e-mails attempt to dupe Internet users into providing important personal information, such as bank accounts, PINs, or Social Security numbers. According to one study by Gartner, deceptive e-mails that harvest financial information, such as Social Security numbers and bank account numbers, cost U.S. users alone more than US$2 billion over the past 12 months.

Techniques for blocking spam, such as blocking all mail from particular domains or examining incoming mail for phrases commonly used by spammers, have not proven permanently successful against spammers, who have quickly developed techniques for bypassing the obstacles in their way. Even legislative efforts such as the U.S. CAN-SPAM Act have had little impact on the problem. (The difficulty of combating spam is discussed in greater depth in the sidebar "Spam Requires Multidimensional Solution".)

Technical Solutions

However, a basic rule of successful spamming—conceal the identity of the sender so that ISPs, police authorities, and offended users cannot counterattack—can be used against it. If an e-mail sender can’t prove that the e-mail is from an authentic e-mail address, he is probably sending e-mail for which he doesn’t want to take responsibility, and that e-mail can either be rejected or flagged and isolated from other mail in a user’s inbox, making it easier for users to identify and delete spam.

That’s the idea behind several different efforts to fight spam with sender authentication protocols, including Sender Policy Framework (SPF), developed by pobox.com, and Microsoft’s Caller ID for E-mail, as well as calls to completely replace the Simple Mail Transport Protocol (SMTP), which transports e-mail over the Internet, but which has no useful facility for authenticating e-mail senders.

However, SMTP is so entrenched as an e-mail protocol that a workable replacement will take years to develop and propagate to all Internet users.

In the meantime, the open-source SPF protocol has attracted a number of advocates, including AOL, and Microsoft’s clout in the e-mail world, as owner of Hotmail and vendor of Outlook Express, sold with every Windows PC, gives Caller ID for E-Mail instant credibility.

Sender ID

However, having multiple solutions to authenticate e-mail will not solve the problem: these solutions rely on the cooperation of both senders and receivers of e-mail, and multiple solutions would mean that sender authentication might not interoperate between e-mail servers, technically known as message transfer agents (MTAs).

In spring 2004, the Internet Engineering Task Force (IETF) established the MTA Authorization Records in DNS (MARID) standards group to bring together a half-dozen proposals and create a single standard that could be used to authenticate e-mail senders.

By August the group had come up with a tentative standard, called Sender ID, which combines three types of tests to e-mail: SPF, Purported Responsible Address (PRA, the key component of Microsoft’s Caller ID for E-mail), and Submitter Optimization.

SPF requires that an organization publish a special record, stored in the Internet’s Domain Name Service (DNS) servers, that specifies which computers on the Internet can send mail as a user from a given domain, such as aol.com. The record does not contain addresses of users’ computers, but those of an organization’s SMTP mail servers. A mail server receiving an e-mail from aol.com can check the IP address of the sending mail server; if it is not listed in AOL’s SPF record, then the message has not been authorized by AOL and can be rejected.

PRA also requires that records be published in DNS servers, and it does a more intensive check of the e-mail to determine who is claiming to be the sender and where the e-mail originated. A mismatch between the claimed sender and the actual sender results in rejection of the e-mail.

Submitter Optimization is an optional approach in which PRA information is included in the header of the e-mail, which is exchanged when e-mail servers negotiate the transmission of e-mail. If the e-mail fails this test, the receiving e-mail server rejects the e-mail before it ever leaves the originating e-mail server.

Hung Up on Patents

The MARID process ground to a stop in late August, however, when Microsoft said that certain aspects of PRA were covered by patent applications the company had submitted to the U.S. Patent and Trademark Office (USPTO). Such disclosures are required as part of the IETF standards process, and while the IETF sometimes incorporates patented technology in its standards (typically, the patent owner grants a free license to any organization that uses patented technology to implement the specific standard), the MARID group was unable to agree on the inclusion of the Microsoft patents.

Initially, Microsoft did not disclose what the patent applications covered—patent applications are not normally made public until 18 months after they are filed—which made it difficult for the working group to determine which aspects of Sender ID were covered by the patents.

Microsoft also said that anyone incorporating Sender ID into their products would need to describe their product to Microsoft and obtain a license, for which the company would not charge. This approach, said open-source advocates, put an unacceptable barrier in the way of open-source developers, who do not want to disclose works-in-progress to Microsoft and whose source code must be usable by other parties without requiring those parties to obtain additional licenses.

Greg Stein, chairman of the Apache Software Foundation, which makes the popular open-source Apache Web server, said Microsoft’s requirements were "generally incompatible with open source, contrary to the practice of open Internet standards," and specifically incompatible with Apache’s open-source license. In addition, U.S.-patented technologies are covered by export restrictions that may not be acceptable to non-U.S. entities.

When Microsoft finally published the details of its patents, one appeared to cover a very broad range of Internet technology, including communication between e-mail servers to authenticate messages, and ways to classify incoming messages based on content. Some experts said the patent is broad enough to cover not just PRA but almost every other authentication scheme, including SPF, as well as a broad range of existing spam filtering technologies.

The committee was unable to continue, given resistance from some members to Microsoft’s licensing requirements, and at the end of Sept. 2004, the committee ceased further work on Sender ID.

Microsoft appeared to be surprised by the reaction. Company spokesperson Sean Sundwall said the patents were filed as a defensive measure so that no one could come along after Sender ID was approved and claim a patent on the technology it included. "Our intentions are 100 percent pure," he said: Microsoft is working hard to come up with broadly acceptable and effective measures against spam, using technology for which it intends never to charge a fee.

Moving Forward with Multiple Standards

The patent controversy appears to have dampened enthusiasm for widespread adoption of Sender ID, but not efforts to defend against spam. Several technologies are already in use or close to adoption by various players in the e-mail field.

AOL, for example, has elected to not proceed with the last (but unofficial) version of the Sender ID specification, but will instead stick with SPF, using it to check inbound e-mail.

Sendmail, which makes a commercial version of the open-source Sendmail server, is testing Sender ID, along with other technologies (although the company has not applied for a license from Microsoft because the patents in question have not yet been granted). Sendmail is also testing DomainKeys, a Yahoo proposal (that incorporates Yahoo-patented technology) in which digital signatures from e-mail servers are used to verify the identity of the sending system. (Yahoo has submitted DomainKeys to the IETF for ratification as a standard.)

Microsoft itself plans to publish both SPF and PRA records for Hotmail and other Microsoft-owned e-mail systems so that systems receiving e-mail from these Microsoft domains can use these types of records to authenticate senders, but Microsoft will check only for PRA on inbound e-mail.

Smaller ISPs and corporate e-mail systems gained little guidance from the wreckage of MARID. They may have to wait and see whether Microsoft’s clout as the owner of Hotmail and vendor of the popular Outlook e-mail clients and the Exchange e-mail server gives the company the ability to establish its preferred solution as the best solution for everyone, in effect turning a Microsoft solution into a de facto standard, regardless of concerns about patents.

The controversy also casts a cloud over an effort at Microsoft to build up a much larger patent portfolio and to license those patents: even when the company offers licenses for free, these licenses can inhibit development of critical Internet standards. In turn, this can slow demand for the company's Internet-enabled products, and Microsoft may get a public-relations black eye over something that was never going to generate revenue for the company.

(For a more extensive discussion of Microsoft’s patent strategy in light of recent developments, see the sidebar "Patent Policy Tested".)

Resources

Microsoft’s patent strategy was described in "Patent Licensing Broadened" on page 23 of the Jan. 2004 Update.

Antispam white papers, Sender ID technical documents, and related information can be found at www.microsoft.com/mscorp/twc/privacy/spam_senderid.mspx.

The Apache Software Foundation’s objections to patent licensing requirements for Sender ID are detailed at www.apache.org/foundation/docs/sender-id-position.html.

Microsoft’s antispam efforts are described at www.microsoft.com/mscorp/twc/privacy/spam.mspx.

Sender Policy Framework is described at www.pobox.com.

The DomainKeys proposal is described at antispam.yahoo.com/domainkeys.