• Chart: 2007 Common Engineering Criteria Additions
• Chart: MOM Management Pack Criteria

The Common Engineering Criteria (CEC), which encourages common design, documentation, and licensing goals for Microsoft's server products, were updated in June 2006 to cover products that will be released in the 2007 product year (July 2006 through June 2007). Microsoft promotes the CEC as a serious engineering-driven program designed to improve the interoperability and lower the cost of deploying and maintaining Microsoft's server products, but the way it publishes the criteria and the number of exemptions granted in the past may have given the program an unfortunate marketing rather than engineering slant.

Common Engineering Roadmap and Criteria

The CEC are the result of a development and evaluation process designed to reduce the total cost of ownership of Microsoft server products and ensure that the products operate well with one another. The criteria include a mechanism for publishing the progress that each product group is making toward meeting the criteria, but this information is sometimes unclear about which server products fall under the CEC, how stringent the criteria are, and how easy it is for a product to gain an exemption for any given criteria.

Servers Covered by the CEC

All of the servers in the Server and Tools division, such as SQL Server, Operations Manager, and BizTalk, have to meet the CEC. In addition, some servers from the Business Division, including the forthcoming SharePoint Server 2007, Office Project Server 2007, Office InfoPath Forms Server 2007, Office Search Server 2007, Live Communications Server, and Exchange Server 2007, also have to meet the CEC. Microsoft Business Solutions products, such as Dynamics, are now being added to the program.

In principle, every server product edition should pass all applicable criteria for the year in which it releases to manufacturing, and if it does not, the product team should explain why it doesn't. For example, Office Project Server 2007, which should ship before the end of 2006, must meet all of the 2005, 2006, and 2007 criteria or apply for an exemption and then provide an explanation for any noncompliance. By the same rule, BizTalk Server 2006 must comply with 2005 and 2006 criteria, but Virtual Server 2005 need only comply with the 2005 criteria. Each product group will need to comply with the criteria added in subsequent years the next time it releases a full edition to manufacturing. (Compliance is not required for a service or feature pack.) An R2 or interim release should comply with all CEC in effect when it releases to manufacturing.

Review Process

Review of a product team's compliance with the CEC occurs at several points in the product development cycle.

• The first review for compliance occurs at the first feature-planning milestone, when the team begins to finalize the features for the next release of the product
• The second review occurs when the product goes out for its first public beta
• A review also occurs annually during each server team's "Running the Business" (RTB) review—an annual review that should occur early enough in the design and development cycle of any major or minor (R2) edition so that any violations of the criteria can be dealt with
• Final review occurs when the product undergoes the "Ship Readiness" review that is part of the release management process.

Detailed Engineering Criteria

Despite the word engineering in the CEC's name, the criteria also cover documentation, training, support, licensing, and branding of server products. The original CEC, first released in 2005, defined 16 criteria for products branded with the "2005" moniker, such as SQL Server 2005.

The 2006 update added seven more criteria. This year, Microsoft added nine criteria, including Web Services adoption and support for IPv6. (For a summary of the new criteria, see the chart "2007 Common Engineering Criteria Additions".)

The main forum for publicly publishing the criteria is the CEC Report, a Web site that shows the criteria by year and the assessed score for each server product. Each product either complies or provides a link to an explanation of why the product was granted an exemption for any criterion.

Microsoft decided not to provide the full technical description and the metric for each criterion, because it assumes customers do not care about the details. For example, the public criteria state the following:

• All server products will have a Microsoft Operations Manager (MOM) management pack available at launch
• A product's management pack will be serviced on the same schedule as the core product
• A product's management pack will provide event and performance processing alerts, basic views that graphically map performance and event trending information, a state-monitoring view (green/yellow/red) for managed entities, and tasks.

The full internal requirements for management packs are more detailed. (For a summary of some specific requirements, see the chart "MOM Management Pack Criteria".)

Program Impact

Although Microsoft will not provide specific examples, it insists that products have been delayed and have not been allowed to ship until they provided a schedule to meet compliance, rather than merely being granted an exemption to meet a release schedule. How effective the program actually is will probably become clearer when the 2007-era products begin to ship.

When the CEC were introduced in 2005, most of the products that would fall under the criteria were already well into the design, development, and testing for the next edition. Given Microsoft's typical development cycle, a product scheduled to ship in 2005, for example, was likely designed and development had started in 2003. This meant that right from the beginning, the evaluation of the CEC was problematic, and in the first several years of the program it was necessary to grant exemptions rather than force product teams to go back and comply with the CEC.

Things are different now. The products shipping in 2007 were designed after the CEC were created, and their teams had more than sufficient time to accommodate the criteria in their plans and schedule. Therefore, if the program is working, the number of exemptions should fall off rapidly with the 2007 products. The reasons for exemptions should also become clearer: For example, Virtual Server might still be exempted from running under Virtual Server.

Microsoft could also enhance the program by publishing the full CEC, not just simplified versions. In particular, disclosure of the full criteria would allow other developers to follow them and thereby improve the quality of all server applications. Like the guidelines published for the Windows Server Logo program, the CEC could become valuable as design criteria even for third-party developers who don't intend to conform to all of them.

Resources

The Common Engineering Criteria for all three years can be found at www.microsoft.com/windowsserversystem/cer/allcriteria.mspx.

The Common Engineering Criteria Report (scorecard), which will be updated in the near future, is available at www.microsoft.com/windowsserversystem/cer/report.mspx.