inset
Windows Compute Cluster Edition Released
Jul. 3, 2006

An edition of Windows Server specifically geared to high-performance computing (HPC) clusters—groups of identical, low-cost servers usually running the same application—was released to manufacturing in June 2006 with general availability anticipated for Aug. 2006. Windows Compute Cluster Server (CCS) 2003 aims to provide a cost-effective, Windows-based solution for HPC clusters in organizations with Windows infrastructures. However, the product shows signs of being a first release—for example, while it does work with new server-based features of Excel, it uses some deprecated server technologies.

High-Performance Computing

HPC systems speed performance of compute-intensive applications by applying large numbers of processors working in parallel. HPC has its roots in academia and research, particularly physical sciences and engineering, where it has been used for applications such as weather simulation and high-energy physics data analysis.

However, HPC is not strictly an idiosyncrasy of academic and research fields. Over the past 10 years, commercial demand for HPC has grown to the point where more than half of the 500 most powerful computing systems (as reported on the site www.top500.org) support nonacademic applications, serving purposes such as quantitative analysis of financial data and analysis of seismic information for oil exploration.

HPC can refer to high-performance computing in several forms, including the following:

  • Parallel vector processing, massively parallel processing, and other designs in which processors connect through a custom high-speed network
  • Grid computing or resource-scavenging, which uses idle desktop computers to analyze data or perform calculations
  • Inexpensive HPC clusters or networks of workstations in which smaller, dedicated servers communicate using standard high-speed network technology, such as gigabit Ethernet.

Windows CCS focuses on HPC clusters, aiming to meet an emerging demand among Microsoft customers for a cost-effective, Windows-based solution for HPC clusters and to take share from the leading HPC platform, Linux. HPC has historically been dominated by Unix, whose tools and development environment are largely mirrored by Linux. Consequently, Linux has captured a large share of the HPC market as legacy Unix systems are upgraded to lower-cost Linux-based clusters. Microsoft was late in both entering this market and delivering its first product. Consequently, without a strong HPC entry Microsoft could see customers who are using Linux-based HPC adopt Linux for other computing applications, giving Linux a toehold in organizations that otherwise use a Microsoft-based computing infrastructure.

Windows Compute Cluster Server 2003

Microsoft's approach with Windows CCS is similar to the approach it initially took with the server market in general. Coming to the market late, Microsoft is trying to convince customers that its offering is simpler to deploy, manage, and use by integrating CCS with other Microsoft products and working with partners and academic institutions to ensure that a variety of hardware and software works with it. (For a diagram showing how CCS works, see the illustration "Windows Compute Cluster Server 2003 Architecture".)

Simplified cluster deployment. CCS uses automated setup routines that require minimal user input. For example, it can use Windows Server Remote Installation Services (RIS) for additional node installation and to ensure that setup is fast and consistent from node to node. Because the job scheduler and message-passing interface (MPI), which are used to manage the distribution of work across the cluster, are integrated in CCS, customers can deploy nodes quickly.

Simplified cluster management. CCS works with existing Windows management tools, such as the Microsoft Management Console (MMC), to give administrators access to CCS management and configuration tools, and with Active Directory to simplify authentication and security setup, which makes it easy to authorize who can submit jobs to the HPC cluster. This should allow experienced Windows Server customers to quickly adapt to managing CCS. Organizations that have invested in advanced Microsoft management tools, such as Systems Management Server (SMS) or Microsoft Operations Manager (MOM), will be able to use those tools for managing updates to the nodes or performing CCS management.

Broad application support. A large number of vendors with applications that target the HPC market, such as Fluent, Livermore Software Technology, MathWorks, Schlumberger, and Wolfram Research, have adapted their software to support CCS. CCS will also support the new server features in Excel 2007, allowing compute-intensive calculations to be offloaded from a desktop to one or more nodes running CCS.

Familiar development environment. Microsoft Visual Studio includes a parallel debugger that makes it easier to debug applications designed to run in parallel across multiple nodes. Microsoft's parallel debugger may reduce the number of instances where developers of applications for HPC still debug by embedding print statements in order to expose application state and the value of variables during execution.

Release Timing Creates Problems

Because CCS is being released between two major editions of Windows Server—Windows Server 2003 and Windows Server "Longhorn"—the first edition of CCS uses some technologies that are deprecated and scheduled to be replaced in the near future. Generally, CCS is based on the x64 code of Windows Server 2003, (this edition included all of the updates in Windows Server 2003 SP1), which explains why it bears the name CCS 2003 even though it is being released in 2006.

Although Microsoft promotes Windows' RIS for CCS deployment, RIS is scheduled for replacement by the newer Windows Deployment Service about the time that Windows Vista ships (late 2006). WDS uses Microsoft's new image-based installation formats and tools, which are not backward compatible with RIS.

In addition, the current command-line interface for CCS is not based on Microsoft's PowerShell (previously code-named Monad), a new shell that offers an improved command line and scripting interface to Windows and server applications. PowerShell is scheduled for release in late 2006 and will be the preferred mechanism for Microsoft applications to expose administrative capabilities for command line, scripting, and even graphical interfaces. Because PowerShell will be Microsoft's preferred scripting environment, customers may want to limit the number of scripts they create based on the current CCS interfaces.

One area of Microsoft research and development that could benefit the broader HPC community is its investment in Message Passing Interface-Channel 2 (MPICH2), a popular job-scheduling interface from Argonne National Labs. Microsoft has based its implementation of MPI for CCS on the MPICH2 open-source code and has indicated that it will put the changes it made to this interface back into the open-source community.

Availability and Resources

Windows Compute Cluster Server 2003 will be available in the volume license channel for an estimated price of US$469 per node, but prices will vary depending on license and volume agreements. Licensing for CCS will be restricted to computers that are used as dedicated computational servers only. Customers wanting to use compute nodes for other purposes, such as using a cluster head node as a database server, should purchase the appropriate version of Windows Server 2003 x64 Editions (Standard, Enterprise, or Datacenter) for installation on those nodes.

Microsoft will be placing sales personnel with HPC expertise into geographic locations that service industries with a focus on HPC, such as automotives, life sciences, or oil and gas.

An evaluation version of Windows Compute Cluster Server 2003 can be downloaded from www.microsoft.com/hpc.

An overview of Windows CCS is available at www.microsoft.com/windowsserver2003/ccs/overview.mspx.

A FAQ about Windows CCS, including both developer and licensing information, is available at www.microsoft.com/windowsserver2003/ccs/faq.mspx.