• Sidebar: Server Virtualization Is Changing Datacenter Architectures
• Sidebar: What Are Intel VT and AMD Pacifica?
• Illustration: Current and Future Microsoft Virtualization Architectures

A second waypoint has been added in Microsoft's route toward a Windows Server product with built-in virtualization capabilities. One more stand-alone server version, in the second half of 2006, will follow Virtual Server 2005 R2, which came out in late 2005. After that, an entirely new virtualization architecture will be integrated into Windows Server after "Longhorn" Server (which is the next major release, due in 2007). All future Microsoft server virtualization technology will exploit upcoming virtualization support in AMD and Intel chipsets, and the roadmap illustrates how Microsoft thinks its customers will soon consider server virtualization a must-have OS feature.

This article deals with server-side virtualization only, but Microsoft also has a client OS virtualization product (Virtual PC) and a corresponding roadmap for it. Even though Microsoft's server and client virtualization technologies are similar, functional requirements and the rationale for using each are significantly different. (For information on Virtual PC, see "First Virtual PC Product Released" on page 11 of the Dec. 2003 Update.)

Virtualization Basics

In the context of computer systems, the term virtual is used in many different ways, such as in Java Virtual Machines or virtual memory. In this article, virtualization refers to a method of abstracting and decoupling a computer's physical hardware from a conventional OS (such as Windows or Linux), thereby allowing additional multiple OSs to run on the same computer at the same time without subdividing the physical hardware resources into separate partitions. In most of today's virtualization technologies, only the host OS (the first OS installed on the computer) is communicating directly with the hardware. All the other OSs (called guest OSs when run this way) and applications installed on them interact with virtual machines (VMs), which emulate the CPU, memory, I/O controller, graphics adapter, network adapter, and other peripherals. A physical server can run multiple VMs simultaneously.

The host OS creates the VMs, and manages and arbitrates each VM's access to the computer's physical resources. Because it is not tied to specific hardware, a guest OS running in a VM, with all of the applications and data installed on it, can be moved to a different machine by simply moving a single file that contains the OS and applications. (In Microsoft's products, this file is called a virtual hard disk, or VHD file). Because the VM is not tied to specific hardware, it does not need to be "installed" on the new machine in the traditional sense; the virtualization software simply has to be configured to load the new VM file.

Such capabilities are important for Microsoft's Dynamic Systems Initiative (DSI), which envisions, among other things, systems that automatically respond to events such as a failing server. A DSI-enabled system could quickly move a VM, including the full system state, to a healthy server.

Current Support for Server Virtualization

In a Mar. 2005 speech, Microsoft CEO Steve Ballmer said virtualization is a topic of "intense interest and activity in the industry and at Microsoft" and the industry should "expect to see a lot of Microsoft innovation" in the area. He also described how, in addition to making sure that today's Virtual Server product is manageable using Microsoft's management products such as Operations Manager, Microsoft will ensure that future DSI technologies fully support and utilize virtualization. Virtualization promises to make it easier to move or clone OSs and applications from one host server to another with little or no downtime and with little administrative effort—it could even be automated based on policy. Virtualization also provides a convenient way to scale out application servers without the need for hardware-specific system images. (See the sidebar "Server Virtualization Is Changing Datacenter Architectures".)

These trends, if not adequately addressed by Microsoft, could pose a serious threat since VMware, now part of EMC, is currently the acknowledged leader in this field and virtualization will be built into Linux distributions shipping by 2007 as part of the XEN Open Source project.

Microsoft has been shipping Virtual Server 2005 since Oct. 2004 and now supports running nearly all of its server applications running in Virtual Server VMs. Microsoft has even begun limited support of its products running on non-Microsoft server virtualization products and recently made its licensing terms much friendlier for virtualization.

Virtual Server 2005 R2: Features and Benefits

Originally planned as the first service pack for Virtual Server 2005, Microsoft added enough new functionality along with the bug fixes that it decided to release the product as Virtual Server 2005 R2 instead. Virtual Server 2005 R2 comes in two editions: Standard Edition, which can be installed on servers with no more than four processors, and Enterprise Edition, which can be installed on servers with up to 32 physical processors. Other than the number of CPUs supported, the two editions are identical. (For a full analysis of R2's predecessor, see "Virtual Server 2005 Supports Testing and Flexibility" on page 7 of the Nov. 2004 Update.)

Released to manufacturing in early Dec. 2005, the new release addresses some shortcomings of the current product, but at a small cost: unlike a service pack, customers without Software Assurance on Virtual Server 2005 will have to buy new licenses to get R2 features. However, that cost will be low: the estimated retail price for Standard Edition is now only US$99 and Enterprise Edition is US$199, roughly a quarter of the price of their predecessors.

Virtual Server 2005 R2 adds the following key features:

64-bit host OS support. R2 supports running 64-bit editions of Windows Server 2003 as the host OS on Intel and AMD x64 CPUs. However, R2 still presents 32-bit VMs to the guest OSs, so customers are limited to installing 32-bit guest OSs on them. Nevertheless, the 64-bit host OS support should increase overall performance when running many memory-intensive VMs.

Virtual Server 2005 was only available in a 32-bit version and required the host OS (Windows Server 2003 or Small Business Server 2003) to be 32-bit as well. Although Virtual Server 2005 ran on Intel and AMD x64 hardware, it could not exploit any 64-bit hardware features. Virtual Server 2005 did support the Address Windowing Extension (AWE) APIs, which allowed it to use the Physical Addressing Extensions (PAE) feature in Windows Server 2003 Enterprise and Datacenter Editions to address up to 64GB of memory, but PAE memory is not equivalent to 64-bit's flat memory space and has limitations that make it less desirable than Virtual Server 2005 R2's native 64-bit support.

Microsoft does not plan to support Virtual Server on 64-bit Intel Itanium-based servers and refers customers to use Hewlett-Packard or SWsoft server virtualization software, which is not based on Microsoft's virtualization technology.

Support for guest OS and host OS clustering via Windows Cluster Service. Although Virtual Server 2005 supported clustering of Windows Server 2003 guest OSs running on the same machine, this scenario was useful only for testing, since it did not provide fault tolerance if the hardware or host OS failed. With Virtual Server 2005 R2, organizations can create Windows Cluster Service clusters at two different levels: between guest OSs on separate physical servers or between host OSs on separate physical servers. Both approaches can be useful in production scenarios.

Guest OS clustering behaves similar to nonvirtual server clusters in that it can provide automatic failover to cluster-aware client applications, but it requires each VM to be independently clustered with a partner VM on another server, and requires Windows Server 2003 Enterprise or Datacenter Edition as the guest OS on each clustered VM. This makes it complex to set up. Furthermore, because the VM must present a virtualized host-bus adapter to the guest OS, Virtual Server 2005 R2 guest OS clustering works only on servers that have iSCSI-based physical connections to the shared storage so that each guest OS can use Virtual Server's virtual network adapter to communicate with the disk array. Guest OS clustering over Fibre Channel connections is not supported.

Host OS clustering is beneficial because it can allow an organization to have a hot-spare server (the cluster's passive node) that can take over hosting all of the cluster's VMs should the host OS or hardware on the active node go down, including being taken down for planned maintenance. Unlike traditional physical server clustering or guest OS clustering, clients connected to applications running in the VMs do not fail over automatically: to them it looks like the server has failed or been shut down as the VMs go offline momentarily while the network reconfigures itself to route VM traffic to the network adapter on the spare server. However, clients can reconnect to the VMs in their new location within a minute or so.

Host clustering requires Windows Server 2003 Enterprise or Datacenter Edition as the host OS and works using SCSI, iSCSI, or Fibre Channel connections to the shared disk storage (typically a storage-area network).

Preboot eXecution Environment (PXE) support. PXE is a small OS that can load out of a system's BIOS, and it enables a computer without an OS on its disk to connect to a network server and begin a bootstrap process that results in OS installation. Virtual Server 2005 R2 provides PXE support for virtual machines, which enables automated OS installation on VMs using Windows' Automated Deployment Services (ADS) in the same manner ADS is used to automate OS installation on physical servers. Without PXE support, administrators have to build a special network boot CD-ROM and manually configure the VM to load it, thereby giving the VM a way to get to an ADS server over the network to obtain a full system image.

Enhanced Linux guest OS support. While not a new technical feature of Virtual Server 2005 R2, Microsoft's Customer Service and Support (formerly Product Support Services) will soon support Virtual Server 2005 R2 customers that run Linux or Sun Solaris as a guest OS in addition to the previously supported Microsoft guest OSs: Windows Server 2003, Small Business Server 2003, Windows 2000 Server, Windows XP SP2, and Windows NT 4.0 SP6a.

Microsoft is also working with partners to extend support for non-Windows guest OSs on Virtual Server 2005 R2. It is working with InnoTek to provide technology that will optimize performance of Linux guest OSs on Virtual Server 2005 R2; this software will be available for download in the first half of 2006. Microsoft is also preparing an arrangement with partner Wipro to provide technical support requests regarding third-party guest OSs to Microsoft Premier or Professional support customers. When customers call Microsoft with issues related to a third-party OS running within Virtual Server, Microsoft will diagnose the problem and then transfer the case to Wipro.

Performance enhancements. Microsoft claims that VMs running on Virtual Server 2005 R2 servers will perform better than on its predecessor when using the same hardware. Furthermore, if the server has CPUs that support Intel's hyperthreading technology, R2 will be able to exploit it the same way that it can exploit multiple CPUs. However, each VM can execute on only half of the processor.

Virtual Server 2005 R2: Remaining Limitations

Although Virtual Server 2005 R2 is a significant improvement, it did not correct all of its predecessor's limitations.

No VHD snapshots. Virtual Server 2005 R2 does not include a Volume Shadow Copy Server (VSS) writer that would allow the Windows Server 2003 host OS to take coherent snapshots of open VHD files while their respective VMs are running. Although this would be a highly desirable feature because it would simplify backup of VMs, the host OS must be able to signal the guest OS to suspend itself for a moment while the snapshot is being taken, and Microsoft's current virtualization products do not support this feature. However, Virtual Server does support a feature called "differencing disks" that provides a way, albeit more cumbersome, to take backups of live VHD files.

No guest OS multiprocessor support. Although the Windows host OS's symmetric multiprocessing (SMP) and non-uniform memory access (NUMA) support allows Virtual Server 2005 R2 to make use of multiple processors, each VM can access only a single processor. If more than one VM is running, Virtual Server will assign them to different processors to balance the workload, but even though the guest OS and most server applications are multithreaded, all threads for each VM execute on only one physical processor. This lack of multiprocessing support limits the processing power available to individual applications.

No USB support. Virtual Server 2005 R2 guest OSs cannot support USB hardware other than USB keyboards and mice. This precludes accessing devices such as USB-attached printers, scanners, uninterruptible power supplies (UPSs), and smart card readers from the guest OSs. However, problems from this limitation are rare: USB-attached printers and scanners are uncommon for servers, and when a UPS signals the server's host OS to shut down, Virtual Server can be configured to gracefully shut down guest OSs first. The lack of smart card support can be worked around by using Remote Desktop (Terminal Services) to log into a Windows guest OS from a computer equipped with a smart card reader.

Licensing and Other Virtualization Policies

Over the past year, Microsoft has made various policy changes that broaden the utility of Microsoft's virtualization technologies, including wider virtualization support in Microsoft's other server products, more liberal licensing provisions, a single format for VHD files, and support for competitive products running on Microsoft's platform.

Virtualization part of Common Engineering Criteria. Microsoft's Common Engineering Criteria (CEC) 2005—a set of product-release requirements designed to ensure consistency and integration across the Windows Server System product line—mandates that compliant products run properly in a Virtual Server VM (with a few exceptions, such as if the software requires hardware that is not currently supported in the VM environment). While only products labeled "2005" and later fall under the CEC, nearly all of Microsoft's current products will run properly in a Virtual Server 2005 VM, with the exception of Speech Server, ISA Server 2004, and SharePoint Portal Server 2003. Speech Server requires telephony hardware not available in a virtualized hardware environment; Virtual Server support and CEC 2006-compliance for the other two will come when Microsoft releases the next versions.

New licensing provisions. Microsoft has made its licensing terms friendlier to its customers who use server virtualization software. (The company has not announced any changes in licensing for desktop OSs and applications running in virtual environments.)

In Oct. 2005, Microsoft announced that it was waiving license requirements for copies of Microsoft server software installed in VMs that are not running; providing users the right to run the host OS plus up to four VMs on Windows Server 2003 R2 Enterprise Edition with a single OS license; and changing the way it counts processors when server applications are running in a VM.

These changes are very important, because they eliminate the licensing "taxes" on server virtualization, making it much more likely that customers will take the plunge into using virtualization for production systems. (For a full analysis of the changes, see "Licensing Retooled for Server Software on Virtual Systems" on page 34 of the Nov. 2005 Update.)

Published VHD format. Microsoft has published the specifications for the VHD file format used by its Virtual PC 2004 and Virtual Server 2005 products. By doing so, it hopes that other software vendors will write management programs that can read information from these files and perform operations, such as installing or patching software packaged in a VHD, while the virtual machine is not loaded and running.

The same VHD format will be used by future Microsoft virtualization products and technologies, which means they will be able to read and run existing VMs either unmodified or after being updated by a conversion tool.

Support for VMware. Just as Microsoft would like other software vendors to support their products running in Virtual Server VMs, Microsoft has begun supporting Premier-level customers that run Microsoft software in other vendors' VMs, particularly that of rival VMware. In these cases, Microsoft will now use "commercially reasonable efforts" to investigate issues, although it may require the customer to reproduce the problem in a nonvirtual environment.

Virtual Server "Next"

Because of delays to the next version of Windows Server (code-named Longhorn) and delays in building native virtualization support into Windows, Microsoft has added one more release of the stand-alone Virtual Server into the roadmap. Still without an announced product name, Virtual Server "Next" will go into beta in the first half of 2006 and ship in the second half. Because of the short and ambitious development cycle, Virtual Server "Next" will not be a major release (and could be thought of as an "R3" release) and will add only a few key features. Although it will not require Longhorn Server (which won't ship earlier than 2007), it will support Longhorn Server as both a host OS and a guest OS.

Virtual Server "Next" will add the following key features:

Exploit AMD Pacifica and Intel VT hardware. In the first half of 2006, new servers containing AMD Pacifica or Intel VT chipsets will hit the market. While the specifics of each technology are different, both provide hardware support for virtualization. (See the sidebar "What Are Intel VT and AMD Pacifica?".) Although Virtual Server "Next" won't require the new Intel or AMD hardware, it will be able to exploit both technologies. In this release, the biggest benefit of the VT/Pacifica support will be substantially improved performance for non-Windows guest OSs.

Host snapshots of VHD files. With Virtual Server "Next," Microsoft plans to add support for taking VSS snapshots of VMs while they are running. This will make it easier to make backups and will make it trivial to roll a VM back to an earlier state. Virtual Server "Next" will include a tool that allows customers to restore individual files from snapshots of VHD files, thereby eliminating the need to run backup agents inside the VMs.

Native Windows Virtualization

Microsoft is so convinced of the strategic importance of server virtualization that in 2004 it decided to make virtualization an integral capability of the Windows OS rather than letting it remain an add-on. Furthermore, while Microsoft undoubtedly has learned a great deal from its Connectix acquisition, it decided not to use the Virtual Server architecture and code base it inherited from Connectix but instead start over with a clean slate. Although Microsoft hoped to have this project done in time to make it into Longhorn Server (planned ship date in 2007), earlier this year the company decided that the feature is too critical to be rushed and would have to come sometime after Longhorn Server. How Microsoft plans to get this feature to Longhorn Server customers has not yet been decided, and while customers might prefer it to be a feature pack, it's conceivable that Microsoft would wait and incorporate it into the Longhorn Server R2 release, since the Virtual Server "Next" product will be Longhorn Server compatible. However, Longhorn Server R2 is not slated for release until 2009.

New Architecture

At the core of the new virtualization architecture is a component called the Windows hypervisor that sits at the lowest level of the host OS and abstracts and controls access to the hardware for multiple guest OSs. (For a graphical description of this new architecture and its components, see the illustration "Current and Future Microsoft Virtualization Architectures".)

The Windows hypervisor is not a full host OS, but it is responsible for abstracting the hardware, creating partitions (somewhat analogous to today's VMs, and completely unrelated to the concept of hardware partitions), and then enforcing memory access rules, CPU usage policies, and device access rules among the partitions. The hypervisor does not host its own hardware drivers. Rather, it passes device communications between the hardware and the partitions running on top of it, which allows Windows Virtualization to leverage the large base of Windows drivers. VMware's current high-end solution, ESX, uses a similar architecture (albeit without the VT/Pacifica hardware support) but the ESX component equivalent to the Windows hypervisor requires its own proprietary drivers, which limits the selection of hardware that ESX can run on. The Windows hypervisor will require hardware based on AMD Pacifica or Intel VT, meaning that customers must purchase new servers to use this technology.

In the new architecture, the concept of a traditional host OS goes away and is replaced by a special partition called the parent. All other partitions are children. Unlike today's host OSs, the parent partition's OS will be specially designed for virtualization and to manage the hypervisor, and will be an updated version of the Windows Longhorn server OS. The parent OS can be installed in a very limited virtual server role that will install and run only those elements it needs to perform its virtualization functions. This will result in a reduced attack surface and reduce the probability of maintenance-related interruptions. Other important OS services, such as Active Directory, Internet Information Services (IIS), or Windows SharePoint Services, will normally run on OSs in child partitions.

The children are similar to today's guest OSs and run in VMs stored on disk as VHD files. However, under the hypervisor architecture, the code execution path between the VMs and the physical hardware will be less complex than Virtual Server or Virtual Server "Next." Most importantly, the child partitions will support 64-bit guest OSs. As with today's VMs, child partitions will run standard, unmodified OSs as guests and will be capable of supporting non-Windows OSs without requiring special paravirtualization extensions—modifications to an OS to make it aware that it's running within a VM and optimize itself accordingly. However, Microsoft plans to offer extensions (which it calls "enlightenments") for Longhorn guest OSs, particularly to its memory manager, to make it run faster in a VM.

The Windows virtualization architecture also creates a new layer called the "VMBus" that sits at the lowest layer of each partition. Although this layer is built and managed by the parent partition, the VMBus in each partition has a mechanism for communicating with the VMBuses in other partitions via the hypervisor. In the VMBus architecture, device communications are provided by a mapping between virtual service providers (VSPs) specific to the physical hardware and corresponding virtual service clients (VSCs) that expose virtual devices to the partitions.

Unlike the Virtual Server architecture (where the host OS owns all the devices), this new architecture results in greater flexibility in the types of devices that can be exposed to child OSs, and it allows each partition to own and selectively share various hardware resources. At the same time, it allows Microsoft to preserve the benefits of VM portability, allowing a partition on one server to be moved unmodified to another. However, child OSs will still have a dependency on the parent OS even after the virtual environment is running, so in this first release of Windows virtualization there will be no way to reboot the parent partition without bringing down all of the children. Microsoft says that it plans to address this limitation in a future release.

The parent OS and hypervisor will employ Windows Management Instrumentation (WMI) so that systems management products such as future versions of Microsoft Operations Manager and Systems Management Server will be able to monitor the health of the system and automate various management activities, such as migrating a partition from one server to another while in use.

Microsoft has committed to making the new architecture backward compatible with older Virtual Server 2005 and Virtual Server "Next" VHDs, so migration of existing VMs to a new Longhorn Server should be relatively painless.

New Features

The current plans call for the new Windows hypervisor virtualization architecture to enable many new features:

64-bit child OSs. In addition to mandating a 64-bit hypervisor and parent OS, Windows virtualization will finally support 64-bit child OSs. Given Microsoft's recent announcements that it will produce only 64-bit versions of upcoming server products, such as the next release of Exchange (code-named Exchange12), this feature is critical. However, all 64-bit support is for Intel and AMD x64-based servers; Microsoft has no plans to support 64-bit virtualization on Itanium-based servers.

Multiprocessing support in VMs. Windows virtualization will at last allow VMs to exploit multiple CPUs on SMP and NUMA servers. This support will make it practical to run large, heavily loaded, multithreaded applications in VMs.

More virtual devices. Windows virtualization will expose more types of devices within the VMs, such as SCSI tape devices and USB devices such as smart card readers.

New administrative user interface. Windows virtualization will dispense with Virtual Server's Web-based administrative interface, which will eliminate the need to run IIS in the parent OS. The new administrative interface will use the enhanced version of Remote Desktop Protocol (RDP) that will come with Longhorn Server Terminal Services. Although details are still forthcoming, the new version of RDP will supposedly support a new hybrid model that melds the advantages of the client-server and terminal server models.

Resource controls. Administrators will be able to control how physical resources, such as memory, processor usage, storage access, and network access, are allocated to different partitions. This will allow them to guarantee that resource demands from one partition do not excessively degrade the performance of other partitions on the same server.

Resources

More on the DSI and Microsoft's overall management roadmap can be found in "Management Product Roadmap Expanding" on page 11 of the June 2005 Update.

A good white paper on Intel's virtualization plans can be found at www.intel.com/business/bss/products/server/virtualization_wp.pdf.

Microsoft's Virtual Server 2005 R2 Web site is at www.microsoft.com/windowsserversystem/virtualserver.