inset
New Generation of Hardware Needed for DataCenter To Scale Beyond Eight CPUs
May 22, 2000

A new generation of Intel-based systems that combine the benefits of both the mainframe and PC server worlds is making its appearance this year. These systems promise to offer the high degree of reliability, availability, and scalability previously available only on mainframe-class systems, yet offer the open architecture, standard components, rapid evolution, and lower cost of the PC server world. This will give OEMs significant opportunities for differentiation, as they move away from Intel’s ProFusion architecture and develop their own methods of balancing memory, processors, cache, and input-output (I/O) to allow their large systems to efficiently scale up to 32 or more processors.

As the number of processors grows beyond four, bus contention between the CPUs and main memory limits the ability of a system to linearly increase its processing power. Intel’s ProFusion architecture increased this limit to eight, but beyond that a different approach is necessary. Some large system vendors, such as IBM’s Sequent division, abandoned the symmetrical approach and adopted a Non-Uniform Memory Access (NUMA) architecture. NUMA uses pods of 4-CPU modules, each with its own memory, all connected via a Scalable Coherent Interconnect (SCI). Unfortunately, this architecture requires extensive changes to the operating system to account for variability in the location and response time of system memory.

Hewlett-Packard, IBM, Unisys, and other vendors are working on better designs based on very fast, low-latency, crossbar switches. Analogous to the way Ethernet switches offer performance superior to Ethernet hubs, these designs give the CPU banks direct, high-speed, point-to-point access to banks of main memory. This approach will allow much better scalability without incurring the drawbacks of NUMA.

Of the hardware manufacturers who have announced participation in the Windows 2000 DataCenter program, it appears that only Unisys will be shipping a platform supporting more than eight processors when Microsoft launches DataCenter. (Compaq may also resell Unisys servers.) Unisys’ new ES7000 server can truly claim to be "enterprise-class" and will compete with machines such as the Sun E10000. The remainder of this article focuses on the ES7000 platform as representative of the new generation of servers.

Overview of the Unisys ES7000 Server

Core architecture. At the core of the ES7000 design is a technology termed Cellular MultiProcessing, or CMP. As the accompanying illustration ("Unisys ES7000 Architecture") shows, up to four 16GB banks of memory can be directly accessed from any bank of processors via four crossbar switches. Unless partitioned, each processor bank has equal access to all memory and must contend with traffic from a maximum of only three other data channels. The processors themselves are organized two to a bus, with each pair of CPU buses connected to its own fast third-level cache.

Input-output. One of the weaknesses of the older PC server designs was its relatively limited I/O capacity compared with that of mainframes and big Unix systems. The ES7000 has come far toward addressing this shortcoming. In addition to the processor pods, each of the four crossbars can accept two Direct I/O bridges (DIB). With each DIB supporting three 64-bit PCI buses, the ES7000 can support a total of 24 PCI buses and 96 PCI cards. Host adapters and controllers supporting Intelligent I/O (I2O) can read and write directly to memory, bypassing the CPUs and other buses. Since all data paths are 64-bit, Unisys states that the ES7000 can be upgraded to IA-64 Itanium processors just by changing out the processor sub-pods—all other components will remain the same.

Monitoring and management. Like its mainframe brethren, the ES7000 uses a single, or a redundant, pair of service processors to maintain and monitor the system. This service processor runs its own operating system independently of Windows 2000 DataCenter and will perform out-of-band functions such as configuration, partitioning, remote console support, and monitoring (with the same sort of "phone home" capabilities as "big iron" systems.)

Partitioning. One of the most powerful capabilities of the ES7000 is support for hardware partitioning. Through the service processor, the main system can be carved up into partitions, isolating CPUs, memory, and I/O buses into virtual systems—each running its own instance of either Windows 2000 DataCenter or SCO UnixWare. In the first release of DataCenter, the service processor statically establishes these partitions before installing Windows 2000. Later versions of DataCenter will negotiate with the hardware to allow dynamic reallocation of system resources without rebooting. If desired, administrators can set up a "cluster in a box," where each partition constitutes a node in an interconnected cluster. Furthermore, through the use of Winsock Direct, a block of shared memory can provide direct low-latency cluster interconnections without using a network.

Storage. All disk storage will be external and connected via Fibre Channel. Unisys has partnered with EMC to offer and support both Symmetrix and Clariion storage arrays, allowing up to nearly 20 terabytes of storage. Of course, administrators can carve up the disk storage among system partitions or among multiple separate (and potentially heterogeneous) systems.

Support. Unisys will sell the ES7000 bundled with installation and configuration services and will offer a single point of support (at extra charge) for both the hardware and operating system. It will also offer optional 99.9% availability service-level agreements for both single-system and clustered configurations. This program includes a financial remedy if the hardware and operating system fails to achieve this level, but it does not count scheduled maintenance downtime in the 9 hours per year allowed. Unisys will not be including application downtime in the first version of this program.

The ES7000 bears little resemblance to today’s generation of servers and is closer to a mainframe than ever before. Yet, because it uses industry standard CPUs, PCI devices, and memory, it will cost dramatically less than comparable Unix and mainframe systems. And it appears that the performance will be extraordinary—Unisys has already benchmarked its smaller sibling, a non-clustered, eight-way ES5085 Xeon system running Windows NT 4.0 Enterprise Edition, at nearly 49,000 TpmC. This is equivalent to the performance of the best eight-way Unix systems at less than half the price. For more benchmark information, see www.unisys.com/news/releases/2000/feb/02166868.asp and www.tpc.org/new_result/c-results.idc. For information on the ES7000 architecture, see www.unisys.com/marketplace/ent/downloads/mainframe-nt.pdf and www.unisys.com/marketplace/ent/downloads/5130-ent.pdf.

          Back to associated article: Windows 2000 DataCenter Targeting High-End Unix Servers