How we tested

by Neil Randall and Allen Fear (3/1/2004; updated 2/28/2005)

Right off the bat, it's important to note that this buying guide is not intended to be a scientific assessment of server performance, at least not in the sense that we've generated replicable benchmark numbers or focused on labs-based testing. Our goal here is to evaluate these machines for first-time buyers in offices that might not have a dedicated IT staff.

That's why we were most interested in finding out how quickly we could set up these boxes and how each one matched up to the others in handling typical daily office activities. We decided, therefore, to keep our performance testing simple.

Note that we performed this testing only on the two Windows towers (HP ProLiant ML110 and the IBM eServer xSeries 225), not on our server alternatives (LaCie Ethernet Disk, Mirra Personal Server, Buffalo LinkStation, Iomega NAS 100d, and Tritton WiFi NAS). Our reasoning: The server alternatives are too different in form and function, compared to the Windows towers. It would have been an apples-to-oranges comparison.

For our testing, we timed the following actions on the Windows servers:

• Transferring a 1.5GB folder (containing more than 4,000 files) from one client to the server.
• Transferring that same 1.5GB folder (each containing those same 4,000 files) from three clients simultaneously to the server.
• Transferring the same 1.5GB folders from five clients simultaneously to the server, at the same time the server was downloading multiple files over the Internet.
• Logging on to the server from a Windows XP client for the first time.
• Logging on to the server from a Windows XP client for the second time.
• Connecting to the Microsoft Exchange Server using Microsoft Outlook.
• Taking control of a client from the server using Windows Remote Desktop.
• Opening the server's case and replacing the network card.
• Opening the server's case and replacing the hard drive.
• Configuring the server as a broadband gateway.

Some chores, such as replacing the network card or the hard drive, were a function of hardware design. Was the hard drive hot-swappable? Was the case easy to open and accessible? We reported anything unusual along those lines in the individual reviews, though the differences were small.

Other tests were more a function of software than hardware. While we were testing the servers themselves, ultimately the operating system had a lot to say about how easy they were to use. Transferring files from a client to the server and back again, for example, involves Windows Explorer, which has its own strengths and weaknesses when it comes to managing such transactions. Similarly, the length of time it takes a client to log on to a server demonstrates the peculiarities of Windows XP Professional connecting to Windows Server 2003 (Standard or Small Business Edition) more than any peculiarities of the server itself.

In the end, the differences between one machine and another were awfully slim, at least for real-world users who just want to get some real work done. Still, we were able to draw some comparisons. To find out what they were, see our overview section, as well as the reviews themselves.

Neil Randall teaches software interface analysis and design at the University of Waterloo, Canada, and has published numerous books and articles covering operating systems, networking, and hardware. Allen Fear is a senior editor at CNET.