IntroductionThis describes how to build a server computer. It discusses hardware and software considerations. The target audience is individuals and small and medium sized businesses. If you are a large business, just ask IBM or other serious vendor to specify, configure and maintain your servers. What Is A ServerThere are many types of servers with different requirements. What they generally have in common is:
These requirements are different from typical computers and the server hardware and software is usually different from typical computers. |
It is desirable that the server not use lots of power. If the server uses lots of power, it will likely require lots of cooling. Cooling costs are higher than the servers electrical costs. If there are many servers in a room, minimizing power use is even more important. Reliability goes down when temperature goes up too high. There are a variety of ways of reducing power usage. There is no need for a fancy graphics card in a server. If the graphics card has a fan, it is time for a different graphics card. I recommend high efficiency power supplies, as they generate less heat. Look for 80+ label, which means the power supply is at least 80% efficient. I like Seasonic power supplies. I will discuss power usage later.
All things made by man will fail in time. The goal is to minimize downtime in a cost effective fashion. If you need 99.999% uptime that means 5 1/4 minutes of downtime a year. This takes serious system engineering, and it is time to call IBM. Look for lots of hot-swap (changeable without powering down) hardware including power supplies, hard drives, fans, PCI cards, and UPS batteries.
If you don't need quite as much reliability, your hardware costs will go down quite a bit. The most unreliable hardware are generally fans, hard drives, and power supplies. Be sure that the ones you use are as reliable as possible. I like Panasonic panaflow fans, as they are quite reliable (as well as quiet). For hard drives, there are some recent papers available which discuss hard drive reliability. (Look for papers by google and by CERN). Historically, SCSI has been more reliable than IDE drives, as well as much more expensive. SAS drives are likely as reliable as SCSI. I wouldn't buy a drive unless it is available with a 5 year warranty.
Memory is a special case. It is quite unlikely to have a permanent error,
however it is quite likely to have non-permanent errors. known as transient
errors. IBM has done lots of research and estimate that with 1 gigabyte of
memory there will be a transient error every week. These are caused by alpha
particles in the memory packaging, cosmic rays, and other stuff. The way to
deal with these transient errors is ECC (error correcting code) memory, which
has extra memory used to detect and correct memory errors. The common ECC
memory will detect all 2 bit errors in 64 bits of memory and correct all 1 bit
errors. There are higher levels of ECC available. See IBM's chipkill
technology for details. If you have a memory error in a part of memory that
isn't being used, it will cause no trouble. If you have a memory error in a
part of memory that will be written before it is read, it will cause no
trouble. Otherwise, the memory error will effect in some way the processing of
the computer, in a bad way. Serious server motherboards will log memory
errors, such as the Intel PR-440FX.
Cheaper motherboards such as the Asus CUR-DLS
and PC-DL will support ECC memory, but won't
log memory errors. There are some CPU 'chipsets' that do not support ECC
memory at all. Motherboards made with these chipsets will not support ECC
memory. I recommend using only motherboards that support ECC memory, and using
only ECC memory for all servers. See the following links for details:
ECC SDRAM Primer
IBM
Chipkill White Paper
EETimes on IBM Chipkill
Soft Errors in Electronic Memory
There are several parts to security. If the server isn't physically secure, then it isn't secure at all. If the server is running software that isn't secure, then the server isn't secure. If the server's operating system isn't secure, then the server isn't secure. If the server isn't secure, then it is at best useless, and quite possibility a liability. Bad people will try to access the server and will try to break into the server, corrupt the server, steal data from the server, and use the server for their own ends. All of this is really bad for you. If the server is only used internally such as a media file server, then security is important, but not mission critical. If the server is used externally (that is accessible to the Internet), then security is mission critical. There are a variety of ways to help make the system secure. I recommend a hardware firewall between the external world and all internal computers. If the server has to be accessible to the outside world, allow just those ports that the server needs to get through the firewall, and to go just to the server that needs those ports. Hardware firewalls are so inexpensive there is no reason to not have one. I recommend a software firewall on all servers that allow access to only the ports that the server needs (this applies to internal as well as external servers).
For the operating system and server software, I recommend choosing reliable software. This does not include any product made by Microsoft. There are far too many vulnerabilities with Microsoft's operating systems as well as server programs such as Microsoft Internet Information Server (web server) and Microsoft Exchange Server (mail server). I am also unimpressed with Sun's recent unreliabilities with Solaris OS and applications. I think OpenBSD is likely the most secure general purpose server OS, although the other BSDs and Linux can be configured to be reasonably secure.
Easy maintenance is important for both the server's hardware and software. The hardware should be easily accessible, well known brands that are generally available, and easy to service. For example, on some Dell workstations that I have seen, the power supply folds out of the computer for easy access to the rest of the system. For an older IBM desktop it took quite a bit of disassembly to access the memory or the second hard drive. If you are unsure about a brand's reputation, you can get recommendations on the Internet. For example, I have had very good support from Adaptec with their SCSI controllers. When mine failed, they cross-shipped me a replacement. I have also had good support for Gateway servers.
For software, it should be easy to install, update, roll back, and check on the status.
First, select a processor and system. There are many options. The lowest power processors x86 processors are made by Via. I decided the Via processors are generally underpowered and somewhat expensive. The next lowest power processors are notebook processors. They are generally more expensive and motherboards for them are limited and expensive. If you had an old notebook lying around, that would make a fine bittorrent server (possibly needing an external disk).
The Pentium III 733 is a reasonably fast, reasonably low power processor. One limitation is there is almost no power saving when the cpu is idle. Virtually all processors made the last 3 years draw less power when idle. In the last two years processors drop their frequency and voltage when load goes down. This was first available with notebook processors, but migrated to the desktop, as desktop processors use more power than notebook processors. If I couldn't locate an old system, I would likely use a low power desktop processor, as they are cheap and readily available. Many motherboards allow the processor voltage to be lowered, or there are software programs such as RM Clock that will allow setting the processor voltage as a function of load. I have used RM Clock to drop my notebook voltages on Turion as well as Core 2 Duo systems. Dropping the voltage n percent will reduce power by approximately n^2 percent. I have dropped my notebook voltage by 20%, which means a 36% reduction in power.
Rather than buy a new computer, I decided to use an existing (free) IBM desktop Pentium III system. The cost is unbeatable, the form factor desirable, and the CPU is relatively low power. I read the IBM manuals and found this particular model doesn't accept ECC memory, though other IBM desktop models do. This isn't ideal, so I am now in the market for another low power system that takes ECC and is cheap. It came with a 20gb hard drive which was dedicated to the OS. A 200gb hard drive was added for storing the files. This machine was old enough that the BIOS didn't recognize the hard drive was over 137gb (new BIOS's know about big hard drives) but it wasn't a problem as the loaded OS didn't use the BIOS to access the big hard drive. The existing graphics card was adequate, as was the 10/100 Ethernet card. I eventually added a low power AGP video card with 32 mbytes of RAM, since I use a KVM switch, and I have a high resolution screen.
For software, a recent version of Linux was selected. In addition to being free, I think that Linux is much more secure than any version of Windows. It also can be configured to not require lots of resources, which is important with older hardware. ssh was enabled to transfer files to and from the server. A firewall was installed, and it allows ssh and the bittorrent protocol in. TCP wrappers was used to limit tcp access. The machine has never crashed.
Power consumption:
I ordered an adapter so I can use a notebook hard drive rather than a 3.5 inch hard drive for the OS. (If you do this, get adapter rails to mount a 2.5 inch drive in a 3.5 inch bay. My solution was a kludge.) Power consumption with the notebook drive:
I may also buy a seasonix SFX form factor 80+ power which should also save a few watts. It won't be very cost effective however.
After about a year the 2.5 inch notebook hard drive became unreliable, which produced a variety of amusing symptoms such as the computer hanging. A friend pointed out that most notebook drives aren't designed for 24x7 operation, which I was subjecting mine to. (The drive itself had come from a 5 year old notebook computer.) I disconnected the drive and reformatted the 200gb hard drive to have the OS as well as the bittorrent storage area. I have re-thought my desire for running a 2.5 inch drive, unless it is rated for 24x7 operation. If I was building a bittorrent server from scratch I might consider one of the 'green' 3.5 inch drives, as they are lower power than most conventional drives.
Why bother backing up data? There are several good reasons. The first is that everything can break, including the hard drive where your data is. The second one is you might delete something important by mistake and want it back later on. The third is in case of disaster recovery, like your computer catching on fire, or being stolen. How do you backup data? There are many ways to backup data. There are flash drives and external drive enclosures. There are prebuilt boxes that do it. There are hardware raid controllers. There are computers dedicated to storing data. Lets narrow down the choices a bit.
The first thing to know where the data you will be backing up is, and how much you will be backing up. On a Unix or Linux system, the typical answer is everything on your personal directory tree. On a windows system, the personal directory tree is typically on the OS partition, and has all kinds of junk as well as useful stuff. I recommend a separate OS partition, and a user partition. That way, if you need to reinstall the OS, you won't lose all your personal data. Of course, computers with the OS pre-installed aren't set up that way. There are two easy solutions. One is to shrink the OS partition and use the rest of the space as the user partition. This is a great solution for notebooks that only have one physical drive. Another solution is to buy a separate hard drive and dedicate it for user files.
The next level of sophistication is to separate out different types of data. Clearly some types of data is more important to backup than others. For me, all my critical data will fit on a big thumb drive. I have several other larger partitions for less critical data.
Flash drives are a great way to store data. They are quite robust, reasonably priced, and very portable. They aren't generally very fast (5-20mbytes/second write speeds are typical), nor are they very big (32 or 64 gigabytes max). If you can fit your date on one, or two if you are worried about a single point of failure, they are a great solution. I recommend name brand flash memory vendors, such as Sandisk, as they tend to use the better grade chips for themselves, and sell the lesser grade chips to their competitors. If you use a fat32 file system, it is not very robust and can get accidentally corrupted. I don't know a universally readable file system that is more robust though.
The next step up is an external hard drive in an enclosure. There are many interface options available, USB, firewire, e-sata, and Ethernet. Each has a potential advantage. Make sure the Ethernet is gigabit Ethernet, or you will be limited to 100mbits/second. USB is found on almost all computers. Firewire is in practice a bit faster, though not as popular. e-sata is the newest external interface. It is faster than any other one, but older computers don't have an external sata port. This can be fixed with an add-on card, but e-sata is useless on all but the newest notebooks. Ethernet's advantage is it can be attached to your network, rather than your computer, so it can be visible from all your computers rather than just the one that it is plugged into. If you have a 3.5 inch hard drive in an enclosure, in my experience it will run quite hot. Hot enough that I am concerned about the drive's reliability. Antec makes a very nice enclosure with a big quiet fan, the MX-1. It has usb and e-sata ports, and the case is designed to absorb noise.
The next step up is a bit more complex as it involves multiple hard drives. You can combine multiple hard drives to look like one big hard drive. You can treat each hard drive separately. Or, you can combine multiple hard drives so that one hard drive's failure can be tolerated by the system. The simplest is called RAID-1 (RAID stands for Redundant Array of Inexpensive Disks). In RAID-1, you copy all data to two disks. If one disk breaks, you get the data from the other disk. The overhead is you need double the amount of hard drive storage you would need without RAID-1. The good news is it is very easy to do. The next level of complexity is called RAID-5. RAID-5 allows for the failure of a single disk in a group of disks to not lose any data. RAID-6 allows for the failure of two disks in a group of disks to not lose any data. Though disks are inexpensive, they aren't very inexpensive. In addition to the price of the disk, the disk takes up space inside the computer, and takes power. Based on my experience with disks, I decided that Raid-5 was the best solution for me.
One thing to watch out for, is RAID-5 protects against one hard drive failing. If you bought a batch of 4 hard drives, and the batch has problems, 3 of the drives might fail within a few weeks. This happened to me. The only protection against this is to buy different brands or models of hard drives. Realize that some companies such as Seagate make drives and label them as Maxtor. So even though the drives have different names and model numbers, they might actually be the same. Just something to keep in mind.
You can buy a system to hold multiple disks and do RAID-5. They are typically called NAS (network attached storage). A prebuilt NAS system is a box that has everything except the hard drives, and you just add hard drives. They are less flexible than a dedicated general purpose computer, but are likely simpler and use less power and volume. Internally, they generally run Linux. I will only consider units with gigabit Ethernet, otherwise file transfer is limited to 12.5mbytes/second, which is too slow for me. I am only going to consider systems that support raid-5 for robustness. The minimum amount of drives you need for raid-5 is 3 drives (with 2 drives you might as well use raid-1). Lets see what is out there. There is the Promise SmartStor NS4300N that holds up to 4 disks. Write speed is around 15 mbytes/second. Cost is $400 without drives. Looking at Small Network Builder's NAS charts, only Thecus 1U4500 and Thecus N5200 have write speeds above 20mbytes/second. Thecus 1U4500 holds up to 4 drives, and has a write speed of 32mbytes/second and costs about $1000 without drives. Thecus NS5200 holds up to 5 drives and has a write speed of 25mbytes/second and costs about $700 without drives. It looks like these systems are not for me. They are somewhat lacking in flexibility, and pretty expensive, none hold more than 5 hard drives, and none have a write speed above 32mbytes/second.
The next big choice is software or hardware raid. There are basically two types of raid controllers, the type that rely on the operating system to do most of the work (known as software raid controllers) and the type that don't rely on the operating system to do most of the work (known as true hardware raid controllers). If a card costs under $100, or only works with Microsoft operating systems, you can be sure it is a software raid controller. For a 8 disk true raid controller, there are several choices: Adaptec has some models around $575, Atto makes a nice model for $1095, ICP has a model for $650, Promise has some nice models starting around $400. In addition to raid controllers, you can do the raid entirely in software. This provides quite a bit of flexibility, although it can take quite a bit of CPU cycles. The good news is you can get a pretty decent motherboard, processor and memory for less than a true hardware raid controller card. Sun has a new filesystem called ZFS which can be configured to have raid-like properties (they call it raid-Z). It is done entirely in software. You can get Solaris for personal use for free. ZFS is also supported under FreeBSD. In some respects ZFS is more robust than other filesystems. If you do use ZFS, there is no reason to buy a hardware raid controller. This is some of the possible flexibility you get when you do raid entirely in software. For me, a hardware controller was too expensive, so I decided to use software raid.
You will need to build or repurpose a computer to be the fileserver. You will need room for all the hard drives, and a way to keep them cool. The power supply must be powerful enough to spin up all the hard drives. You will need one or more gigabit Ethernet interfaces.
The motherboard should support gigabit Ethernet. If it doesn't, it is likely too old, or too cheap to use. For older pentium III servers, either get a 64 bit PCI Ethernet card, or a 32 bit PCI card running on its own PCI bus (server motherboards often have more than one bus). You will want to use ECC memory. If the motherboard or processor doesn't support ECC, it is the wrong choice for a fileserver. The last feature is a decent amount of I/O speed. Even though my Asus NCCH-DL motherboard has two PCI-X slots (64bits and 66mhz which totals 528 mbytes/second), the southbridge-northbridge interface is only 266 mbytes/second. This is one place where a hardware raid card might be faster than software raid, as much of the I/O can happen on the card, and only some would get to the PCI-X bus. Any opteron system that I am familiar with will have plenty of I/O capability.
You will need enough I/O ports for all of your hard drives. You will also need one for the OS disk, and likely one for the optical drive. I suppose you could use a USB optical drive if you ran out of ports. If the motherboard doesn't have enough ports, you will need a card with some ports. If you have pci-x slots, supermicro has a sat2-mv8, 8 port sata II controller card that works well. If you have pci-e slots, try to find a sata II card without raid hardware or software. I don't recommend parallel ata, unless you only have one drive per controller, as a failed disk can make the other disk on the cable inaccessible.
I recommend a 64 bit processor, as the xor's of raid-5 will be twice as fast (assuming a 64 bit OS). Any dual core processor is likely fast enough, so select a low power one. An AMD phenom is plenty fast, reasonably low power, and stupidly cheap right now.
It is important to keep the disks reasonably cool. The coolermaster 4-in-3 module is the best way I have found to do so. It has a 120mm fan in front of the hard drives. The fan doesn't have a rpm sensor. It is reasonably priced. The only downside is that to change a hard drive means disconnecting all the cables from all the drives, and removing the module. If that doesn't bother you, buy one or more of these. If that does bother you, I recommend the supermicro 5 bay hotswap rack. Each hard drive is accessible from the front, and can be removed while the computer is on (assuming you have sataII or SAS and software support). The 92mm fan in the back is quite loud, but you can get an inline 3 pin adjustable fan speed controller and slow it down so it isn't so bothersome. The rack has a fan failure alarm, an over-temperature alarm, and hard drive activity lights, and fault lights. It isn't cheap, but I really like them. You will want to monitor the hard drive temperature with some software tool. For windows, speedfan works great.
I wanted a fileserver to back up the rest of my computers to, at a low cost. I had an older Gateway 6400 server motherboard that was used. It has two Pentium III 933 processors, over a gigabyte of ECC memory, and some 64 bit PCI slots (in addition to the usual 32 bit PCI slots). Since hardware RAID is generally expensive, slow, and can be difficult to repair (if the controller fails you usually need to replace it with the exact same part), I choose software RAID. The main CPU(s) are used, and if some part fails, there is no need to find an exact replacement for it. Since the motherboard has a problem with its IDE controllers Asus CUR-DLS, I used two Promise Ultra TX 100 IDE controllers. These controllers each have two UDMA-133 connectors, however you cannot boot to a CDROM drive attached to them to install the OS. I could have used a SCSI drive attached to the motherboards SCSI controller, but I decided to use a generic IDE hard drive instead. So I attached a CDROM to the motherboard's IDE controller, and six 250gb hard drives for the RAID partition, and one IDE hard drive for the OS to the Promise controllers.
(Note it is generally accepted that it is bad to use the master and slave on a single IDE controller because if the master drive fails, it often takes the slave drive offline as well. This is why people use SCSI which doesn't have this problem, or SATA which doesn't have master and slave, but uses point to point. However, given the low failure rate of hard drives, I decided to risk it. Since this is a backup file server, its failure wouldn't jeopardize anything.)
As more hard drives became available, they would be added to the RAID array. Eventually, SATA drives would be added to the array, minimizing the risks of a master drive causing a double failure. After operating the system for a while, I realized that the 100mbit Ethernet was a bottleneck. I got a Broadcom gigabit Ethernet server card, which has a PCI-X interface. Since the motherboard has two 64 bit PCI slots, which are separate from the other PCI slots, the disk traffic wouldn't affect the Ethernet traffic. The array can sustain reading and writing at 50 mbytes/second according to the Bonnie+ benchmarks, included below. When backing up files from other computers, I can copy at about 20-30mbytes/second across the Ethernet.
Eventually a 300gb drive was added to the array, though the array only could use 250gb. A CoolerMaster Stacker case was chosen in order to provide good cooling for the seven hard drives.
For software, I chose Linux because it had mature RAID software. We configured Linux with ssh as well as samba, which emulates the windows SMB protocol used by windows file servers. That way, windows machines could see the files on the backup file server, and files could be copied back and forth using the windows GUI.
Here is how I configured the RAID-5 array:
# make a device-special file
# Not needed with most 2008+ Linuxes # mknod /dev/md0 b 9 0
# make the raid array with 6 hard drives
mdadm --create /dev/md0 --verbose --chunk=32 --level=5 --raid-devices=6 /dev/hd[abcdef]
# This will make a 232gb size set, and a chunk size of 32k
mdadm --detail --scan
# make the file system on the raid array
mkreiserfs /dev/md0
# make the mount point for the raid array
make directory /data
#the fstab entry for the raid array
/dev/md0 /data reiserfs notail 1 2
# to create mdadm.conf, as root run
#!/bin/sh
grep -qs '/^[[:space:]]*DEVICE' /etc/mdadm.conf || \
echo "DEVICE partitions" >> /etc/mdadm.conf
echo "MAILADDR foo@bar.com" >> /etc/mdadm.conf
mdadm -Esc partitions | awk '
/^ARRAY[[:space:]]/ {
print $0, "auto=yes"
}
' >> /etc/mdadm.conf
# This will make /etc/mdadm.conf, which looks like:
# DEVICE partitions
# MAILADDR foo@bar.com
# ARRAY /dev/md0 level=raid5 num-devices=5 UUID=ce5e9366:d83a232c:3da68804:f6814703 auto=yes
If you happen to be running Mandriva 2008 spring, fedora 9, or possibly some other 2008+ version of linux, you should read bug 40023 if your raid array doesn't mount when you reboot your computer.
Power consumption:
Here are some performance numbers:
[fileserver ~]$ /sbin/bonnie++ -d /mnt/data/tmp
Version 1.03 ------Sequential Output------ --Sequential Input- --Random-
-Per Chr- --Block-- -Rewrite- -Per Chr- --Block-- --Seeks--
Machine Size K/sec %CP K/sec %CP K/sec %CP K/sec %CP K/sec %CP /sec %CP
fileserver 2272M 13919 96 60733 89 14963 20 13685 86 35304 36 384.7 3
------Sequential Create------ --------Random Create--------
-Create-- --Read--- -Delete-- -Create-- --Read--- -Delete--
files /sec %CP /sec %CP /sec %CP /sec %CP /sec %CP /sec %CP
16 9601 100 +++++ +++ 3626 94 5255 97 +++++ +++ 7231 99
fileserver,2272M,13919,96,60733,89,14963,20,13685,86,35304,36,384.7,3,16,9601,100,+++++,+++,3626,94,5255,97,+++++,+++,7231,99
[fileserver ~]$ /sbin/bonnie++ -d /mnt/data/tmp -s 10000
Version 1.03 ------Sequential Output------ --Sequential Input- --Random-
-Per Chr- --Block-- -Rewrite- -Per Chr- --Block-- --Seeks--
Machine Size K/sec %CP K/sec %CP K/sec %CP K/sec %CP K/sec %CP /sec %CP
fileserver 10000M 14254 96 57165 78 14495 21 15121 92 53172 50 173.3 2
------Sequential Create------ --------Random Create--------
-Create-- --Read--- -Delete-- -Create-- --Read--- -Delete--
files /sec %CP /sec %CP /sec %CP /sec %CP /sec %CP /sec %CP
16 7523 77 +++++ +++ 8500 99 9494 99 +++++ +++ 6762 92
fileserver,10000M,14254,96,57165,78,14495,21,15121,92,53172,50,173.3,2,16,7523,77,+++++,+++,8500,99,9494,99,+++++,+++,6762,92
I learned a lot building my first backup fileserver. It was time to build another. I bought a used Intel STL-2 motherboard, two Intel Pentium III 933 processors, and 4gb of ECC memory for $130. This is a really serious server motherboard. It has onboard ultra-160 scsi, and 2 pci-x slots (64 bit, 66mhz), which is faster than my gateway 6400 64 bit slots. I added a supermicro pci-x 8 port sata II controller, and an Intel pro 1000-mt pci-x gigabit Ethernet card. I added four sata II 500gb hard drives. I used a promise Ultra TX 100 IDE controller for the OS disk.
Power consumption:
Here are some performance numbers:
/usr/local/sbin/bonnie++ -d /mnt/data/tmp
Version 1.03 ------Sequential Output------ --Sequential Input- --Random-
-Per Chr- --Block-- -Rewrite- -Per Chr- --Block-- --Seeks--
Machine Size K/sec %CP K/sec %CP K/sec %CP K/sec %CP K/sec %CP /sec %CP
sata_fileserver 6G 11719 97 46545 73 33499 50 15310 98 102930 87 369.4 2
------Sequential Create------ --------Random Create--------
-Create-- --Read--- -Delete-- -Create-- --Read--- -Delete--
files /sec %CP /sec %CP /sec %CP /sec %CP /sec %CP /sec %CP
16 10307 99 +++++ +++ 8693 98 9832 99 +++++ +++ 7549 99
sata_fileserver,6G,11719,97,46545,73,33499,50,15310,98,102930,87,369.4,2,16,10307,99,+++++,+++,8693,98,9832,99,+++++,+++,7549,99
The writing speed is slower than backup file server V1, but I only have 4 drives in the array. Hopefully it will speed up when I add more hard drives. The reading speed is about twice as fast as the old backup file server. It is possible I was saturating the 33mhz pci bus, or perhaps I was having master/slave issues (which isn't a concern with sata).
I learned a lot building my first two backup fileservers. It is now clear to me that my first two were limited by relatively slow main memory speed. Using PC133 memory, the main memory speed is roughly 400 mbytes/second. Since there is no easy way to speed that up using pentium III processors, it was time for something newer. I bought a used Asus NCCH-DL motherboard and two LV Xeon 2.8ghz processors. Though the Xeons can draw more power than the Pentium III, they draw a variable amount of power depending on load, unlike the Pentium III which always draws the same amount of power. This motherboard, like the Intel STL-2, features 2 pci-x slots (64 bit, 66mhz), as well as built in Intel gigabit Ethernet. It has a bunch of other nifty features as well, the best ones are the ability to set the clock speed and cpu multiplier as well as fan speed based on system and cpu temperatures. I added 2gb of pc3200 memory. I used the same supermicro pci-x 8 port sata II controller, and four 750gb hard drives.
Power consumption (with 4 disk array, and 1.6ghz processors (1.86ghz)):
Here are some performance numbers:
/usr/local/sbin/bonnie++ -d /data/tmp
Version 1.03 ------Sequential Output------ --Sequential Input- --Random-
-Per Chr- --Block-- -Rewrite- -Per Chr- --Block-- --Seeks--
Machine Size K/sec %CP K/sec %CP K/sec %CP K/sec %CP K/sec %CP /sec %CP
sata_fileserver 2G 20822 99 72873 70 39750 40 23786 97 119355 62 437.7 9
------Sequential Create------ --------Random Create--------
-Create-- --Read--- -Delete-- -Create-- --Read--- -Delete--
files /sec %CP /sec %CP /sec %CP /sec %CP /sec %CP /sec %CP
16 362 32 +++++ +++ 10892 98 267 34 +++++ +++ 7270 98
sata_fileserver,2G,20822,99,72873,70,39750,40,23786,97,119355,62,437.7,9,16,362,32,+++++,+++,10892,98,267,34,+++++,+++,7270,98
The writing speed is 72.9mb/sec (mega bytes per second) which is faster than V2 which was 46.5mb/second and faster than V1 which was 60.7mb/sec. The reading speed is 119mb/second which is faster than V2 which was 102mb/second and faster than V1 which was 35mb/sec or 53mb/sec (benchmark was run twice). I hope the faster writes translate into faster performance using samba. With the V2 system, I could write between 20 and 25 mb/second. With the V3 system, I can write between 25 and 35 mb/second. I decided to test by copying two disks over at a time. I had tried this before with the V1 and V2 system, but the overall speed didn't increase (as it was maxed out). With the V3 system, I can write to the fileserver with an aggregate speed of between 40 and 50 mb/sec. This is a huge increase over the V2 system. Perhaps I will increase the memory speed and see what that does for performance, but this version is almost twice as fast as the V2 system, which is a big win already.
I bought two more drives for my raid array. You can 'grow' the array without destroying the data. First you say:
mdadm --add /dev/md0 drive1 drive2
to add the drives to the array. Next you say:
mdadm --grow /dev/md0
To grow the array with the added drives. I did this. What I didn't know was the process takes over 100 hours when I am adding two 500gb drives. The system hung during this process, so I built the array from scratch.
The next step is to grow the filesystem via the command:
resize_reiserfs /dev/md0
The there will be more free disk space.
Power consumption (with 6 disk array and 2.8ghz processors):
Well, what did the extra two disks do for filesystem performance? Here are some performance numbers:
/usr/local/sbin/bonnie++ -d /data/tmp
Version 1.03c ------Sequential Output------ --Sequential Input- --Random-
-Per Chr- --Block-- -Rewrite- -Per Chr- --Block-- --Seeks--
Machine Size K/sec %CP K/sec %CP K/sec %CP K/sec %CP K/sec %CP /sec %CP
localhost 2G 21890 99 92074 91 52511 36 27457 95 191331 79 511.9 15
------Sequential Create------ --------Random Create--------
-Create-- --Read--- -Delete-- -Create-- --Read--- -Delete--
files /sec %CP /sec %CP /sec %CP /sec %CP /sec %CP /sec %CP
16 15239 100 +++++ +++ 11601 94 14777 99 +++++ +++ 11679 100
localhost,2G,21890,99,92074,91,52511,36,27457,95,191331,79,511.9,15,16,15239,100,+++++,+++,11601,94,14777,99,+++++,+++,11679,100
So, sequential writes went from 73mbytes/second to 92mbytes/second. Sequential reads went from 119mbytes/second to 191mbytes/second. This is close enough to saturating gigabyte Ethernet from my perspective. No need to crank up the cpu or memory speed.
I decided to build a media server. There are somewhat different requirements compared to most other servers. You need audio and video outputs. Being quiet is very important. Reliability is less important.
I started off with an E-Machines AMD-2000 based system, because it was free. I added a dedicated video card since I wanted S-Video output for my TV set. It worked pretty well, but had some limitations. One significant limitation was the form factor. It was a mini-tower which didn't fit inside my video rack. I ended up buying an Antec NSK-2400 media computer case. It is a very well designed case, though a bit expensive. I used a wireless NIC card, as I had cleverly not put an Ethernet drop near the TV.
The next limitation was the E-Machines motherboard. It had a fault where it would reboot when the system was shut down. I had a powerstrip that I would turn off when I turned the computer off. Not very convenient. Also it was a bit finicky with power supplies. It wouldn't work with any Antec high efficiency power supplies. I tested it with the two Antec SU-380 power supplies and two Antec EarthWatts 380 power supplies. The existing power supplies I had were a bit noisy, which isn't ideal for a media server.
I ended up buying a new motherboard and processor, the ECS P4M900T-M motherboard and an Intel E-4500 Core 2 Duo processor. I needed a motherboard that had a micro-atx form factor, and I decided that I wanted pci-e video. The board was inexpensive, but worked ok. I did have some blue screens of death that was caused by poorly seated memory, but once I fixed that, it worked fine. I used the same wireless NIC card, and an ASUS 7200 pci-e video card that has component video output. I hoped that the component video output would be significantly better than S-Video, but I was only a little better (sharper colors mainly). Another advantage of the new motherboard was the processor runs much cooler and quieter than the old AMD-2000 processor, and can be made to run even cooler. I was able to use an efficient, quiet Antec power supply. The new media computer is much quieter than the original one, and is faster (not that that is needed).
If you have comments or suggestions, Email me at turbo-www@weasel.com
Created with gnu emacs and template-toolkit, not some sissy HTML editor.
No Java or javascript needed to view my web pages. They
both have significant security issues.