iostat

One of the client's news portal has a surged of visitors due to a very interesting local news. The traffic doubled up overnight and the server was struggling to cope with it. The specs of the server is as below:-

Dual Xeon 5405

16GB DDR2 ECC RAM

2 x 320GB SATA 7200 (s/w RAID1)

I have tuned the webserver and mysql to allow more connections but the load was too high. The average load can go up as high as 40 during peak hours. The webserver can accept up to 352 concurrent connections now, up from 152 before the tuning. Memory usage should be still fine as only about 30MB swap being used as shown from "top" and "free".

The output from iostat is as below:-

Linux 2.6.26-2-amd64 (zz) 09/15/2010 _x86_64_

avg-cpu: %user %nice %system %iowait %steal %idle

23.95 0.00 4.44 0.80 0.00 70.82

Device: tps Blk_read/s Blk_wrtn/s Blk_read Blk_wrtn

sda 28.06 1000.72 1154.44 1459026910 1683138720

sda1 28.05 992.46 1154.38 1446969730 1683053488

sda2 0.00 0.00 0.00 4 0

sda5 0.01 8.27 0.06 12056864 85232

sdb 30.07 1078.20 1154.44 1571984044 1683138720

sdb1 30.04 1069.93 1154.38 1559927008 1683053488

sdb2 0.00 0.00 0.00 4 0

sdb5 0.04 8.27 0.06 12056720 85232

md0 163.84 1221.37 1143.97 1780718146 1667876624

md1 0.01 0.01 0.05 16216 79440

Is this bad? or normal for a 3 years old server and hard disk?

Is a hardware upgrade an imminent solution for this problem? I'm thinking of an upgrade to the latest Nehalem with SAS (Hardware RAID 10), what do you think?