Read the output under "iowait" section, for each CPU. In ideal situations, it should be near to 0%. If you do however are scrutinizing at time of a load spike, consider rechecking these values multiple times to reach a fine conclusion. Anything above 15% is worrysome. Next, you can check the speed of your hard disk drive to see if it's really lagging:
If you do know your hard disk exists on /dev/sda or /dev/hda, just perform the following. Or execute "df -h" command to check which is the drive that your data resides on.
# hdparm -Tt /dev/sda
Timing cached reads: 1484 MB in 2.01 seconds = 739.00 MB/sec
Timing buffered disk reads: 62 MB in 3.00 seconds = 20.66 MB/sec
It was awesome at the buffer-cache reads, most probably because of the disk's onboard cache, however, buffered disk reads is just at 20.66 MB / sec. Anything below 25MB is something you should worry about.
3. CPU power is all consumed?
Check the top output to find out if you're using too much CPU power. You should be looking the value under idle besides each CPU entry. Anything below 45% is something you should really worry about.
III) Problem identified, What's the solution?
To wrap it up, let me offer a few solutions for each problem:
A global solution to all problems is to optimize MySQL, and Webserver including PHP/Perl scripts and queries. Or the least you can do is to optimize Apache and MySQL server parameters to perform better.
1. Too much CPU usage
In "ps -auxf" or "top" look for processes that use too much CPU. If it's HTTP or MySQL, you better optimize your scripts and queries, if possible. In most cases, it's extremely difficult to optimize all the scripts and queries and a better option is to just go for a CPU change/upgrade. A dual CPU should perform better, but what kind of upgrade you're looking for depends on your current CPU.
2. RAM's all exhausted