On the Drupal site, there's a handy article instructing you on getting the most performance out of your server for the Drupal software.
The performance of your Drupal site is dependent on three main factors: the goals of your site, the resource demands of your site traffic, and the system performance and configuration of underlying technologies.
They seperate it out into three different sections - setting out your performance goals, analysing your site for current traffic/resource consumption, and the actual implementation of the performance settings. They give a few steps here to follow to check what your server is currently using and some links to other tips on tuning the various pieces of the puzzle.
One thing that they mention that's worth repeating to any and all web developers out there: "Apache is bandwidth limited, PHP is CPU limited, and MySQL is memory limited and disk I/O bound".
