Have you tried taking all the plugins out of the plugin folder and same with the themes?
Even though the plugins aren't activated WP still scans through the files. I would start there and run another test.
Other then that I would run a phpinfo and compare the two server configurations to find what the difference is or try a 3rd server just to make sure it's not the codebase. If the codebase is exactly the same then it has to be a server config issue... take a closer look at php accelerators as well.