I have the same problem!
The code is not working for me. Deactivating the email function cannot be the solution, have not tried it yet but I want to keep this function alive.
The load time of the admin-ajax.php takes somewhere between 1 and 1,6 seconds!
Have also a VPS 2gb ram and wordpress gets 128mb of it.
Would be nice to get help on this issue.
