I might have found a solution, in my case the website indeed slowed down and this was caused by the admin-ajax.php.. In case you are using WPML to translate your website, deactivate two options:
Under the tab Stringtranslations and at the bottom of that page:
- disable "register strings automaticly for translation"
- disable "Keep track of were strings appear on the website"
The two options to disable might have a slight different naming (I'm using a dutch version so I don't know the exact english title).