almarma on "[Plugin: Jetpack by WordPress.com] Jetpack uses a lot of resources from the server"

I've been using jetpack in several blogs I run. I was very happy with the features it provides, but in the last weeks my hosting provider alerted me about high CPU usage. I monitored the blogs, and using p3 plugin profiler, to check the resources usage of each plugin I have installed, I discovered that Jetpack uses A LOT of resources. It makes my sites load 2 seconds slower, and that's too much.
Without jetpack, the impact of the plugins on my site load time is 30%. With jetpack it's 65%! And there's no different on how many jetpack plugins I enable or disable. I've disable almost all jetpack plugins, and it's the same delay, so I have them disabled. It's a pity, because there are some that I really like and would love to be able to use :(. Any other with the same problem? Any idea about how I could debug more to see if there is a problem or incompatibility with any other plugin I use?

http://wordpress.org/plugins/jetpack/