Do you have another working email script on your website that you could compare results with? Aside from Jetpack, I mean? I've had up to a 12 hour delay on WordPress email simply due to server load sometimes. If you use another form to test with that uses the wp_mail(); function, we could rule out server delay on that function at least.
For what it's worth, I'm running two websites with the Jetpack contact form enabled, neither of which run any other plugins. One is running WordPress 3.5.1, one running WordPress 3.5.2, and both send to multiple email addresses.