So I think the issue is, in fact, the WordPress SEO plugin. After installing 2.8, it wouldn't work; with WordPress SEO out of the loop, it does work.
Jetpack 2.8 is compatible with WordPress SEO, so I'm not sure what happened there. You might want to give it another try, and make sure that your WordPress SEO settings are correct under SEO > Social in your dashboard. You'll need to activate the OPen Graph meta tags there, or WordPress SEO won't add any tags to your posts.
still not 100% happy about the random picture that Facebook picks, though. Got to be a way to fix that.
When you share a post on Facebook, or when Jetpack Publicize publishes a post to your Facebook page, Facebook crawls the page and looks for Open Graph meta tags in the head to build a complete post preview (with an image, title, description, ...).
Jetpack and WordPress SEO use similar methods to create an image meta tag for Facebook to use; the 2 plugins look for an image attached or inserted into your post.
If you want to display a specific image on Facebook, you'll consequently need to insert an image into your post, or define a Featured Image. It's worth noting that this image has to be larger than 200x200px, or Facebook will simply ignore it.
If you don't wish to add an image into your posts, you could follow the instructions here to force Jetpack to add a specific fallback image tag when no other image can be found in the post:
http://jetpack.me/2013/10/15/add-a-default-fallback-image-if-no-image/
If you use WordPress SEO, it offers a similar alternative under SEO > Social, in the "Default settings" section.
I hope this helps.