Photon only caches your site's images. CSS and JS libraries won't be cached using Photon. You should use another CDN service if you don't want to serve your CSS and JS files from your domain.
You can read more about Photon here:
http://jetpack.me/support/photon/