Indeed, it's the latency, not the size, that's the problem. While almost everything is cached for users today, most CDNs are configured in such a way that the browsers still try to validate the cache often, paying almost all the latency.
One 250KB file that contains everything (html, js, possibly even images in data;base64) is -- for many users -- much faster than 5 10KB files each pulled from a different CDN/server (and possibly even from the same server). Also, making sure your cache is valid essentially forever (and breaking it via embedding e.g. hash in the URL) is very helpful and rarely practiced when considering caches.
