I don't use the download() function anymore and serve static files straight from Nginx and it works very well.
I am also looking at a very cool new service, CloudFlare that is free! What you do is switch your DNS entires to CloudFlare-provided DNS servers and then CloudFlare caches all your images as well as provides threat-protection and other good stuff. It's basically a free way to speed up your web site and only takes about 5 minutes to set up. The one caveat is that you have to pay for SSL.
