On Mon, Nov 27, 2017, at 04:22, Kevin A. McGrail wrote: > On 11/27/2017 12:06 AM, Dave Warren wrote: >> I’m not currently behind CloudFlare, but I already wrote code to >> purge their cache whenever mirrored content is rsync’d in case I do >> move anything under CloudFlare in the future, or use any other CDN. >> I’m automating a couple mirrors to flip to CloudFlare when there is a >> spike, but I have not enabled this code for SpamAssassin.>> >> Are there cases where any files are updated other than the MIRROR* >> files? Or does this mirror only add files? Basically I’m wondering if >> I should dump the entire cache or just these specific files?> Because the >> items are release artifacts, they are never altered or > removed, just added.> > If you can have a < 10 min cache on these files, that would be fine > > GPG.KEY > index.html > MIRROR.CHECK > MIRRORED.BY > robots.txt >
tl;dr: I set up another mirror that uses CloudFlare's cache, feel free to add it as a mirror, I'd love to compare the results of a traditional host vs the CloudFlare cached version. The new mirror is: sa-update-cf.razx.cloud The details: I've been mucking around with the Cloudflare configuration and my web server's caching rules and I think I've got it. Currently the above list of files are cached but update in realtime, while everything else (.gz, .gz.asc, and .gz.sha1 files in particular) will cache indefinitely if the file exists, or a maximum of 5 minutes for a 404. I've got CloudFlare's security settings and browser verification disabled as it seems unlikely that sa-update can complete a CAPTCHA. For whatever it is worth, traffic through Tor is whitelisted as well, not that I expect to see any sa-update traffic through Tor. This may or may not actually be useful as a sa-mirror, it's as much a learning exercise as anything else for me as I have limited experience with reverse proxies doing anything other than load balancing and SSL termination to private LAN destinations. Plus, I have more and more clients using CloudFlare now, so it's nice to dig into their infrastructure. If you want to see how this works in production, I now have a secondary hostname/mirror routing traffic via CloudFlare, feel free to add it as a mirror and I can report back about the results of the directly hosted sa-update.razx.cloud vs the CloudFlare enabled version. sa-update-cf.razx.cloud Or, if this is a bad/stupid idea, just say so. I'm doing the same for another mirror I operate, so the effort was not wasted, but that mirror is infrequently downloaded large files so it's quite a different testcase.
