Jenkins handles a lot of download requests for core as well as plugins.
So we rely on our mirrors to actually deliver bits.
Unfortunately, by the nature of our mirrors, there's some time lag
between the time we stage our master copy of files (which gets created
as a part of the update center metadata generation process) to the time
mirrors pick up those files.
But as soon as bits are staged in the master copy, those URLs get
advertised (download link from http://jenkins-ci.org/ and update center
metadata.) Right now, download requests that happened during this window
will result in 404.
What can we do to fix this? Does anyone have any thoughts?
We used to serve binaries from our master server in this situation
without forwarding requests to mirrors, but this was the root cause why
we ended up with the $5000 hole in our budget last year. So naturally I
suspect Tyler will be afraid of enabling this again.
If someone is willing to be a mirror that allows us to push files,
that'd be ideal, as we can ensure that there's always at least one
mirror that has all the copies all the time. We can set the priority
very low to make sure that it only gets used very rarely.
Another possibility is maybe to run some kind of bandwidth-restricted
HTTP server. In that way, maybe we can convince ourselves that even in
the worst case we won't go bankrupt.
The third possibility might be to buy a bigger disk to
cabbage.jenkins-ci.org. My understanding is that this machine is in
OSUOSL, so we can use it to serve bytes without worrying about incurring
overcharge.
BTW, right now, we have about 58GB of data that needs to be mirrored.
--
Kohsuke Kawaguchi | CloudBees, Inc. | http://cloudbees.com/
Try Nectar, our professional version of Jenkins
