Kohsuke asked, "What can we do to fix this? Does anyone have any thoughts?"
Proposal with steps 1. Rename host jenkins-ci.org to master.jenkins-ci.org 2. Change jenkins-ci.org to a CNAME (if not already CNAME) 3. Set jenkins-ci.org CNAME to master.jenkins-ci.org 4. Before staging job, set jenkins-ci.org CNAMfE to mirrors.jenkins-ci.org(or cucumber.jenkins-ci.org, etc.) 5. mirrors.jenkins-ci.org push notification to staging job process, or staging job process pull status from mirrors.jenkins-ci.org 6. After staging job recognizes that mirrors have pick up files from master, set jenkins-ci.org CNAME back to master.jenkins-ci.org Thanks, Lloyd On Fri, Sep 7, 2012 at 7:05 PM, Kohsuke Kawaguchi <[email protected]>wrote: > > Jenkins handles a lot of download requests for core as well as plugins. So > we rely on our mirrors to actually deliver bits. > > Unfortunately, by the nature of our mirrors, there's some time lag between > the time we stage our master copy of files (which gets created as a part of > the update center metadata generation process) to the time mirrors pick up > those files. > > But as soon as bits are staged in the master copy, those URLs get > advertised (download link from http://jenkins-ci.org/ and update center > metadata.) Right now, download requests that happened during this window > will result in 404. > > What can we do to fix this? Does anyone have any thoughts? > > > We used to serve binaries from our master server in this situation without > forwarding requests to mirrors, but this was the root cause why we ended up > with the $5000 hole in our budget last year. So naturally I suspect Tyler > will be afraid of enabling this again. > > If someone is willing to be a mirror that allows us to push files, that'd > be ideal, as we can ensure that there's always at least one mirror that has > all the copies all the time. We can set the priority very low to make sure > that it only gets used very rarely. > > Another possibility is maybe to run some kind of bandwidth-restricted HTTP > server. In that way, maybe we can convince ourselves that even in the worst > case we won't go bankrupt. > > The third possibility might be to buy a bigger disk to > cabbage.jenkins-ci.org. My understanding is that this machine is in > OSUOSL, so we can use it to serve bytes without worrying about incurring > overcharge. > > > BTW, right now, we have about 58GB of data that needs to be mirrored. > > -- > Kohsuke Kawaguchi | CloudBees, Inc. | http://cloudbees.com/ > Try Nectar, our professional version of Jenkins >
