On 2014-01-09 14:59:45 +0800 (+0800), James E. Blair wrote: [...] > I'd like to move the Zuul git merging component into a separate process > that can be located on a separate host (or hosts) and scaled out. > > The current zuul-server would continue to manage the queue and launch > jobs, but as it processes the queue and decides which changes should be > composed and built into zuul git refs, it would package the info about > each ref and put it on the gearman queue as a work item. An instance of > the new component (zuul-merger) would fetch that job and fetch the > needed refs from Gerrit, and merge them. It would also serve the > resulting git repo in the same way that Zuul does now. > > Zuul would not have to wait for a response before continuing to process > the queue, and since it's not doing any actual work, will be able to > move through the queue _much_ faster than currently. Once Zuul _does_ > receive a completion response from a zuul-merger, it can then launch the > jobs for that change. It will pass the URL for that particular > zuul-merger (as ZUUL_URL) to the jobs so that they know from which > merger to fetch the zuul ref. We can also use the cancel job > functionality in gearman if Zuul decides to reorder the queue. [...]
Aside from a little extra latency queuing each change, I see no obvious downsides. I expect it should overcome the current scaling problem nicely. -- Jeremy Stanley _______________________________________________ OpenStack-Infra mailing list [email protected] http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-infra
