Hi, There are some changes to Zuul's behavior that will be arriving beginning this Friday, July 19th. Here's what you need to know:
* Zuul will be more aggressive at dequeuing changes in the gate pipeline when there are merge conflicts. * Zuul will prioritize jobs for the gate pipeline ahead of all others, followed by the post-commit, and then check pipelines Merge conflicts: Currently our projects move so quickly that something like one in every 6 changes that is approved has a merge conflict. Generally this means that Zuul can start testing only about 5 changes before it runs into a change that can't be merged with the changes ahead of it. It then waits until that change reaches the head of the queue, just in case a change ahead fails to merge, and the merge conflict becomes moot. That doesn't usually happen, so the result is that Zuul does not test as many changes as it could in parallel, and developers have to wait quite a while to learn that their change did not merge simply because of a merge conflict. At the Havana summit, we decided it would be better for Zuul to immediately dequeue changes in the gate queue that have a merge conflict with the changes ahead of it. This means that you'll immediately learn that there is a conflict with other changes that have been approved ahead of yours, and Zuul can test and merge more changes in parallel (lowering overall time-to-merge for all changes). If this happens, Zuul will leave a message in Gerrit indicating which changes were enqueued ahead which may have caused the merge conflict: This change was unable to be automatically merged with the current state of the repository and the following changes which were enqueued ahead of it: 1234, 5678. Please rebase your change and upload a new patchset. If you see a message like that, you can either wait for those changes to merge (or not) and rebase on the tip, or you can rebase your change on the changes currently in the queue to be certain it can be merged. Job precedence: We recently changed Zuul to use Gearman as a job dispatching system. Zuul now submits job requests to Gearman, and Jenkins (using the gearman-plugin written by Khai Do) picks up those jobs and runs them. This gives us a lot of new flexibility of which we're going to start taking advantage in the future. One of the first new features this system gives us is the ability to set the precedence of jobs based on our own criteria. We plan on giving jobs in the gate pipeline the highest precedence, so that as long as there are outstanding gate jobs, they will be run as soon as resources are available. Next, we'll allocate resources to post-merge jobs (such as tarball and docs builds and uploads), and finally to the check pipeline. We have a number of other cool features planned, such as reporting on build progress directly on the Zuul status page, and perhaps most importantly, Gearman lets us scale out our Jenkins setup to multiple masters, and even run jobs that aren't in Jenkins at all. As usual, the Infrastructure Team will be available in #openstack-infra to provide assistance as these new features are implemented. -Jim _______________________________________________ OpenStack-dev mailing list [email protected] http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
