Antoine Musso <[email protected]> writes: > Seems the bottleneck is 400 or so fetches against a single host (Zuul). > Maybe you could use the Gerrit replication system to replicate the git > repositories to some mirror hosts then have Zuul push its references to > them. > > Since Zuul publish the git repository to fetch from (ZUUL_URL), Zuul > could push the reference to a single mirror and give that mirror to the > job. One thing you would need is make sure the replication is properly > working though.
We actually added the ability for Zuul to push to any location. We already know that having 400 hosts hit gerrit would just kill Gerrit, so we pushed to our load-balanced farm of 4 git servers. However, just pushing all those refs to servers over ssh took an additional 20 seconds for each change. We could look into making ssh more efficient, but I think that the separate merger solution is better because not only does it help with the load for git fetches, but it can make zuul's queue processing itself much faster and more responsive (it currently takes zuul several minutes to process a gate reset for a very deep queue, and it doesn't do anything else during that time but with this change, gate resets should take seconds). -Jim _______________________________________________ OpenStack-Infra mailing list [email protected] http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-infra
