On 24 August 2016 at 10:30, Armando M. <arma...@gmail.com> wrote: > > > On 22 August 2016 at 18:25, Armando M. <arma...@gmail.com> wrote: > >> >> >> On 22 August 2016 at 09:51, Armando M. <arma...@gmail.com> wrote: >> >>> Neutrinos, >>> >>> Since the merge of test [1], a race has surfaced, which leads to failure >>> as reported in [2]. A number of Neutron's jobs are affected, and even >>> though the failure is only intermittent, it is particularly bad for the >>> linux bridge job. >>> >>> If I can ask you to hold your +2/+W until [3] puts the fire out, zuul >>> will be grateful. >>> >> >> An update: >> >> We have [1] in the gate queue, and Kevin is going full steam with [2], >> which will mitigate the other intermittent issues we're facing in >> functional and unit tests. Please bear with us. >> >> Cheers, >> Armando >> >> [1] https://review.openstack.org/#/c/358753/ >> [2] https://review.openstack.org/#/c/356530/ >> >> >>> >>> Thanks, >>> Armando >>> >>> [1] https://review.openstack.org/#/c/327191/ >>> [2] https://bugs.launchpad.net/neutron/+bug/1615710 >>> [3] https://review.openstack.org/#/c/358753/ >>> >> > > One more update: > > We're not quite out of the woods, yet so approve/recheck gently please. > > Things have become slightly better but we still have a pending fix for > Neutron [1]. Recently we also observed zuul nodes dying unexpectedly on the > OSIC cloud (that runs Neutron on IPv6) and that had an impact on the > overall sluggishness of the gate [2]. Once all of the fixes are in, we > should be back in business...until the next fire! >
Another (final) update: Changes [1] have merged. That should take care of the infra node resets. At this point, there's [2] to wrap up, but things look largely recovered [3,4] (though we need [5] to fully see a current picture). Please keep an eye on [6], and help report/address critical fixes in the busy times ahead. Cheers, Armando [1] https://review.openstack.org/#/q/Ia044fff2a1731ab6c04f82aea47096b425e0c0a0,n,z [2] https://review.openstack.org/#/c/356530/ [3] http://grafana.openstack.org/dashboard/db/neutron-failure-rate?panelId=3&fullscreen [4] http://grafana.openstack.org/dashboard/db/neutron-failure-rate?panelId=5&fullscreen [5] https://review.openstack.org/#/c/358462/ [6] https://bugs.launchpad.net/neutron/+bugs?field.status%3Alist=NEW&field.status%3Alist=CONFIRMED&field.tag=gate-failure > > Cheers, > Armando > > [1] https://review.openstack.org/#/c/356530/ > [2] https://bugs.launchpad.net/neutron/+bug/1616282 > >
__________________________________________________________________________ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev