Re: [openstack-dev] Top Gate Bugs

2013-12-06 Thread Matt Riedemann
On Wednesday, December 04, 2013 7:22:23 AM, Joe Gordon wrote: TL;DR: Gate is failing 23% of the time due to bugs in nova, neutron and tempest. We need help fixing these bugs. Hi All, Before going any further we have a bug that is affecting gate and stable, so its getting top priority here.

Re: [openstack-dev] Top Gate Bugs

2013-12-06 Thread Davanum Srinivas
Joe, Looks like we may be a bit more stable now? Short URL: http://bit.ly/18qq4q2 Long URL :

Re: [openstack-dev] Top Gate Bugs

2013-12-06 Thread Davanum Srinivas
I had the labels wrong - here's a slightly better link - http://bit.ly/1gdxYeg On Fri, Dec 6, 2013 at 4:31 PM, Davanum Srinivas dava...@gmail.com wrote: Joe, Looks like we may be a bit more stable now? Short URL: http://bit.ly/18qq4q2 Long URL :

[openstack-dev] Top Gate Bugs

2013-12-04 Thread Joe Gordon
TL;DR: Gate is failing 23% of the time due to bugs in nova, neutron and tempest. We need help fixing these bugs. Hi All, Before going any further we have a bug that is affecting gate and stable, so its getting top priority here. elastic-recheck currently doesn't track unit tests because we

Re: [openstack-dev] Top Gate Bugs

2013-11-21 Thread Ken'ichi Ohmichi
Hi Clark, 2013/11/21 Clark Boylan clark.boy...@gmail.com: Joe seemed to be on the same track with https://review.openstack.org/#/q/status:open+project:openstack/tempest+branch:master+topic:57578,n,z but went far enough to revert the change that introduced that test. A couple people were

Re: [openstack-dev] Top Gate Bugs

2013-11-21 Thread Matt Riedemann
On Wednesday, November 20, 2013 11:53:45 PM, Clark Boylan wrote: On Wed, Nov 20, 2013 at 9:43 PM, Ken'ichi Ohmichi ken1ohmi...@gmail.com wrote: Hi Joe, 2013/11/20 Joe Gordon joe.gord...@gmail.com: Hi All, As many of you have noticed the gate has been in very bad shape over the past few

Re: [openstack-dev] Top Gate Bugs

2013-11-21 Thread Christopher Yeoh
On Fri, Nov 22, 2013 at 2:28 AM, Matt Riedemann mrie...@linux.vnet.ibm.comwrote: On Wednesday, November 20, 2013 11:53:45 PM, Clark Boylan wrote: On Wed, Nov 20, 2013 at 9:43 PM, Ken'ichi Ohmichi ken1ohmi...@gmail.com wrote: Hi Joe, 2013/11/20 Joe Gordon joe.gord...@gmail.com: Hi All,

Re: [openstack-dev] Top Gate Bugs

2013-11-20 Thread Anita Kuno
Thanks for posting this, Joe. It really helps to create focus so we can address these bugs. We are chatting in #openstack-neutron about 1251784, 1249065, and 1251448. We are looking for someone to work on 1251784 - I had mentioned it at Monday's Neutron team meeting and am trying to shop it

Re: [openstack-dev] Top Gate Bugs

2013-11-20 Thread Derek Higgins
On 20/11/13 14:21, Anita Kuno wrote: Thanks for posting this, Joe. It really helps to create focus so we can address these bugs. We are chatting in #openstack-neutron about 1251784, 1249065, and 1251448. We are looking for someone to work on 1251784 - I had mentioned it at Monday's

Re: [openstack-dev] Top Gate Bugs

2013-11-20 Thread Roman Podoliaka
Hey all, I think I found a serious bug in our usage of eventlet thread local storage. Please check out this snippet [1]. This is how we use eventlet TLS in Nova and common Oslo code [2]. This could explain how [3] actually breaks TripleO devtest story and our gates. Am I right? Or I am missing

Re: [openstack-dev] Top Gate Bugs

2013-11-20 Thread Alex Gaynor
Nope, you're totally right, corolocal.local is a class, whose instances are the actual coroutine local storage. Alex On Wed, Nov 20, 2013 at 9:11 AM, Roman Podoliaka rpodoly...@mirantis.comwrote: Hey all, I think I found a serious bug in our usage of eventlet thread local storage. Please

Re: [openstack-dev] Top Gate Bugs

2013-11-20 Thread Russell Bryant
On 11/20/2013 12:21 PM, Alex Gaynor wrote: Nope, you're totally right, corolocal.local is a class, whose instances are the actual coroutine local storage. But I don't think his example is what is being used. Here is an example using the openstack.common.local module, which is what nova uses

Re: [openstack-dev] Top Gate Bugs

2013-11-20 Thread Robert Collins
We settled on 1251920. https://review.openstack.org/57509 is the fix for that bug. Note that Oslo was fixed on Jun 28th, nova hasn't synced since then. If we were using oslo as a library we would have had the fix as soon as olso did a release. These are the references to strong_store - and thus

Re: [openstack-dev] Top Gate Bugs

2013-11-20 Thread Ken'ichi Ohmichi
Hi Joe, 2013/11/20 Joe Gordon joe.gord...@gmail.com: Hi All, As many of you have noticed the gate has been in very bad shape over the past few days. Here is a list of some of the top open bugs (without pending patches, and many recent hits) that we are hitting. Gate won't be stable, and

Re: [openstack-dev] Top Gate Bugs

2013-11-20 Thread Clark Boylan
On Wed, Nov 20, 2013 at 9:43 PM, Ken'ichi Ohmichi ken1ohmi...@gmail.com wrote: Hi Joe, 2013/11/20 Joe Gordon joe.gord...@gmail.com: Hi All, As many of you have noticed the gate has been in very bad shape over the past few days. Here is a list of some of the top open bugs (without pending

[openstack-dev] Top Gate Bugs

2013-11-19 Thread Joe Gordon
Hi All, As many of you have noticed the gate has been in very bad shape over the past few days. Here is a list of some of the top open bugs (without pending patches, and many recent hits) that we are hitting. Gate won't be stable, and it will be hard to get your code merged, until we fix these