Re: [openstack-dev] Top Gate Bugs

2013-12-06 Thread Matt Riedemann
On Wednesday, December 04, 2013 7:22:23 AM, Joe Gordon wrote: TL;DR: Gate is failing 23% of the time due to bugs in nova, neutron and tempest. We need help fixing these bugs. Hi All, Before going any further we have a bug that is affecting gate and stable, so its getting top priority here.

Re: [openstack-dev] Top Gate Bugs

2013-12-06 Thread Davanum Srinivas
Joe, Looks like we may be a bit more stable now? Short URL: http://bit.ly/18qq4q2 Long URL :

Re: [openstack-dev] Top Gate Bugs

2013-12-06 Thread Davanum Srinivas
I had the labels wrong - here's a slightly better link - http://bit.ly/1gdxYeg On Fri, Dec 6, 2013 at 4:31 PM, Davanum Srinivas dava...@gmail.com wrote: Joe, Looks like we may be a bit more stable now? Short URL: http://bit.ly/18qq4q2 Long URL :

Re: [openstack-dev] Top Gate Bugs

2013-11-21 Thread Ken'ichi Ohmichi
Hi Clark, 2013/11/21 Clark Boylan clark.boy...@gmail.com: Joe seemed to be on the same track with https://review.openstack.org/#/q/status:open+project:openstack/tempest+branch:master+topic:57578,n,z but went far enough to revert the change that introduced that test. A couple people were

Re: [openstack-dev] Top Gate Bugs

2013-11-21 Thread Matt Riedemann
On Wednesday, November 20, 2013 11:53:45 PM, Clark Boylan wrote: On Wed, Nov 20, 2013 at 9:43 PM, Ken'ichi Ohmichi ken1ohmi...@gmail.com wrote: Hi Joe, 2013/11/20 Joe Gordon joe.gord...@gmail.com: Hi All, As many of you have noticed the gate has been in very bad shape over the past few

Re: [openstack-dev] Top Gate Bugs

2013-11-21 Thread Christopher Yeoh
On Fri, Nov 22, 2013 at 2:28 AM, Matt Riedemann mrie...@linux.vnet.ibm.comwrote: On Wednesday, November 20, 2013 11:53:45 PM, Clark Boylan wrote: On Wed, Nov 20, 2013 at 9:43 PM, Ken'ichi Ohmichi ken1ohmi...@gmail.com wrote: Hi Joe, 2013/11/20 Joe Gordon joe.gord...@gmail.com: Hi All,

Re: [openstack-dev] Top Gate Bugs

2013-11-20 Thread Anita Kuno
Thanks for posting this, Joe. It really helps to create focus so we can address these bugs. We are chatting in #openstack-neutron about 1251784, 1249065, and 1251448. We are looking for someone to work on 1251784 - I had mentioned it at Monday's Neutron team meeting and am trying to shop it

Re: [openstack-dev] Top Gate Bugs

2013-11-20 Thread Derek Higgins
On 20/11/13 14:21, Anita Kuno wrote: Thanks for posting this, Joe. It really helps to create focus so we can address these bugs. We are chatting in #openstack-neutron about 1251784, 1249065, and 1251448. We are looking for someone to work on 1251784 - I had mentioned it at Monday's

Re: [openstack-dev] Top Gate Bugs

2013-11-20 Thread Roman Podoliaka
Hey all, I think I found a serious bug in our usage of eventlet thread local storage. Please check out this snippet [1]. This is how we use eventlet TLS in Nova and common Oslo code [2]. This could explain how [3] actually breaks TripleO devtest story and our gates. Am I right? Or I am missing

Re: [openstack-dev] Top Gate Bugs

2013-11-20 Thread Alex Gaynor
Nope, you're totally right, corolocal.local is a class, whose instances are the actual coroutine local storage. Alex On Wed, Nov 20, 2013 at 9:11 AM, Roman Podoliaka rpodoly...@mirantis.comwrote: Hey all, I think I found a serious bug in our usage of eventlet thread local storage. Please

Re: [openstack-dev] Top Gate Bugs

2013-11-20 Thread Russell Bryant
On 11/20/2013 12:21 PM, Alex Gaynor wrote: Nope, you're totally right, corolocal.local is a class, whose instances are the actual coroutine local storage. But I don't think his example is what is being used. Here is an example using the openstack.common.local module, which is what nova uses

Re: [openstack-dev] Top Gate Bugs

2013-11-20 Thread Robert Collins
We settled on 1251920. https://review.openstack.org/57509 is the fix for that bug. Note that Oslo was fixed on Jun 28th, nova hasn't synced since then. If we were using oslo as a library we would have had the fix as soon as olso did a release. These are the references to strong_store - and thus

Re: [openstack-dev] Top Gate Bugs

2013-11-20 Thread Ken'ichi Ohmichi
Hi Joe, 2013/11/20 Joe Gordon joe.gord...@gmail.com: Hi All, As many of you have noticed the gate has been in very bad shape over the past few days. Here is a list of some of the top open bugs (without pending patches, and many recent hits) that we are hitting. Gate won't be stable, and

Re: [openstack-dev] Top Gate Bugs

2013-11-20 Thread Clark Boylan
On Wed, Nov 20, 2013 at 9:43 PM, Ken'ichi Ohmichi ken1ohmi...@gmail.com wrote: Hi Joe, 2013/11/20 Joe Gordon joe.gord...@gmail.com: Hi All, As many of you have noticed the gate has been in very bad shape over the past few days. Here is a list of some of the top open bugs (without pending