Re: [openstack-dev] [nova] massive number of new errors in logs with oslo.messaging

2014-02-04 Thread Doug Hellmann
On Sun, Feb 2, 2014 at 7:13 AM, Sean Dague s...@dague.net wrote: Just noticed this at the end of a successful run: http://logs.openstack.org/15/63215/13/check/check-tempest-dsvm-full/2636cae/console.html#_2014-02-02_12_02_44_422 It looks like the merge of oslo.messaging brings a huge amount

Re: [openstack-dev] [nova] massive number of new errors in logs with oslo.messaging

2014-02-04 Thread Dan Smith
And the error messages, which look like this: Returning exception Unexpected task state: expecting [u'scheduling', None] but the actual state is deleting to caller don't make sense -- at least in the English language. It's missing some grouping operator to help with order of operations.

Re: [openstack-dev] [nova] massive number of new errors in logs with oslo.messaging

2014-02-04 Thread Dan Smith
The inner exception is a thing and the outer pieces are a thing. The inner means that some instance update was attempted, but should be aborted if the instance state is not what we think it is. And looking a little closer, I think it means that the messaging.expected_exceptions decorator isn't

Re: [openstack-dev] [nova] massive number of new errors in logs with oslo.messaging

2014-02-04 Thread Andrew Laski
On 02/04/14 at 10:39am, Dan Smith wrote: The inner exception is a thing and the outer pieces are a thing. The inner means that some instance update was attempted, but should be aborted if the instance state is not what we think it is. And looking a little closer, I think it means that the

Re: [openstack-dev] [nova] massive number of new errors in logs with oslo.messaging

2014-02-04 Thread Boris Pavlovic
Monty, One of the bugs that can produce random issues=) https://github.com/openstack/oslo.messaging/blob/master/oslo/messaging/_drivers/impl_rabbit.py#L548 Best regards, Boris Pavlovic On Tue, Feb 4, 2014 at 11:05 PM, Andrew Laski andrew.la...@rackspace.comwrote: On 02/04/14 at 10:39am,

Re: [openstack-dev] [nova] massive number of new errors in logs with oslo.messaging

2014-02-03 Thread Monty Taylor
On 02/02/2014 08:33 PM, Jay Pipes wrote: On Sun, 2014-02-02 at 07:13 -0500, Sean Dague wrote: Just noticed this at the end of a successful run: http://logs.openstack.org/15/63215/13/check/check-tempest-dsvm-full/2636cae/console.html#_2014-02-02_12_02_44_422 It looks like the merge of

Re: [openstack-dev] [nova] massive number of new errors in logs with oslo.messaging

2014-02-03 Thread David Kranz
On 02/03/2014 07:36 AM, Monty Taylor wrote: On 02/02/2014 08:33 PM, Jay Pipes wrote: On Sun, 2014-02-02 at 07:13 -0500, Sean Dague wrote: Just noticed this at the end of a successful run:

[openstack-dev] [nova] massive number of new errors in logs with oslo.messaging

2014-02-02 Thread Sean Dague
Just noticed this at the end of a successful run: http://logs.openstack.org/15/63215/13/check/check-tempest-dsvm-full/2636cae/console.html#_2014-02-02_12_02_44_422 It looks like the merge of oslo.messaging brings a huge amount of false negative error messages in the logs. Would be good if we

Re: [openstack-dev] [nova] massive number of new errors in logs with oslo.messaging

2014-02-02 Thread Jay Pipes
On Sun, 2014-02-02 at 07:13 -0500, Sean Dague wrote: Just noticed this at the end of a successful run: http://logs.openstack.org/15/63215/13/check/check-tempest-dsvm-full/2636cae/console.html#_2014-02-02_12_02_44_422 It looks like the merge of oslo.messaging brings a huge amount of false