Public bug reported:

There have been several occurences of this bug in check/gate queues.

http://logstash.openstack.org/#eyJzZWFyY2giOiJtZXNzYWdlOlwiKE9wZXJhdGlvbmFsRXJyb3IpICgxMjA1LCAnTG9jayB3YWl0IHRpbWVvdXQgZXhjZWVkZWQ7IHRyeSByZXN0YXJ0aW5nIHRyYW5zYWN0aW9uJylcIiAgQU5EIG1lc3NhZ2U6XCJVUERBVEUgcG9ydHMgU0VUIHN0YXR1c1wiIEFORCBOT1QgbWVzc2FnZTpcIlRyYWNlYmFjayAobW9zdCByZWNlbnQgY2FsbCBsYXN0XCIgQU5EIHRhZ3M6XCJzY3JlZW4tcS1zdmMudHh0XCIgQU5EIE5PVCBidWlsZF9uYW1lOlwiY2hlY2stdGVtcGVzdC1tYXN0ZXItZHN2bS1uZXV0cm9uLWhhdmFuYVwiIiwiZmllbGRzIjpbXSwib2Zmc2V0IjowLCJ0aW1lZnJhbWUiOiI2MDQ4MDAiLCJncmFwaG1vZGUiOiJjb3VudCIsInRpbWUiOnsidXNlcl9pbnRlcnZhbCI6MH0sInN0YW1wIjoxMzk4NDM5MjQ0NDY2fQ==


266 hits in 7 days at the time of bug report, excluding havana jobs (which are 
susceptible to lock wait timeout errors anyway because of fixes not 
backportable from icehouse)
Build failure rate: 20.6%
Hits in gate queue: 26
Failures in gate queue: 1

Notes:
1) Occurrences and failure rate in gate queue are lower because most of the 
failure happen with the full job, which is not yet voting.
2) Even if failure rate is generally low, a lock wait timeout should be always 
considered an error, regardless of the outcome of the build job.
3) A detailed look at the logs reveals a pattern similar to bug 1283522, whose 
fingerprint is being matched. It seems the semaphore lock is ignored, but 
lockutils lacks the necessary logging to reveal whether a semaphore has been 
released or not; more investigations are in progress.

** Affects: neutron
     Importance: High
     Assignee: Salvatore Orlando (salvatore-orlando)
         Status: New

** Changed in: neutron
     Assignee: (unassigned) => Salvatore Orlando (salvatore-orlando)

** Changed in: neutron
   Importance: Undecided => High

-- 
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to neutron.
https://bugs.launchpad.net/bugs/1312964

Title:
  lock wait timeout in update_port_status

Status in OpenStack Neutron (virtual network service):
  New

Bug description:
  There have been several occurences of this bug in check/gate queues.

  
http://logstash.openstack.org/#eyJzZWFyY2giOiJtZXNzYWdlOlwiKE9wZXJhdGlvbmFsRXJyb3IpICgxMjA1LCAnTG9jayB3YWl0IHRpbWVvdXQgZXhjZWVkZWQ7IHRyeSByZXN0YXJ0aW5nIHRyYW5zYWN0aW9uJylcIiAgQU5EIG1lc3NhZ2U6XCJVUERBVEUgcG9ydHMgU0VUIHN0YXR1c1wiIEFORCBOT1QgbWVzc2FnZTpcIlRyYWNlYmFjayAobW9zdCByZWNlbnQgY2FsbCBsYXN0XCIgQU5EIHRhZ3M6XCJzY3JlZW4tcS1zdmMudHh0XCIgQU5EIE5PVCBidWlsZF9uYW1lOlwiY2hlY2stdGVtcGVzdC1tYXN0ZXItZHN2bS1uZXV0cm9uLWhhdmFuYVwiIiwiZmllbGRzIjpbXSwib2Zmc2V0IjowLCJ0aW1lZnJhbWUiOiI2MDQ4MDAiLCJncmFwaG1vZGUiOiJjb3VudCIsInRpbWUiOnsidXNlcl9pbnRlcnZhbCI6MH0sInN0YW1wIjoxMzk4NDM5MjQ0NDY2fQ==

  
  266 hits in 7 days at the time of bug report, excluding havana jobs (which 
are susceptible to lock wait timeout errors anyway because of fixes not 
backportable from icehouse)
  Build failure rate: 20.6%
  Hits in gate queue: 26
  Failures in gate queue: 1

  Notes:
  1) Occurrences and failure rate in gate queue are lower because most of the 
failure happen with the full job, which is not yet voting.
  2) Even if failure rate is generally low, a lock wait timeout should be 
always considered an error, regardless of the outcome of the build job.
  3) A detailed look at the logs reveals a pattern similar to bug 1283522, 
whose fingerprint is being matched. It seems the semaphore lock is ignored, but 
lockutils lacks the necessary logging to reveal whether a semaphore has been 
released or not; more investigations are in progress.

To manage notifications about this bug go to:
https://bugs.launchpad.net/neutron/+bug/1312964/+subscriptions

-- 
Mailing list: https://launchpad.net/~yahoo-eng-team
Post to     : yahoo-eng-team@lists.launchpad.net
Unsubscribe : https://launchpad.net/~yahoo-eng-team
More help   : https://help.launchpad.net/ListHelp

Reply via email to