On Wednesday, November 20, 2013 11:53:45 PM, Clark Boylan wrote:
On Wed, Nov 20, 2013 at 9:43 PM, Ken'ichi Ohmichi <ken1ohmi...@gmail.com> wrote:
Hi Joe,
2013/11/20 Joe Gordon <joe.gord...@gmail.com>:
Hi All,
As many of you have noticed the gate has been in very bad shape over the
past few days. Here is a list of some of the top open bugs (without pending
patches, and many recent hits) that we are hitting. Gate won't be stable,
and it will be hard to get your code merged, until we fix these bugs.
1) https://bugs.launchpad.net/bugs/1251920
nova
468 Hits
Can we know the frequency of each failure?
I'm trying 1251920 and putting the investigation tempest patch.
https://review.openstack.org/#/c/57193/
The patch can avoid this problem 4 times, but I am not sure this is
worth or not.
Thanks
Ken'ichi Ohmichi
---
2) https://bugs.launchpad.net/bugs/1251784
neutron, Nova
328 Hits
3) https://bugs.launchpad.net/bugs/1249065
neutron
122 hits
4) https://bugs.launchpad.net/bugs/1251448
neutron
65 Hits
Raw Data:
Note: If a bug has any hits for anything besides failure, it means the
fingerprint isn't perfect.
Elastic recheck known issues
Bug: https://bugs.launchpad.net/bugs/1251920 => message:"assertionerror:
console output was empty" AND filename:"console.html" Title: Tempest
failures due to failure to return console logs from an instance Project:
Status nova: Confirmed Hits FAILURE: 468 Bug:
https://bugs.launchpad.net/bugs/1251784 => message:"Connection to neutron
failed: Maximum attempts reached" AND filename:"logs/screen-n-cpu.txt"
Title: nova+neutron scheduling error: Connection to neutron failed: Maximum
attempts reached Project: Status neutron: New nova: New Hits FAILURE: 328
UNSTABLE: 13 SUCCESS: 275 Bug: https://bugs.launchpad.net/bugs/1240256 =>
message:" 503" AND filename:"logs/syslog.txt" AND
syslog_program:"proxy-server" Title: swift proxy-server returning 503 during
tempest run Project: Status openstack-ci: Incomplete swift: New tempest: New
Hits FAILURE: 136 SUCCESS: 83
Pending Patch Bug: https://bugs.launchpad.net/bugs/1249065 => message:"No
nw_info cache associated with instance" AND filename:"logs/screen-n-api.txt"
Title: Tempest failure: tempest/scenario/test_snapshot_pattern.py Project:
Status neutron: New nova: Confirmed Hits FAILURE: 122 Bug:
https://bugs.launchpad.net/bugs/1252514 => message:"Got error from Swift:
put_object" AND filename:"logs/screen-g-api.txt" Title: glance doesn't
recover if Swift returns an error Project: Status devstack: New glance: New
swift: New Hits FAILURE: 95
Pending Patch Bug: https://bugs.launchpad.net/bugs/1244255 =>
message:"NovaException: Unexpected vif_type=binding_failed" AND
filename:"logs/screen-n-cpu.txt" Title: binding_failed because of l2 agent
assumed down Project: Status neutron: Fix Committed Hits FAILURE: 92
SUCCESS: 29 Bug: https://bugs.launchpad.net/bugs/1251448 => message:"
possible networks found, use a Network ID to be more specific. (HTTP 400)"
AND filename:"console.html" Title: BadRequest: Multiple possible networks
found, use a Network ID to be more specific. Project: Status neutron: New
Hits FAILURE: 65 Bug: https://bugs.launchpad.net/bugs/1239856 =>
message:"tempest/services" AND message:"/images_client.py" AND
message:"wait_for_image_status" AND filename:"console.html" Title:
"TimeoutException: Request timed out" on
tempest.api.compute.images.test_list_image_filters.ListImageFiltersTestXML
Project: Status glance: New Hits FAILURE: 62 Bug:
https://bugs.launchpad.net/bugs/1235435 => message:"One or more ports have
an IP allocation from this subnet" AND message:" SubnetInUse: Unable to
complete operation on subnet" AND filename:"logs/screen-q-svc.txt" Title:
'SubnetInUse: Unable to complete operation on subnet UUID. One or more ports
have an IP allocation from this subnet.' Project: Status neutron: Incomplete
nova: Fix Committed tempest: New Hits FAILURE: 48 Bug:
https://bugs.launchpad.net/bugs/1224001 =>
message:"tempest.scenario.test_network_basic_ops AssertionError: Timed out
waiting for" AND filename:"console.html" Title: test_network_basic_ops fails
waiting for network to become available Project: Status neutron: In Progress
swift: Invalid tempest: Invalid Hits FAILURE: 42 Bug:
https://bugs.launchpad.net/bugs/1218391 => message:"Cannot 'createImage'"
AND filename:"console.html" Title:
tempest.api.compute.images.test_images_oneserver.ImagesOneServerTestXML.test_delete_image_that_is_not_yet_active
spurious failure Project: Status nova: Confirmed swift: Confirmed tempest:
Confirmed Hits FAILURE: 25
best,
Joe Gordon
_______________________________________________
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
_______________________________________________
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Joe seemed to be on the same track with
https://review.openstack.org/#/q/status:open+project:openstack/tempest+branch:master+topic:57578,n,z
but went far enough to revert the change that introduced that test. A
couple people were going to keep hitting those changes to run them
through more tests and see if 1251920 goes away.
I don't quite understand why this test is problematic (Joe indicated
it went in at about the time 1251920 became a problem). I would be
very interested in finding out why this caused a problem.
You can see frequencies for bugs with known signatures at
http://status.openstack.org/elastic-recheck/
Hope this helps.
Clark
_______________________________________________
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Joe is tracking some notes in an etherpad here:
https://etherpad.openstack.org/p/critical-patches-gatecrash-November-2013
I've added https://review.openstack.org/#/c/57069/ and
https://review.openstack.org/#/c/57042/ to the list.
--
Thanks,
Matt Riedemann
_______________________________________________
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
