This has returned in liberty and is currently blocking grenade in much the same way as last time.
https://review.openstack.org/#/c/190175/ is most likely part of the problem. It looks like after we ensured that SIGTERM actually terminated the program at the kilo release, folks from hds.com complained that some work units weren't processed after that patch, so it was flipped back to a graceful version which is possible will never exit. Current unit tests in oslo.service have gotten much flakier since then as well, which may be related. ** Also affects: oslo.service Importance: Undecided Status: New ** Changed in: oslo.service Importance: Undecided => Critical ** Summary changed: - services no longer reliably stop in stable/kilo + services no longer reliably stop in stable/liberty / master ** Description changed: In attempting to upgrade the upgrade branch structure to support stable/kilo -> master in devstack gate, we found the project could no longer pass Grenade testing. The reason is because pkill -g is no longer reliably killing off the services: http://logs.openstack.org/91/175391/5/gate/gate-grenade- dsvm/0ad4a94/logs/grenade.sh.txt.gz#_2015-04-21_03_15_31_436 It has been seen with keystone-all and cinder-api on this patch series: http://logstash.openstack.org/#eyJzZWFyY2giOiJtZXNzYWdlOlwiVGhlIGZvbGxvd2luZyBzZXJ2aWNlcyBhcmUgc3RpbGwgcnVubmluZ1wiIEFORCBtZXNzYWdlOlwiZGllXCIiLCJmaWVsZHMiOltdLCJvZmZzZXQiOjAsInRpbWVmcmFtZSI6IjE3MjgwMCIsImdyYXBobW9kZSI6ImNvdW50IiwidGltZSI6eyJ1c2VyX2ludGVydmFsIjowfSwic3RhbXAiOjE0Mjk2MTU0NTQ2MzB9 There were a number of changes to the oslo-incubator service.py code during kilo, it's unclear at this point which is the issue. + + Note: this has returned in stable/liberty / master and oslo.service, see + comment #50 for where this reemerges. -- You received this bug notification because you are a member of Yahoo! Engineering Team, which is subscribed to OpenStack Compute (nova). https://bugs.launchpad.net/bugs/1446583 Title: services no longer reliably stop in stable/liberty / master Status in Cinder: Fix Released Status in Cinder kilo series: Fix Released Status in Keystone: Fix Released Status in Keystone kilo series: Fix Released Status in OpenStack Compute (nova): Fix Released Status in OpenStack Compute (nova) kilo series: Fix Released Status in oslo-incubator: Fix Released Status in oslo.service: New Bug description: In attempting to upgrade the upgrade branch structure to support stable/kilo -> master in devstack gate, we found the project could no longer pass Grenade testing. The reason is because pkill -g is no longer reliably killing off the services: http://logs.openstack.org/91/175391/5/gate/gate-grenade- dsvm/0ad4a94/logs/grenade.sh.txt.gz#_2015-04-21_03_15_31_436 It has been seen with keystone-all and cinder-api on this patch series: http://logstash.openstack.org/#eyJzZWFyY2giOiJtZXNzYWdlOlwiVGhlIGZvbGxvd2luZyBzZXJ2aWNlcyBhcmUgc3RpbGwgcnVubmluZ1wiIEFORCBtZXNzYWdlOlwiZGllXCIiLCJmaWVsZHMiOltdLCJvZmZzZXQiOjAsInRpbWVmcmFtZSI6IjE3MjgwMCIsImdyYXBobW9kZSI6ImNvdW50IiwidGltZSI6eyJ1c2VyX2ludGVydmFsIjowfSwic3RhbXAiOjE0Mjk2MTU0NTQ2MzB9 There were a number of changes to the oslo-incubator service.py code during kilo, it's unclear at this point which is the issue. Note: this has returned in stable/liberty / master and oslo.service, see comment #50 for where this reemerges. To manage notifications about this bug go to: https://bugs.launchpad.net/cinder/+bug/1446583/+subscriptions -- Mailing list: https://launchpad.net/~yahoo-eng-team Post to : [email protected] Unsubscribe : https://launchpad.net/~yahoo-eng-team More help : https://help.launchpad.net/ListHelp
