[Yahoo-eng-team] [Bug 1783565] Re: ServerGroupTestV21.test_evacuate_with_anti_affinity_no_valid_host intermittently fails with "Instance compute service state on host2 expected to be down, but it was
** Also affects: nova/stein
Importance: Undecided
Status: New
** Also affects: nova/rocky
Importance: Undecided
Status: New
** Also affects: nova/train
Importance: Undecided
Status: New
** Changed in: nova/train
Status: New => Fix Released
** Changed in: nova/train
Importance: Undecided => Low
** Changed in: nova/rocky
Status: New => Confirmed
** Changed in: nova/rocky
Importance: Undecided => Low
** Changed in: nova/stein
Importance: Undecided => Low
--
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to OpenStack Compute (nova).
https://bugs.launchpad.net/bugs/1783565
Title:
ServerGroupTestV21.test_evacuate_with_anti_affinity_no_valid_host
intermittently fails with "Instance compute service state on host2
expected to be down, but it was up."
Status in OpenStack Compute (nova):
Fix Released
Status in OpenStack Compute (nova) rocky series:
Confirmed
Status in OpenStack Compute (nova) stein series:
New
Status in OpenStack Compute (nova) train series:
Fix Released
Bug description:
http://logs.openstack.org/32/584032/5/check/nova-tox-functional-
py35/7061ec1/job-output.txt.gz#_2018-07-25_03_16_46_462415
18-07-25 03:16:46.418499 | ubuntu-xenial | {5}
nova.tests.functional.test_server_group.ServerGroupTestV21.test_evacuate_with_anti_affinity_no_valid_host
[14.070214s] ... FAILED
2018-07-25 03:16:46.418582 | ubuntu-xenial |
2018-07-25 03:16:46.418645 | ubuntu-xenial | Captured traceback:
2018-07-25 03:16:46.418705 | ubuntu-xenial | ~~~
2018-07-25 03:16:46.418798 | ubuntu-xenial | b'Traceback (most recent
call last):'
2018-07-25 03:16:46.419095 | ubuntu-xenial | b' File
"/home/zuul/src/git.openstack.org/openstack/nova/nova/tests/functional/test_server_group.py",
line 456, in test_evacuate_with_anti_affinity_no_valid_host'
2018-07-25 03:16:46.419232 | ubuntu-xenial | b"
self.admin_api.post_server_action(servers[1]['id'], post)"
2018-07-25 03:16:46.419471 | ubuntu-xenial | b' File
"/home/zuul/src/git.openstack.org/openstack/nova/nova/tests/functional/api/client.py",
line 294, in post_server_action'
2018-07-25 03:16:46.419602 | ubuntu-xenial | b"'/servers/%s/action' %
server_id, data, **kwargs).body"
2018-07-25 03:16:46.419841 | ubuntu-xenial | b' File
"/home/zuul/src/git.openstack.org/openstack/nova/nova/tests/functional/api/client.py",
line 235, in api_post'
2018-07-25 03:16:46.419975 | ubuntu-xenial | b'return
APIResponse(self.api_request(relative_uri, **kwargs))'
2018-07-25 03:16:46.420187 | ubuntu-xenial | b' File
"/home/zuul/src/git.openstack.org/openstack/nova/nova/tests/functional/api/client.py",
line 213, in api_request'
2018-07-25 03:16:46.420263 | ubuntu-xenial | b'response=response)'
2018-07-25 03:16:46.420545 | ubuntu-xenial |
b'nova.tests.functional.api.client.OpenStackApiException: Unexpected status
code: {"badRequest": {"message": "Compute service of host2 is still in use.",
"code": 400}}'
2018-07-25 03:16:46.420581 | ubuntu-xenial | b''
2018-07-25 03:16:46.420606 | ubuntu-xenial |
2018-07-25 03:16:46.420654 | ubuntu-xenial | Captured stderr:
2018-07-25 03:16:46.420702 | ubuntu-xenial |
2018-07-25 03:16:46.421102 | ubuntu-xenial |
b'/home/zuul/src/git.openstack.org/openstack/nova/.tox/functional-py35/lib/python3.5/site-packages/oslo_db/sqlalchemy/enginefacade.py:350:
OsloDBDeprecationWarning: EngineFacade is deprecated; please use
oslo_db.sqlalchemy.enginefacade'
2018-07-25 03:16:46.421240 | ubuntu-xenial | b' self._legacy_facade =
LegacyEngineFacade(None, _factory=self)'
2018-07-25 03:16:46.421623 | ubuntu-xenial |
b'/home/zuul/src/git.openstack.org/openstack/nova/.tox/functional-py35/lib/python3.5/site-packages/oslo_db/sqlalchemy/enginefacade.py:350:
OsloDBDeprecationWarning: EngineFacade is deprecated; please use
oslo_db.sqlalchemy.enginefacade'
2018-07-25 03:16:46.421751 | ubuntu-xenial | b' self._legacy_facade =
LegacyEngineFacade(None, _factory=self)'
2018-07-25 03:16:46.422054 | ubuntu-xenial |
b"/home/zuul/src/git.openstack.org/openstack/nova/nova/test.py:323:
DeprecationWarning: Using class 'MoxStubout' (either directly or via
inheritance) is deprecated in version '3.5.0'"
2018-07-25 03:16:46.422174 | ubuntu-xenial | b' mox_fixture =
self.useFixture(moxstubout.MoxStubout())'
2018-07-25 03:16:46.422537 | ubuntu-xenial |
b'/home/zuul/src/git.openstack.org/openstack/nova/.tox/functional-py35/lib/python3.5/site-packages/paste/deploy/loadwsgi.py:22:
DeprecationWarning: Parameters to load are deprecated. Call .resolve and
.require separately.'
2018-07-25 03:16:46.422664 | ubuntu-xenial | b' return
pkg_resources.EntryPoint.parse("x=" + s).load(False)'
2018-07-25 03:16:46.422928 | ubuntu-xenial |
[Yahoo-eng-team] [Bug 1783565] Re: ServerGroupTestV21.test_evacuate_with_anti_affinity_no_valid_host intermittently fails with "Instance compute service state on host2 expected to be down, but it was
Reviewed: https://review.opendev.org/692252
Committed:
https://git.openstack.org/cgit/openstack/nova/commit/?id=1c93ca82b8fb8e39787020decfe335d45f8732e7
Submitter: Zuul
Branch:master
commit 1c93ca82b8fb8e39787020decfe335d45f8732e7
Author: melanie witt
Date: Thu Oct 31 01:57:41 2019 +
Replace time.sleep(10) with service forced_down in tests
The server group functional tests are doing time.sleep(10) in order
to make sure a stopped compute service is considered "down" by the nova
compute API.
Instead of sleeping, we can set the service as "forced_down" to get the
desired "down" compute service status and avoid unnecessary delays in
these tests.
Unnecessary service start() calls are also removed in this change. They
appear at the end of tests and services are started during each test
setUp() and killed during each test tearDown() via the ServiceFixture.
Closes-Bug: #1783565
Change-Id: I74f64b68e4b33ee0f8c45fdc5f570c7e12e05d3b
** Changed in: nova
Status: In Progress => Fix Released
--
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to OpenStack Compute (nova).
https://bugs.launchpad.net/bugs/1783565
Title:
ServerGroupTestV21.test_evacuate_with_anti_affinity_no_valid_host
intermittently fails with "Instance compute service state on host2
expected to be down, but it was up."
Status in OpenStack Compute (nova):
Fix Released
Bug description:
http://logs.openstack.org/32/584032/5/check/nova-tox-functional-
py35/7061ec1/job-output.txt.gz#_2018-07-25_03_16_46_462415
18-07-25 03:16:46.418499 | ubuntu-xenial | {5}
nova.tests.functional.test_server_group.ServerGroupTestV21.test_evacuate_with_anti_affinity_no_valid_host
[14.070214s] ... FAILED
2018-07-25 03:16:46.418582 | ubuntu-xenial |
2018-07-25 03:16:46.418645 | ubuntu-xenial | Captured traceback:
2018-07-25 03:16:46.418705 | ubuntu-xenial | ~~~
2018-07-25 03:16:46.418798 | ubuntu-xenial | b'Traceback (most recent
call last):'
2018-07-25 03:16:46.419095 | ubuntu-xenial | b' File
"/home/zuul/src/git.openstack.org/openstack/nova/nova/tests/functional/test_server_group.py",
line 456, in test_evacuate_with_anti_affinity_no_valid_host'
2018-07-25 03:16:46.419232 | ubuntu-xenial | b"
self.admin_api.post_server_action(servers[1]['id'], post)"
2018-07-25 03:16:46.419471 | ubuntu-xenial | b' File
"/home/zuul/src/git.openstack.org/openstack/nova/nova/tests/functional/api/client.py",
line 294, in post_server_action'
2018-07-25 03:16:46.419602 | ubuntu-xenial | b"'/servers/%s/action' %
server_id, data, **kwargs).body"
2018-07-25 03:16:46.419841 | ubuntu-xenial | b' File
"/home/zuul/src/git.openstack.org/openstack/nova/nova/tests/functional/api/client.py",
line 235, in api_post'
2018-07-25 03:16:46.419975 | ubuntu-xenial | b'return
APIResponse(self.api_request(relative_uri, **kwargs))'
2018-07-25 03:16:46.420187 | ubuntu-xenial | b' File
"/home/zuul/src/git.openstack.org/openstack/nova/nova/tests/functional/api/client.py",
line 213, in api_request'
2018-07-25 03:16:46.420263 | ubuntu-xenial | b'response=response)'
2018-07-25 03:16:46.420545 | ubuntu-xenial |
b'nova.tests.functional.api.client.OpenStackApiException: Unexpected status
code: {"badRequest": {"message": "Compute service of host2 is still in use.",
"code": 400}}'
2018-07-25 03:16:46.420581 | ubuntu-xenial | b''
2018-07-25 03:16:46.420606 | ubuntu-xenial |
2018-07-25 03:16:46.420654 | ubuntu-xenial | Captured stderr:
2018-07-25 03:16:46.420702 | ubuntu-xenial |
2018-07-25 03:16:46.421102 | ubuntu-xenial |
b'/home/zuul/src/git.openstack.org/openstack/nova/.tox/functional-py35/lib/python3.5/site-packages/oslo_db/sqlalchemy/enginefacade.py:350:
OsloDBDeprecationWarning: EngineFacade is deprecated; please use
oslo_db.sqlalchemy.enginefacade'
2018-07-25 03:16:46.421240 | ubuntu-xenial | b' self._legacy_facade =
LegacyEngineFacade(None, _factory=self)'
2018-07-25 03:16:46.421623 | ubuntu-xenial |
b'/home/zuul/src/git.openstack.org/openstack/nova/.tox/functional-py35/lib/python3.5/site-packages/oslo_db/sqlalchemy/enginefacade.py:350:
OsloDBDeprecationWarning: EngineFacade is deprecated; please use
oslo_db.sqlalchemy.enginefacade'
2018-07-25 03:16:46.421751 | ubuntu-xenial | b' self._legacy_facade =
LegacyEngineFacade(None, _factory=self)'
2018-07-25 03:16:46.422054 | ubuntu-xenial |
b"/home/zuul/src/git.openstack.org/openstack/nova/nova/test.py:323:
DeprecationWarning: Using class 'MoxStubout' (either directly or via
inheritance) is deprecated in version '3.5.0'"
2018-07-25 03:16:46.422174 | ubuntu-xenial | b' mox_fixture =
self.useFixture(moxstubout.MoxStubout())'
2018-07-25 03:16:46.422537 | ubuntu-xenial |
[Yahoo-eng-team] [Bug 1783565] Re: ServerGroupTestV21.test_evacuate_with_anti_affinity_no_valid_host intermittently fails with "Instance compute service state on host2 expected to be down, but it was
gibi has a fix here: https://review.opendev.org/692252
** Changed in: nova
Assignee: (unassigned) => Balazs Gibizer (balazs-gibizer)
** Changed in: nova
Status: Invalid => In Progress
** Changed in: nova
Importance: Undecided => Low
--
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to OpenStack Compute (nova).
https://bugs.launchpad.net/bugs/1783565
Title:
ServerGroupTestV21.test_evacuate_with_anti_affinity_no_valid_host
intermittently fails with "Instance compute service state on host2
expected to be down, but it was up."
Status in OpenStack Compute (nova):
In Progress
Bug description:
http://logs.openstack.org/32/584032/5/check/nova-tox-functional-
py35/7061ec1/job-output.txt.gz#_2018-07-25_03_16_46_462415
18-07-25 03:16:46.418499 | ubuntu-xenial | {5}
nova.tests.functional.test_server_group.ServerGroupTestV21.test_evacuate_with_anti_affinity_no_valid_host
[14.070214s] ... FAILED
2018-07-25 03:16:46.418582 | ubuntu-xenial |
2018-07-25 03:16:46.418645 | ubuntu-xenial | Captured traceback:
2018-07-25 03:16:46.418705 | ubuntu-xenial | ~~~
2018-07-25 03:16:46.418798 | ubuntu-xenial | b'Traceback (most recent
call last):'
2018-07-25 03:16:46.419095 | ubuntu-xenial | b' File
"/home/zuul/src/git.openstack.org/openstack/nova/nova/tests/functional/test_server_group.py",
line 456, in test_evacuate_with_anti_affinity_no_valid_host'
2018-07-25 03:16:46.419232 | ubuntu-xenial | b"
self.admin_api.post_server_action(servers[1]['id'], post)"
2018-07-25 03:16:46.419471 | ubuntu-xenial | b' File
"/home/zuul/src/git.openstack.org/openstack/nova/nova/tests/functional/api/client.py",
line 294, in post_server_action'
2018-07-25 03:16:46.419602 | ubuntu-xenial | b"'/servers/%s/action' %
server_id, data, **kwargs).body"
2018-07-25 03:16:46.419841 | ubuntu-xenial | b' File
"/home/zuul/src/git.openstack.org/openstack/nova/nova/tests/functional/api/client.py",
line 235, in api_post'
2018-07-25 03:16:46.419975 | ubuntu-xenial | b'return
APIResponse(self.api_request(relative_uri, **kwargs))'
2018-07-25 03:16:46.420187 | ubuntu-xenial | b' File
"/home/zuul/src/git.openstack.org/openstack/nova/nova/tests/functional/api/client.py",
line 213, in api_request'
2018-07-25 03:16:46.420263 | ubuntu-xenial | b'response=response)'
2018-07-25 03:16:46.420545 | ubuntu-xenial |
b'nova.tests.functional.api.client.OpenStackApiException: Unexpected status
code: {"badRequest": {"message": "Compute service of host2 is still in use.",
"code": 400}}'
2018-07-25 03:16:46.420581 | ubuntu-xenial | b''
2018-07-25 03:16:46.420606 | ubuntu-xenial |
2018-07-25 03:16:46.420654 | ubuntu-xenial | Captured stderr:
2018-07-25 03:16:46.420702 | ubuntu-xenial |
2018-07-25 03:16:46.421102 | ubuntu-xenial |
b'/home/zuul/src/git.openstack.org/openstack/nova/.tox/functional-py35/lib/python3.5/site-packages/oslo_db/sqlalchemy/enginefacade.py:350:
OsloDBDeprecationWarning: EngineFacade is deprecated; please use
oslo_db.sqlalchemy.enginefacade'
2018-07-25 03:16:46.421240 | ubuntu-xenial | b' self._legacy_facade =
LegacyEngineFacade(None, _factory=self)'
2018-07-25 03:16:46.421623 | ubuntu-xenial |
b'/home/zuul/src/git.openstack.org/openstack/nova/.tox/functional-py35/lib/python3.5/site-packages/oslo_db/sqlalchemy/enginefacade.py:350:
OsloDBDeprecationWarning: EngineFacade is deprecated; please use
oslo_db.sqlalchemy.enginefacade'
2018-07-25 03:16:46.421751 | ubuntu-xenial | b' self._legacy_facade =
LegacyEngineFacade(None, _factory=self)'
2018-07-25 03:16:46.422054 | ubuntu-xenial |
b"/home/zuul/src/git.openstack.org/openstack/nova/nova/test.py:323:
DeprecationWarning: Using class 'MoxStubout' (either directly or via
inheritance) is deprecated in version '3.5.0'"
2018-07-25 03:16:46.422174 | ubuntu-xenial | b' mox_fixture =
self.useFixture(moxstubout.MoxStubout())'
2018-07-25 03:16:46.422537 | ubuntu-xenial |
b'/home/zuul/src/git.openstack.org/openstack/nova/.tox/functional-py35/lib/python3.5/site-packages/paste/deploy/loadwsgi.py:22:
DeprecationWarning: Parameters to load are deprecated. Call .resolve and
.require separately.'
2018-07-25 03:16:46.422664 | ubuntu-xenial | b' return
pkg_resources.EntryPoint.parse("x=" + s).load(False)'
2018-07-25 03:16:46.422928 | ubuntu-xenial |
b"/home/zuul/src/git.openstack.org/openstack/nova/nova/db/sqlalchemy/api.py:205:
DeprecationWarning: Property 'async_compat' has moved to 'function.async_'"
2018-07-25 03:16:46.423038 | ubuntu-xenial | b' reader_mode =
get_context_manager(context).async'
2018-07-25 03:16:46.423301 | ubuntu-xenial |
b"/home/zuul/src/git.openstack.org/openstack/nova/nova/db/sqlalchemy/api.py:205:
DeprecationWarning: Property 'async_compat' has moved to 'function.async_'"
2018-07-25
[Yahoo-eng-team] [Bug 1783565] Re: ServerGroupTestV21.test_evacuate_with_anti_affinity_no_valid_host intermittently fails with "Instance compute service state on host2 expected to be down, but it was
We don't seem to be hitting this in the gate anymore so I'm not sure if
it's just rare now or if it's resolved some other way:
http://status.openstack.org/elastic-recheck/#1783565
I'm marking invalid for now though. We can re-open if necessary.
** Changed in: nova
Assignee: Zhenyu Zheng (zhengzhenyu) => (unassigned)
** Changed in: nova
Status: In Progress => Invalid
--
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to OpenStack Compute (nova).
https://bugs.launchpad.net/bugs/1783565
Title:
ServerGroupTestV21.test_evacuate_with_anti_affinity_no_valid_host
intermittently fails with "Instance compute service state on host2
expected to be down, but it was up."
Status in OpenStack Compute (nova):
Invalid
Bug description:
http://logs.openstack.org/32/584032/5/check/nova-tox-functional-
py35/7061ec1/job-output.txt.gz#_2018-07-25_03_16_46_462415
18-07-25 03:16:46.418499 | ubuntu-xenial | {5}
nova.tests.functional.test_server_group.ServerGroupTestV21.test_evacuate_with_anti_affinity_no_valid_host
[14.070214s] ... FAILED
2018-07-25 03:16:46.418582 | ubuntu-xenial |
2018-07-25 03:16:46.418645 | ubuntu-xenial | Captured traceback:
2018-07-25 03:16:46.418705 | ubuntu-xenial | ~~~
2018-07-25 03:16:46.418798 | ubuntu-xenial | b'Traceback (most recent
call last):'
2018-07-25 03:16:46.419095 | ubuntu-xenial | b' File
"/home/zuul/src/git.openstack.org/openstack/nova/nova/tests/functional/test_server_group.py",
line 456, in test_evacuate_with_anti_affinity_no_valid_host'
2018-07-25 03:16:46.419232 | ubuntu-xenial | b"
self.admin_api.post_server_action(servers[1]['id'], post)"
2018-07-25 03:16:46.419471 | ubuntu-xenial | b' File
"/home/zuul/src/git.openstack.org/openstack/nova/nova/tests/functional/api/client.py",
line 294, in post_server_action'
2018-07-25 03:16:46.419602 | ubuntu-xenial | b"'/servers/%s/action' %
server_id, data, **kwargs).body"
2018-07-25 03:16:46.419841 | ubuntu-xenial | b' File
"/home/zuul/src/git.openstack.org/openstack/nova/nova/tests/functional/api/client.py",
line 235, in api_post'
2018-07-25 03:16:46.419975 | ubuntu-xenial | b'return
APIResponse(self.api_request(relative_uri, **kwargs))'
2018-07-25 03:16:46.420187 | ubuntu-xenial | b' File
"/home/zuul/src/git.openstack.org/openstack/nova/nova/tests/functional/api/client.py",
line 213, in api_request'
2018-07-25 03:16:46.420263 | ubuntu-xenial | b'response=response)'
2018-07-25 03:16:46.420545 | ubuntu-xenial |
b'nova.tests.functional.api.client.OpenStackApiException: Unexpected status
code: {"badRequest": {"message": "Compute service of host2 is still in use.",
"code": 400}}'
2018-07-25 03:16:46.420581 | ubuntu-xenial | b''
2018-07-25 03:16:46.420606 | ubuntu-xenial |
2018-07-25 03:16:46.420654 | ubuntu-xenial | Captured stderr:
2018-07-25 03:16:46.420702 | ubuntu-xenial |
2018-07-25 03:16:46.421102 | ubuntu-xenial |
b'/home/zuul/src/git.openstack.org/openstack/nova/.tox/functional-py35/lib/python3.5/site-packages/oslo_db/sqlalchemy/enginefacade.py:350:
OsloDBDeprecationWarning: EngineFacade is deprecated; please use
oslo_db.sqlalchemy.enginefacade'
2018-07-25 03:16:46.421240 | ubuntu-xenial | b' self._legacy_facade =
LegacyEngineFacade(None, _factory=self)'
2018-07-25 03:16:46.421623 | ubuntu-xenial |
b'/home/zuul/src/git.openstack.org/openstack/nova/.tox/functional-py35/lib/python3.5/site-packages/oslo_db/sqlalchemy/enginefacade.py:350:
OsloDBDeprecationWarning: EngineFacade is deprecated; please use
oslo_db.sqlalchemy.enginefacade'
2018-07-25 03:16:46.421751 | ubuntu-xenial | b' self._legacy_facade =
LegacyEngineFacade(None, _factory=self)'
2018-07-25 03:16:46.422054 | ubuntu-xenial |
b"/home/zuul/src/git.openstack.org/openstack/nova/nova/test.py:323:
DeprecationWarning: Using class 'MoxStubout' (either directly or via
inheritance) is deprecated in version '3.5.0'"
2018-07-25 03:16:46.422174 | ubuntu-xenial | b' mox_fixture =
self.useFixture(moxstubout.MoxStubout())'
2018-07-25 03:16:46.422537 | ubuntu-xenial |
b'/home/zuul/src/git.openstack.org/openstack/nova/.tox/functional-py35/lib/python3.5/site-packages/paste/deploy/loadwsgi.py:22:
DeprecationWarning: Parameters to load are deprecated. Call .resolve and
.require separately.'
2018-07-25 03:16:46.422664 | ubuntu-xenial | b' return
pkg_resources.EntryPoint.parse("x=" + s).load(False)'
2018-07-25 03:16:46.422928 | ubuntu-xenial |
b"/home/zuul/src/git.openstack.org/openstack/nova/nova/db/sqlalchemy/api.py:205:
DeprecationWarning: Property 'async_compat' has moved to 'function.async_'"
2018-07-25 03:16:46.423038 | ubuntu-xenial | b' reader_mode =
get_context_manager(context).async'
2018-07-25 03:16:46.423301 | ubuntu-xenial |
