Re: [openstack-dev] [CI][Keystone][Requirements][Release] What happened to the gate on Feb 4th?
On 02/04/2018 10:44 PM, Qiming Teng wrote: > Starting about 24 hours ago, we have been notified CI gate failure > although we haven't changed anything to our project locally. Before that > we have spent quite some time making the out-of-tree tempest plugins > work on gate. > > After checking the log again and again ... we found the following logs > from Keystone: > > Feb 05 03:31:12.609492 ubuntu-xenial-ovh-gra1-0002362092 > devstack@keystone.service[24845]: WARNING keystone.common.wsgi [None > req-dfcbf106-fbf5-41bd-9012-3c65d1de5f9a None admin] Could not find > project: service.: ProjectNotFound: Could not find project: service. > > Feb 05 03:31:13.845694 ubuntu-xenial-ovh-gra1-0002362092 > devstack@keystone.service[24845]: WARNING keystone.common.wsgi [None > req-50feed46-7c15-425d-bec7-1b4a7ccf6859 None admin] Could not find > service: clustering.: ServiceNotFound: Could not find service: > clustering. > > Feb 05 03:31:12.552647 ubuntu-xenial-ovh-gra1-0002362092 > devstack@keystone.service[24845]: WARNING keystone.common.wsgi [None > req-0a5e660f-dad6-4779-aea4-dd6969c728e6 None admin] Could not find > domain: Default.: DomainNotFound: Could not find domain: Default. > > Feb 05 03:31:12.441128 ubuntu-xenial-ovh-gra1-0002362092 > devstack@keystone.service[24845]: WARNING keystone.common.wsgi [None > req-7eb9ed90-28fc-40aa-8a41-d560f7a156c9 None admin] Could not find > user: senlin.: UserNotFound: Could not find user: senlin. > > Feb 05 03:31:12.336572 ubuntu-xenial-ovh-gra1-0002362092 > devstack@keystone.service[24845]: WARNING keystone.common.wsgi [None > req-19e52d02-5471-49a2-8acd-360199d8c6e0 None admin] Could not find > role: admin.: RoleNotFound: Could not find role: admin. > > Feb 05 03:28:33.797665 ubuntu-xenial-ovh-gra1-0002362092 > devstack@keystone.service[24845]: WARNING keystone.common.wsgi [None > req-544cd822-18a4-4f7b-913d-297716418239 None admin] Could not find > user: glance.: UserNotFound: Could not find user: glance. > > Feb 05 03:28:29.993214 ubuntu-xenial-ovh-gra1-0002362092 > devstack@keystone.service[24845]: WARNING py.warnings [None > req-dc411d9c-6ab9-44e3-9afb-20e5e7034f12 None admin] > /usr/local/lib/python2.7/dist-packages/oslo_policy/policy.py:865: > UserWarning: Policy identity:create_endpoint failed scope check. The > token used to make the request was project scoped but the policy > requires ['system'] scope. This behavior may change in the future where > using the intended scope is required > > Feb 05 03:28:29.920892 ubuntu-xenial-ovh-gra1-0002362092 > devstack@keystone.service[24845]: WARNING keystone.common.wsgi [None > req-32a4a378-d6d3-411e-9842-2178e577af27 None admin] Could not find > service: compute.: ServiceNotFound: Could not find service: compute. These are all WARNINGS messages. If this is a tempest run, these are probably from negative testing [0], in which case keystone is doing the correct thing. The warnings you've pasted are also present in successful tempest runs [1]. Can you provide a link to a patch that's failing? What project do you work on? [0] https://github.com/openstack/tempest/blob/master/tempest/api/identity/admin/v3/test_projects_negative.py [1] http://logs.openstack.org/57/540557/2/check/tempest-full/bbd7cdd/controller/logs/screen-keystone.txt.gz?level=WARNING > > > > -- > > So I'm wondering what the hack happened? Keystone version bump? > Devstack changed? Tempest settings changed? > Why are we merging these changes near the end of a cycle when people are > focusing on stabilizing things? The original feature freeze date was 10 days ago [2] and with the condition of the gate during that time, there were several projects trailing with merging feature. Keystone was one of them and we issued feature freeze exceptions for those efforts [3] [4] [5]. Based on the warnings you've reported, I'm not convinced any of those efforts are affecting CI in a negative way, especially since we're still getting support into tempest to test those features. [2] https://releases.openstack.org/queens/schedule.html#q-keystone-ffreeze [3] http://lists.openstack.org/pipermail/openstack-dev/2018-January/126587.html [4] http://lists.openstack.org/pipermail/openstack-dev/2018-January/126588.html [5] http://lists.openstack.org/pipermail/openstack-dev/2018-January/126589.html > Any hints on these are highly appreciated. > > - Qiming > > > __ > OpenStack Development Mailing List (not for usage questions) > Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev signature.asc Description: OpenPGP digital signature __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack
Re: [openstack-dev] [CI][Keystone][Requirements][Release] What happened to the gate on Feb 4th?
On 2018-02-05 05:44, Qiming Teng wrote: > Starting about 24 hours ago, we have been notified CI gate failure > although we haven't changed anything to our project locally. Before that > we have spent quite some time making the out-of-tree tempest plugins > work on gate. What is *your* project? PLease provide also links to full failure logs, Andreas > After checking the log again and again ... we found the following logs > from Keystone: > > Feb 05 03:31:12.609492 ubuntu-xenial-ovh-gra1-0002362092 > devstack@keystone.service[24845]: WARNING keystone.common.wsgi [None > req-dfcbf106-fbf5-41bd-9012-3c65d1de5f9a None admin] Could not find > project: service.: ProjectNotFound: Could not find project: service. > > Feb 05 03:31:13.845694 ubuntu-xenial-ovh-gra1-0002362092 > devstack@keystone.service[24845]: WARNING keystone.common.wsgi [None > req-50feed46-7c15-425d-bec7-1b4a7ccf6859 None admin] Could not find > service: clustering.: ServiceNotFound: Could not find service: > clustering. > > Feb 05 03:31:12.552647 ubuntu-xenial-ovh-gra1-0002362092 > devstack@keystone.service[24845]: WARNING keystone.common.wsgi [None > req-0a5e660f-dad6-4779-aea4-dd6969c728e6 None admin] Could not find > domain: Default.: DomainNotFound: Could not find domain: Default. > > Feb 05 03:31:12.441128 ubuntu-xenial-ovh-gra1-0002362092 > devstack@keystone.service[24845]: WARNING keystone.common.wsgi [None > req-7eb9ed90-28fc-40aa-8a41-d560f7a156c9 None admin] Could not find > user: senlin.: UserNotFound: Could not find user: senlin. > > Feb 05 03:31:12.336572 ubuntu-xenial-ovh-gra1-0002362092 > devstack@keystone.service[24845]: WARNING keystone.common.wsgi [None > req-19e52d02-5471-49a2-8acd-360199d8c6e0 None admin] Could not find > role: admin.: RoleNotFound: Could not find role: admin. > > Feb 05 03:28:33.797665 ubuntu-xenial-ovh-gra1-0002362092 > devstack@keystone.service[24845]: WARNING keystone.common.wsgi [None > req-544cd822-18a4-4f7b-913d-297716418239 None admin] Could not find > user: glance.: UserNotFound: Could not find user: glance. > > Feb 05 03:28:29.993214 ubuntu-xenial-ovh-gra1-0002362092 > devstack@keystone.service[24845]: WARNING py.warnings [None > req-dc411d9c-6ab9-44e3-9afb-20e5e7034f12 None admin] > /usr/local/lib/python2.7/dist-packages/oslo_policy/policy.py:865: > UserWarning: Policy identity:create_endpoint failed scope check. The > token used to make the request was project scoped but the policy > requires ['system'] scope. This behavior may change in the future where > using the intended scope is required > > Feb 05 03:28:29.920892 ubuntu-xenial-ovh-gra1-0002362092 > devstack@keystone.service[24845]: WARNING keystone.common.wsgi [None > req-32a4a378-d6d3-411e-9842-2178e577af27 None admin] Could not find > service: compute.: ServiceNotFound: Could not find service: compute. > > > > -- > > So I'm wondering what the hack happened? Keystone version bump? > Devstack changed? Tempest settings changed? > Why are we merging these changes near the end of a cycle when people are > focusing on stabilizing things? > > Any hints on these are highly appreciated. -- Andreas Jaeger aj@{suse.com,opensuse.org} Twitter: jaegerandi SUSE LINUX GmbH, Maxfeldstr. 5, 90409 Nürnberg, Germany GF: Felix Imendörffer, Jane Smithard, Graham Norton, HRB 21284 (AG Nürnberg) GPG fingerprint = 93A3 365E CE47 B889 DF7F FED1 389A 563C C272 A126 __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
[openstack-dev] [CI][Keystone][Requirements][Release] What happened to the gate on Feb 4th?
Starting about 24 hours ago, we have been notified CI gate failure although we haven't changed anything to our project locally. Before that we have spent quite some time making the out-of-tree tempest plugins work on gate. After checking the log again and again ... we found the following logs from Keystone: Feb 05 03:31:12.609492 ubuntu-xenial-ovh-gra1-0002362092 devstack@keystone.service[24845]: WARNING keystone.common.wsgi [None req-dfcbf106-fbf5-41bd-9012-3c65d1de5f9a None admin] Could not find project: service.: ProjectNotFound: Could not find project: service. Feb 05 03:31:13.845694 ubuntu-xenial-ovh-gra1-0002362092 devstack@keystone.service[24845]: WARNING keystone.common.wsgi [None req-50feed46-7c15-425d-bec7-1b4a7ccf6859 None admin] Could not find service: clustering.: ServiceNotFound: Could not find service: clustering. Feb 05 03:31:12.552647 ubuntu-xenial-ovh-gra1-0002362092 devstack@keystone.service[24845]: WARNING keystone.common.wsgi [None req-0a5e660f-dad6-4779-aea4-dd6969c728e6 None admin] Could not find domain: Default.: DomainNotFound: Could not find domain: Default. Feb 05 03:31:12.441128 ubuntu-xenial-ovh-gra1-0002362092 devstack@keystone.service[24845]: WARNING keystone.common.wsgi [None req-7eb9ed90-28fc-40aa-8a41-d560f7a156c9 None admin] Could not find user: senlin.: UserNotFound: Could not find user: senlin. Feb 05 03:31:12.336572 ubuntu-xenial-ovh-gra1-0002362092 devstack@keystone.service[24845]: WARNING keystone.common.wsgi [None req-19e52d02-5471-49a2-8acd-360199d8c6e0 None admin] Could not find role: admin.: RoleNotFound: Could not find role: admin. Feb 05 03:28:33.797665 ubuntu-xenial-ovh-gra1-0002362092 devstack@keystone.service[24845]: WARNING keystone.common.wsgi [None req-544cd822-18a4-4f7b-913d-297716418239 None admin] Could not find user: glance.: UserNotFound: Could not find user: glance. Feb 05 03:28:29.993214 ubuntu-xenial-ovh-gra1-0002362092 devstack@keystone.service[24845]: WARNING py.warnings [None req-dc411d9c-6ab9-44e3-9afb-20e5e7034f12 None admin] /usr/local/lib/python2.7/dist-packages/oslo_policy/policy.py:865: UserWarning: Policy identity:create_endpoint failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Feb 05 03:28:29.920892 ubuntu-xenial-ovh-gra1-0002362092 devstack@keystone.service[24845]: WARNING keystone.common.wsgi [None req-32a4a378-d6d3-411e-9842-2178e577af27 None admin] Could not find service: compute.: ServiceNotFound: Could not find service: compute. -- So I'm wondering what the hack happened? Keystone version bump? Devstack changed? Tempest settings changed? Why are we merging these changes near the end of a cycle when people are focusing on stabilizing things? Any hints on these are highly appreciated. - Qiming __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev