Re: [openstack-dev] [nova][placement][oslo] Excessive WARNING level log messages in placement-api
On 8/9/2018 12:53 PM, Lance Bragstad wrote: I could have sworn I created a bug in oslo.policy for this at one point for the same reason Jay mentions it, but I guess not. This?https://bugs.launchpad.net/oslo.policy/+bug/1421863 Not unless I was time traveling (note the date that was reported). -- Thanks, Matt __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [nova][placement][oslo] Excessive WARNING level log messages in placement-api
Excerpts from Lance Bragstad's message of 2018-08-09 12:55:52 -0500: > > On 08/09/2018 12:48 PM, Doug Hellmann wrote: > > Excerpts from Matt Riedemann's message of 2018-08-09 12:18:14 -0500: > >> On 8/9/2018 11:47 AM, Doug Hellmann wrote: > >>> Excerpts from Jay Pipes's message of 2018-08-08 22:53:54 -0400: > For evidence, see: > > http://logs.openstack.org/41/590041/1/check/tempest-full-py3/db08dec/controller/logs/screen-placement-api.txt.gz?level=WARNING > > thousands of these are filling the logs with WARNING-level log messages, > making it difficult to find anything: > > Aug 08 22:17:30.837557 ubuntu-xenial-inap-mtl01-0001226060 > devstack@placement-api.service[14403]: WARNING py.warnings > [req-a809b022-59af-4628-be73-488cfec3187d > req-d46cb1f0-431f-490f-955b-b9c2cd9f6437 service placement] > /usr/local/lib/python3.5/dist-packages/oslo_policy/policy.py:896: > UserWarning: Policy placement:resource_providers:list failed scope > check. The token used to make the request was project scoped but the > policy requires ['system'] scope. This behavior may change in the future > where using the intended scope is required > Aug 08 22:17:30.837800 ubuntu-xenial-inap-mtl01-0001226060 > devstack@placement-api.service[14403]: warnings.warn(msg) > Aug 08 22:17:30.838067 ubuntu-xenial-inap-mtl01-0001226060 > devstack@placement-api.service[14403]: > > Is there any way we can get rid of these? > > Thanks, > -jay > > >>> It looks like those are coming out of the policy library? Maybe file a > >>> bug there. I added "oslo" to the subject line to get the team's > >>> attention. > >>> > >>> This feels like something we could fix and backport to rocky. > >>> > >>> Doug > >> I could have sworn I created a bug in oslo.policy for this at one point > >> for the same reason Jay mentions it, but I guess not. > >> > >> We could simply, on the nova side, add a warnings filter to only log > >> this once. > >> > > What level should it be logged at in the policy library? Should it be > > logged there at all? > > The initial intent behind logging was to make sure operators knew that > they needed to make a role assignment adjustment in order to be > compatible moving forward. I can investigate a way to log things at > least once in oslo.policy though. I fear not logging it at all would > cause failures in upgrade since operators wouldn't know they need to > make that adjustment. That sounds like a good check to add to the upgrade test tools as part of the goal for Stein. Doug __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [nova][placement][oslo] Excessive WARNING level log messages in placement-api
On 08/09/2018 12:48 PM, Doug Hellmann wrote: > Excerpts from Matt Riedemann's message of 2018-08-09 12:18:14 -0500: >> On 8/9/2018 11:47 AM, Doug Hellmann wrote: >>> Excerpts from Jay Pipes's message of 2018-08-08 22:53:54 -0400: For evidence, see: http://logs.openstack.org/41/590041/1/check/tempest-full-py3/db08dec/controller/logs/screen-placement-api.txt.gz?level=WARNING thousands of these are filling the logs with WARNING-level log messages, making it difficult to find anything: Aug 08 22:17:30.837557 ubuntu-xenial-inap-mtl01-0001226060 devstack@placement-api.service[14403]: WARNING py.warnings [req-a809b022-59af-4628-be73-488cfec3187d req-d46cb1f0-431f-490f-955b-b9c2cd9f6437 service placement] /usr/local/lib/python3.5/dist-packages/oslo_policy/policy.py:896: UserWarning: Policy placement:resource_providers:list failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 08 22:17:30.837800 ubuntu-xenial-inap-mtl01-0001226060 devstack@placement-api.service[14403]: warnings.warn(msg) Aug 08 22:17:30.838067 ubuntu-xenial-inap-mtl01-0001226060 devstack@placement-api.service[14403]: Is there any way we can get rid of these? Thanks, -jay >>> It looks like those are coming out of the policy library? Maybe file a >>> bug there. I added "oslo" to the subject line to get the team's >>> attention. >>> >>> This feels like something we could fix and backport to rocky. >>> >>> Doug >> I could have sworn I created a bug in oslo.policy for this at one point >> for the same reason Jay mentions it, but I guess not. >> >> We could simply, on the nova side, add a warnings filter to only log >> this once. >> > What level should it be logged at in the policy library? Should it be > logged there at all? The initial intent behind logging was to make sure operators knew that they needed to make a role assignment adjustment in order to be compatible moving forward. I can investigate a way to log things at least once in oslo.policy though. I fear not logging it at all would cause failures in upgrade since operators wouldn't know they need to make that adjustment. > > Doug > > __ > OpenStack Development Mailing List (not for usage questions) > Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev signature.asc Description: OpenPGP digital signature __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [nova][placement][oslo] Excessive WARNING level log messages in placement-api
On 08/09/2018 12:18 PM, Matt Riedemann wrote: > On 8/9/2018 11:47 AM, Doug Hellmann wrote: >> Excerpts from Jay Pipes's message of 2018-08-08 22:53:54 -0400: >>> For evidence, see: >>> >>> http://logs.openstack.org/41/590041/1/check/tempest-full-py3/db08dec/controller/logs/screen-placement-api.txt.gz?level=WARNING >>> >>> >>> thousands of these are filling the logs with WARNING-level log >>> messages, >>> making it difficult to find anything: >>> >>> Aug 08 22:17:30.837557 ubuntu-xenial-inap-mtl01-0001226060 >>> devstack@placement-api.service[14403]: WARNING py.warnings >>> [req-a809b022-59af-4628-be73-488cfec3187d >>> req-d46cb1f0-431f-490f-955b-b9c2cd9f6437 service placement] >>> /usr/local/lib/python3.5/dist-packages/oslo_policy/policy.py:896: >>> UserWarning: Policy placement:resource_providers:list failed scope >>> check. The token used to make the request was project scoped but the >>> policy requires ['system'] scope. This behavior may change in the >>> future >>> where using the intended scope is required >>> Aug 08 22:17:30.837800 ubuntu-xenial-inap-mtl01-0001226060 >>> devstack@placement-api.service[14403]: warnings.warn(msg) >>> Aug 08 22:17:30.838067 ubuntu-xenial-inap-mtl01-0001226060 >>> devstack@placement-api.service[14403]: >>> >>> Is there any way we can get rid of these? >>> >>> Thanks, >>> -jay >>> >> It looks like those are coming out of the policy library? Maybe file a >> bug there. I added "oslo" to the subject line to get the team's >> attention. >> >> This feels like something we could fix and backport to rocky. >> >> Doug > > I could have sworn I created a bug in oslo.policy for this at one > point for the same reason Jay mentions it, but I guess not. > This? https://bugs.launchpad.net/oslo.policy/+bug/1421863 > > We could simply, on the nova side, add a warnings filter to only log > this once. > signature.asc Description: OpenPGP digital signature __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [nova][placement][oslo] Excessive WARNING level log messages in placement-api
Excerpts from Matt Riedemann's message of 2018-08-09 12:18:14 -0500: > On 8/9/2018 11:47 AM, Doug Hellmann wrote: > > Excerpts from Jay Pipes's message of 2018-08-08 22:53:54 -0400: > >> For evidence, see: > >> > >> http://logs.openstack.org/41/590041/1/check/tempest-full-py3/db08dec/controller/logs/screen-placement-api.txt.gz?level=WARNING > >> > >> thousands of these are filling the logs with WARNING-level log messages, > >> making it difficult to find anything: > >> > >> Aug 08 22:17:30.837557 ubuntu-xenial-inap-mtl01-0001226060 > >> devstack@placement-api.service[14403]: WARNING py.warnings > >> [req-a809b022-59af-4628-be73-488cfec3187d > >> req-d46cb1f0-431f-490f-955b-b9c2cd9f6437 service placement] > >> /usr/local/lib/python3.5/dist-packages/oslo_policy/policy.py:896: > >> UserWarning: Policy placement:resource_providers:list failed scope > >> check. The token used to make the request was project scoped but the > >> policy requires ['system'] scope. This behavior may change in the future > >> where using the intended scope is required > >> Aug 08 22:17:30.837800 ubuntu-xenial-inap-mtl01-0001226060 > >> devstack@placement-api.service[14403]: warnings.warn(msg) > >> Aug 08 22:17:30.838067 ubuntu-xenial-inap-mtl01-0001226060 > >> devstack@placement-api.service[14403]: > >> > >> Is there any way we can get rid of these? > >> > >> Thanks, > >> -jay > >> > > It looks like those are coming out of the policy library? Maybe file a > > bug there. I added "oslo" to the subject line to get the team's > > attention. > > > > This feels like something we could fix and backport to rocky. > > > > Doug > > I could have sworn I created a bug in oslo.policy for this at one point > for the same reason Jay mentions it, but I guess not. > > We could simply, on the nova side, add a warnings filter to only log > this once. > What level should it be logged at in the policy library? Should it be logged there at all? Doug __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [nova][placement][oslo] Excessive WARNING level log messages in placement-api
On 8/9/2018 12:18 PM, Matt Riedemann wrote: We could simply, on the nova side, add a warnings filter to only log this once. Let's see if this works: https://review.openstack.org/#/c/590445/ -- Thanks, Matt __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [nova][placement][oslo] Excessive WARNING level log messages in placement-api
On 8/9/2018 11:47 AM, Doug Hellmann wrote: Excerpts from Jay Pipes's message of 2018-08-08 22:53:54 -0400: For evidence, see: http://logs.openstack.org/41/590041/1/check/tempest-full-py3/db08dec/controller/logs/screen-placement-api.txt.gz?level=WARNING thousands of these are filling the logs with WARNING-level log messages, making it difficult to find anything: Aug 08 22:17:30.837557 ubuntu-xenial-inap-mtl01-0001226060 devstack@placement-api.service[14403]: WARNING py.warnings [req-a809b022-59af-4628-be73-488cfec3187d req-d46cb1f0-431f-490f-955b-b9c2cd9f6437 service placement] /usr/local/lib/python3.5/dist-packages/oslo_policy/policy.py:896: UserWarning: Policy placement:resource_providers:list failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 08 22:17:30.837800 ubuntu-xenial-inap-mtl01-0001226060 devstack@placement-api.service[14403]: warnings.warn(msg) Aug 08 22:17:30.838067 ubuntu-xenial-inap-mtl01-0001226060 devstack@placement-api.service[14403]: Is there any way we can get rid of these? Thanks, -jay It looks like those are coming out of the policy library? Maybe file a bug there. I added "oslo" to the subject line to get the team's attention. This feels like something we could fix and backport to rocky. Doug I could have sworn I created a bug in oslo.policy for this at one point for the same reason Jay mentions it, but I guess not. We could simply, on the nova side, add a warnings filter to only log this once. -- Thanks, Matt __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [nova][placement][oslo] Excessive WARNING level log messages in placement-api
Excerpts from Jay Pipes's message of 2018-08-08 22:53:54 -0400: > For evidence, see: > > http://logs.openstack.org/41/590041/1/check/tempest-full-py3/db08dec/controller/logs/screen-placement-api.txt.gz?level=WARNING > > thousands of these are filling the logs with WARNING-level log messages, > making it difficult to find anything: > > Aug 08 22:17:30.837557 ubuntu-xenial-inap-mtl01-0001226060 > devstack@placement-api.service[14403]: WARNING py.warnings > [req-a809b022-59af-4628-be73-488cfec3187d > req-d46cb1f0-431f-490f-955b-b9c2cd9f6437 service placement] > /usr/local/lib/python3.5/dist-packages/oslo_policy/policy.py:896: > UserWarning: Policy placement:resource_providers:list failed scope > check. The token used to make the request was project scoped but the > policy requires ['system'] scope. This behavior may change in the future > where using the intended scope is required > Aug 08 22:17:30.837800 ubuntu-xenial-inap-mtl01-0001226060 > devstack@placement-api.service[14403]: warnings.warn(msg) > Aug 08 22:17:30.838067 ubuntu-xenial-inap-mtl01-0001226060 > devstack@placement-api.service[14403]: > > Is there any way we can get rid of these? > > Thanks, > -jay > It looks like those are coming out of the policy library? Maybe file a bug there. I added "oslo" to the subject line to get the team's attention. This feels like something we could fix and backport to rocky. Doug __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
