Re: [openstack-dev] [oslo] oslo.context and name fields

[Ronald Bradford]

Sean,

I've taken your WIP https://review.openstack.org/#/c/302744/ a step forward
and its in review now.
I'd value Jamie's input on your proposed from_environ signature change to
see if that does meet his indented goal.

Regards

Ronald



Ronald Bradford

Web Site: http://ronaldbradford.com
LinkedIn:  http://www.linkedin.com/in/ronaldbradford
Twitter:@RonaldBradford 
Skype: RonaldBradford
GTalk: Ronald.Bradford
IRC: rbradfor


On Tue, Apr 5, 2016 at 9:53 PM, Jamie Lennox  wrote:

> from_environ was mine, it's reasonably new and at the time i was blocked
> upon getting a release before pushing it out to services. Since then i've
> been distracted with other things. The intent at the time was exactly this
> to standardize the values on the context object, though in my case i was
> particularly interested with how we could handle authentication plugins.
>
> The problems i hit were specifically around how we could seperate values
> that were relevant to things like policy from values that were relevant for
> RPC etc rather that the big to_dict that is used for everything at the
> moment.
>
> There were a number of problems with this, however nothing that would
> prevent more standardization of the base attributes and using from_environ
> now.
>
>
> On 6 April 2016 at 07:39, Ronald Bradford  wrote:
>
>> I have created a version that use constructor arguments. [5]
>> I will review in more detail across projects the use of keystone
>> middleware to see if we can utilize a constructor environment attribute to
>> simply constructor usage.
>>
>> [5] https://review.openstack.org/301918
>>
>> Ronald Bradford
>>
>> Web Site: http://ronaldbradford.com
>> LinkedIn:  http://www.linkedin.com/in/ronaldbradford
>> Twitter:@RonaldBradford 
>> Skype: RonaldBradford
>> GTalk: Ronald.Bradford
>> IRC: rbradfor
>>
>>
>> On Tue, Apr 5, 2016 at 3:49 PM, Sean Dague  wrote:
>>
>>> Cool. Great.
>>>
>>> In looking at this code a bit more I think we're missing out on some
>>> commonality by the fact that this nice bit of common parsing -
>>>
>>> https://github.com/openstack/oslo.context/blob/c63a359094907bc50cc5e1be716508ddee825dfa/oslo_context/context.py#L138-L161
>>> is actually hidden behind a factory pattern, and not used by anyone in
>>> OpenStack -
>>> http://codesearch.openstack.org/?q=from_environ=nope==
>>>
>>> If instead of standardizing the args to the context constructor, we
>>> could remove a bunch of them and extract that data from a passed
>>> environment during the constructor that should remove a bunch of parsing
>>> code in every project. It would also mean that we could easily add
>>> things like project_name and user_name in, and they would be available
>>> to all consumers.
>>>
>>> -Sean
>>>
>>> On 04/05/2016 03:39 PM, Ronald Bradford wrote:
>>> > Sean,
>>> >
>>> > I cannot speak to historically why there were not there, but I am
>>> > working through the app-agnostic-logging-parameters blueprint [1] right
>>> > now and it's very related to this.  As part of this work I would be
>>> > reviewing attributes that are more commonly used in subclassed context
>>> > objects for inclusion into the base oslo.context class, a step before a
>>> > more kwargs init() approach that many subclassed context object
>>> utilize now.
>>> >
>>> > I am also proposing the standardization of context arguments [2]
>>> > (specifically ids), and names are not mentioned but I would like to
>>> > follow the proposed convention.
>>> >
>>> > However, as you point out in the middleware [3], if the information is
>>> > already available I see no reason not to facilite the base oslo.context
>>> > class to consume this for subsequent use by logging.  FYI the
>>> > get_logging_values() work in [4] is specially to add logging only
>>> values
>>> > and this can be the first use case.
>>> >
>>> > While devstack uses these logging format string options, the defaults
>>> > (which I presume are operator centric do not).  One of my goals of the
>>> > Austin Ops summit it to get to talk with actual operators and find out
>>> > what is really in use.   Regardless, the capacity to choose should be
>>> > available when possible if the information is already identified
>>> without
>>> > subsequent lookup.
>>> >
>>> >
>>> > Ronald
>>> >
>>> >
>>> > [1]
>>> https://blueprints.launchpad.net/oslo.log/+spec/app-agnostic-logging-parameters
>>> > [2] https://review.openstack.org/#/c/290907/
>>> > [3]
>>> http://docs.openstack.org/developer/keystonemiddleware/api/keystonemiddleware.auth_token.html#what-auth-token-adds-to-the-request-for-use-by-the-openstack-service
>>> > [4] https://review.openstack.org/#/c/274186/
>>> >
>>> >
>>> >
>>> >
>>> >
>>> > On Tue, Apr 5, 2016 at 2:31 PM, Sean Dague >> > > wrote:
>>> >
>>> > I was trying to clean up 

Re: [openstack-dev] [oslo] oslo.context and name fields

[Jamie Lennox]

from_environ was mine, it's reasonably new and at the time i was blocked
upon getting a release before pushing it out to services. Since then i've
been distracted with other things. The intent at the time was exactly this
to standardize the values on the context object, though in my case i was
particularly interested with how we could handle authentication plugins.

The problems i hit were specifically around how we could seperate values
that were relevant to things like policy from values that were relevant for
RPC etc rather that the big to_dict that is used for everything at the
moment.

There were a number of problems with this, however nothing that would
prevent more standardization of the base attributes and using from_environ
now.


On 6 April 2016 at 07:39, Ronald Bradford  wrote:

> I have created a version that use constructor arguments. [5]
> I will review in more detail across projects the use of keystone
> middleware to see if we can utilize a constructor environment attribute to
> simply constructor usage.
>
> [5] https://review.openstack.org/301918
>
> Ronald Bradford
>
> Web Site: http://ronaldbradford.com
> LinkedIn:  http://www.linkedin.com/in/ronaldbradford
> Twitter:@RonaldBradford 
> Skype: RonaldBradford
> GTalk: Ronald.Bradford
> IRC: rbradfor
>
>
> On Tue, Apr 5, 2016 at 3:49 PM, Sean Dague  wrote:
>
>> Cool. Great.
>>
>> In looking at this code a bit more I think we're missing out on some
>> commonality by the fact that this nice bit of common parsing -
>>
>> https://github.com/openstack/oslo.context/blob/c63a359094907bc50cc5e1be716508ddee825dfa/oslo_context/context.py#L138-L161
>> is actually hidden behind a factory pattern, and not used by anyone in
>> OpenStack -
>> http://codesearch.openstack.org/?q=from_environ=nope==
>>
>> If instead of standardizing the args to the context constructor, we
>> could remove a bunch of them and extract that data from a passed
>> environment during the constructor that should remove a bunch of parsing
>> code in every project. It would also mean that we could easily add
>> things like project_name and user_name in, and they would be available
>> to all consumers.
>>
>> -Sean
>>
>> On 04/05/2016 03:39 PM, Ronald Bradford wrote:
>> > Sean,
>> >
>> > I cannot speak to historically why there were not there, but I am
>> > working through the app-agnostic-logging-parameters blueprint [1] right
>> > now and it's very related to this.  As part of this work I would be
>> > reviewing attributes that are more commonly used in subclassed context
>> > objects for inclusion into the base oslo.context class, a step before a
>> > more kwargs init() approach that many subclassed context object utilize
>> now.
>> >
>> > I am also proposing the standardization of context arguments [2]
>> > (specifically ids), and names are not mentioned but I would like to
>> > follow the proposed convention.
>> >
>> > However, as you point out in the middleware [3], if the information is
>> > already available I see no reason not to facilite the base oslo.context
>> > class to consume this for subsequent use by logging.  FYI the
>> > get_logging_values() work in [4] is specially to add logging only values
>> > and this can be the first use case.
>> >
>> > While devstack uses these logging format string options, the defaults
>> > (which I presume are operator centric do not).  One of my goals of the
>> > Austin Ops summit it to get to talk with actual operators and find out
>> > what is really in use.   Regardless, the capacity to choose should be
>> > available when possible if the information is already identified without
>> > subsequent lookup.
>> >
>> >
>> > Ronald
>> >
>> >
>> > [1]
>> https://blueprints.launchpad.net/oslo.log/+spec/app-agnostic-logging-parameters
>> > [2] https://review.openstack.org/#/c/290907/
>> > [3]
>> http://docs.openstack.org/developer/keystonemiddleware/api/keystonemiddleware.auth_token.html#what-auth-token-adds-to-the-request-for-use-by-the-openstack-service
>> > [4] https://review.openstack.org/#/c/274186/
>> >
>> >
>> >
>> >
>> >
>> > On Tue, Apr 5, 2016 at 2:31 PM, Sean Dague > > > wrote:
>> >
>> > I was trying to clean up the divergent logging definitions in
>> devstack
>> > as part of scrubbing out 'tenant' references -
>> > https://review.openstack.org/#/c/301801/ and in doing so stumbled
>> over
>> > the fact that the extremely useful project_name and user_name
>> fields are
>> > not in base oslo.context.
>> >
>> >
>> https://github.com/openstack/oslo.context/blob/c63a359094907bc50cc5e1be716508ddee825dfa/oslo_context/context.py#L148-L159
>> >
>> > These are always available to be set -
>> >
>> http://docs.openstack.org/developer/keystonemiddleware/api/keystonemiddleware.auth_token.html#what-auth-token-adds-to-the-request-for-use-by-the-openstack-service
>> >
>> > And they 

Re: [openstack-dev] [oslo] oslo.context and name fields

[Ronald Bradford]

I have created a version that use constructor arguments. [5]
I will review in more detail across projects the use of keystone middleware
to see if we can utilize a constructor environment attribute to simply
constructor usage.

[5] https://review.openstack.org/301918

Ronald Bradford

Web Site: http://ronaldbradford.com
LinkedIn:  http://www.linkedin.com/in/ronaldbradford
Twitter:@RonaldBradford 
Skype: RonaldBradford
GTalk: Ronald.Bradford
IRC: rbradfor


On Tue, Apr 5, 2016 at 3:49 PM, Sean Dague  wrote:

> Cool. Great.
>
> In looking at this code a bit more I think we're missing out on some
> commonality by the fact that this nice bit of common parsing -
>
> https://github.com/openstack/oslo.context/blob/c63a359094907bc50cc5e1be716508ddee825dfa/oslo_context/context.py#L138-L161
> is actually hidden behind a factory pattern, and not used by anyone in
> OpenStack -
> http://codesearch.openstack.org/?q=from_environ=nope==
>
> If instead of standardizing the args to the context constructor, we
> could remove a bunch of them and extract that data from a passed
> environment during the constructor that should remove a bunch of parsing
> code in every project. It would also mean that we could easily add
> things like project_name and user_name in, and they would be available
> to all consumers.
>
> -Sean
>
> On 04/05/2016 03:39 PM, Ronald Bradford wrote:
> > Sean,
> >
> > I cannot speak to historically why there were not there, but I am
> > working through the app-agnostic-logging-parameters blueprint [1] right
> > now and it's very related to this.  As part of this work I would be
> > reviewing attributes that are more commonly used in subclassed context
> > objects for inclusion into the base oslo.context class, a step before a
> > more kwargs init() approach that many subclassed context object utilize
> now.
> >
> > I am also proposing the standardization of context arguments [2]
> > (specifically ids), and names are not mentioned but I would like to
> > follow the proposed convention.
> >
> > However, as you point out in the middleware [3], if the information is
> > already available I see no reason not to facilite the base oslo.context
> > class to consume this for subsequent use by logging.  FYI the
> > get_logging_values() work in [4] is specially to add logging only values
> > and this can be the first use case.
> >
> > While devstack uses these logging format string options, the defaults
> > (which I presume are operator centric do not).  One of my goals of the
> > Austin Ops summit it to get to talk with actual operators and find out
> > what is really in use.   Regardless, the capacity to choose should be
> > available when possible if the information is already identified without
> > subsequent lookup.
> >
> >
> > Ronald
> >
> >
> > [1]
> https://blueprints.launchpad.net/oslo.log/+spec/app-agnostic-logging-parameters
> > [2] https://review.openstack.org/#/c/290907/
> > [3]
> http://docs.openstack.org/developer/keystonemiddleware/api/keystonemiddleware.auth_token.html#what-auth-token-adds-to-the-request-for-use-by-the-openstack-service
> > [4] https://review.openstack.org/#/c/274186/
> >
> >
> >
> >
> >
> > On Tue, Apr 5, 2016 at 2:31 PM, Sean Dague  > > wrote:
> >
> > I was trying to clean up the divergent logging definitions in
> devstack
> > as part of scrubbing out 'tenant' references -
> > https://review.openstack.org/#/c/301801/ and in doing so stumbled
> over
> > the fact that the extremely useful project_name and user_name fields
> are
> > not in base oslo.context.
> >
> >
> https://github.com/openstack/oslo.context/blob/c63a359094907bc50cc5e1be716508ddee825dfa/oslo_context/context.py#L148-L159
> >
> > These are always available to be set -
> >
> http://docs.openstack.org/developer/keystonemiddleware/api/keystonemiddleware.auth_token.html#what-auth-token-adds-to-the-request-for-use-by-the-openstack-service
> >
> > And they are extremely valuable when a human is looking at logs, as
> you
> > actually can remember names when looking at various services to cross
> > reference things. Nova has a custom context that sets these things -
> >
> https://github.com/openstack/nova/blob/d57a4e8be9147bd79be12d3f5adccc9289a375b6/nova/api/auth.py#L114-L115
> >
> > I would really like these to be available in all services using
> > oslo.context.
> >
> > So the question is, were these not implemented on purpose? Is the
> > opposition to putting them into oslo.context?
> >
> > Please let me know before I start going down this path.
> >
> > -Sean
> >
> > --
> > Sean Dague
> > http://dague.net
> >
> >
>  __
> > OpenStack Development Mailing List (not for usage questions)
> > Unsubscribe:
> > 

Re: [openstack-dev] [oslo] oslo.context and name fields

[Sean Dague]

Cool. Great.

In looking at this code a bit more I think we're missing out on some
commonality by the fact that this nice bit of common parsing -
https://github.com/openstack/oslo.context/blob/c63a359094907bc50cc5e1be716508ddee825dfa/oslo_context/context.py#L138-L161
is actually hidden behind a factory pattern, and not used by anyone in
OpenStack -
http://codesearch.openstack.org/?q=from_environ=nope==

If instead of standardizing the args to the context constructor, we
could remove a bunch of them and extract that data from a passed
environment during the constructor that should remove a bunch of parsing
code in every project. It would also mean that we could easily add
things like project_name and user_name in, and they would be available
to all consumers.

-Sean

On 04/05/2016 03:39 PM, Ronald Bradford wrote:
> Sean,
> 
> I cannot speak to historically why there were not there, but I am
> working through the app-agnostic-logging-parameters blueprint [1] right
> now and it's very related to this.  As part of this work I would be
> reviewing attributes that are more commonly used in subclassed context
> objects for inclusion into the base oslo.context class, a step before a
> more kwargs init() approach that many subclassed context object utilize now.
> 
> I am also proposing the standardization of context arguments [2]
> (specifically ids), and names are not mentioned but I would like to
> follow the proposed convention.
> 
> However, as you point out in the middleware [3], if the information is
> already available I see no reason not to facilite the base oslo.context
> class to consume this for subsequent use by logging.  FYI the
> get_logging_values() work in [4] is specially to add logging only values
> and this can be the first use case.
> 
> While devstack uses these logging format string options, the defaults
> (which I presume are operator centric do not).  One of my goals of the
> Austin Ops summit it to get to talk with actual operators and find out
> what is really in use.   Regardless, the capacity to choose should be
> available when possible if the information is already identified without
> subsequent lookup.
> 
> 
> Ronald 
> 
> 
> [1] 
> https://blueprints.launchpad.net/oslo.log/+spec/app-agnostic-logging-parameters
> [2] https://review.openstack.org/#/c/290907/
> [3] 
> http://docs.openstack.org/developer/keystonemiddleware/api/keystonemiddleware.auth_token.html#what-auth-token-adds-to-the-request-for-use-by-the-openstack-service
> [4] https://review.openstack.org/#/c/274186/
> 
> 
> 
> 
> 
> On Tue, Apr 5, 2016 at 2:31 PM, Sean Dague  > wrote:
> 
> I was trying to clean up the divergent logging definitions in devstack
> as part of scrubbing out 'tenant' references -
> https://review.openstack.org/#/c/301801/ and in doing so stumbled over
> the fact that the extremely useful project_name and user_name fields are
> not in base oslo.context.
> 
> 
> https://github.com/openstack/oslo.context/blob/c63a359094907bc50cc5e1be716508ddee825dfa/oslo_context/context.py#L148-L159
> 
> These are always available to be set -
> 
> http://docs.openstack.org/developer/keystonemiddleware/api/keystonemiddleware.auth_token.html#what-auth-token-adds-to-the-request-for-use-by-the-openstack-service
> 
> And they are extremely valuable when a human is looking at logs, as you
> actually can remember names when looking at various services to cross
> reference things. Nova has a custom context that sets these things -
> 
> https://github.com/openstack/nova/blob/d57a4e8be9147bd79be12d3f5adccc9289a375b6/nova/api/auth.py#L114-L115
> 
> I would really like these to be available in all services using
> oslo.context.
> 
> So the question is, were these not implemented on purpose? Is the
> opposition to putting them into oslo.context?
> 
> Please let me know before I start going down this path.
> 
> -Sean
> 
> --
> Sean Dague
> http://dague.net
> 
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe:
> openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> 
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
> 
> 
> 
> 
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
> 


-- 
Sean Dague
http://dague.net

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe

Re: [openstack-dev] [oslo] oslo.context and name fields

[Ronald Bradford]

Sean,

I cannot speak to historically why there were not there, but I am working
through the app-agnostic-logging-parameters blueprint [1] right now and
it's very related to this.  As part of this work I would be reviewing
attributes that are more commonly used in subclassed context objects for
inclusion into the base oslo.context class, a step before a more kwargs
init() approach that many subclassed context object utilize now.

I am also proposing the standardization of context arguments [2]
(specifically ids), and names are not mentioned but I would like to follow
the proposed convention.

However, as you point out in the middleware [3], if the information is
already available I see no reason not to facilite the base oslo.context
class to consume this for subsequent use by logging.  FYI the
get_logging_values() work in [4] is specially to add logging only values
and this can be the first use case.

While devstack uses these logging format string options, the defaults
(which I presume are operator centric do not).  One of my goals of the
Austin Ops summit it to get to talk with actual operators and find out what
is really in use.   Regardless, the capacity to choose should be available
when possible if the information is already identified without subsequent
lookup.


Ronald


[1]
https://blueprints.launchpad.net/oslo.log/+spec/app-agnostic-logging-parameters
[2] https://review.openstack.org/#/c/290907/
[3]
http://docs.openstack.org/developer/keystonemiddleware/api/keystonemiddleware.auth_token.html#what-auth-token-adds-to-the-request-for-use-by-the-openstack-service
[4] https://review.openstack.org/#/c/274186/





On Tue, Apr 5, 2016 at 2:31 PM, Sean Dague  wrote:

> I was trying to clean up the divergent logging definitions in devstack
> as part of scrubbing out 'tenant' references -
> https://review.openstack.org/#/c/301801/ and in doing so stumbled over
> the fact that the extremely useful project_name and user_name fields are
> not in base oslo.context.
>
>
> https://github.com/openstack/oslo.context/blob/c63a359094907bc50cc5e1be716508ddee825dfa/oslo_context/context.py#L148-L159
>
> These are always available to be set -
>
> http://docs.openstack.org/developer/keystonemiddleware/api/keystonemiddleware.auth_token.html#what-auth-token-adds-to-the-request-for-use-by-the-openstack-service
>
> And they are extremely valuable when a human is looking at logs, as you
> actually can remember names when looking at various services to cross
> reference things. Nova has a custom context that sets these things -
>
> https://github.com/openstack/nova/blob/d57a4e8be9147bd79be12d3f5adccc9289a375b6/nova/api/auth.py#L114-L115
>
> I would really like these to be available in all services using
> oslo.context.
>
> So the question is, were these not implemented on purpose? Is the
> opposition to putting them into oslo.context?
>
> Please let me know before I start going down this path.
>
> -Sean
>
> --
> Sean Dague
> http://dague.net
>
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [oslo] oslo.context and name fields

[Doug Hellmann]

Excerpts from Sean Dague's message of 2016-04-05 14:31:46 -0400:
> I was trying to clean up the divergent logging definitions in devstack
> as part of scrubbing out 'tenant' references -
> https://review.openstack.org/#/c/301801/ and in doing so stumbled over
> the fact that the extremely useful project_name and user_name fields are
> not in base oslo.context.
> 
> https://github.com/openstack/oslo.context/blob/c63a359094907bc50cc5e1be716508ddee825dfa/oslo_context/context.py#L148-L159
> 
> These are always available to be set -
> http://docs.openstack.org/developer/keystonemiddleware/api/keystonemiddleware.auth_token.html#what-auth-token-adds-to-the-request-for-use-by-the-openstack-service
> 
> And they are extremely valuable when a human is looking at logs, as you
> actually can remember names when looking at various services to cross
> reference things. Nova has a custom context that sets these things -
> https://github.com/openstack/nova/blob/d57a4e8be9147bd79be12d3f5adccc9289a375b6/nova/api/auth.py#L114-L115
> 
> I would really like these to be available in all services using
> oslo.context.
> 
> So the question is, were these not implemented on purpose? Is the
> opposition to putting them into oslo.context?

It's probably just a matter of not getting to them. We have a couple of
incomplete specs related to contexts and logging. Ronald and I were
going to try to make some progress on those during Newton.

http://specs.openstack.org/openstack/oslo-specs/specs/kilo/app-agnostic-logging-parameters.html
http://specs.openstack.org/openstack/oslo-specs/specs/liberty/oslo.log-user-identity-format-flexibility.html

Doug

> 
> Please let me know before I start going down this path.
> 
> -Sean
> 

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

[openstack-dev] [oslo] oslo.context and name fields

[Sean Dague]

I was trying to clean up the divergent logging definitions in devstack
as part of scrubbing out 'tenant' references -
https://review.openstack.org/#/c/301801/ and in doing so stumbled over
the fact that the extremely useful project_name and user_name fields are
not in base oslo.context.

https://github.com/openstack/oslo.context/blob/c63a359094907bc50cc5e1be716508ddee825dfa/oslo_context/context.py#L148-L159

These are always available to be set -
http://docs.openstack.org/developer/keystonemiddleware/api/keystonemiddleware.auth_token.html#what-auth-token-adds-to-the-request-for-use-by-the-openstack-service

And they are extremely valuable when a human is looking at logs, as you
actually can remember names when looking at various services to cross
reference things. Nova has a custom context that sets these things -
https://github.com/openstack/nova/blob/d57a4e8be9147bd79be12d3f5adccc9289a375b6/nova/api/auth.py#L114-L115

I would really like these to be available in all services using
oslo.context.

So the question is, were these not implemented on purpose? Is the
opposition to putting them into oslo.context?

Please let me know before I start going down this path.

-Sean

-- 
Sean Dague
http://dague.net

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev