Re: [openstack-dev] Proposed Logging Standards

[Macdonald-Wallace, Matthew]

> -Original Message-
> From: Ben Nemec [mailto:openst...@nemebean.com]
> Sent: 31 January 2014 16:01
> To: openstack-dev@lists.openstack.org
> Subject: Re: [openstack-dev] Proposed Logging Standards
> > I'd just love to see the ability in the python logger to include the
> > application name, not just the class/module that created the log
> > message (it's in 3. but I don't think we can justify a
> > switch to Python 3 just based on logging!):
> >
> > 
> >   
> >
> > At the moment, all of the above is possible except for the
> >  part.  Is there anything we can do to add this to the
> > context or similar?
> >
> > Matt
> 
> Wish granted. :-)

YAY! \o/
 
> https://github.com/openstack/oslo-
> incubator/commit/8c3046b78dca8eae1d911e3421b5938c19f20c37
> 
> The plan is to turn that on by default as soon as we can get through the
> deprecation process for the existing format.

Awesome, good to hear.

Thanks,

Matt

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] Proposed Logging Standards

[Ben Nemec]

On 2014-01-31 07:00, Macdonald-Wallace, Matthew wrote:

-Original Message-
From: Sean Dague [mailto:s...@dague.net]
Sent: 31 January 2014 12:29
To: OpenStack Development Mailing List (not for usage questions)
Subject: Re: [openstack-dev] Proposed Logging Standards

On 01/30/2014 07:15 PM, John Dickinson wrote:
> 1) Every log message is one line (ends with \n) and the log fields are
> space-delineated. eg (`log_line = ' '.join(urllib.quote(x) for x in
> log_fields_list)`)


+1 for this - multiple lines (even in in DEBUG mode!) are a PITA to
handle with most log analyser software.



> 2) The only definition of a log format is the prefix and the message is a set 
of
fields defined by the service actually doing the logging.

So, actually, most of my concern at this point wasn't the line format.
It was the concern about when projects were calling the loggers, and 
what kind

of information should be logged at each level.

Given that most projects are using the oslo defaults today, much of 
the line
format is handled. I think that if you have concerns on that front, 
it's probably a

different conversation with the oslo team.

I do agree we should standards a little more on project (i.e. logger 
"name"),

because in most projects this is just defaulting to module.
Which is fine for debug level, but not very user friendly at ops 
levels.


I'd just love to see the ability in the python logger to include the
application name, not just the class/module that created the log
message (it's in 3. but I don't think we can justify a
switch to Python 3 just based on logging!):


  

At the moment, all of the above is possible except for the
 part.  Is there anything we can do to add this to the
context or similar?

Matt


Wish granted. :-)

https://github.com/openstack/oslo-incubator/commit/8c3046b78dca8eae1d911e3421b5938c19f20c37

The plan is to turn that on by default as soon as we can get through the 
deprecation process for the existing format.


Related to John's comments on the line format, I know that has come up 
before.  I gather the default log format in Oslo started life as a 
direct copy of the Nova log code, so there seem to be some nova-isms 
left there.  I'm open to suggestions on how to make the default better, 
and probably how to allow each project to specify its own default format 
since I doubt we're going to find one that satisfies everyone.


-Ben

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] Proposed Logging Standards

[Macdonald-Wallace, Matthew]

> -Original Message-
> From: Sean Dague [mailto:s...@dague.net]
> Sent: 31 January 2014 12:29
> To: OpenStack Development Mailing List (not for usage questions)
> Subject: Re: [openstack-dev] Proposed Logging Standards
> 
> On 01/30/2014 07:15 PM, John Dickinson wrote:
> > 1) Every log message is one line (ends with \n) and the log fields are
> > space-delineated. eg (`log_line = ' '.join(urllib.quote(x) for x in
> > log_fields_list)`)

+1 for this - multiple lines (even in in DEBUG mode!) are a PITA to handle with 
most log analyser software.

>
> > 2) The only definition of a log format is the prefix and the message is a 
> > set of
> fields defined by the service actually doing the logging.
> 
> So, actually, most of my concern at this point wasn't the line format.
> It was the concern about when projects were calling the loggers, and what kind
> of information should be logged at each level.
> 
> Given that most projects are using the oslo defaults today, much of the line
> format is handled. I think that if you have concerns on that front, it's 
> probably a
> different conversation with the oslo team.
> 
> I do agree we should standards a little more on project (i.e. logger "name"),
> because in most projects this is just defaulting to module.
> Which is fine for debug level, but not very user friendly at ops levels.

I'd just love to see the ability in the python logger to include the 
application name, not just the class/module that created the log message (it's 
in 3. but I don't think we can justify a switch to Python 3 just 
based on logging!):

  
 

At the moment, all of the above is possible except for the  part. 
 Is there anything we can do to add this to the context or similar?

Matt

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] Proposed Logging Standards

[Sean Dague]

On 01/30/2014 07:15 PM, John Dickinson wrote:
> I've been keeping an eye on this thread, and it seems I actually have a few 
> minutes to spend on a response today.
> 
> To first answer the specific question, while there are some minor technical 
> concerns about oslo logging, the bigger concerns are non-technical. Some 
> things I'm concerned about from a technical perspective are that it's not a 
> separate module or package that can be imported, so it would probably 
> currently require copy/paste code into the Swift codebase. My second concern 
> is that there are log line elements that just don't seem to make sense like 
> "instance". I'd be happy to be wrong on both of these items, and I want to 
> make clear that these are not long-term issues. They are both solvable.
> 
> My bigger concern with using oslo logging in Swift is simply changing the 
> request log format is something that cannot be done lightly. Request logs are 
> a very real interface into the system, and changing the log format in a 
> breaking way can cause major headaches for people relying on those logs for 
> system health, billing, and other operational concerns.
> 
> One possible solution to this is to keep requests logged the same way, but 
> add configuration options for all of the other things that are logged. Having 
> two different logging systems (or multiple configurable log handlers) to do 
> this seems to add a fair bit of complexity to me, especially when I'm not 
> quite sure of the actual problem that's being solved. That said, adding in a 
> different log format into Swift isn't a terrible idea by itself, but 
> migration is a big concern of any implementation (and I know you'll find very 
> strong feelings on this in gerrit if/when something is proposed).
> 
> 
> 
> Now back to the original topic of actual logging formats.
> 
> Here's (something like) what I'd like to see for a common log standard (ie 
> Sean, what I think you were asking for comments on):
> 
> log_line = prefix message
> prefix = timestamp project log_level
> message = bytestream
> timestamp = `eg the output of time.time()`
> project = `one of {nova,swift,neutron,cinder,glance,etc}`
> 
> Now, there's plenty of opportunity to bikeshed what the actual log line would 
> look like, but the general idea of what I want to see has 2 major parts:
> 
> 1) Every log message is one line (ends with \n) and the log fields are 
> space-delineated. eg (`log_line = ' '.join(urllib.quote(x) for x in 
> log_fields_list)`)
> 
> 2) The only definition of a log format is the prefix and the message is a set 
> of fields defined by the service actually doing the logging.

So, actually, most of my concern at this point wasn't the line format.
It was the concern about when projects were calling the loggers, and
what kind of information should be logged at each level.

Given that most projects are using the oslo defaults today, much of the
line format is handled. I think that if you have concerns on that front,
it's probably a different conversation with the oslo team.

I do agree we should standards a little more on project (i.e. logger
"name"), because in most projects this is just defaulting to module.
Which is fine for debug level, but not very user friendly at ops levels.

-Sean

-- 
Sean Dague
Samsung Research America
s...@dague.net / sean.da...@samsung.com
http://dague.net



signature.asc
Description: OpenPGP digital signature
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] Proposed Logging Standards

[John Dickinson]

I've been keeping an eye on this thread, and it seems I actually have a few 
minutes to spend on a response today.

To first answer the specific question, while there are some minor technical 
concerns about oslo logging, the bigger concerns are non-technical. Some things 
I'm concerned about from a technical perspective are that it's not a separate 
module or package that can be imported, so it would probably currently require 
copy/paste code into the Swift codebase. My second concern is that there are 
log line elements that just don't seem to make sense like "instance". I'd be 
happy to be wrong on both of these items, and I want to make clear that these 
are not long-term issues. They are both solvable.

My bigger concern with using oslo logging in Swift is simply changing the 
request log format is something that cannot be done lightly. Request logs are a 
very real interface into the system, and changing the log format in a breaking 
way can cause major headaches for people relying on those logs for system 
health, billing, and other operational concerns.

One possible solution to this is to keep requests logged the same way, but add 
configuration options for all of the other things that are logged. Having two 
different logging systems (or multiple configurable log handlers) to do this 
seems to add a fair bit of complexity to me, especially when I'm not quite sure 
of the actual problem that's being solved. That said, adding in a different log 
format into Swift isn't a terrible idea by itself, but migration is a big 
concern of any implementation (and I know you'll find very strong feelings on 
this in gerrit if/when something is proposed).




Now back to the original topic of actual logging formats.

Here's (something like) what I'd like to see for a common log standard (ie 
Sean, what I think you were asking for comments on):

log_line = prefix message
prefix = timestamp project log_level
message = bytestream
timestamp = `eg the output of time.time()`
project = `one of {nova,swift,neutron,cinder,glance,etc}`

Now, there's plenty of opportunity to bikeshed what the actual log line would 
look like, but the general idea of what I want to see has 2 major parts:

1) Every log message is one line (ends with \n) and the log fields are 
space-delineated. eg (`log_line = ' '.join(urllib.quote(x) for x in 
log_fields_list)`)

2) The only definition of a log format is the prefix and the message is a set 
of fields defined by the service actually doing the logging.


--John




On Jan 30, 2014, at 10:11 AM, Sanchez, Cristian A 
 wrote:

> Is there any technical reason of why Swift does not use oslo logging?
> If not, I can work on incorporating that to Swift.
> 
> Thanks
> 
> Cristian
> 
> On 30/01/14 11:12, "Sean Dague"  wrote:
> 
>> For all projects that use oslo logging (which is currently everything
>> except swift), this works.
>> 
>>  -Sean
>> 
>> On 01/30/2014 09:07 AM, Macdonald-Wallace, Matthew wrote:
>>> No idea, I only really work on Nova, but as this is in Oslo I expect so!
>>> 
>>> Matt
>>> 
>>>> -Original Message-
>>>> From: Sanchez, Cristian A [mailto:cristian.a.sanc...@intel.com]
>>>> Sent: 30 January 2014 13:44
>>>> To: OpenStack Development Mailing List (not for usage questions)
>>>> Subject: Re: [openstack-dev] Proposed Logging Standards
>>>> 
>>>> Hi Matt,
>>>> What about the rest of the components? Do they also have this
>>>> capability?
>>>> Thanks
>>>> 
>>>> Cristian
>>>> 
>>>> On 30/01/14 04:59, "Macdonald-Wallace, Matthew"
>>>>  wrote:
>>>> 
>>>>> Hi Cristian,
>>>>> 
>>>>> The functionality already exists within Openstack (certainly it's
>>>>> there
>>>>> in Nova) it's just not very well documented (something I keep meaning
>>>>> to
>>>>> do!)
>>>>> 
>>>>> Basically you need to add the following to your nova.conf file:
>>>>> 
>>>>> log_config=/etc/nova/logging.conf
>>>>> 
>>>>> And then create /etc/nova/logging.conf with the configuration you want
>>>>> to use based on the Python Logging Module's "ini" configuration
>>>>> format.
>>>>> 
>>>>> Hope that helps,
>>>>> 
>>>>> Matt
>>>>> 
>>>>>> -Original Message-
>>>>>> From: Sanchez, Cristian A [mailto:cristian.a.sanc...@intel.com]
>>>>>> Sent: 29 Ja

Re: [openstack-dev] Proposed Logging Standards

[Sanchez, Cristian A]

Is there any technical reason of why Swift does not use oslo logging?
If not, I can work on incorporating that to Swift.

Thanks

Cristian

On 30/01/14 11:12, "Sean Dague"  wrote:

>For all projects that use oslo logging (which is currently everything
>except swift), this works.
>
>   -Sean
>
>On 01/30/2014 09:07 AM, Macdonald-Wallace, Matthew wrote:
>> No idea, I only really work on Nova, but as this is in Oslo I expect so!
>> 
>> Matt
>> 
>>> -Original Message-
>>> From: Sanchez, Cristian A [mailto:cristian.a.sanc...@intel.com]
>>> Sent: 30 January 2014 13:44
>>> To: OpenStack Development Mailing List (not for usage questions)
>>> Subject: Re: [openstack-dev] Proposed Logging Standards
>>>
>>> Hi Matt,
>>> What about the rest of the components? Do they also have this
>>>capability?
>>> Thanks
>>>
>>> Cristian
>>>
>>> On 30/01/14 04:59, "Macdonald-Wallace, Matthew"
>>>  wrote:
>>>
>>>> Hi Cristian,
>>>>
>>>> The functionality already exists within Openstack (certainly it's
>>>>there
>>>> in Nova) it's just not very well documented (something I keep meaning
>>>> to
>>>> do!)
>>>>
>>>> Basically you need to add the following to your nova.conf file:
>>>>
>>>> log_config=/etc/nova/logging.conf
>>>>
>>>> And then create /etc/nova/logging.conf with the configuration you want
>>>> to use based on the Python Logging Module's "ini" configuration
>>>>format.
>>>>
>>>> Hope that helps,
>>>>
>>>> Matt
>>>>
>>>>> -Original Message-
>>>>> From: Sanchez, Cristian A [mailto:cristian.a.sanc...@intel.com]
>>>>> Sent: 29 January 2014 17:57
>>>>> To: OpenStack Development Mailing List (not for usage questions)
>>>>> Subject: Re: [openstack-dev] Proposed Logging Standards
>>>>>
>>>>> Hi Matthew,
>>>>> I¹m interested to help in this switch to python logging framework for
>>>>> shipping to  logstash/etc. Are you working on a blueprint for this?
>>>>> Cheers,
>>>>>
>>>>> Cristian
>>>>>
>>>>> On 27/01/14 11:07, "Macdonald-Wallace, Matthew"
>>>>>  wrote:
>>>>>
>>>>>> Hi Sean,
>>>>>>
>>>>>> I'm currently working on moving away from the "built-in" logging to
>>>>>> use log_config= and the python logging framework so that
>>>>>> we can start shipping to logstash/sentry/>>>>>here>.
>>>>>>
>>>>>> I'd be very interested in getting involved in this, especially from
>>>>>> a "why do we have log messages that are split across multiple lines"
>>>>>> perspective!
>>>>>>
>>>>>> Cheers,
>>>>>>
>>>>>> Matt
>>>>>>
>>>>>> P.S. FWIW, I'd also welcome details on what the "Audit" level gives
>>>>>> us that the others don't... :)
>>>>>>
>>>>>>> -Original Message-
>>>>>>> From: Sean Dague [mailto:s...@dague.net]
>>>>>>> Sent: 27 January 2014 13:08
>>>>>>> To: OpenStack Development Mailing List
>>>>>>> Subject: [openstack-dev] Proposed Logging Standards
>>>>>>>
>>>>>>> Back at the beginning of the cycle, I pushed for the idea of doing
>>>>>>> some log  harmonization, so that the OpenStack logs, across
>>>>>>> services, made sense.
>>>>>>> I've
>>>>>>> pushed a proposed changes to Nova and Keystone over the past
>>>>>>> couple of days.
>>>>>>>
>>>>>>> This is going to be a long process, so right now I want to just
>>>>>>> focus on making  INFO level sane, because as someone that spends a
>>>>>>> lot of time staring at logs in  test failures, I can tell you it
>>>>>>> currently isn't.
>>>>>>>
>>>>>>> https://wiki.openstack.org/wiki/LoggingStandards is a few things
>>>>>>> I've written  down so far, comments welc

Re: [openstack-dev] Proposed Logging Standards

[Sean Dague]

On 01/28/2014 06:28 PM, Scott Devoid wrote:
> 
> A big part of my interest here is to make INFO a useful informational
> level for operators. That means getting a bunch of messages out of it
> that don't belong.
> 
> 
> +1 to that! How should I open / tag bugs for this?

I'm thinking right now we should probably call out specifically
unhelpful messages in the wiki -
https://wiki.openstack.org/wiki/LoggingStandards  (possibly create a new
page? https://wiki.openstack.org/wiki/LoggingStandardsBadLogMessages ?)

With a suggestion on whether we should either:
 * fix the message to be useful (the secret decoder ring problem)
 * push it to DEBUG

Right now straight out deleting messages is not my intent, it's make
them useful, or put them to DEBUG. We'll audit DEBUG later.

I am very much interested in getting feedback from large operators like
yourself on this, as I think that's a really important voice in this
discussion.

> We should be logging user / tenant on every wsgi request, so that should
> be parsable out of INFO. If not, we should figure out what is falling
> down there.
> 
> 
> At the moment we're not automatically parsing logs (just collecting via
> syslog and logstash).

Well for logstash purposes, the standard format should give you user /
tenant

> 
> Follow on question: do you primarily use the EC2 or OSAPI? As there are
> some current short comings on the EC2 logging, and figuring out
> normalizing those would be good as well.
> 
>  
> Most of our users work through Horizon or the nova CLI. Good to know
> about the EC2 issues though.

Thanks for the feedback.

-Sean

-- 
Sean Dague
Samsung Research America
s...@dague.net / sean.da...@samsung.com
http://dague.net



signature.asc
Description: OpenPGP digital signature
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] Proposed Logging Standards

[Sean Dague]

On 01/27/2014 11:03 AM, Macdonald-Wallace, Matthew wrote:
>>> I've also noticed just now that we appear to be "re-inventing" some parts of
>> the logging framework (openstack.common.log.WriteableLogger for example
>> appears to be a "catchall" when we should just be handing off to the default
>> logger and letting the python logging framework decide what to do IMHO).
>>
>> WriteableLogger exists for a very specific reason: eventlet. Eventlet 
>> assumes a
>> file object for logging, not a python logger.
>>
>> I've proposed a change for that -
>> https://github.com/eventlet/eventlet/pull/75 - but it's not yet upstream.
> 
> Thanks for clearing that up, makes a lot more sense now!
> 
> So when the change is merged upstream we can get rid of that in our code as 
> well?

I'm pretty sure that's the only place it's used, so my hope is yes.

-Sean

-- 
Sean Dague
Samsung Research America
s...@dague.net / sean.da...@samsung.com
http://dague.net



signature.asc
Description: OpenPGP digital signature
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] Proposed Logging Standards

[Sean Dague]

For all projects that use oslo logging (which is currently everything
except swift), this works.

-Sean

On 01/30/2014 09:07 AM, Macdonald-Wallace, Matthew wrote:
> No idea, I only really work on Nova, but as this is in Oslo I expect so!
> 
> Matt
> 
>> -Original Message-
>> From: Sanchez, Cristian A [mailto:cristian.a.sanc...@intel.com]
>> Sent: 30 January 2014 13:44
>> To: OpenStack Development Mailing List (not for usage questions)
>> Subject: Re: [openstack-dev] Proposed Logging Standards
>>
>> Hi Matt,
>> What about the rest of the components? Do they also have this capability?
>> Thanks
>>
>> Cristian
>>
>> On 30/01/14 04:59, "Macdonald-Wallace, Matthew"
>>  wrote:
>>
>>> Hi Cristian,
>>>
>>> The functionality already exists within Openstack (certainly it's there
>>> in Nova) it's just not very well documented (something I keep meaning
>>> to
>>> do!)
>>>
>>> Basically you need to add the following to your nova.conf file:
>>>
>>> log_config=/etc/nova/logging.conf
>>>
>>> And then create /etc/nova/logging.conf with the configuration you want
>>> to use based on the Python Logging Module's "ini" configuration format.
>>>
>>> Hope that helps,
>>>
>>> Matt
>>>
>>>> -Original Message-
>>>> From: Sanchez, Cristian A [mailto:cristian.a.sanc...@intel.com]
>>>> Sent: 29 January 2014 17:57
>>>> To: OpenStack Development Mailing List (not for usage questions)
>>>> Subject: Re: [openstack-dev] Proposed Logging Standards
>>>>
>>>> Hi Matthew,
>>>> I¹m interested to help in this switch to python logging framework for
>>>> shipping to  logstash/etc. Are you working on a blueprint for this?
>>>> Cheers,
>>>>
>>>> Cristian
>>>>
>>>> On 27/01/14 11:07, "Macdonald-Wallace, Matthew"
>>>>  wrote:
>>>>
>>>>> Hi Sean,
>>>>>
>>>>> I'm currently working on moving away from the "built-in" logging to
>>>>> use log_config= and the python logging framework so that
>>>>> we can start shipping to logstash/sentry/.
>>>>>
>>>>> I'd be very interested in getting involved in this, especially from
>>>>> a "why do we have log messages that are split across multiple lines"
>>>>> perspective!
>>>>>
>>>>> Cheers,
>>>>>
>>>>> Matt
>>>>>
>>>>> P.S. FWIW, I'd also welcome details on what the "Audit" level gives
>>>>> us that the others don't... :)
>>>>>
>>>>>> -Original Message-
>>>>>> From: Sean Dague [mailto:s...@dague.net]
>>>>>> Sent: 27 January 2014 13:08
>>>>>> To: OpenStack Development Mailing List
>>>>>> Subject: [openstack-dev] Proposed Logging Standards
>>>>>>
>>>>>> Back at the beginning of the cycle, I pushed for the idea of doing
>>>>>> some log  harmonization, so that the OpenStack logs, across
>>>>>> services, made sense.
>>>>>> I've
>>>>>> pushed a proposed changes to Nova and Keystone over the past
>>>>>> couple of days.
>>>>>>
>>>>>> This is going to be a long process, so right now I want to just
>>>>>> focus on making  INFO level sane, because as someone that spends a
>>>>>> lot of time staring at logs in  test failures, I can tell you it
>>>>>> currently isn't.
>>>>>>
>>>>>> https://wiki.openstack.org/wiki/LoggingStandards is a few things
>>>>>> I've written  down so far, comments welcomed.
>>>>>>
>>>>>> We kind of need to solve this set of recommendations once and for
>>>>>> all up front,  because negotiating each change, with each project,
>>>>>> isn't going to work (e.g -
>>>>>> https://review.openstack.org/#/c/69218/)
>>>>>>
>>>>>> What I'd like to find out now:
>>>>>>
>>>>>> 1) who's interested in this topic?
>>>>>> 2) who's interested in helping flesh out the guidelines for
>>>>>> various log levels?
&g

Re: [openstack-dev] Proposed Logging Standards

[Macdonald-Wallace, Matthew]

No idea, I only really work on Nova, but as this is in Oslo I expect so!

Matt

> -Original Message-
> From: Sanchez, Cristian A [mailto:cristian.a.sanc...@intel.com]
> Sent: 30 January 2014 13:44
> To: OpenStack Development Mailing List (not for usage questions)
> Subject: Re: [openstack-dev] Proposed Logging Standards
> 
> Hi Matt,
> What about the rest of the components? Do they also have this capability?
> Thanks
> 
> Cristian
> 
> On 30/01/14 04:59, "Macdonald-Wallace, Matthew"
>  wrote:
> 
> >Hi Cristian,
> >
> >The functionality already exists within Openstack (certainly it's there
> >in Nova) it's just not very well documented (something I keep meaning
> >to
> >do!)
> >
> >Basically you need to add the following to your nova.conf file:
> >
> >log_config=/etc/nova/logging.conf
> >
> >And then create /etc/nova/logging.conf with the configuration you want
> >to use based on the Python Logging Module's "ini" configuration format.
> >
> >Hope that helps,
> >
> >Matt
> >
> >> -Original Message-
> >> From: Sanchez, Cristian A [mailto:cristian.a.sanc...@intel.com]
> >> Sent: 29 January 2014 17:57
> >> To: OpenStack Development Mailing List (not for usage questions)
> >> Subject: Re: [openstack-dev] Proposed Logging Standards
> >>
> >> Hi Matthew,
> >> I¹m interested to help in this switch to python logging framework for
> >>shipping to  logstash/etc. Are you working on a blueprint for this?
> >> Cheers,
> >>
> >> Cristian
> >>
> >> On 27/01/14 11:07, "Macdonald-Wallace, Matthew"
> >>  wrote:
> >>
> >> >Hi Sean,
> >> >
> >> >I'm currently working on moving away from the "built-in" logging to
> >> >use log_config= and the python logging framework so that
> >> >we can start shipping to logstash/sentry/.
> >> >
> >> >I'd be very interested in getting involved in this, especially from
> >> >a "why do we have log messages that are split across multiple lines"
> >> >perspective!
> >> >
> >> >Cheers,
> >> >
> >> >Matt
> >> >
> >> >P.S. FWIW, I'd also welcome details on what the "Audit" level gives
> >> >us that the others don't... :)
> >> >
> >> >> -Original Message-
> >> >> From: Sean Dague [mailto:s...@dague.net]
> >> >> Sent: 27 January 2014 13:08
> >> >> To: OpenStack Development Mailing List
> >> >> Subject: [openstack-dev] Proposed Logging Standards
> >> >>
> >> >> Back at the beginning of the cycle, I pushed for the idea of doing
> >> >>some log  harmonization, so that the OpenStack logs, across
> >> >>services, made sense.
> >> >>I've
> >> >> pushed a proposed changes to Nova and Keystone over the past
> >> >>couple of days.
> >> >>
> >> >> This is going to be a long process, so right now I want to just
> >> >>focus on making  INFO level sane, because as someone that spends a
> >> >>lot of time staring at logs in  test failures, I can tell you it
> >> >>currently isn't.
> >> >>
> >> >> https://wiki.openstack.org/wiki/LoggingStandards is a few things
> >> >>I've written  down so far, comments welcomed.
> >> >>
> >> >> We kind of need to solve this set of recommendations once and for
> >> >>all up front,  because negotiating each change, with each project,
> >> >>isn't going to work (e.g -
> >> >> https://review.openstack.org/#/c/69218/)
> >> >>
> >> >> What I'd like to find out now:
> >> >>
> >> >> 1) who's interested in this topic?
> >> >> 2) who's interested in helping flesh out the guidelines for
> >> >>various log levels?
> >> >> 3) who's interested in helping get these kinds of patches into
> >> >>various projects in  OpenStack?
> >> >> 4) which projects are interested in participating (i.e. interested
> >> >>in prioritizing  landing these kinds of UX improvements)
> >> >>
> >> >> This is going to be progressive and iterative. And will require
> >> >>lots of folks  involved.
> >> >>
> >> >> -Sean
> >> >>
> >> >> --
> >> >> Sean Dague
> >> >> Samsung Research America
> >> >> s...@dague.net / sean.da...@samsung.com http://dague.net
> >> >
> >> >___
> >> >OpenStack-dev mailing list
> >> >OpenStack-dev@lists.openstack.org
> >> >http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
> >>
> >>
> >> ___
> >> OpenStack-dev mailing list
> >> OpenStack-dev@lists.openstack.org
> >> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
> >
> >___
> >OpenStack-dev mailing list
> >OpenStack-dev@lists.openstack.org
> >http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
> 
> ___
> OpenStack-dev mailing list
> OpenStack-dev@lists.openstack.org
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] Proposed Logging Standards

[Sanchez, Cristian A]

Hi Matt, 
What about the rest of the components? Do they also have this capability?
Thanks

Cristian

On 30/01/14 04:59, "Macdonald-Wallace, Matthew"
 wrote:

>Hi Cristian,
>
>The functionality already exists within Openstack (certainly it's there
>in Nova) it's just not very well documented (something I keep meaning to
>do!)
>
>Basically you need to add the following to your nova.conf file:
>
>log_config=/etc/nova/logging.conf
>
>And then create /etc/nova/logging.conf with the configuration you want to
>use based on the Python Logging Module's "ini" configuration format.
>
>Hope that helps,
>
>Matt
>
>> -Original Message-
>> From: Sanchez, Cristian A [mailto:cristian.a.sanc...@intel.com]
>> Sent: 29 January 2014 17:57
>> To: OpenStack Development Mailing List (not for usage questions)
>> Subject: Re: [openstack-dev] Proposed Logging Standards
>> 
>> Hi Matthew,
>> I¹m interested to help in this switch to python logging framework for
>>shipping to
>> logstash/etc. Are you working on a blueprint for this?
>> Cheers,
>> 
>> Cristian
>> 
>> On 27/01/14 11:07, "Macdonald-Wallace, Matthew"
>>  wrote:
>> 
>> >Hi Sean,
>> >
>> >I'm currently working on moving away from the "built-in" logging to use
>> >log_config= and the python logging framework so that we can
>> >start shipping to logstash/sentry/.
>> >
>> >I'd be very interested in getting involved in this, especially from a
>> >"why do we have log messages that are split across multiple lines"
>> >perspective!
>> >
>> >Cheers,
>> >
>> >Matt
>> >
>> >P.S. FWIW, I'd also welcome details on what the "Audit" level gives us
>> >that the others don't... :)
>> >
>> >> -Original Message-
>> >> From: Sean Dague [mailto:s...@dague.net]
>> >> Sent: 27 January 2014 13:08
>> >> To: OpenStack Development Mailing List
>> >> Subject: [openstack-dev] Proposed Logging Standards
>> >>
>> >> Back at the beginning of the cycle, I pushed for the idea of doing
>> >>some log  harmonization, so that the OpenStack logs, across services,
>> >>made sense.
>> >>I've
>> >> pushed a proposed changes to Nova and Keystone over the past couple
>> >>of days.
>> >>
>> >> This is going to be a long process, so right now I want to just focus
>> >>on making  INFO level sane, because as someone that spends a lot of
>> >>time staring at logs in  test failures, I can tell you it currently
>> >>isn't.
>> >>
>> >> https://wiki.openstack.org/wiki/LoggingStandards is a few things I've
>> >>written  down so far, comments welcomed.
>> >>
>> >> We kind of need to solve this set of recommendations once and for all
>> >>up front,  because negotiating each change, with each project, isn't
>> >>going to work (e.g -
>> >> https://review.openstack.org/#/c/69218/)
>> >>
>> >> What I'd like to find out now:
>> >>
>> >> 1) who's interested in this topic?
>> >> 2) who's interested in helping flesh out the guidelines for various
>> >>log levels?
>> >> 3) who's interested in helping get these kinds of patches into
>> >>various projects in  OpenStack?
>> >> 4) which projects are interested in participating (i.e. interested in
>> >>prioritizing  landing these kinds of UX improvements)
>> >>
>> >> This is going to be progressive and iterative. And will require lots
>> >>of folks  involved.
>> >>
>> >>   -Sean
>> >>
>> >> --
>> >> Sean Dague
>> >> Samsung Research America
>> >> s...@dague.net / sean.da...@samsung.com http://dague.net
>> >
>> >___
>> >OpenStack-dev mailing list
>> >OpenStack-dev@lists.openstack.org
>> >http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>> 
>> 
>> ___
>> OpenStack-dev mailing list
>> OpenStack-dev@lists.openstack.org
>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
>___
>OpenStack-dev mailing list
>OpenStack-dev@lists.openstack.org
>http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] Proposed Logging Standards

[Macdonald-Wallace, Matthew]

Hi Cristian,

The functionality already exists within Openstack (certainly it's there in 
Nova) it's just not very well documented (something I keep meaning to do!)

Basically you need to add the following to your nova.conf file:

log_config=/etc/nova/logging.conf

And then create /etc/nova/logging.conf with the configuration you want to use 
based on the Python Logging Module's "ini" configuration format.

Hope that helps,

Matt

> -Original Message-
> From: Sanchez, Cristian A [mailto:cristian.a.sanc...@intel.com]
> Sent: 29 January 2014 17:57
> To: OpenStack Development Mailing List (not for usage questions)
> Subject: Re: [openstack-dev] Proposed Logging Standards
> 
> Hi Matthew,
> I¹m interested to help in this switch to python logging framework for 
> shipping to
> logstash/etc. Are you working on a blueprint for this?
> Cheers,
> 
> Cristian
> 
> On 27/01/14 11:07, "Macdonald-Wallace, Matthew"
>  wrote:
> 
> >Hi Sean,
> >
> >I'm currently working on moving away from the "built-in" logging to use
> >log_config= and the python logging framework so that we can
> >start shipping to logstash/sentry/.
> >
> >I'd be very interested in getting involved in this, especially from a
> >"why do we have log messages that are split across multiple lines"
> >perspective!
> >
> >Cheers,
> >
> >Matt
> >
> >P.S. FWIW, I'd also welcome details on what the "Audit" level gives us
> >that the others don't... :)
> >
> >> -Original Message-
> >> From: Sean Dague [mailto:s...@dague.net]
> >> Sent: 27 January 2014 13:08
> >> To: OpenStack Development Mailing List
> >> Subject: [openstack-dev] Proposed Logging Standards
> >>
> >> Back at the beginning of the cycle, I pushed for the idea of doing
> >>some log  harmonization, so that the OpenStack logs, across services,
> >>made sense.
> >>I've
> >> pushed a proposed changes to Nova and Keystone over the past couple
> >>of days.
> >>
> >> This is going to be a long process, so right now I want to just focus
> >>on making  INFO level sane, because as someone that spends a lot of
> >>time staring at logs in  test failures, I can tell you it currently
> >>isn't.
> >>
> >> https://wiki.openstack.org/wiki/LoggingStandards is a few things I've
> >>written  down so far, comments welcomed.
> >>
> >> We kind of need to solve this set of recommendations once and for all
> >>up front,  because negotiating each change, with each project, isn't
> >>going to work (e.g -
> >> https://review.openstack.org/#/c/69218/)
> >>
> >> What I'd like to find out now:
> >>
> >> 1) who's interested in this topic?
> >> 2) who's interested in helping flesh out the guidelines for various
> >>log levels?
> >> 3) who's interested in helping get these kinds of patches into
> >>various projects in  OpenStack?
> >> 4) which projects are interested in participating (i.e. interested in
> >>prioritizing  landing these kinds of UX improvements)
> >>
> >> This is going to be progressive and iterative. And will require lots
> >>of folks  involved.
> >>
> >>-Sean
> >>
> >> --
> >> Sean Dague
> >> Samsung Research America
> >> s...@dague.net / sean.da...@samsung.com http://dague.net
> >
> >___
> >OpenStack-dev mailing list
> >OpenStack-dev@lists.openstack.org
> >http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
> 
> 
> ___
> OpenStack-dev mailing list
> OpenStack-dev@lists.openstack.org
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] Proposed Logging Standards

[Sanchez, Cristian A]

Hi Matthew, 
I¹m interested to help in this switch to python logging framework for
shipping to logstash/etc. Are you working on a blueprint for this?
Cheers, 

Cristian

On 27/01/14 11:07, "Macdonald-Wallace, Matthew"
 wrote:

>Hi Sean,
>
>I'm currently working on moving away from the "built-in" logging to use
>log_config= and the python logging framework so that we can
>start shipping to logstash/sentry/.
>
>I'd be very interested in getting involved in this, especially from a
>"why do we have log messages that are split across multiple lines"
>perspective!
>
>Cheers,
>
>Matt
>
>P.S. FWIW, I'd also welcome details on what the "Audit" level gives us
>that the others don't... :)
>
>> -Original Message-
>> From: Sean Dague [mailto:s...@dague.net]
>> Sent: 27 January 2014 13:08
>> To: OpenStack Development Mailing List
>> Subject: [openstack-dev] Proposed Logging Standards
>> 
>> Back at the beginning of the cycle, I pushed for the idea of doing some
>>log
>> harmonization, so that the OpenStack logs, across services, made sense.
>>I've
>> pushed a proposed changes to Nova and Keystone over the past couple of
>>days.
>> 
>> This is going to be a long process, so right now I want to just focus
>>on making
>> INFO level sane, because as someone that spends a lot of time staring
>>at logs in
>> test failures, I can tell you it currently isn't.
>> 
>> https://wiki.openstack.org/wiki/LoggingStandards is a few things I've
>>written
>> down so far, comments welcomed.
>> 
>> We kind of need to solve this set of recommendations once and for all
>>up front,
>> because negotiating each change, with each project, isn't going to work
>>(e.g -
>> https://review.openstack.org/#/c/69218/)
>> 
>> What I'd like to find out now:
>> 
>> 1) who's interested in this topic?
>> 2) who's interested in helping flesh out the guidelines for various log
>>levels?
>> 3) who's interested in helping get these kinds of patches into various
>>projects in
>> OpenStack?
>> 4) which projects are interested in participating (i.e. interested in
>>prioritizing
>> landing these kinds of UX improvements)
>> 
>> This is going to be progressive and iterative. And will require lots of
>>folks
>> involved.
>> 
>>  -Sean
>> 
>> --
>> Sean Dague
>> Samsung Research America
>> s...@dague.net / sean.da...@samsung.com
>> http://dague.net
>
>___
>OpenStack-dev mailing list
>OpenStack-dev@lists.openstack.org
>http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] Proposed Logging Standards

[Sean Dague]

On 01/28/2014 10:53 PM, Vishvananda Ishaya wrote:
> 
> On Jan 27, 2014, at 6:57 PM, Christopher Yeoh  > wrote:
> 
>> On Tue, Jan 28, 2014 at 12:55 AM, Sean Dague > > wrote:
>>
>> On 01/27/2014 09:07 AM, Macdonald-Wallace, Matthew wrote:
>> > Hi Sean,
>> >
>> > I'm currently working on moving away from the "built-in" logging
>> to use log_config= and the python logging framework so
>> that we can start shipping to logstash/sentry/> tool here>.
>> >
>> > I'd be very interested in getting involved in this, especially
>> from a "why do we have log messages that are split across multiple
>> lines" perspective!
>>
>> Do we have many that aren't either DEBUG or TRACE? I thought we were
>> pretty clean there.
>>
>> > Cheers,
>> >
>> > Matt
>> >
>> > P.S. FWIW, I'd also welcome details on what the "Audit" level
>> gives us that the others don't... :)
>>
>> Well as far as I can tell the AUDIT level was a prior drive by
>> contribution that's not being actively maintained. Honestly, I
>> think we
>> should probably rip it out, because I don't see any in tree tooling to
>> use it, and it's horribly inconsistent.
>>
>>
>> For the uses I've seen of it in the nova api code INFO would be
>> perfectly fine in place of AUDIT.
> 
> +1 AUDIT was added for a specific NASA use case because we needed a
> clean feed of important actions for security compliance and many
> upstream libraries were putting out INFO logs that we did not want
> included.  Safe to rip it out IMO.

Cool.

Thanks for the context Vish, very helpful in understanding why some of
this stuff is there so we can unwind it sanely.

-Sean

-- 
Sean Dague
Samsung Research America
s...@dague.net / sean.da...@samsung.com
http://dague.net



signature.asc
Description: OpenPGP digital signature
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] Proposed Logging Standards

[Vishvananda Ishaya]

On Jan 27, 2014, at 6:57 PM, Christopher Yeoh  wrote:

> On Tue, Jan 28, 2014 at 12:55 AM, Sean Dague  wrote:
> On 01/27/2014 09:07 AM, Macdonald-Wallace, Matthew wrote:
> > Hi Sean,
> >
> > I'm currently working on moving away from the "built-in" logging to use 
> > log_config= and the python logging framework so that we can start 
> > shipping to logstash/sentry/.
> >
> > I'd be very interested in getting involved in this, especially from a "why 
> > do we have log messages that are split across multiple lines" perspective!
> 
> Do we have many that aren't either DEBUG or TRACE? I thought we were
> pretty clean there.
> 
> > Cheers,
> >
> > Matt
> >
> > P.S. FWIW, I'd also welcome details on what the "Audit" level gives us that 
> > the others don't... :)
> 
> Well as far as I can tell the AUDIT level was a prior drive by
> contribution that's not being actively maintained. Honestly, I think we
> should probably rip it out, because I don't see any in tree tooling to
> use it, and it's horribly inconsistent.
> 
> 
> For the uses I've seen of it in the nova api code INFO would be perfectly 
> fine in place of AUDIT.

+1 AUDIT was added for a specific NASA use case because we needed a clean feed 
of important actions for security compliance and many upstream libraries were 
putting out INFO logs that we did not want included.  Safe to rip it out IMO.

Vish

> 
> I'd be happy to help out with patches to cleanup the logging in n-api.
> 
> One other thing to look at - I've noticed with logs is that when something 
> like glanceclient code (just as an example) is called from nova,
> we can get ERROR level messages for say image not found when its actually 
> perfectly expected that this will occur.
> I'm not sure if we should be changing the error level in glanceclient or just 
> forcing any error logging in glanceclient when
> called from Nova to a lower level though.
> 
> Chris
> ___
> OpenStack-dev mailing list
> OpenStack-dev@lists.openstack.org
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev



signature.asc
Description: Message signed with OpenPGP using GPGMail
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] Proposed Logging Standards

[Scott Devoid]

> A big part of my interest here is to make INFO a useful informational
> level for operators. That means getting a bunch of messages out of it
> that don't belong.


+1 to that! How should I open / tag bugs for this?

We should be logging user / tenant on every wsgi request, so that should
> be parsable out of INFO. If not, we should figure out what is falling
> down there.
>

At the moment we're not automatically parsing logs (just collecting via
syslog and logstash).

Follow on question: do you primarily use the EC2 or OSAPI? As there are
> some current short comings on the EC2 logging, and figuring out
> normalizing those would be good as well.


Most of our users work through Horizon or the nova CLI. Good to know about
the EC2 issues though.


On Tue, Jan 28, 2014 at 1:46 PM, Sean Dague  wrote:

> On 01/28/2014 12:41 PM, Scott Devoid wrote:
> > For the uses I've seen of it in the nova api code INFO would be
> > perfectly fine in place of AUDIT.
> >
> >
> > We've found the AUDIT logs in nova useful for tracking which user
> > initiated a particular request (e.g. delete this instance). AUDIT had a
> > much better signal to noise ratio than INFO or DEBUG. Although this
> > seems to have changed since Essex. For example nova-compute spits out
> > "AUDIT nova.compute.resource_tracker" messages every minute even if
> > there are no changes :-/
>
> A big part of my interest here is to make INFO a useful informational
> level for operators. That means getting a bunch of messages out of it
> that don't belong.
>
> We should be logging user / tenant on every wsgi request, so that should
> be parsable out of INFO. If not, we should figure out what is falling
> down there.
>
> Follow on question: do you primarily use the EC2 or OSAPI? As there are
> some current short comings on the EC2 logging, and figuring out
> normalizing those would be good as well.
>
> -Sean
>
> >
> > ~ Scott
> >
> >
> > On Tue, Jan 28, 2014 at 11:11 AM, Everett Toews
> > mailto:everett.to...@rackspace.com>>
> wrote:
> >
> > Hi Sean,
> >
> > Could 1.1.1 "Every Inbound WSGI request should be logged Exactly
> > Once" be used to track API call data in order to discover which API
> > calls are being made most frequently?
> >
> > It certainly seems like it could but I want to confirm. I ask
> > because this came up as B "Get aggregate API call data from
> > companies willing to share it." in the user survey discussion [1].
> >
> > Thanks,
> > Everett
> >
> > [1]
> >
> http://lists.openstack.org/pipermail/user-committee/2014-January/000214.html
> >
> >
> > On Jan 27, 2014, at 7:07 AM, Sean Dague wrote:
> >
> > > Back at the beginning of the cycle, I pushed for the idea of doing
> > some
> > > log harmonization, so that the OpenStack logs, across services,
> made
> > > sense. I've pushed a proposed changes to Nova and Keystone over
> > the past
> > > couple of days.
> > >
> > > This is going to be a long process, so right now I want to just
> > focus on
> > > making INFO level sane, because as someone that spends a lot of
> time
> > > staring at logs in test failures, I can tell you it currently
> isn't.
> > >
> > > https://wiki.openstack.org/wiki/LoggingStandards is a few things
> I've
> > > written down so far, comments welcomed.
> > >
> > > We kind of need to solve this set of recommendations once and for
> > all up
> > > front, because negotiating each change, with each project, isn't
> going
> > > to work (e.g - https://review.openstack.org/#/c/69218/)
> > >
> > > What I'd like to find out now:
> > >
> > > 1) who's interested in this topic?
> > > 2) who's interested in helping flesh out the guidelines for
> > various log
> > > levels?
> > > 3) who's interested in helping get these kinds of patches into
> various
> > > projects in OpenStack?
> > > 4) which projects are interested in participating (i.e. interested
> in
> > > prioritizing landing these kinds of UX improvements)
> > >
> > > This is going to be progressive and iterative. And will require
> > lots of
> > > folks involved.
> > >
> > >   -Sean
> > >
> > > --
> > > Sean Dague
> > > Samsung Research America
> > > s...@dague.net  / sean.da...@samsung.com
> > 
> > > http://dague.net
> > >
> > > ___
> > > OpenStack-dev mailing list
> > > OpenStack-dev@lists.openstack.org
> > 
> > > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
> >
> >
> > ___
> > OpenStack-dev mailing list
> > OpenStack-dev@lists.openstack.org
> > 
> > http://lists.openstack.org/cgi-bin/mailman/listinfo/op

Re: [openstack-dev] Proposed Logging Standards

[Sean Dague]

On 01/28/2014 05:56 PM, Jay Pipes wrote:
> On Tue, 2014-01-28 at 16:25 -0500, Brad Topol wrote:
>> So we are starting to add more cloud audit (aka CADF) support to
>> OpenStack.  We have support in Nova and infrastructure added to
>> Ceilometer and I am starting to add this capability to keystone.  This
>> work is based on sending events to ceilometer.  If this is related to
>> the audit work below I would like to be included.
> 
> I don't believe Sean is talking about either notifications or
> tenant-facing activity. He is talking about Python log message
> consistency and cleanup across OpenStack projects.

Correct, this is just about the per service logging, which might be to
files, syslog, or other logging services.

-Sean

-- 
Sean Dague
Samsung Research America
s...@dague.net / sean.da...@samsung.com
http://dague.net



signature.asc
Description: OpenPGP digital signature
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] Proposed Logging Standards

[Jay Pipes]

On Tue, 2014-01-28 at 16:25 -0500, Brad Topol wrote:
> So we are starting to add more cloud audit (aka CADF) support to
> OpenStack.  We have support in Nova and infrastructure added to
> Ceilometer and I am starting to add this capability to keystone.  This
> work is based on sending events to ceilometer.  If this is related to
> the audit work below I would like to be included.

I don't believe Sean is talking about either notifications or
tenant-facing activity. He is talking about Python log message
consistency and cleanup across OpenStack projects.

Best,
-jay


___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] Proposed Logging Standards

[Brad Topol]

So we are starting to add more cloud audit (aka CADF) support to 
OpenStack.  We have support in Nova and infrastructure added to Ceilometer 
and I am starting to add this capability to keystone.  This work is based 
on sending events to ceilometer.  If this is related to the audit work 
below I would like to be included. 

Thanks,

Brad

Brad Topol, Ph.D.
IBM Distinguished Engineer
OpenStack
(919) 543-0646
Internet:  bto...@us.ibm.com
Assistant: Kendra Witherspoon (919) 254-0680



From:   Scott Devoid 
To: "OpenStack Development Mailing List (not for usage questions)" 

Date:   01/28/2014 12:47 PM
Subject:    Re: [openstack-dev] Proposed Logging Standards



For the uses I've seen of it in the nova api code INFO would be perfectly 
fine in place of AUDIT.

We've found the AUDIT logs in nova useful for tracking which user 
initiated a particular request (e.g. delete this instance). AUDIT had a 
much better signal to noise ratio than INFO or DEBUG. Although this seems 
to have changed since Essex. For example nova-compute spits out 
"AUDIT nova.compute.resource_tracker" messages every minute even if there 
are no changes :-/

~ Scott


On Tue, Jan 28, 2014 at 11:11 AM, Everett Toews <
everett.to...@rackspace.com> wrote:
Hi Sean,

Could 1.1.1 "Every Inbound WSGI request should be logged Exactly Once" be 
used to track API call data in order to discover which API calls are being 
made most frequently?

It certainly seems like it could but I want to confirm. I ask because this 
came up as B "Get aggregate API call data from companies willing to share 
it." in the user survey discussion [1].

Thanks,
Everett

[1] 
http://lists.openstack.org/pipermail/user-committee/2014-January/000214.html


On Jan 27, 2014, at 7:07 AM, Sean Dague wrote:

> Back at the beginning of the cycle, I pushed for the idea of doing some
> log harmonization, so that the OpenStack logs, across services, made
> sense. I've pushed a proposed changes to Nova and Keystone over the past
> couple of days.
>
> This is going to be a long process, so right now I want to just focus on
> making INFO level sane, because as someone that spends a lot of time
> staring at logs in test failures, I can tell you it currently isn't.
>
> https://wiki.openstack.org/wiki/LoggingStandards is a few things I've
> written down so far, comments welcomed.
>
> We kind of need to solve this set of recommendations once and for all up
> front, because negotiating each change, with each project, isn't going
> to work (e.g - https://review.openstack.org/#/c/69218/)
>
> What I'd like to find out now:
>
> 1) who's interested in this topic?
> 2) who's interested in helping flesh out the guidelines for various log
> levels?
> 3) who's interested in helping get these kinds of patches into various
> projects in OpenStack?
> 4) which projects are interested in participating (i.e. interested in
> prioritizing landing these kinds of UX improvements)
>
> This is going to be progressive and iterative. And will require lots of
> folks involved.
>
>   -Sean
>
> --
> Sean Dague
> Samsung Research America
> s...@dague.net / sean.da...@samsung.com
> http://dague.net
>
> ___
> OpenStack-dev mailing list
> OpenStack-dev@lists.openstack.org
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] Proposed Logging Standards

[Sean Dague]

On 01/28/2014 12:41 PM, Scott Devoid wrote:
> For the uses I've seen of it in the nova api code INFO would be
> perfectly fine in place of AUDIT.
> 
> 
> We've found the AUDIT logs in nova useful for tracking which user
> initiated a particular request (e.g. delete this instance). AUDIT had a
> much better signal to noise ratio than INFO or DEBUG. Although this
> seems to have changed since Essex. For example nova-compute spits out
> "AUDIT nova.compute.resource_tracker" messages every minute even if
> there are no changes :-/

A big part of my interest here is to make INFO a useful informational
level for operators. That means getting a bunch of messages out of it
that don't belong.

We should be logging user / tenant on every wsgi request, so that should
be parsable out of INFO. If not, we should figure out what is falling
down there.

Follow on question: do you primarily use the EC2 or OSAPI? As there are
some current short comings on the EC2 logging, and figuring out
normalizing those would be good as well.

-Sean

> 
> ~ Scott
> 
> 
> On Tue, Jan 28, 2014 at 11:11 AM, Everett Toews
> mailto:everett.to...@rackspace.com>> wrote:
> 
> Hi Sean,
> 
> Could 1.1.1 "Every Inbound WSGI request should be logged Exactly
> Once" be used to track API call data in order to discover which API
> calls are being made most frequently?
> 
> It certainly seems like it could but I want to confirm. I ask
> because this came up as B "Get aggregate API call data from
> companies willing to share it." in the user survey discussion [1].
> 
> Thanks,
> Everett
> 
> [1]
> 
> http://lists.openstack.org/pipermail/user-committee/2014-January/000214.html
> 
> 
> On Jan 27, 2014, at 7:07 AM, Sean Dague wrote:
> 
> > Back at the beginning of the cycle, I pushed for the idea of doing
> some
> > log harmonization, so that the OpenStack logs, across services, made
> > sense. I've pushed a proposed changes to Nova and Keystone over
> the past
> > couple of days.
> >
> > This is going to be a long process, so right now I want to just
> focus on
> > making INFO level sane, because as someone that spends a lot of time
> > staring at logs in test failures, I can tell you it currently isn't.
> >
> > https://wiki.openstack.org/wiki/LoggingStandards is a few things I've
> > written down so far, comments welcomed.
> >
> > We kind of need to solve this set of recommendations once and for
> all up
> > front, because negotiating each change, with each project, isn't going
> > to work (e.g - https://review.openstack.org/#/c/69218/)
> >
> > What I'd like to find out now:
> >
> > 1) who's interested in this topic?
> > 2) who's interested in helping flesh out the guidelines for
> various log
> > levels?
> > 3) who's interested in helping get these kinds of patches into various
> > projects in OpenStack?
> > 4) which projects are interested in participating (i.e. interested in
> > prioritizing landing these kinds of UX improvements)
> >
> > This is going to be progressive and iterative. And will require
> lots of
> > folks involved.
> >
> >   -Sean
> >
> > --
> > Sean Dague
> > Samsung Research America
> > s...@dague.net  / sean.da...@samsung.com
> 
> > http://dague.net
> >
> > ___
> > OpenStack-dev mailing list
> > OpenStack-dev@lists.openstack.org
> 
> > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
> 
> 
> ___
> OpenStack-dev mailing list
> OpenStack-dev@lists.openstack.org
> 
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
> 
> 
> 
> 
> ___
> OpenStack-dev mailing list
> OpenStack-dev@lists.openstack.org
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
> 


-- 
Sean Dague
Samsung Research America
s...@dague.net / sean.da...@samsung.com
http://dague.net



signature.asc
Description: OpenPGP digital signature
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] Proposed Logging Standards

[Scott Devoid]

>
> For the uses I've seen of it in the nova api code INFO would be perfectly
> fine in place of AUDIT.
>

We've found the AUDIT logs in nova useful for tracking which user initiated
a particular request (e.g. delete this instance). AUDIT had a much better
signal to noise ratio than INFO or DEBUG. Although this seems to have
changed since Essex. For example nova-compute spits out
"AUDIT nova.compute.resource_tracker" messages every minute even if there
are no changes :-/

~ Scott


On Tue, Jan 28, 2014 at 11:11 AM, Everett Toews  wrote:

> Hi Sean,
>
> Could 1.1.1 "Every Inbound WSGI request should be logged Exactly Once" be
> used to track API call data in order to discover which API calls are being
> made most frequently?
>
> It certainly seems like it could but I want to confirm. I ask because this
> came up as B "Get aggregate API call data from companies willing to share
> it." in the user survey discussion [1].
>
> Thanks,
> Everett
>
> [1]
> http://lists.openstack.org/pipermail/user-committee/2014-January/000214.html
>
>
> On Jan 27, 2014, at 7:07 AM, Sean Dague wrote:
>
> > Back at the beginning of the cycle, I pushed for the idea of doing some
> > log harmonization, so that the OpenStack logs, across services, made
> > sense. I've pushed a proposed changes to Nova and Keystone over the past
> > couple of days.
> >
> > This is going to be a long process, so right now I want to just focus on
> > making INFO level sane, because as someone that spends a lot of time
> > staring at logs in test failures, I can tell you it currently isn't.
> >
> > https://wiki.openstack.org/wiki/LoggingStandards is a few things I've
> > written down so far, comments welcomed.
> >
> > We kind of need to solve this set of recommendations once and for all up
> > front, because negotiating each change, with each project, isn't going
> > to work (e.g - https://review.openstack.org/#/c/69218/)
> >
> > What I'd like to find out now:
> >
> > 1) who's interested in this topic?
> > 2) who's interested in helping flesh out the guidelines for various log
> > levels?
> > 3) who's interested in helping get these kinds of patches into various
> > projects in OpenStack?
> > 4) which projects are interested in participating (i.e. interested in
> > prioritizing landing these kinds of UX improvements)
> >
> > This is going to be progressive and iterative. And will require lots of
> > folks involved.
> >
> >   -Sean
> >
> > --
> > Sean Dague
> > Samsung Research America
> > s...@dague.net / sean.da...@samsung.com
> > http://dague.net
> >
> > ___
> > OpenStack-dev mailing list
> > OpenStack-dev@lists.openstack.org
> > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
>
> ___
> OpenStack-dev mailing list
> OpenStack-dev@lists.openstack.org
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] Proposed Logging Standards

[Everett Toews]

Hi Sean,

Could 1.1.1 "Every Inbound WSGI request should be logged Exactly Once" be used 
to track API call data in order to discover which API calls are being made most 
frequently?

It certainly seems like it could but I want to confirm. I ask because this came 
up as B "Get aggregate API call data from companies willing to share it." in 
the user survey discussion [1].

Thanks,
Everett

[1] http://lists.openstack.org/pipermail/user-committee/2014-January/000214.html


On Jan 27, 2014, at 7:07 AM, Sean Dague wrote:

> Back at the beginning of the cycle, I pushed for the idea of doing some
> log harmonization, so that the OpenStack logs, across services, made
> sense. I've pushed a proposed changes to Nova and Keystone over the past
> couple of days.
> 
> This is going to be a long process, so right now I want to just focus on
> making INFO level sane, because as someone that spends a lot of time
> staring at logs in test failures, I can tell you it currently isn't.
> 
> https://wiki.openstack.org/wiki/LoggingStandards is a few things I've
> written down so far, comments welcomed.
> 
> We kind of need to solve this set of recommendations once and for all up
> front, because negotiating each change, with each project, isn't going
> to work (e.g - https://review.openstack.org/#/c/69218/)
> 
> What I'd like to find out now:
> 
> 1) who's interested in this topic?
> 2) who's interested in helping flesh out the guidelines for various log
> levels?
> 3) who's interested in helping get these kinds of patches into various
> projects in OpenStack?
> 4) which projects are interested in participating (i.e. interested in
> prioritizing landing these kinds of UX improvements)
> 
> This is going to be progressive and iterative. And will require lots of
> folks involved.
> 
>   -Sean
> 
> -- 
> Sean Dague
> Samsung Research America
> s...@dague.net / sean.da...@samsung.com
> http://dague.net
> 
> ___
> OpenStack-dev mailing list
> OpenStack-dev@lists.openstack.org
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] Proposed Logging Standards

[iKhan]

I haven't started contributing to openstack, looks like this might be good
opportunity. Count me in. I had the same doubt about AUDIT level log in
cinder.


On Tue, Jan 28, 2014 at 8:16 PM, Alexander Tivelkov
wrote:

> Very interested, thanks a lot for this topic.
> Will work on bringing all of this to Murano
>
> --
> Regards,
> Alexander Tivelkov
>
>
> On Tue, Jan 28, 2014 at 6:45 AM, Sergey Lukjanov 
> wrote:
>
>> FYI it was added to the project meeting agenda -
>> https://wiki.openstack.org/wiki/Meetings/ProjectMeeting
>>
>>
>> On Tue, Jan 28, 2014 at 3:42 PM, Sergey Lukjanov 
>> wrote:
>>
>>> Hi Sean,
>>>
>>> it's great that you're catching this up.
>>>
>>> I'd like to participate. I don't know how much time I'll be able to
>>> dedicate on it, but at least I'm ready for reviews and pushing it to
>>> Savanna.
>>>
>>> Thanks!
>>>
>>>
>>> On Tue, Jan 28, 2014 at 3:21 PM, Sean Dague  wrote:
>>>
 On 01/27/2014 09:57 PM, Christopher Yeoh wrote:
 > On Tue, Jan 28, 2014 at 12:55 AM, Sean Dague >>> > > wrote:
 >
 > On 01/27/2014 09:07 AM, Macdonald-Wallace, Matthew wrote:
 > > Hi Sean,
 > >
 > > I'm currently working on moving away from the "built-in" logging
 > to use log_config= and the python logging framework so
 > that we can start shipping to logstash/sentry/>>> > tool here>.
 > >
 > > I'd be very interested in getting involved in this, especially
 > from a "why do we have log messages that are split across multiple
 > lines" perspective!
 >
 > Do we have many that aren't either DEBUG or TRACE? I thought we
 were
 > pretty clean there.
 >
 > > Cheers,
 > >
 > > Matt
 > >
 > > P.S. FWIW, I'd also welcome details on what the "Audit" level
 > gives us that the others don't... :)
 >
 > Well as far as I can tell the AUDIT level was a prior drive by
 > contribution that's not being actively maintained. Honestly, I
 think we
 > should probably rip it out, because I don't see any in tree
 tooling to
 > use it, and it's horribly inconsistent.
 >
 >
 > For the uses I've seen of it in the nova api code INFO would be
 > perfectly fine in place of AUDIT.
 >
 > I'd be happy to help out with patches to cleanup the logging in n-api.
 >
 > One other thing to look at - I've noticed with logs is that when
 > something like glanceclient code (just as an example) is called from
 nova,
 > we can get ERROR level messages for say image not found when its
 > actually perfectly expected that this will occur.
 > I'm not sure if we should be changing the error level in glanceclient
 or
 > just forcing any error logging in glanceclient when
 > called from Nova to a lower level though.

 It's now changed in glanceclient -
 https://review.openstack.org/#/c/67744/ - it should be gone in the gate
 logs, and will be gone for everyone once a new release is out.

 -Sean

 --
 Sean Dague
 Samsung Research America
 s...@dague.net / sean.da...@samsung.com
 http://dague.net


 ___
 OpenStack-dev mailing list
 OpenStack-dev@lists.openstack.org
 http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


>>>
>>>
>>> --
>>> Sincerely yours,
>>> Sergey Lukjanov
>>> Savanna Technical Lead
>>> Mirantis Inc.
>>>
>>
>>
>>
>> --
>> Sincerely yours,
>> Sergey Lukjanov
>> Savanna Technical Lead
>> Mirantis Inc.
>>
>> ___
>> OpenStack-dev mailing list
>> OpenStack-dev@lists.openstack.org
>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>
>>
>
> ___
> OpenStack-dev mailing list
> OpenStack-dev@lists.openstack.org
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
>


-- 
Thanks,
Ibad Khan
9686594607
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] Proposed Logging Standards

[Alexander Tivelkov]

Very interested, thanks a lot for this topic.
Will work on bringing all of this to Murano

--
Regards,
Alexander Tivelkov


On Tue, Jan 28, 2014 at 6:45 AM, Sergey Lukjanov wrote:

> FYI it was added to the project meeting agenda -
> https://wiki.openstack.org/wiki/Meetings/ProjectMeeting
>
>
> On Tue, Jan 28, 2014 at 3:42 PM, Sergey Lukjanov 
> wrote:
>
>> Hi Sean,
>>
>> it's great that you're catching this up.
>>
>> I'd like to participate. I don't know how much time I'll be able to
>> dedicate on it, but at least I'm ready for reviews and pushing it to
>> Savanna.
>>
>> Thanks!
>>
>>
>> On Tue, Jan 28, 2014 at 3:21 PM, Sean Dague  wrote:
>>
>>> On 01/27/2014 09:57 PM, Christopher Yeoh wrote:
>>> > On Tue, Jan 28, 2014 at 12:55 AM, Sean Dague >> > > wrote:
>>> >
>>> > On 01/27/2014 09:07 AM, Macdonald-Wallace, Matthew wrote:
>>> > > Hi Sean,
>>> > >
>>> > > I'm currently working on moving away from the "built-in" logging
>>> > to use log_config= and the python logging framework so
>>> > that we can start shipping to logstash/sentry/>> > tool here>.
>>> > >
>>> > > I'd be very interested in getting involved in this, especially
>>> > from a "why do we have log messages that are split across multiple
>>> > lines" perspective!
>>> >
>>> > Do we have many that aren't either DEBUG or TRACE? I thought we
>>> were
>>> > pretty clean there.
>>> >
>>> > > Cheers,
>>> > >
>>> > > Matt
>>> > >
>>> > > P.S. FWIW, I'd also welcome details on what the "Audit" level
>>> > gives us that the others don't... :)
>>> >
>>> > Well as far as I can tell the AUDIT level was a prior drive by
>>> > contribution that's not being actively maintained. Honestly, I
>>> think we
>>> > should probably rip it out, because I don't see any in tree
>>> tooling to
>>> > use it, and it's horribly inconsistent.
>>> >
>>> >
>>> > For the uses I've seen of it in the nova api code INFO would be
>>> > perfectly fine in place of AUDIT.
>>> >
>>> > I'd be happy to help out with patches to cleanup the logging in n-api.
>>> >
>>> > One other thing to look at - I've noticed with logs is that when
>>> > something like glanceclient code (just as an example) is called from
>>> nova,
>>> > we can get ERROR level messages for say image not found when its
>>> > actually perfectly expected that this will occur.
>>> > I'm not sure if we should be changing the error level in glanceclient
>>> or
>>> > just forcing any error logging in glanceclient when
>>> > called from Nova to a lower level though.
>>>
>>> It's now changed in glanceclient -
>>> https://review.openstack.org/#/c/67744/ - it should be gone in the gate
>>> logs, and will be gone for everyone once a new release is out.
>>>
>>> -Sean
>>>
>>> --
>>> Sean Dague
>>> Samsung Research America
>>> s...@dague.net / sean.da...@samsung.com
>>> http://dague.net
>>>
>>>
>>> ___
>>> OpenStack-dev mailing list
>>> OpenStack-dev@lists.openstack.org
>>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>>
>>>
>>
>>
>> --
>> Sincerely yours,
>> Sergey Lukjanov
>> Savanna Technical Lead
>> Mirantis Inc.
>>
>
>
>
> --
> Sincerely yours,
> Sergey Lukjanov
> Savanna Technical Lead
> Mirantis Inc.
>
> ___
> OpenStack-dev mailing list
> OpenStack-dev@lists.openstack.org
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
>
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] Proposed Logging Standards

[Sergey Lukjanov]

FYI it was added to the project meeting agenda -
https://wiki.openstack.org/wiki/Meetings/ProjectMeeting


On Tue, Jan 28, 2014 at 3:42 PM, Sergey Lukjanov wrote:

> Hi Sean,
>
> it's great that you're catching this up.
>
> I'd like to participate. I don't know how much time I'll be able to
> dedicate on it, but at least I'm ready for reviews and pushing it to
> Savanna.
>
> Thanks!
>
>
> On Tue, Jan 28, 2014 at 3:21 PM, Sean Dague  wrote:
>
>> On 01/27/2014 09:57 PM, Christopher Yeoh wrote:
>> > On Tue, Jan 28, 2014 at 12:55 AM, Sean Dague > > > wrote:
>> >
>> > On 01/27/2014 09:07 AM, Macdonald-Wallace, Matthew wrote:
>> > > Hi Sean,
>> > >
>> > > I'm currently working on moving away from the "built-in" logging
>> > to use log_config= and the python logging framework so
>> > that we can start shipping to logstash/sentry/> > tool here>.
>> > >
>> > > I'd be very interested in getting involved in this, especially
>> > from a "why do we have log messages that are split across multiple
>> > lines" perspective!
>> >
>> > Do we have many that aren't either DEBUG or TRACE? I thought we were
>> > pretty clean there.
>> >
>> > > Cheers,
>> > >
>> > > Matt
>> > >
>> > > P.S. FWIW, I'd also welcome details on what the "Audit" level
>> > gives us that the others don't... :)
>> >
>> > Well as far as I can tell the AUDIT level was a prior drive by
>> > contribution that's not being actively maintained. Honestly, I
>> think we
>> > should probably rip it out, because I don't see any in tree tooling
>> to
>> > use it, and it's horribly inconsistent.
>> >
>> >
>> > For the uses I've seen of it in the nova api code INFO would be
>> > perfectly fine in place of AUDIT.
>> >
>> > I'd be happy to help out with patches to cleanup the logging in n-api.
>> >
>> > One other thing to look at - I've noticed with logs is that when
>> > something like glanceclient code (just as an example) is called from
>> nova,
>> > we can get ERROR level messages for say image not found when its
>> > actually perfectly expected that this will occur.
>> > I'm not sure if we should be changing the error level in glanceclient or
>> > just forcing any error logging in glanceclient when
>> > called from Nova to a lower level though.
>>
>> It's now changed in glanceclient -
>> https://review.openstack.org/#/c/67744/ - it should be gone in the gate
>> logs, and will be gone for everyone once a new release is out.
>>
>> -Sean
>>
>> --
>> Sean Dague
>> Samsung Research America
>> s...@dague.net / sean.da...@samsung.com
>> http://dague.net
>>
>>
>> ___
>> OpenStack-dev mailing list
>> OpenStack-dev@lists.openstack.org
>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>
>>
>
>
> --
> Sincerely yours,
> Sergey Lukjanov
> Savanna Technical Lead
> Mirantis Inc.
>



-- 
Sincerely yours,
Sergey Lukjanov
Savanna Technical Lead
Mirantis Inc.
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] Proposed Logging Standards

[Sergey Lukjanov]

Hi Sean,

it's great that you're catching this up.

I'd like to participate. I don't know how much time I'll be able to
dedicate on it, but at least I'm ready for reviews and pushing it to
Savanna.

Thanks!


On Tue, Jan 28, 2014 at 3:21 PM, Sean Dague  wrote:

> On 01/27/2014 09:57 PM, Christopher Yeoh wrote:
> > On Tue, Jan 28, 2014 at 12:55 AM, Sean Dague  > > wrote:
> >
> > On 01/27/2014 09:07 AM, Macdonald-Wallace, Matthew wrote:
> > > Hi Sean,
> > >
> > > I'm currently working on moving away from the "built-in" logging
> > to use log_config= and the python logging framework so
> > that we can start shipping to logstash/sentry/ > tool here>.
> > >
> > > I'd be very interested in getting involved in this, especially
> > from a "why do we have log messages that are split across multiple
> > lines" perspective!
> >
> > Do we have many that aren't either DEBUG or TRACE? I thought we were
> > pretty clean there.
> >
> > > Cheers,
> > >
> > > Matt
> > >
> > > P.S. FWIW, I'd also welcome details on what the "Audit" level
> > gives us that the others don't... :)
> >
> > Well as far as I can tell the AUDIT level was a prior drive by
> > contribution that's not being actively maintained. Honestly, I think
> we
> > should probably rip it out, because I don't see any in tree tooling
> to
> > use it, and it's horribly inconsistent.
> >
> >
> > For the uses I've seen of it in the nova api code INFO would be
> > perfectly fine in place of AUDIT.
> >
> > I'd be happy to help out with patches to cleanup the logging in n-api.
> >
> > One other thing to look at - I've noticed with logs is that when
> > something like glanceclient code (just as an example) is called from
> nova,
> > we can get ERROR level messages for say image not found when its
> > actually perfectly expected that this will occur.
> > I'm not sure if we should be changing the error level in glanceclient or
> > just forcing any error logging in glanceclient when
> > called from Nova to a lower level though.
>
> It's now changed in glanceclient -
> https://review.openstack.org/#/c/67744/ - it should be gone in the gate
> logs, and will be gone for everyone once a new release is out.
>
> -Sean
>
> --
> Sean Dague
> Samsung Research America
> s...@dague.net / sean.da...@samsung.com
> http://dague.net
>
>
> ___
> OpenStack-dev mailing list
> OpenStack-dev@lists.openstack.org
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
>


-- 
Sincerely yours,
Sergey Lukjanov
Savanna Technical Lead
Mirantis Inc.
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] Proposed Logging Standards

[Sean Dague]

On 01/27/2014 09:57 PM, Christopher Yeoh wrote:
> On Tue, Jan 28, 2014 at 12:55 AM, Sean Dague  > wrote:
> 
> On 01/27/2014 09:07 AM, Macdonald-Wallace, Matthew wrote:
> > Hi Sean,
> >
> > I'm currently working on moving away from the "built-in" logging
> to use log_config= and the python logging framework so
> that we can start shipping to logstash/sentry/ tool here>.
> >
> > I'd be very interested in getting involved in this, especially
> from a "why do we have log messages that are split across multiple
> lines" perspective!
> 
> Do we have many that aren't either DEBUG or TRACE? I thought we were
> pretty clean there.
> 
> > Cheers,
> >
> > Matt
> >
> > P.S. FWIW, I'd also welcome details on what the "Audit" level
> gives us that the others don't... :)
> 
> Well as far as I can tell the AUDIT level was a prior drive by
> contribution that's not being actively maintained. Honestly, I think we
> should probably rip it out, because I don't see any in tree tooling to
> use it, and it's horribly inconsistent.
> 
> 
> For the uses I've seen of it in the nova api code INFO would be
> perfectly fine in place of AUDIT.
> 
> I'd be happy to help out with patches to cleanup the logging in n-api.
> 
> One other thing to look at - I've noticed with logs is that when
> something like glanceclient code (just as an example) is called from nova,
> we can get ERROR level messages for say image not found when its
> actually perfectly expected that this will occur.
> I'm not sure if we should be changing the error level in glanceclient or
> just forcing any error logging in glanceclient when
> called from Nova to a lower level though.

It's now changed in glanceclient -
https://review.openstack.org/#/c/67744/ - it should be gone in the gate
logs, and will be gone for everyone once a new release is out.

-Sean

-- 
Sean Dague
Samsung Research America
s...@dague.net / sean.da...@samsung.com
http://dague.net



signature.asc
Description: OpenPGP digital signature
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] Proposed Logging Standards

[Thierry Carrez]

Sean Dague wrote:
> Back at the beginning of the cycle, I pushed for the idea of doing some
> log harmonization, so that the OpenStack logs, across services, made
> sense. I've pushed a proposed changes to Nova and Keystone over the past
> couple of days.
> 
> This is going to be a long process, so right now I want to just focus on
> making INFO level sane, because as someone that spends a lot of time
> staring at logs in test failures, I can tell you it currently isn't.
> 
> https://wiki.openstack.org/wiki/LoggingStandards is a few things I've
> written down so far, comments welcomed.
> 
> We kind of need to solve this set of recommendations once and for all up
> front, because negotiating each change, with each project, isn't going
> to work (e.g - https://review.openstack.org/#/c/69218/)
> 
> What I'd like to find out now:
> 
> 1) who's interested in this topic?
> 2) who's interested in helping flesh out the guidelines for various log
> levels?
> 3) who's interested in helping get these kinds of patches into various
> projects in OpenStack?
> 4) which projects are interested in participating (i.e. interested in
> prioritizing landing these kinds of UX improvements)
> 
> This is going to be progressive and iterative. And will require lots of
> folks involved.

I'm interested too (though I have no idea how much time I'll be able to
dedicate to it).

Sounds like a good area for new contributors too.

-- 
Thierry Carrez (ttx)



signature.asc
Description: OpenPGP digital signature
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] Proposed Logging Standards

[Christopher Yeoh]

On Tue, Jan 28, 2014 at 12:55 AM, Sean Dague  wrote:

> On 01/27/2014 09:07 AM, Macdonald-Wallace, Matthew wrote:
> > Hi Sean,
> >
> > I'm currently working on moving away from the "built-in" logging to use
> log_config= and the python logging framework so that we can start
> shipping to logstash/sentry/.
> >
> > I'd be very interested in getting involved in this, especially from a
> "why do we have log messages that are split across multiple lines"
> perspective!
>
> Do we have many that aren't either DEBUG or TRACE? I thought we were
> pretty clean there.
>
> > Cheers,
> >
> > Matt
> >
> > P.S. FWIW, I'd also welcome details on what the "Audit" level gives us
> that the others don't... :)
>
> Well as far as I can tell the AUDIT level was a prior drive by
> contribution that's not being actively maintained. Honestly, I think we
> should probably rip it out, because I don't see any in tree tooling to
> use it, and it's horribly inconsistent.
>
>
For the uses I've seen of it in the nova api code INFO would be perfectly
fine in place of AUDIT.

I'd be happy to help out with patches to cleanup the logging in n-api.

One other thing to look at - I've noticed with logs is that when something
like glanceclient code (just as an example) is called from nova,
we can get ERROR level messages for say image not found when its actually
perfectly expected that this will occur.
I'm not sure if we should be changing the error level in glanceclient or
just forcing any error logging in glanceclient when
called from Nova to a lower level though.

Chris
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] Proposed Logging Standards

[Jay S Bryant]

Sean and John,

I would be happy to help out with this for Cinder.

Let me know how I can help.


Jay S. Bryant
   IBM Cinder Subject Matter Expert  &  Cinder Core Member
Department 7YLA, Building 015-2, Office E125, Rochester, MN
Telephone: (507) 253-4270, FAX (507) 253-6410
TIE Line: 553-4270
E-Mail:  jsbry...@us.ibm.com

 All the world's a stage and most of us are desperately unrehearsed.
   -- Sean O'Casey




From:   John Griffith 
To: "OpenStack Development Mailing List (not for usage questions)" 
, 
Date:   01/27/2014 02:52 PM
Subject:    Re: [openstack-dev] Proposed Logging Standards



On Mon, Jan 27, 2014 at 6:07 AM, Sean Dague  wrote:
> Back at the beginning of the cycle, I pushed for the idea of doing some
> log harmonization, so that the OpenStack logs, across services, made
> sense. I've pushed a proposed changes to Nova and Keystone over the past
> couple of days.
>
> This is going to be a long process, so right now I want to just focus on
> making INFO level sane, because as someone that spends a lot of time
> staring at logs in test failures, I can tell you it currently isn't.
>
> https://wiki.openstack.org/wiki/LoggingStandards is a few things I've
> written down so far, comments welcomed.
>
> We kind of need to solve this set of recommendations once and for all up
> front, because negotiating each change, with each project, isn't going
> to work (e.g - https://review.openstack.org/#/c/69218/)
>
> What I'd like to find out now:
>
> 1) who's interested in this topic?
> 2) who's interested in helping flesh out the guidelines for various log
> levels?
> 3) who's interested in helping get these kinds of patches into various
> projects in OpenStack?
> 4) which projects are interested in participating (i.e. interested in
> prioritizing landing these kinds of UX improvements)
>
> This is going to be progressive and iterative. And will require lots of
> folks involved.
>
> -Sean
>
> --
> Sean Dague
> Samsung Research America
> s...@dague.net / sean.da...@samsung.com
> http://dague.net
>
>
> ___
> OpenStack-dev mailing list
> OpenStack-dev@lists.openstack.org
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
Very interested in all of the above.

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] Proposed Logging Standards

[John Griffith]

On Mon, Jan 27, 2014 at 6:07 AM, Sean Dague  wrote:
> Back at the beginning of the cycle, I pushed for the idea of doing some
> log harmonization, so that the OpenStack logs, across services, made
> sense. I've pushed a proposed changes to Nova and Keystone over the past
> couple of days.
>
> This is going to be a long process, so right now I want to just focus on
> making INFO level sane, because as someone that spends a lot of time
> staring at logs in test failures, I can tell you it currently isn't.
>
> https://wiki.openstack.org/wiki/LoggingStandards is a few things I've
> written down so far, comments welcomed.
>
> We kind of need to solve this set of recommendations once and for all up
> front, because negotiating each change, with each project, isn't going
> to work (e.g - https://review.openstack.org/#/c/69218/)
>
> What I'd like to find out now:
>
> 1) who's interested in this topic?
> 2) who's interested in helping flesh out the guidelines for various log
> levels?
> 3) who's interested in helping get these kinds of patches into various
> projects in OpenStack?
> 4) which projects are interested in participating (i.e. interested in
> prioritizing landing these kinds of UX improvements)
>
> This is going to be progressive and iterative. And will require lots of
> folks involved.
>
> -Sean
>
> --
> Sean Dague
> Samsung Research America
> s...@dague.net / sean.da...@samsung.com
> http://dague.net
>
>
> ___
> OpenStack-dev mailing list
> OpenStack-dev@lists.openstack.org
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
Very interested in all of the above.

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] Proposed Logging Standards

[Jay Pipes]

On Mon, 2014-01-27 at 08:07 -0500, Sean Dague wrote:
> Back at the beginning of the cycle, I pushed for the idea of doing some
> log harmonization, so that the OpenStack logs, across services, made
> sense. I've pushed a proposed changes to Nova and Keystone over the past
> couple of days.
> 
> This is going to be a long process, so right now I want to just focus on
> making INFO level sane, because as someone that spends a lot of time
> staring at logs in test failures, I can tell you it currently isn't.
> 
> https://wiki.openstack.org/wiki/LoggingStandards is a few things I've
> written down so far, comments welcomed.
> 
> We kind of need to solve this set of recommendations once and for all up
> front, because negotiating each change, with each project, isn't going
> to work (e.g - https://review.openstack.org/#/c/69218/)
> 
> What I'd like to find out now:
> 
> 1) who's interested in this topic?
> 2) who's interested in helping flesh out the guidelines for various log
> levels?
> 3) who's interested in helping get these kinds of patches into various
> projects in OpenStack?
> 4) which projects are interested in participating (i.e. interested in
> prioritizing landing these kinds of UX improvements)
> 
> This is going to be progressive and iterative. And will require lots of
> folks involved.

I'm interested, can contribute patches (feel free to assign me) and can
do reviews.

Best,
-jay


___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] Proposed Logging Standards

[Susanne Balle]

What I'd like to find out now:

1) who's interested in this topic?

Please include me.

2) who's interested in helping flesh out the guidelines for various log
levels?

Please include me.

3) who's interested in helping get these kinds of patches into various
projects in OpenStack?
4) which projects are interested in participating (i.e. interested in
prioritizing landing these kinds of UX improvements)

This is going to be progressive and iterative. And will require lots of
folks involved.

Regards Susanne

>
>
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] Proposed Logging Standards

[Macdonald-Wallace, Matthew]

> > I've also noticed just now that we appear to be "re-inventing" some parts of
> the logging framework (openstack.common.log.WriteableLogger for example
> appears to be a "catchall" when we should just be handing off to the default
> logger and letting the python logging framework decide what to do IMHO).
> 
> WriteableLogger exists for a very specific reason: eventlet. Eventlet assumes 
> a
> file object for logging, not a python logger.
> 
> I've proposed a change for that -
> https://github.com/eventlet/eventlet/pull/75 - but it's not yet upstream.

Thanks for clearing that up, makes a lot more sense now!

So when the change is merged upstream we can get rid of that in our code as 
well?

Matt
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] Proposed Logging Standards

[Sean Dague]

On 01/27/2014 09:44 AM, Macdonald-Wallace, Matthew wrote:
>> -Original Message-
>> From: Sean Dague [mailto:s...@dague.net]
>> Sent: 27 January 2014 14:26
>> To: OpenStack Development Mailing List (not for usage questions)
>> Subject: Re: [openstack-dev] Proposed Logging Standards
>>
>> On 01/27/2014 09:07 AM, Macdonald-Wallace, Matthew wrote:
>>> Hi Sean,
>>>
>>> I'm currently working on moving away from the "built-in" logging to use
>> log_config= and the python logging framework so that we can start
>> shipping to logstash/sentry/.
>>>
>>> I'd be very interested in getting involved in this, especially from a "why 
>>> do we
>> have log messages that are split across multiple lines" perspective!
>>
>> Do we have many that aren't either DEBUG or TRACE? I thought we were pretty
>> clean there.
> 
> True, most (all?!!) are DEBUG/TRACE and mainly from calling out to other 
> clients (Neutron/Glance/Cinder/etc), but if you're sending DEBUG somewhere 
> useful for future processing then trying to glue the split-lines back 
> together again can be "interesting".
> 
> At the moment we are assuming that anything that doesn't start with a 
> date-stamp is associated with the line above it.  This is probably OK for 
> now, however if anything changes in future that negates this rule we won't 
> catch it until it's too late!
> 
>>>
>>> P.S. FWIW, I'd also welcome details on what the "Audit" level gives us
>>> that the others don't... :)
>>
>> Well as far as I can tell the AUDIT level was a prior drive by contribution 
>> that's
>> not being actively maintained. Honestly, I think we should probably rip it 
>> out,
>> because I don't see any in tree tooling to use it, and it's horribly 
>> inconsistent.
> 
> +1 for this, I wondered if it was something to do with Ceilometer but I'm 
> guessing probably not from your comment here.
> 
> I've also noticed just now that we appear to be "re-inventing" some parts of 
> the logging framework (openstack.common.log.WriteableLogger for example 
> appears to be a "catchall" when we should just be handing off to the default 
> logger and letting the python logging framework decide what to do IMHO).

WriteableLogger exists for a very specific reason: eventlet. Eventlet
assumes a file object for logging, not a python logger.

I've proposed a change for that -
https://github.com/eventlet/eventlet/pull/75 - but it's not yet upstream.

-Sean

-- 
Sean Dague
Samsung Research America
s...@dague.net / sean.da...@samsung.com
http://dague.net



signature.asc
Description: OpenPGP digital signature
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] Proposed Logging Standards

[Macdonald-Wallace, Matthew]

> -Original Message-
> From: Sean Dague [mailto:s...@dague.net]
> Sent: 27 January 2014 14:26
> To: OpenStack Development Mailing List (not for usage questions)
> Subject: Re: [openstack-dev] Proposed Logging Standards
> 
> On 01/27/2014 09:07 AM, Macdonald-Wallace, Matthew wrote:
> > Hi Sean,
> >
> > I'm currently working on moving away from the "built-in" logging to use
> log_config= and the python logging framework so that we can start
> shipping to logstash/sentry/.
> >
> > I'd be very interested in getting involved in this, especially from a "why 
> > do we
> have log messages that are split across multiple lines" perspective!
> 
> Do we have many that aren't either DEBUG or TRACE? I thought we were pretty
> clean there.
> 
> > Cheers,
> >
> > Matt
> >
> > P.S. FWIW, I'd also welcome details on what the "Audit" level gives us
> > that the others don't... :)
> 
> Well as far as I can tell the AUDIT level was a prior drive by contribution 
> that's
> not being actively maintained. Honestly, I think we should probably rip it 
> out,
> because I don't see any in tree tooling to use it, and it's horribly 
> inconsistent.
> 
>   -Sean

Just as an aside, AUDIT was introduced to the Nova code base as part of 
05ccbb75c45aa3c348162043495e1a3d279e5b06 however a "grep -r AUDIT *" (yes, I 
know, crude but it does work! :P ) across the nova codebase only returns the 
openstack.common.log libraries as having it listed in the code.

I don't know if other projects are making use of it, but if not then I agree 
that it should probably be removed from Oslo

Matt
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] Proposed Logging Standards

[Macdonald-Wallace, Matthew]

> -Original Message-
> From: Sean Dague [mailto:s...@dague.net]
> Sent: 27 January 2014 14:26
> To: OpenStack Development Mailing List (not for usage questions)
> Subject: Re: [openstack-dev] Proposed Logging Standards
> 
> On 01/27/2014 09:07 AM, Macdonald-Wallace, Matthew wrote:
> > Hi Sean,
> >
> > I'm currently working on moving away from the "built-in" logging to use
> log_config= and the python logging framework so that we can start
> shipping to logstash/sentry/.
> >
> > I'd be very interested in getting involved in this, especially from a "why 
> > do we
> have log messages that are split across multiple lines" perspective!
> 
> Do we have many that aren't either DEBUG or TRACE? I thought we were pretty
> clean there.

True, most (all?!!) are DEBUG/TRACE and mainly from calling out to other 
clients (Neutron/Glance/Cinder/etc), but if you're sending DEBUG somewhere 
useful for future processing then trying to glue the split-lines back together 
again can be "interesting".

At the moment we are assuming that anything that doesn't start with a 
date-stamp is associated with the line above it.  This is probably OK for now, 
however if anything changes in future that negates this rule we won't catch it 
until it's too late!

> >
> > P.S. FWIW, I'd also welcome details on what the "Audit" level gives us
> > that the others don't... :)
> 
> Well as far as I can tell the AUDIT level was a prior drive by contribution 
> that's
> not being actively maintained. Honestly, I think we should probably rip it 
> out,
> because I don't see any in tree tooling to use it, and it's horribly 
> inconsistent.

+1 for this, I wondered if it was something to do with Ceilometer but I'm 
guessing probably not from your comment here.

I've also noticed just now that we appear to be "re-inventing" some parts of 
the logging framework (openstack.common.log.WriteableLogger for example appears 
to be a "catchall" when we should just be handing off to the default logger and 
letting the python logging framework decide what to do IMHO).

Cheers,

Matt
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] Proposed Logging Standards

[Peter Portante]

On Mon, Jan 27, 2014 at 8:07 AM, Sean Dague  wrote:

> Back at the beginning of the cycle, I pushed for the idea of doing some
> log harmonization, so that the OpenStack logs, across services, made
> sense. I've pushed a proposed changes to Nova and Keystone over the past
> couple of days.
>
> This is going to be a long process, so right now I want to just focus on
> making INFO level sane, because as someone that spends a lot of time
> staring at logs in test failures, I can tell you it currently isn't.
>
> https://wiki.openstack.org/wiki/LoggingStandards is a few things I've
> written down so far, comments welcomed.
>
> We kind of need to solve this set of recommendations once and for all up
> front, because negotiating each change, with each project, isn't going
> to work (e.g - https://review.openstack.org/#/c/69218/)
>
> What I'd like to find out now:
>
> 1) who's interested in this topic?
>

Interested.


> 2) who's interested in helping flesh out the guidelines for various log
> levels?
>

Interested.


> 3) who's interested in helping get these kinds of patches into various
> projects in OpenStack?
>

Interested, but too much already on plate, so can't guarantee consistent
help.


> 4) which projects are interested in participating (i.e. interested in
> prioritizing landing these kinds of UX improvements)
>
> This is going to be progressive and iterative. And will require lots of
> folks involved.
>
> -Sean
>
> --
> Sean Dague
> Samsung Research America
> s...@dague.net / sean.da...@samsung.com
> http://dague.net
>
>
> ___
> OpenStack-dev mailing list
> OpenStack-dev@lists.openstack.org
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
>
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] Proposed Logging Standards

[Sean Dague]

On 01/27/2014 09:07 AM, Macdonald-Wallace, Matthew wrote:
> Hi Sean,
> 
> I'm currently working on moving away from the "built-in" logging to use 
> log_config= and the python logging framework so that we can start 
> shipping to logstash/sentry/.
> 
> I'd be very interested in getting involved in this, especially from a "why do 
> we have log messages that are split across multiple lines" perspective!

Do we have many that aren't either DEBUG or TRACE? I thought we were
pretty clean there.

> Cheers,
> 
> Matt
> 
> P.S. FWIW, I'd also welcome details on what the "Audit" level gives us that 
> the others don't... :)

Well as far as I can tell the AUDIT level was a prior drive by
contribution that's not being actively maintained. Honestly, I think we
should probably rip it out, because I don't see any in tree tooling to
use it, and it's horribly inconsistent.

-Sean

-- 
Sean Dague
Samsung Research America
s...@dague.net / sean.da...@samsung.com
http://dague.net



signature.asc
Description: OpenPGP digital signature
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] Proposed Logging Standards

[Macdonald-Wallace, Matthew]

Hi Sean,

I'm currently working on moving away from the "built-in" logging to use 
log_config= and the python logging framework so that we can start 
shipping to logstash/sentry/.

I'd be very interested in getting involved in this, especially from a "why do 
we have log messages that are split across multiple lines" perspective!

Cheers,

Matt

P.S. FWIW, I'd also welcome details on what the "Audit" level gives us that the 
others don't... :)

> -Original Message-
> From: Sean Dague [mailto:s...@dague.net]
> Sent: 27 January 2014 13:08
> To: OpenStack Development Mailing List
> Subject: [openstack-dev] Proposed Logging Standards
> 
> Back at the beginning of the cycle, I pushed for the idea of doing some log
> harmonization, so that the OpenStack logs, across services, made sense. I've
> pushed a proposed changes to Nova and Keystone over the past couple of days.
> 
> This is going to be a long process, so right now I want to just focus on 
> making
> INFO level sane, because as someone that spends a lot of time staring at logs 
> in
> test failures, I can tell you it currently isn't.
> 
> https://wiki.openstack.org/wiki/LoggingStandards is a few things I've written
> down so far, comments welcomed.
> 
> We kind of need to solve this set of recommendations once and for all up 
> front,
> because negotiating each change, with each project, isn't going to work (e.g -
> https://review.openstack.org/#/c/69218/)
> 
> What I'd like to find out now:
> 
> 1) who's interested in this topic?
> 2) who's interested in helping flesh out the guidelines for various log 
> levels?
> 3) who's interested in helping get these kinds of patches into various 
> projects in
> OpenStack?
> 4) which projects are interested in participating (i.e. interested in 
> prioritizing
> landing these kinds of UX improvements)
> 
> This is going to be progressive and iterative. And will require lots of folks
> involved.
> 
>   -Sean
> 
> --
> Sean Dague
> Samsung Research America
> s...@dague.net / sean.da...@samsung.com
> http://dague.net

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev