- openstack-dev + fuel-core team Hello. In order to make possible to implement logging improvements (https://blueprints.launchpad.net/fuel/+spec/refactor-logging-puppet-openstack-services) in Fuel manifests for puppet-openstack modules, I'd like to suggest us: 1) do not wait for Oslo patch for https://bugs.launchpad.net/nova/+bug/904307 will be included for all Openstack projects in the later I releases (if it will...). Why? There is a chance some Openstack projects (deployed by Fuel as well) will not sync this patch from Oslo in I, or even J, despite it had been accepted in Oslu-incubatuor, thus - we could not provide then new logging 'use_syslog_rfc_format' config option for such services. (Just for note, this option is heavily needed to reflect app/service names in Fuel UI logging section.)
2) improve our hardened Mirantis Openstack packages by including this patch from Oslo-incubator by our own initiative, for any release we want. I believe OSCI team can do this (see How incubation works in Oslo https://wiki.openstack.org/wiki/Oslo#Incubation). What do you think? -------- Original Message -------- Subject: Fw: [Openstack-dev] [Fuel] [Oslo] Add APP-NAME (RFC5424) for Oslo syslog logging Date: Mon, 03 Feb 2014 16:58:40 +0200 From: Bogdan Dobrelya <[email protected]> Organization: Mirantis To: [email protected], [email protected] On 12/20/2013 08:21 PM, Bogdan Dobrelya wrote: > *Preamble* > Hi stackers, I was trying to implement correct APP-NAME tags for remote > logging in Fuel for Openstack, and faced the > https://bugs.launchpad.net/nova/+bug/904307 issue. There are no logging > options in Python 2.6/2.7 to address this APP-NAME in logging formats or > configs (log_format, log_config(_append)). > > Just look at the log file names, and you will understand me: > cinder-cinder.api.extensions.log > cinder-cinder.db.sqlalchemy.session.log > cinder-cinder.log > cinder-cinder.openstack.common.rpc.common.log > cinder-eventlet.wsgi.server.log > cinder-keystoneclient.middleware.auth_token.log > glance-eventlet.wsgi.server.log > glance-glance.api.middleware.cache.log > glance-glance.api.middleware.cache_manage.log > glance-glance.image_cache.log > glance-keystoneclient.middleware.auth_token.log > keystone-root.log > nova-keystoneclient.middleware.auth_token.log > nova-nova.api.openstack.compute.extensions.log > nova-nova.api.openstack.extensions.log > nova-nova.ec2.wsgi.server.log > nova-nova.log > nova-nova.metadata.wsgi.server.log > nova-nova.network.driver.log > nova-nova.osapi_compute.wsgi.server.log > nova-nova.S3.log > quantum-eventlet.wsgi.server.log > quantum-keystoneclient.middleware.auth_token.log > quantum-quantum.api.extensions.log > quantum-quantum.manager.log > quantum-quantum.openstack.common.rpc.amqp.log > quantum-quantum.plugins.openvswitch.ovs_quantum_plugin.log > > But I actually want to see something like this: > cinder-api.log > cinder-volume.log > glance-api.log > glance-manage.log > glance-registry.log > keystone-all.log > nova-api.log > nova-conductor.log > nova-consoleauth.log > nova-objectstore.log > nova-scheduler.log > ...and so on. > > Another words, logging should honor RFC3164 & RFC5424, here are some quotes: > "The MSG part has two fields known as the TAG field and the CONTENT > field. The value in the TAG field will be the name of the program or > process that generated the message. The CONTENT contains the details of > the message..." > "The APP-NAME field SHOULD identify the device or application that > originated the message..." > > I see two solutions for this issue. > > *Solution 1* > The one of possible solutions is to use new key for log_format (i.e. > %(binary_name)s) to extract application/service name for log records. > The implementation could be like patch #4: > https://review.openstack.org/#/c/63094/4 > And the log_format could be like this: > log_format=%(asctime)s %(binary_name)s %(levelname)s: %(name)s: %(message)s > > The patch is applicable to other Openstack services, which did not moved > to Oslo yet. > I tested it with nova services, and all services can start with > log_format using %(binary_name)s, but nova-api. Looks like > /keystoneclient/middleware/auth_token.py is unhappy with this patch, see > the trace http://paste.openstack.org/show/55519/ > > *Solution 2* > The other and only option I can suggest, is to backport ‘ident’ from > python 3.3, see http://hg.python.org/cpython/rev/6baa90fa2b6d > The implementation could be like this: > https://review.openstack.org/#/c/63094 > To ensure we will have APP-NAME in message we can set use_syslog = true > and check the results. > If we’re using log_config_append, the formatters and handlers could be > like this: > [formatter_normal] > format = %(levelname)s: %(message)s > [handler_production] > class = openstack.common.log.RFCSysLogHandler > level = INFO > formatter = normal > args = ('/dev/log', handlers.SysLogHandler.LOG_LOCAL6) > > The patch is also applicable to other Openstack services, which did not > moved to Oslo yet. > For syslog logging, the application/service/process name (aka APP-NAME, > see RFC5424) would be added before the MSG part, right after it has been > formatted, and there is no need to use any special log_format settings > as well. > > *Conclusion* > I vote for implement solution 2 for Oslo logging, and for those > Openstack services, which don’t use Oslo for logging yet. That would not > require any changes outside of the Openstack modules, thus looks like a > good compromise for backporting ‘ident’ feature for APP-NAME tags from > Python 3.3. What do you think? > Hi, stackers. FYI, the patch https://bugs.launchpad.net/oslo/+bug/904307 had been merged into Oslo-incubator to implement Option 2 (see above). According to the docs provided, to honor RFC5424 APP-NAME for syslog messages, the existing syslog format is DEPRECATED during I and will be removed in J. New use_syslog_rfc_format option was introduced: (Optional) use syslog rfc5424 format for logging. If enabled, will add APP-NAME (RFC5424) before the MSG part of the syslog message. The old format without APP-NAME is deprecated in I, and will be removed in J. 1) Please consider to sync the patch to any affected Openstack projects (i.e. all projects with use_syslog option in their configs) to ensure the application/service name will be present in the syslog messages, if use_syslog_rfc_format = true. 2) Please consider to add this info to the Syslog section of the https://wiki.openstack.org/wiki/LoggingStandards > -- > Best regards, > Bogdan Dobrelya, > Researcher TechLead, Mirantis, Inc. > +38 (066) 051 07 53 > Skype bogdando_at_yahoo.com > Irc #bogdando > 38, Lenina ave. > Kharkov, Ukraine > www.mirantis.com > www.mirantis.ru > [email protected] > -- Best regards, Bogdan Dobrelya, Researcher TechLead, Mirantis, Inc. +38 (066) 051 07 53 Skype bogdando_at_yahoo.com Irc #bogdando 38, Lenina ave. Kharkov, Ukraine www.mirantis.com www.mirantis.ru [email protected] -- Mailing list: https://launchpad.net/~fuel-dev Post to : [email protected] Unsubscribe : https://launchpad.net/~fuel-dev More help : https://help.launchpad.net/ListHelp
