----- Original Message -----
> OK, unless anyone has anything else to add I'm going to assume that
> the
> project maintainers are OK with message IDs and I'll start the
> process.

Danken, where do you stand on this?

> 
> To summarize:
> 1. I'm going to do some minor surgery to the logger so that the log
> format is pinned an not user modifiable.  This is necessary to ensure
> that message IDs can be substituted into the string.
> 2. Message IDs will have the following format: VDSM#####
> 3. Message IDs will just be a simple up-counter across all of VDSM.
> 4. Existing strings will be converted to translatable strings.  The
> string itself won't be changed it will just be wrapped by _(...) so
> that
> gettext will work.
> 5. Message IDs may be documented, somewhere not sure where yet, with
> a
> brief explanation (if an explanation is appropriate).  This part
> might
> take some and; hopefully, the explanations will evolve.  I expect
>  many
> of the IDs to not have explanations right away though and I
> definitely
> don't want to put any bad explanations up or ones that haven't been
> vetted.
> 
> Cheers,
> Keith
> 
> On 10/06/2011 03:29 PM, Keith Robertson wrote:
> > On 10/06/2011 03:11 PM, Ayal Baron wrote:
> >>>>>>>>> I would like to propose enhancing the VDSM source to emit
> >>>>>>>>> message
> >>>>>>>>> IDs in
> >>>>>>>>> log messages with levels greater than DEBUG (i.e. INFO,
> >>>>>>>>> ERROR,
> >>>>>>>>> WARN,
> >>>>>>>>> EXCEPTION, and CRITICAL). Messages IDs have been used
> >>>>>>>>> successfully in
> >>>>>>>>> many products and they can aid in debugging, documentation,
> >>>>>>>>> root
> >>>>>>>>> cause
> >>>>>>>>> analysis, and internationalization.
> >>>>>>>>>
> >>>>>>>>> I have scanned the source and have determined that there
> >>>>>>>>> are
> >>>>>>>>> roughly 525
> >>>>>>>>> calls to log.[error|warn|info|critical|exception] and I am
> >>>>>>>>> volunteering
> >>>>>>>>> to go through and add IDs to them all but, before I do I
> >>>>>>>>> want
> >>>>>>>>> to get
> >>>>>>>>> buy-in from the project maintainers. While adding IDs to
> >>>>>>>>> the
> >> wrt i18n, I'm not entirely convinced.
> > Yeah, i18n is one reason to do it but it is only relevant if you
> > take
> > the time to create lang-packs for different nationalities (which
> > can
> > be expensive). However, it is a cakewalk to code for in Python
> > _("string here").  So you can code for it ... and if someone ever
> > wanted to take the time create a lang-pack you'd be set.
> >>   Every line we log has the module name and line (never localized)
> >> which logged it so effectively we already have this info.
> >> However, the ID would remain constant where lines will change
> >> place
> >> between versions so it makes it easier.
> >> I also agree about the lookup and being able to write simpler
> >> parsers.
> >>
> >> So I'm in.
> >>
> > Cool.
> >>>>>>>>> messages I
> >>>>>>>>> would also enable them for i18n.
> >>>>>>>>>
> >>>>>>>>> I have written a rough plan for adding messages IDs to VDSM
> >>>>>>>>> below
> >>>>>>>>> and
> >>>>>>>>> I've provided some examples of products/projects that use
> >>>>>>>>> message
> >>>>>>>>> IDs.
> >>>>>>>>>
> >>>>>>>>> Examples of Projects that Use Message IDs:
> >>>>>>>>> - JBoss 7 is adopting Message IDs for all of its logging.
> >>>>>>>>> See
> >>>>>>>>> the
> >>>>>>>>> annotations (e.g. @Message(id = 12000, ) in the following
> >>>>>>>>> piece of
> >>>>>>>>> source:
> >>>>>>>>> https://github.com/jbossas/jboss-as/blob/master/process-controller/src/main/java/org/jboss/as/process/ProcessLogger.java
> >>>>>>>>>
> >>>>>>>>>
> >>>>>>>>>
> >>>>>>>>>
> >>>>>>>>> - Almost all IBM products use message IDs and they do a
> >>>>>>>>> fairly
> >>>>>>>>> good
> >>>>>>>>> job
> >>>>>>>>> of documenting them. Google has indexed them all and you
> >>>>>>>>> can
> >>>>>>>>> simply
> >>>>>>>>> type
> >>>>>>>>> them in to a search and get a description of the problem
> >>>>>>>>> that
> >>>>>>>>> is
> >>>>>>>>> associated with the ID.
> >>>>>>>>> http://publib.boulder.ibm.com/infocenter/tivihelp/v42r1/topic/com.ibm.omegamon.stor.doc/GC23-9705-0484.htm
> >>>>>>>>>
> >>>>>>>>>
> >>>>>>>>>
> >>>>>>>>>
> >>>>>>>>>
> >>>>>>>>>
> >>>>>>>>> Example of a Log Statement with a Message ID in Python:
> >>>>>>>>> FORMAT = '%(asctime)s %(lineno)s
> >>>>>>>>> %(levelname)s(%(messageid)s):
> >>>>>>>>> %(message)s'
> >>>>>>>>> logging.basicConfig(format=FORMAT)
> >>>>>>>>> logger = logging.getLogger()
> >>>>>>>>> logger.warning('NFS problem. Unable to mount.', extra={
> >>>>>>>>> 'messageid' :
> >>>>>>>>> 'VDSM1000'})
> >>>>>>>>>
> >>>>>>>>> // Produces
> >>>>>>>>> 2011-10-05 13:24:43,180 11 WARNING(VDSM1000): NFS problem.
> >>>>>>>>> Unable
> >>>>>>>>> to mount.
> >>>>>>>>>
> >>>>>>>>>
> >>>>>>>>> Example of a Log Statement with a Message ID in Python with
> >>>>>>>>> i18n:
> >>>>>>>>> logger.warning(_('NFS problem. Unable to mount.'), extra={
> >>>>>>>>> 'messageid' :
> >>>>>>>>> 'VDSM1000'})
> >>>>>>>>>
> >>>>>>>>> Example of an i18n Log Message:
> >>>>>>>>> Chinese: 2011年10月5日13:24:43,18011警告(VDSM1000):NFS问
> >>>>>>>>> 题。无法
> >>>>>>>>> 安 装。
> >>>>>>>>> English: 2011-10-05 13:24:43,180 11 WARNING(VDSM1000): NFS
> >>>>>>>>> problem.
> >>>>>>>>> Unable to mount.
> >>>>>>>>>
> >>>>>>>>>
> >>>>>>>>>
> >>>>>>>>> A Rough Plan for adding Message IDs to VDSM:
> >>>>>>>>>
> >>>>>>>>> Step 0: Decide on a logging format of the ID itself:
> >>>>>>>>> - JBoss is using "JBAS#####". I am thinking that VDSM could
> >>>>>>>>> probably do
> >>>>>>>>> something similar "VDSM####".
> >>>>>>>>>
> >>>>>>>>> Step 1: Decide where to list and document all of the IDs.
> >>>>>>>>> - It could be as simple as a page on VDSM's fedorahosted
> >>>>>>>>> wiki.
> >>>>>>>>>
> >>>>>>>>> Step 2: Pin the format of the log messages:
> >>>>>>>>> - Currently VDSM allows a user configurable Python log
> >>>>>>>>> format.
> >>>>>>>>> We
> >>>>>>>>> would
> >>>>>>>>> need to pin the format so that it includes a substitution
> >>>>>>>>> variable for
> >>>>>>>>> the message ID.
> >>>>>>>>> Example: '%(asctime)s %(lineno)s
> >>>>>>>>> %(levelname)s(%(messageid)s):
> >>>>>>>>> %(message)s'
> >>>>>>>>>
> >>>>>>>>> Step 4: Plow through the VDSM source and add message IDs.
> >>>>>>>>>
> >>>>>>>>>
> >>>>>>>>> Cheers,
> >>>>>>>>> Keith
> >>>>>>>>>
> >>>>>>>>>
> >>>>>>>>>
> >>>>>>>>>
> >>>>>>>>>
> >>>>>>>>> _______________________________________________
> >>>>>>>>> vdsm-devel mailing list
> >>>>>>>>> vdsm-devel@lists.fedorahosted.org
> >>>>>>>>> https://fedorahosted.org/mailman/listinfo/vdsm-devel
> >>>>>>>> Seems like a nice idea. But how does it affect developers.
> >>>>>>>> How
> >>>>>>>> do
> >>>>>>>> they choose mIDs? How do you prevent collisions? How are
> >>>>>>>> they
> >>>>>>>> different from simple gettext?
> >>>>>> As I never used this myself so I can't really imagine the all
> >>>>>> benefits
> >>>>>> of using such a system, at least enough to warrant all the
> >>>>>> tedious
> >>>>>> work. If you could just list out some of the use cases it'll
> >>>>>> be
> >>>>>> most
> >>>>>> helpful.
> >>>>>>
> >>>>>> For instance I assume grepping for an ID is easier then
> >>>>>> grepping
> >>>>>> for a
> >>>>>> message especially in multiple versions where the message
> >>>>>> might
> >>>>>> have
> >>>>>> changed.
> >>>>> Sure.  As far as the tedious part, yes, adding message IDs to
> >>>>> an
> >>>>> existing source tree is a bit of a PITA but the 525 invocations
> >>>>> in
> >>>>> VDSM
> >>>>> isn't all that bad.  I've seen much worse.  Plus, once it's
> >>>>> done
> >>>>> the
> >>>>> effort is much smaller (new lines of code only).
> >>>>>
> >>>>> As for the value (I've tried to itemize a few common reasons
> >>>>> for
> >>>>> doing it):
> >>>>>
> >>>>> Value: Message IDs allow you to use gettext globalize your
> >>>>> messages. If
> >>>>> you just used gettext and pulled the strings out of your code
> >>>>> for
> >>>>> globalization then you might end up with something like this in
> >>>>> vdsm.log...
> >>>>>       错误:问题安装驱动器
> >>>>> the English translation (according to google) is:
> >>>>>       ERROR: Problem mounting drive
> >>>>> Unless you can read Chinese, then the message above is
> >>>>> completely
> >>>>> unhelpful to a support engineer.  The converse is true also.
> >>>>>   Consider a
> >>>>> non-english speaking person trying to decipher "ERROR: Problem
> >>>>> mounting
> >>>>> drive".  With a message ID; however, you get this...
> >>>>>       错误(VDSM1000):安装驱动问题
> >>>> Is it possible to just hash the main string before
> >>>> internationalization?
> >>>>
> >>>> So basically:
> >>>>
> >>>> fprintf(f, "[%s] ERROR(VDSM%04x) %s", date_str, crc32(message),
> >>>> message);
> >>> Yes, a hash of the string is possible.  However, would make it
> >>> hard
> >>> for
> >>> someone to document the ID because they would have to hash the
> >>> string
> >>> to
> >>> get the ID.
> >>>
> >>> One of the features of message ID/codes is that they are
> >>> documentable.
> >>>> Or the equivalent Python.
> >>>>
> >>>>> Value: Message IDs allow other applications to perform actions
> >>>>> based on
> >>>>> what is in your logs.  On the mainframe, this is extremely
> >>>>> common.
> >>>>> You'll have one program watching the logs for a particular
> >>>>> ID(s).
> >>>>>   When
> >>>>> the program sees the ID it will look in a dictionary and decide
> >>>>> what (if
> >>>>> anything) needs to be done.  On linux, logwatch, is frequently
> >>>>> used for
> >>>>> this purpose and message IDs make this process much simpler
> >>>>> because the
> >>>>> program doesn't have to write complicated REGEXs to look for
> >>>>> arbitrary
> >>>>> strings it can just scan for an ID.
> >>>>>
> >>>>> Value: Message IDs generally have the same meaning across
> >>>>> releases
> >>>>> of
> >>>>> the product.  This means that the developer can change the text
> >>>>> of
> >>>>> the
> >>>>> log message without worrying about breaking someone's
> >>>>> automation.
> >>>>> Example:
> >>>>>       Version 1 of the product- ERROR(VDSM1000): Problem
> >>>>>       mounting
> >>>>>       drive.
> >>>>>       Version 2 of the product- ERROR(VDSM1000): unable to
> >>>>>       contact
> >>>>> 127.0.0.1 for mounting. Try blah blah.
> >>>> Does this mean that logging messages are an ABI that needs to be
> >>>> maintained over time?
> >>> The degree to which message IDs need to be maintained is
> >>> generally a
> >>> function of how dynamic the code is and how much you want to
> >>> document
> >>> what the ID actually means.  If, on the WIKI wherein you define
> >>> messages, you elect to say [1] then this "ABI" doesn't need any
> >>> maintenance whatsoever.
> >>>
> >>> [1] VDSM1000 == "Call support or post to the forum to figure out
> >>> what
> >>> the heck the program just did"
> >>>
> >>>> That's probably very difficult to do right.
> >>> Lots of other software programs have been successful at it [1],
> >>> [2].
> >>>   I
> >>> would prefer to think of it in terms of how easy we want to make
> >>> it
> >>> for
> >>> people to self diagnose.  Also, without message IDs supporting
> >>> source
> >>> that is i18n is nearly impossible unless everyone is
> >>> multilingual.
> >>>
> >>> [1]
> >>> http://www.cisco.com/en/US/products/sw/netmgtsw/ps1982/products_command_reference_chapter09186a0080087439.html
> >>>
> >>> [2] http://mail.google.com/support/bin/answer.py?answer=12336
> >>>
> >>>
> >>>> Regards,
> >>>>
> >>>> Anthony Liguori
> >>>>
> >>>>> Value: If you do it right search engines will index your
> >>>>> message
> >>>>> IDs and
> >>>>> take users to a page where they can hopefully get more detail
> >>>>> on
> >>>>> the
> >>>>> issue and potentially "self solve" the problem without calling
> >>>>> support.
> >>>>> Put this into a search engine... "KDFAP0020S"
> >>>>>
> >>>>> Value: Message IDs allow you to easily describe flows.  You
> >>>>> might
> >>>>> be
> >>>>> able to say... "if you saw ID123 then look farther back for
> >>>>> ID124
> >>>>> and if
> >>>>> you see ID124 then the problem is _probably_ XYZ."
> >>>>>
> >>>>> I am probably not doing enough justice to the value of message
> >>>>> IDs
> >>>>> but,
> >>>>> I will say that lots of very large companies use them in
> >>>>> complex
> >>>>> software applications to solve these and other issues.
> >>>>>
> >>>>> Cheers,
> >>>>> Keith
> >>>>>
> >>>>>
> >>>>>
> >>>>>
> >>>>>
> >>>>>
> >>>>>
> >>>>>
> >>>>>
> >>>>>
> >>>>>
> >>>>>
> >>>>> _______________________________________________
> >>>>> vdsm-devel mailing list
> >>>>> vdsm-devel@lists.fedorahosted.org
> >>>>> https://fedorahosted.org/mailman/listinfo/vdsm-devel
> >>> _______________________________________________
> >>> vdsm-devel mailing list
> >>> vdsm-devel@lists.fedorahosted.org
> >>> https://fedorahosted.org/mailman/listinfo/vdsm-devel
> >>>
> >> _______________________________________________
> >> vdsm-devel mailing list
> >> vdsm-devel@lists.fedorahosted.org
> >> https://fedorahosted.org/mailman/listinfo/vdsm-devel
> >
> 
> _______________________________________________
> vdsm-devel mailing list
> vdsm-devel@lists.fedorahosted.org
> https://fedorahosted.org/mailman/listinfo/vdsm-devel
> 
_______________________________________________
vdsm-devel mailing list
vdsm-devel@lists.fedorahosted.org
https://fedorahosted.org/mailman/listinfo/vdsm-devel

Reply via email to