On 10/06/2011 03:11 PM, Ayal Baron wrote:
>>>>>>>> I would like to propose enhancing the VDSM source to emit
>>>>>>>> message
>>>>>>>> IDs in
>>>>>>>> log messages with levels greater than DEBUG (i.e. INFO, ERROR,
>>>>>>>> WARN,
>>>>>>>> EXCEPTION, and CRITICAL). Messages IDs have been used
>>>>>>>> successfully in
>>>>>>>> many products and they can aid in debugging, documentation,
>>>>>>>> root
>>>>>>>> cause
>>>>>>>> analysis, and internationalization.
>>>>>>>>
>>>>>>>> I have scanned the source and have determined that there are
>>>>>>>> roughly 525
>>>>>>>> calls to log.[error|warn|info|critical|exception] and I am
>>>>>>>> volunteering
>>>>>>>> to go through and add IDs to them all but, before I do I want
>>>>>>>> to get
>>>>>>>> buy-in from the project maintainers. While adding IDs to the
> wrt i18n, I'm not entirely convinced.
Yeah, i18n is one reason to do it but it is only relevant if you take 
the time to create lang-packs for different nationalities (which can be 
expensive). However, it is a cakewalk to code for in Python _("string 
here").  So you can code for it ... and if someone ever wanted to take 
the time create a lang-pack you'd be set.
>   Every line we log has the module name and line (never localized) which 
> logged it so effectively we already have this info.
> However, the ID would remain constant where lines will change place between 
> versions so it makes it easier.
> I also agree about the lookup and being able to write simpler parsers.
>
> So I'm in.
>
Cool.
>>>>>>>> messages I
>>>>>>>> would also enable them for i18n.
>>>>>>>>
>>>>>>>> I have written a rough plan for adding messages IDs to VDSM
>>>>>>>> below
>>>>>>>> and
>>>>>>>> I've provided some examples of products/projects that use
>>>>>>>> message
>>>>>>>> IDs.
>>>>>>>>
>>>>>>>> Examples of Projects that Use Message IDs:
>>>>>>>> - JBoss 7 is adopting Message IDs for all of its logging. See
>>>>>>>> the
>>>>>>>> annotations (e.g. @Message(id = 12000, ) in the following
>>>>>>>> piece of
>>>>>>>> source:
>>>>>>>> https://github.com/jbossas/jboss-as/blob/master/process-controller/src/main/java/org/jboss/as/process/ProcessLogger.java
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> - Almost all IBM products use message IDs and they do a fairly
>>>>>>>> good
>>>>>>>> job
>>>>>>>> of documenting them. Google has indexed them all and you can
>>>>>>>> simply
>>>>>>>> type
>>>>>>>> them in to a search and get a description of the problem that
>>>>>>>> is
>>>>>>>> associated with the ID.
>>>>>>>> http://publib.boulder.ibm.com/infocenter/tivihelp/v42r1/topic/com.ibm.omegamon.stor.doc/GC23-9705-0484.htm
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> Example of a Log Statement with a Message ID in Python:
>>>>>>>> FORMAT = '%(asctime)s %(lineno)s %(levelname)s(%(messageid)s):
>>>>>>>> %(message)s'
>>>>>>>> logging.basicConfig(format=FORMAT)
>>>>>>>> logger = logging.getLogger()
>>>>>>>> logger.warning('NFS problem. Unable to mount.', extra={
>>>>>>>> 'messageid' :
>>>>>>>> 'VDSM1000'})
>>>>>>>>
>>>>>>>> // Produces
>>>>>>>> 2011-10-05 13:24:43,180 11 WARNING(VDSM1000): NFS problem.
>>>>>>>> Unable
>>>>>>>> to mount.
>>>>>>>>
>>>>>>>>
>>>>>>>> Example of a Log Statement with a Message ID in Python with
>>>>>>>> i18n:
>>>>>>>> logger.warning(_('NFS problem. Unable to mount.'), extra={
>>>>>>>> 'messageid' :
>>>>>>>> 'VDSM1000'})
>>>>>>>>
>>>>>>>> Example of an i18n Log Message:
>>>>>>>> Chinese: 2011年10月5日13:24:43,18011警告(VDSM1000):NFS问题。无法
>>>>>>>> 安 装。
>>>>>>>> English: 2011-10-05 13:24:43,180 11 WARNING(VDSM1000): NFS
>>>>>>>> problem.
>>>>>>>> Unable to mount.
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> A Rough Plan for adding Message IDs to VDSM:
>>>>>>>>
>>>>>>>> Step 0: Decide on a logging format of the ID itself:
>>>>>>>> - JBoss is using "JBAS#####". I am thinking that VDSM could
>>>>>>>> probably do
>>>>>>>> something similar "VDSM####".
>>>>>>>>
>>>>>>>> Step 1: Decide where to list and document all of the IDs.
>>>>>>>> - It could be as simple as a page on VDSM's fedorahosted wiki.
>>>>>>>>
>>>>>>>> Step 2: Pin the format of the log messages:
>>>>>>>> - Currently VDSM allows a user configurable Python log format.
>>>>>>>> We
>>>>>>>> would
>>>>>>>> need to pin the format so that it includes a substitution
>>>>>>>> variable for
>>>>>>>> the message ID.
>>>>>>>> Example: '%(asctime)s %(lineno)s %(levelname)s(%(messageid)s):
>>>>>>>> %(message)s'
>>>>>>>>
>>>>>>>> Step 4: Plow through the VDSM source and add message IDs.
>>>>>>>>
>>>>>>>>
>>>>>>>> Cheers,
>>>>>>>> Keith
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> _______________________________________________
>>>>>>>> vdsm-devel mailing list
>>>>>>>> vdsm-devel@lists.fedorahosted.org
>>>>>>>> https://fedorahosted.org/mailman/listinfo/vdsm-devel
>>>>>>> Seems like a nice idea. But how does it affect developers. How
>>>>>>> do
>>>>>>> they choose mIDs? How do you prevent collisions? How are they
>>>>>>> different from simple gettext?
>>>>> As I never used this myself so I can't really imagine the all
>>>>> benefits
>>>>> of using such a system, at least enough to warrant all the
>>>>> tedious
>>>>> work. If you could just list out some of the use cases it'll be
>>>>> most
>>>>> helpful.
>>>>>
>>>>> For instance I assume grepping for an ID is easier then grepping
>>>>> for a
>>>>> message especially in multiple versions where the message might
>>>>> have
>>>>> changed.
>>>> Sure.  As far as the tedious part, yes, adding message IDs to an
>>>> existing source tree is a bit of a PITA but the 525 invocations in
>>>> VDSM
>>>> isn't all that bad.  I've seen much worse.  Plus, once it's done
>>>> the
>>>> effort is much smaller (new lines of code only).
>>>>
>>>> As for the value (I've tried to itemize a few common reasons for
>>>> doing it):
>>>>
>>>> Value: Message IDs allow you to use gettext globalize your
>>>> messages. If
>>>> you just used gettext and pulled the strings out of your code for
>>>> globalization then you might end up with something like this in
>>>> vdsm.log...
>>>>       错误:问题安装驱动器
>>>> the English translation (according to google) is:
>>>>       ERROR: Problem mounting drive
>>>> Unless you can read Chinese, then the message above is completely
>>>> unhelpful to a support engineer.  The converse is true also.
>>>>   Consider a
>>>> non-english speaking person trying to decipher "ERROR: Problem
>>>> mounting
>>>> drive".  With a message ID; however, you get this...
>>>>       错误(VDSM1000):安装驱动问题
>>> Is it possible to just hash the main string before
>>> internationalization?
>>>
>>> So basically:
>>>
>>> fprintf(f, "[%s] ERROR(VDSM%04x) %s", date_str, crc32(message),
>>> message);
>> Yes, a hash of the string is possible.  However, would make it hard
>> for
>> someone to document the ID because they would have to hash the string
>> to
>> get the ID.
>>
>> One of the features of message ID/codes is that they are
>> documentable.
>>> Or the equivalent Python.
>>>
>>>> Value: Message IDs allow other applications to perform actions
>>>> based on
>>>> what is in your logs.  On the mainframe, this is extremely common.
>>>> You'll have one program watching the logs for a particular ID(s).
>>>>   When
>>>> the program sees the ID it will look in a dictionary and decide
>>>> what (if
>>>> anything) needs to be done.  On linux, logwatch, is frequently
>>>> used for
>>>> this purpose and message IDs make this process much simpler
>>>> because the
>>>> program doesn't have to write complicated REGEXs to look for
>>>> arbitrary
>>>> strings it can just scan for an ID.
>>>>
>>>> Value: Message IDs generally have the same meaning across releases
>>>> of
>>>> the product.  This means that the developer can change the text of
>>>> the
>>>> log message without worrying about breaking someone's automation.
>>>> Example:
>>>>       Version 1 of the product- ERROR(VDSM1000): Problem mounting
>>>>       drive.
>>>>       Version 2 of the product- ERROR(VDSM1000): unable to contact
>>>> 127.0.0.1 for mounting. Try blah blah.
>>> Does this mean that logging messages are an ABI that needs to be
>>> maintained over time?
>> The degree to which message IDs need to be maintained is generally a
>> function of how dynamic the code is and how much you want to document
>> what the ID actually means.  If, on the WIKI wherein you define
>> messages, you elect to say [1] then this "ABI" doesn't need any
>> maintenance whatsoever.
>>
>> [1] VDSM1000 == "Call support or post to the forum to figure out what
>> the heck the program just did"
>>
>>> That's probably very difficult to do right.
>> Lots of other software programs have been successful at it [1], [2].
>>   I
>> would prefer to think of it in terms of how easy we want to make it
>> for
>> people to self diagnose.  Also, without message IDs supporting source
>> that is i18n is nearly impossible unless everyone is multilingual.
>>
>> [1]
>> http://www.cisco.com/en/US/products/sw/netmgtsw/ps1982/products_command_reference_chapter09186a0080087439.html
>> [2] http://mail.google.com/support/bin/answer.py?answer=12336
>>
>>
>>> Regards,
>>>
>>> Anthony Liguori
>>>
>>>> Value: If you do it right search engines will index your message
>>>> IDs and
>>>> take users to a page where they can hopefully get more detail on
>>>> the
>>>> issue and potentially "self solve" the problem without calling
>>>> support.
>>>> Put this into a search engine... "KDFAP0020S"
>>>>
>>>> Value: Message IDs allow you to easily describe flows.  You might
>>>> be
>>>> able to say... "if you saw ID123 then look farther back for ID124
>>>> and if
>>>> you see ID124 then the problem is _probably_ XYZ."
>>>>
>>>> I am probably not doing enough justice to the value of message IDs
>>>> but,
>>>> I will say that lots of very large companies use them in complex
>>>> software applications to solve these and other issues.
>>>>
>>>> Cheers,
>>>> Keith
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>> _______________________________________________
>>>> vdsm-devel mailing list
>>>> vdsm-devel@lists.fedorahosted.org
>>>> https://fedorahosted.org/mailman/listinfo/vdsm-devel
>> _______________________________________________
>> vdsm-devel mailing list
>> vdsm-devel@lists.fedorahosted.org
>> https://fedorahosted.org/mailman/listinfo/vdsm-devel
>>
> _______________________________________________
> vdsm-devel mailing list
> vdsm-devel@lists.fedorahosted.org
> https://fedorahosted.org/mailman/listinfo/vdsm-devel

_______________________________________________
vdsm-devel mailing list
vdsm-devel@lists.fedorahosted.org
https://fedorahosted.org/mailman/listinfo/vdsm-devel

Reply via email to