Message IDs do require some discipline on the part of the developer to ensure uniqueness. They also force the developer to think about whether or not the text that is being emitted is actually worthy of [INFO|ERROR|WARN|EXCEPTION|CRITICAL] instead of the the lowly DEBUG.

As to how it is done, I have seen this done a few different ways but the most popular seems to be to assign a range of numbers to a particular component/package. In this manner, a developer can quickly scan that package and choose the next higher number. Some projects, like my last one, were fairly monolithic in design and we just used an up-counter for the whole source tree (a little harder but not impossible with grep and sort).

All solutions generally involve a central document/registry where the IDs are listed and *can* be documented (generally the documentation evolves over time). For VDSM, I would probably recommend a simple up-counter. Take a look at how JBoss is breaking up their product.
http://community.jboss.org/wiki/LoggingIds

As for gettext, message IDs don't really affect gettext. gettext is a vehicle that is used to enable i18n. Message IDs are typically coded as symbols directly in the source while gettext handles the internationalization of all things that are not substitution variables (you don't want to translate them). I've attached the makefile that I wrote for the rhevm-iso-uploader to show the flow for using gettext in a project.

Here is an example of a logging statement that would work with gettext and a message ID:
  // Assume the log format is this...
FORMAT = '%(asctime)s %(lineno)s %(levelname)s(%(messageid)s): %(message)s'
  // An invocation
logger.warning(_("A log message with a substitution string, %s, that doesn't get translated.") % "woweee", extra={ 'messageid' : 'VDSM1000'})
  // The result
2011-10-06 11:13:09,458 14 WARNING(VDSM1000): A log message with a substitution string, woweee, that doesn't get translated.

Things to notice:
1. The substitution variable "woweee" doesn't get translated or picked up by gettext because it isn't surrounded by _(...) which is what you want. 2. The message ID (i.e. VDSM1000) doesn't get translated either for the same reason.

Cheers,
Keith






On 10/06/2011 10:02 AM, Saggi Mizrahi wrote:
On Wed 05 Oct 2011 08:24:13 PM IST, Keith Robertson wrote:
Hi,

I would like to propose enhancing the VDSM source to emit message IDs in
log messages with levels greater than DEBUG (i.e. INFO, ERROR, WARN,
EXCEPTION, and CRITICAL). Messages IDs have been used successfully in
many products and they can aid in debugging, documentation, root cause
analysis, and internationalization.

I have scanned the source and have determined that there are roughly 525
calls to log.[error|warn|info|critical|exception] and I am volunteering
to go through and add IDs to them all but, before I do I want to get
buy-in from the project maintainers. While adding IDs to the messages I
would also enable them for i18n.

I have written a rough plan for adding messages IDs to VDSM below and
I've provided some examples of products/projects that use message IDs.

Examples of Projects that Use Message IDs:
- JBoss 7 is adopting Message IDs for all of its logging. See the
annotations (e.g. @Message(id = 12000, ) in the following piece of source: https://github.com/jbossas/jboss-as/blob/master/process-controller/src/main/java/org/jboss/as/process/ProcessLogger.java

- Almost all IBM products use message IDs and they do a fairly good job
of documenting them. Google has indexed them all and you can simply type
them in to a search and get a description of the problem that is
associated with the ID.
http://publib.boulder.ibm.com/infocenter/tivihelp/v42r1/topic/com.ibm.omegamon.stor.doc/GC23-9705-0484.htm



Example of a Log Statement with a Message ID in Python:
FORMAT = '%(asctime)s %(lineno)s %(levelname)s(%(messageid)s): %(message)s'
logging.basicConfig(format=FORMAT)
logger = logging.getLogger()
logger.warning('NFS problem. Unable to mount.', extra={ 'messageid' :
'VDSM1000'})

// Produces
2011-10-05 13:24:43,180 11 WARNING(VDSM1000): NFS problem. Unable to mount.


Example of a Log Statement with a Message ID in Python with i18n:
logger.warning(_('NFS problem. Unable to mount.'), extra={ 'messageid' :
'VDSM1000'})

Example of an i18n Log Message:
Chinese: 2011年10月5日13:24:43,18011警告(VDSM1000):NFS问题。无法安 装。
English: 2011-10-05 13:24:43,180 11 WARNING(VDSM1000): NFS problem.
Unable to mount.



A Rough Plan for adding Message IDs to VDSM:

Step 0: Decide on a logging format of the ID itself:
- JBoss is using "JBAS#####". I am thinking that VDSM could probably do
something similar "VDSM####".

Step 1: Decide where to list and document all of the IDs.
- It could be as simple as a page on VDSM's fedorahosted wiki.

Step 2: Pin the format of the log messages:
- Currently VDSM allows a user configurable Python log format. We would
need to pin the format so that it includes a substitution variable for
the message ID.
Example: '%(asctime)s %(lineno)s %(levelname)s(%(messageid)s): %(message)s'

Step 4: Plow through the VDSM source and add message IDs.


Cheers,
Keith





_______________________________________________
vdsm-devel mailing list
vdsm-devel@lists.fedorahosted.org
https://fedorahosted.org/mailman/listinfo/vdsm-devel

Seems like a nice idea. But how does it affect developers. How do they choose mIDs? How do you prevent collisions? How are they different from simple gettext?

PYTHON=python
XGETTEXT=xgettext
MSGINIT=msginit
MSGFMT=msgfmt
EXCLUDES=__init__.py
LOCALES=en_US en_CA
TMP_LOCALE_DIR=tmp_locale

.SILENT:  # don't echo commands executed 

# Step 1 of translation
create-pot:
        for p in $(filter-out $(EXCLUDES), $(wildcard *.py)) ; do \
                echo Generating Portable Object Template for $$p with 
$(XGETTEXT); \
                $(XGETTEXT) --language=Python --keyword=_ --output=`basename 
$$p .py`.pot $$p; \
        done
        
# Step 2 of translation.        
create-po: 
        for p in $(filter-out $(EXCLUDES), $(wildcard *.pot)) ; do \
                for l in $(LOCALES) ; do \
                        echo "Generating Portable Object(s) for locale $$l from 
$$p with $(MSGINIT)"; \
                        $(MSGINIT) --no-translator --output-file=`basename $$p 
.pot`-$$l.po --input=$$p --locale=$$l; \
                done \
        done
        echo "Please send the .po files to a translation center for 
translation, place the resulting files \
        back in this directory, and then run the makemo target."

# Step 3 of translation 
create-mo: 
        for l in $(LOCALES) ; do \
                echo "Generating .mo for $$l with $(MSGFMT) and placing results 
in $(TMP_LOCALE_DIR)/$$l/LC_MESSAGES"; \
                mkdir -p $(TMP_LOCALE_DIR)/$$l/LC_MESSAGES; \
                $(MSGFMT) 
--output-file=$(TMP_LOCALE_DIR)/$$l/LC_MESSAGES/`basename *-$$l.po -$$l.po`.mo 
*-$$l.po; \
        done
        
test:
        # To run this simple test...
        # 1. Run create-pot
        # 2. Run create-po
        # 3. Edit the "version" msgstr in rhevm-iso-uploader-en_CA.po with some 
sort of fake message
        # 4. Run this target 
        LC_MESSAGES=$PWD/$(TMP_LOCALE_DIR)/en_CA/LC_MESSAGES; LANG=en_CA; 
python rhevm-iso-uploader.py --version

man:
        gzip -c rhevm-iso-uploader.8  > rhevm-iso-uploader.8.gz 
        
all: man
        $(PYTHON) -m compileall -x $(EXCLUDES) .
        $(PYTHON) -OO -m compileall -x $(EXCLUDES) .    
        
clean:
        rm -rf *.pyc *.pyo *~ *.po *.pot $(TMP_LOCALE_DIR)
        


_______________________________________________
vdsm-devel mailing list
vdsm-devel@lists.fedorahosted.org
https://fedorahosted.org/mailman/listinfo/vdsm-devel

Reply via email to