[issue20918] Logging of logging exceptions can fail in 3.4 if args are unprintable
the mulhern added the comment: Thanks for the fix. When you say having repr raise is a pretty unusual occurrence you probably mean having repr raise should be a pretty unusual occurrence. I think its more usual than you realize and the regression in 3.4 will have consequences. -- status: closed - open ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue20918 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue20918] Logging of logging exceptions can fail in 3.4 if args are unprintable
R. David Murray added the comment: Did you reopen the issue accidentally? The bug has been fixed. -- status: open - closed ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue20918 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue20918] Logging of logging exceptions can fail in 3.4 if args are unprintable
R. David Murray added the comment: Yes on 3.3 fixes, but you are right that it doesn't need fixed there. This appears to be a 3.4 regression. I used exactly the test you suggest to reproduce it on 3.4...there there is a chained traceback and Done does not get printed. So, the original report is indeed invalid, but we've uncovered a regression. Fortunately it is a low impact regression. I'm guessing this arises from the improvements you made to the fallback logging of these errors. -- keywords: +3.4regression resolution: invalid - status: closed - open title: LogRecord.__init__ should handle exception if % operation fails - Logging of logging exceptions can fail in 3.4 if args are unprintable versions: -Python 2.7, Python 3.3 ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue20918 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue20918] Logging of logging exceptions can fail in 3.4 if args are unprintable
the mulhern added the comment: Yes, I really misinterpreted what I saw. So glad it's a bug anyway and I'm not just wasting your time ;) -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue20918 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue20918] Logging of logging exceptions can fail in 3.4 if args are unprintable
Roundup Robot added the comment: New changeset 73c2a70e4b35 by Vinay Sajip in branch 'default': Closes #20918: Added handling for exceptions during fallback output of logging exceptions. http://hg.python.org/cpython/rev/73c2a70e4b35 -- nosy: +python-dev resolution: - fixed stage: needs patch - committed/rejected status: open - closed ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue20918 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue20918] Logging of logging exceptions can fail in 3.4 if args are unprintable
Vinay Sajip added the comment: This fix doesn't seem to meet the criteria for cherry-picking for 3.4 - Only important interface changes, new features, or bugfixes should be checked in now - so I haven't created a separate cherry-pick issue for it. But I will add larry to nosy in case he thinks I should do so for 73c2a70e4b35. -- nosy: +larry ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue20918 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue20918] Logging of logging exceptions can fail in 3.4 if args are unprintable
R. David Murray added the comment: At this point only things that would make it a brown bag release (broken out of the box) would get cherry picked. I agree that this doesn't qualify...having repr raise is a pretty unusual occurrence. -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue20918 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
Re: Repeated output when logging exceptions
Vinay Sajip wrote: I'm not sure why you need all the code you've posted. The logging package allows you to add tracebacks to your logs by using the exception() method, which logs an ERROR with a traceback and is specifically intended for use from within exception handlers. You can also use the exc_info=True parameter with any logging method... Chris -- Simplistix - Content Management, Batch Processing Python Consulting - http://www.simplistix.co.uk -- http://mail.python.org/mailman/listinfo/python-list
Re: Repeated output when logging exceptions
John Gordon wrote: If I didn't do all that in a class, where would I do it? I find the configureLoggers method of ZConfig most convenient for this: http://pypi.python.org/pypi/ZConfig cheers, Chris -- Simplistix - Content Management, Batch Processing Python Consulting - http://www.simplistix.co.uk -- http://mail.python.org/mailman/listinfo/python-list
Re: Repeated output when logging exceptions
On Sep 28, 9:38 pm, John Gordon gor...@panix.com wrote:
If I didn't do all that in a class, where would I do it?
You could, for example, use the basicConfig() function to do it all
for you.
import logging
logging.basicConfig(filename='/path/to/my/log',level=logging.DEBUG)
logging.debug('This message should go to the log file')
Here's the documentation link for simple examples:
http://docs.python.org/library/logging.html#simple-examples
Here's the link for basicConfig:
http://docs.python.org/library/logging.html#logging.basicConfig
Regards,
Vinay Sajip
--
http://mail.python.org/mailman/listinfo/python-list
Re: Repeated output when logging exceptions
In 6bce12c3-f2d9-450c-89ee-afa4f21d5...@h30g2000vbr.googlegroups.com Vinay Sajip vinay_sa...@yahoo.co.uk writes: The logging package allows you to add tracebacks to your logs by using the exception() method, which logs an ERROR with a traceback and is specifically intended for use from within exception handlers. All that stuff with temporary files seems completely unnecessary, even with Python 2.3. I didn't know about the exception() method. Thanks! In general, avoid adding handlers more than once - which you are almost guaranteed to not avoid if you do this kind of processing in a constructor. If you write your applications without using the exceptionLogger class (not really needed, since logging exceptions with tracebacks is part and parcel of the logging package's functionality since its introduction with Python 2.3), what functionality do you lose? I'm trying to encapsulate all the setup work that must be done before any actual logging occurs: setting the debugging level, choosing the output filename, declaring a format string, adding the handler, etc. If I didn't do all that in a class, where would I do it? -- John Gordon A is for Amy, who fell down the stairs gor...@panix.com B is for Basil, assaulted by bears -- Edward Gorey, The Gashlycrumb Tinies -- http://mail.python.org/mailman/listinfo/python-list
Re: Repeated output when logging exceptions
On Mon, Sep 28, 2009 at 4:38 PM, John Gordon gor...@panix.com wrote: In 6bce12c3-f2d9-450c-89ee-afa4f21d5...@h30g2000vbr.googlegroups.com Vinay Sajip vinay_sa...@yahoo.co.uk writes: The logging package allows you to add tracebacks to your logs by using the exception() method, which logs an ERROR with a traceback and is specifically intended for use from within exception handlers. All that stuff with temporary files seems completely unnecessary, even with Python 2.3. I didn't know about the exception() method. Thanks! In general, avoid adding handlers more than once - which you are almost guaranteed to not avoid if you do this kind of processing in a constructor. If you write your applications without using the exceptionLogger class (not really needed, since logging exceptions with tracebacks is part and parcel of the logging package's functionality since its introduction with Python 2.3), what functionality do you lose? I'm trying to encapsulate all the setup work that must be done before any actual logging occurs: setting the debugging level, choosing the output filename, declaring a format string, adding the handler, etc. If I didn't do all that in a class, where would I do it? Put it in a module. :] -- John Gordon A is for Amy, who fell down the stairs gor...@panix.com B is for Basil, assaulted by bears -- Edward Gorey, The Gashlycrumb Tinies -- http://mail.python.org/mailman/listinfo/python-list
Re: Repeated output when logging exceptions
On Sep 24, 8:43 pm, John Gordon gor...@panix.com wrote: Why is this happening? I suspect it's because I'm declaring two instances of the exceptionLogger class, which ends up calling logger.addHandler() twice. Is that right? Yes, that's why you get duplicated lines in the log. What would be a better way to handle this? I'm not sure why you need all the code you've posted. The logging package allows you to add tracebacks to your logs by using the exception() method, which logs an ERROR with a traceback and is specifically intended for use from within exception handlers. All that stuff with temporary files seems completely unnecessary, even with Python 2.3. In general, avoid adding handlers more than once - which you are almost guaranteed to not avoid if you do this kind of processing in a constructor. If you write your applications without using the exceptionLogger class (not really needed, since logging exceptions with tracebacks is part and parcel of the logging package's functionality since its introduction with Python 2.3), what functionality do you lose? Regards, Vinay Sajip -- http://mail.python.org/mailman/listinfo/python-list
Repeated output when logging exceptions
I wrote some code to handle and log exceptions in my application. It works well, but it produces double output for each exception. How can I fix this? Here's the pared-down code: - main.py import exceptionLogger import doStuff exlog = exceptionLogger.exceptionLogger() stuff = doStuff.doStuff() try: stuff.doit() except Exception,msg: exlog.logException() - exceptionLogger.py import traceback import logging import os class exceptionLogger: loggingLevel = logger = None def __init__(self): self.logger = logging.getLogger(foo) self.loggingLevel = DEBUG if self.loggingLevel == DEBUG: self.logger.setLevel(logging.DEBUG) handler = logging.FileHandler(error.txt) format = %(asctime)s %(levelname)-8s %(message)s formatter = logging.Formatter(format) handler.setFormatter(formatter) self.logger.addHandler(handler) def logMsg(self,message): try: if self.loggingLevel == DEBUG: self.logger.debug(message) except: self.stopLogging() def logException(self): Write an exception traceback to the logfile. # save the stack trace to a file and then copy the contents of that # file to the log. (traceback.format_exc would allow us to avoid # using a temporary file, but it is not available in python 2.3) TMPFILE = os.tmpfile() traceback.print_exc(None, TMPFILE) TMPFILE.flush() self.logMsg( ** EXCEPTION BEGINS **) self.logMsgFromFile(TMPFILE) self.logMsg( ** EXCEPTION ENDS **) TMPFILE.close() def logMsgFromFile(self,file): Read the contents of a file into a string and log the string. file.seek(0) msg = for line in file: msg = msg + line self.logMsg(msg) def stopLogging(self): logging.shutdown() - doStuff.py import exceptionLogger class doStuff: def doit(self): exlog = exceptionLogger.exceptionLogger() try: raise ValueError, hi there except Exception: exlog.logException() And here's the contents of error.txt: - errror.txt 2009-09-24 14:22:23,288 DEBUG ** EXCEPTION BEGINS ** 2009-09-24 14:22:23,288 DEBUG ** EXCEPTION BEGINS ** 2009-09-24 14:22:23,288 DEBUGTraceback (most recent call last): File /home/gordonj/exception/doStuff.py, line 8, in doit raise ValueError, hi there ValueError: hi there 2009-09-24 14:22:23,288 DEBUGTraceback (most recent call last): File /home/gordonj/exception/doStuff.py, line 8, in doit raise ValueError, hi there ValueError: hi there 2009-09-24 14:22:23,288 DEBUG ** EXCEPTION ENDS ** 2009-09-24 14:22:23,288 DEBUG ** EXCEPTION ENDS ** As you can see, all of the output is doubled. Header lines, footer lines, and body text. Why is this happening? I suspect it's because I'm declaring two instances of the exceptionLogger class, which ends up calling logger.addHandler() twice. Is that right? What would be a better way to handle this? Thanks. -- John Gordon A is for Amy, who fell down the stairs gor...@panix.com B is for Basil, assaulted by bears -- Edward Gorey, The Gashlycrumb Tinies -- http://mail.python.org/mailman/listinfo/python-list
Re: Logging exceptions to a file
In message 597627b8- d30b-4b74-9202-9cd46fb1d...@s28g2000vbp.googlegroups.com, Pierre GM wrote: ... I'd like to record (possibly unhandled) exceptions in the logfile. python myscript.py 2error.log -- http://mail.python.org/mailman/listinfo/python-list
Re: Logging exceptions to a file
Pierre GM wrote:
All,
I need to log messages to both the console and a given file. I use the
following code (on Python 2.5)
import logging
#
logging.basicConfig(level=logging.DEBUG,)
logfile = logging.FileHandler('log.log')
logfile.setLevel(level=logging.INFO)
logging.getLogger('').addHandler(logfile)
#
mylogger = logging.getLogger('mylogger')
#
mylogger.info(an info message)
So far so good, but I'd like to record (possibly unhandled) exceptions
in the logfile.
* Do I need to explicitly trap every single exception ?
* In that case, won't I get 2 log messages on the console (as
illustrated in the code below:
try:
1/0
except ZeroDivisionError:
mylogger.exception(:()
raise
Any comments/idea welcomed
Cheers.
Although it is usually not recommended to use a catch-all except, this
is the case where it might be useful. JUST DON'T FORGET TO RE-RAISE THE
EXCEPTION.
if __name__ == '__main__':
try:
main():
except Exception, e:
# log('Unhandled Exception', e)
raise
--
http://mail.python.org/mailman/listinfo/python-list
Re: Logging exceptions to a file
On May 7, 5:32 am, Lie Ryan lie.1...@gmail.com wrote:
Pierre GM wrote:
All,
I need to log messages to both the console and a given file. I use the
following code (on Python 2.5)
import logging
#
logging.basicConfig(level=logging.DEBUG,)
logfile = logging.FileHandler('log.log')
logfile.setLevel(level=logging.INFO)
logging.getLogger('').addHandler(logfile)
#
mylogger = logging.getLogger('mylogger')
#
mylogger.info(an info message)
So far so good, but I'd like to record (possibly unhandled) exceptions
in the logfile.
* Do I need to explicitly trap every single exception ?
* In that case, won't I get 2 log messages on the console (as
illustrated in the code below:
try:
1/0
except ZeroDivisionError:
mylogger.exception(:()
raise
Any comments/idea welcomed
Cheers.
Although it is usually not recommended to use a catch-all except, this
is the case where it might be useful. JUST DON'T FORGET TO RE-RAISE THE
EXCEPTION.
if __name__ == '__main__':
try:
main():
except Exception, e:
# log('Unhandled Exception', e)
raise
OK for a simple script, but the (unhandled) exceptions need to be
caught at the module level. Any idea?
--
http://mail.python.org/mailman/listinfo/python-list
Re: Logging exceptions to a file
So far so good, but I'd like to record (possibly unhandled) exceptions in the logfile. * Do I need to explicitly trap every single exception ? * In that case, won't I get 2 log messages on the console (as illustrated in the code below: Check out sys.excepthook, something vaguely like: import sys def handle_exceptions(exctype, value, traceback): logging.exception(An unhandled exception was detected.) sys.excepthook = handle_exceptions -- http://mail.python.org/mailman/listinfo/python-list
Re: Logging exceptions to a file
On May 7, 1:19 pm, Pierre GM pierregmc...@gmail.com wrote:
On May 7, 5:32 am, Lie Ryan lie.1...@gmail.com wrote:
Pierre GM wrote:
All,
I need to log messages to both the console and a given file. I use the
following code (on Python 2.5)
import logging
#
logging.basicConfig(level=logging.DEBUG,)
logfile = logging.FileHandler('log.log')
logfile.setLevel(level=logging.INFO)
logging.getLogger('').addHandler(logfile)
#
mylogger = logging.getLogger('mylogger')
#
mylogger.info(an info message)
So far so good, but I'd like to record (possibly unhandled) exceptions
in the logfile.
* Do I need to explicitly trap every single exception ?
* In that case, won't I get 2 log messages on the console (as
illustrated in the code below:
try:
1/0
except ZeroDivisionError:
mylogger.exception(:()
raise
Any comments/idea welcomed
Cheers.
Although it is usually not recommended to use a catch-all except, this
is the case where it might be useful. JUST DON'T FORGET TO RE-RAISE THE
EXCEPTION.
if __name__ == '__main__':
try:
main():
except Exception, e:
# log('Unhandled Exception', e)
raise
OK for a simple script, but the (unhandled) exceptions need to be
caught at the module level. Any idea?
Override sys.excepthook. http://docs.python.org/library/sys.html#sys.excepthook
--
http://mail.python.org/mailman/listinfo/python-list
Logging exceptions to a file
All,
I need to log messages to both the console and a given file. I use the
following code (on Python 2.5)
import logging
#
logging.basicConfig(level=logging.DEBUG,)
logfile = logging.FileHandler('log.log')
logfile.setLevel(level=logging.INFO)
logging.getLogger('').addHandler(logfile)
#
mylogger = logging.getLogger('mylogger')
#
mylogger.info(an info message)
So far so good, but I'd like to record (possibly unhandled) exceptions
in the logfile.
* Do I need to explicitly trap every single exception ?
* In that case, won't I get 2 log messages on the console (as
illustrated in the code below:
try:
1/0
except ZeroDivisionError:
mylogger.exception(:()
raise
Any comments/idea welcomed
Cheers.
--
http://mail.python.org/mailman/listinfo/python-list
Re: logging exceptions
On Aug 26, 10:36 am, Alexandru Mosoi [EMAIL PROTECTED] wrote:
why doesn'tloggingthrow any exception when it should? how do I
configureloggingto throw exceptions?
try:
... logging.fatal('asdf %d', '123')
... except:
... print 'this line is never printed'
...
Traceback (most recent call last):
File /Library/Frameworks/Python.framework/Versions/2.5/lib/
python2.5/logging/__init__.py, line 744, in emit
msg = self.format(record)
File /Library/Frameworks/Python.framework/Versions/2.5/lib/
python2.5/logging/__init__.py, line 630, in format
return fmt.format(record)
File /Library/Frameworks/Python.framework/Versions/2.5/lib/
python2.5/logging/__init__.py, line 418, in format
record.message = record.getMessage()
File /Library/Frameworks/Python.framework/Versions/2.5/lib/
python2.5/logging/__init__.py, line 288, in getMessage
msg = msg % self.args
TypeError: int argument required
Was your traceback from the snippet you posted? If it was, then the
exception (a TypeError) *is* being raised from logging. So I don't
understand your question why doesn't logging throw any exception when
it should?, because logging is raising an exception here.
To cause logging to *not* raise exceptions, set
logging.raiseExceptions to 0 (default is 1). The raiseExceptions
variable would normally be set to 0 in a production environment, where
you don't want logging-related exceptions to bring an application
down.
Regards,
Vinay Sajip
--
http://mail.python.org/mailman/listinfo/python-list
Re: logging exceptions
Vinay Sajip napisał(a):
On Aug 26, 10:36 am, Alexandru Mosoi [EMAIL PROTECTED] wrote:
why doesn'tloggingthrow any exception when it should? how do I
configureloggingto throw exceptions?
try:
... logging.fatal('asdf %d', '123')
... except:
... print 'this line is never printed'
...
Traceback (most recent call last):
File /Library/Frameworks/Python.framework/Versions/2.5/lib/
python2.5/logging/__init__.py, line 744, in emit
msg = self.format(record)
File /Library/Frameworks/Python.framework/Versions/2.5/lib/
python2.5/logging/__init__.py, line 630, in format
return fmt.format(record)
File /Library/Frameworks/Python.framework/Versions/2.5/lib/
python2.5/logging/__init__.py, line 418, in format
record.message = record.getMessage()
File /Library/Frameworks/Python.framework/Versions/2.5/lib/
python2.5/logging/__init__.py, line 288, in getMessage
msg = msg % self.args
TypeError: int argument required
Was your traceback from the snippet you posted? If it was, then the
exception (a TypeError) *is* being raised from logging. So I don't
understand your question why doesn't logging throw any exception when
it should?, because logging is raising an exception here.
No, it isn't.
This traceback is *printed* in `Handler.handleError` method:
code from logging
__version__ = 0.5.0.2
[...]
def handleError(self, record):
Handle errors which occur during an emit() call.
This method should be called from handlers when an exception
is
encountered during an emit() call. If raiseExceptions is
false,
exceptions get silently ignored. This is what is mostly wanted
for a logging system - most users will not care about errors
in
the logging system, they are more interested in application
errors.
You could, however, replace this with a custom handler if you
wish.
The record which was being processed is passed in to this
method.
if raiseExceptions:
ei = sys.exc_info()
traceback.print_exception(ei[0], ei[1], ei[2], None,
sys.stderr)
del ei
/code from logging
To cause logging to *not* raise exceptions, set
logging.raiseExceptions to 0 (default is 1). The raiseExceptions
variable would normally be set to 0 in a production environment, where
you don't want logging-related exceptions to bring an application
down.
Well, I think that it will not help. The exception will be printed
not raised. The only solution is to redefine `handleError` method
or maybe I've missed something?
Br,
Rob
--
http://mail.python.org/mailman/listinfo/python-list
Re: logging exceptions
On Aug 27, 10:30 am, Rob Wolfe [EMAIL PROTECTED] wrote:
Vinay Sajip napisa³(a):
On Aug 26, 10:36 am, Alexandru Mosoi [EMAIL PROTECTED] wrote:
why doesn'tloggingthrow any exception when it should? how do I
configureloggingto throw exceptions?
try:
... logging.fatal('asdf %d', '123')
... except:
... print 'this line is never printed'
...
Traceback (most recent call last):
File /Library/Frameworks/Python.framework/Versions/2.5/lib/
python2.5/logging/__init__.py, line 744, in emit
msg = self.format(record)
File /Library/Frameworks/Python.framework/Versions/2.5/lib/
python2.5/logging/__init__.py, line 630, in format
return fmt.format(record)
File /Library/Frameworks/Python.framework/Versions/2.5/lib/
python2.5/logging/__init__.py, line 418, in format
record.message = record.getMessage()
File /Library/Frameworks/Python.framework/Versions/2.5/lib/
python2.5/logging/__init__.py, line 288, in getMessage
msg = msg % self.args
TypeError: int argument required
Was your traceback from the snippet you posted? If it was, then the
exception (a TypeError) *is* being raised fromlogging. So I don't
understand your question why doesn'tloggingthrow any exception when
it should?, becauseloggingis raising an exception here.
No, it isn't.
This traceback is *printed* in `Handler.handleError` method:
You're right - silly me. I wasn't paying attention :-(
variable would normally be set to 0 in a production environment, where
you don't wantlogging-related exceptions to bring an application
down.
Well, I think that it will not help. The exception will be printed
not raised. The only solution is to redefine `handleError` method
or maybe I've missed something?
No, you're right. To do custom handling of exceptions, handleError
needs to be overridden.
Regards,
Vinay
--
http://mail.python.org/mailman/listinfo/python-list
logging exceptions
why doesn't logging throw any exception when it should? how do I
configure logging to throw exceptions?
try:
... logging.fatal('asdf %d', '123')
... except:
... print 'this line is never printed'
...
Traceback (most recent call last):
File /Library/Frameworks/Python.framework/Versions/2.5/lib/
python2.5/logging/__init__.py, line 744, in emit
msg = self.format(record)
File /Library/Frameworks/Python.framework/Versions/2.5/lib/
python2.5/logging/__init__.py, line 630, in format
return fmt.format(record)
File /Library/Frameworks/Python.framework/Versions/2.5/lib/
python2.5/logging/__init__.py, line 418, in format
record.message = record.getMessage()
File /Library/Frameworks/Python.framework/Versions/2.5/lib/
python2.5/logging/__init__.py, line 288, in getMessage
msg = msg % self.args
TypeError: int argument required
--
http://mail.python.org/mailman/listinfo/python-list
Re: logging exceptions
Alexandru Mosoi napisał(a):
why doesn't logging throw any exception when it should? how do I
configure logging to throw exceptions?
try:
... logging.fatal('asdf %d', '123')
... except:
... print 'this line is never printed'
...
[...]
You need to subclass your handler and redefine `handleError` method.
See: http://docs.python.org/lib/node409.html
HTH,
Rob
--
http://mail.python.org/mailman/listinfo/python-list
problem with logging exceptions with non-ASCII __str__ result
Dear all,
I have a problem with logging an exception.
environment:
Python 2.4, Debian testing
${LANGUAGE} not set
${LC_ALL} not set
${LC_CTYPE} not set
${LANG}=de_DE.UTF-8
activating user-default locale with locale.setlocale(locale.LC_ALL,
'') returns: [de_DE.UTF-8]
locale.getdefaultlocale() - default (user) locale: ('de_DE', 'utf-8')
encoding sanity check (also check locale.nl_langinfo(CODESET) below):
sys.getdefaultencoding(): [ascii]
locale.getpreferredencoding(): [UTF-8]
locale.getlocale()[1]: [utf-8]
sys.getfilesystemencoding(): [UTF-8]
_logfile = codecs.open(filename = _logfile_name, mode = 'wb', encoding
= 'utf8', errors = 'replace')
logging.basicConfig (
format = fmt,
datefmt = '%Y-%m-%d %H:%M:%S',
level = logging.DEBUG,
stream = _logfile
)
I am using psycopg2 which in turn uses libpq. When trying to
connect to the database and providing faulty authentication
information:
try:
... try to connect ...
except StandardError, e:
_log.error(ulogin attempt %s/%s failed:, attempt+1,
max_attempts)
print exception type :, type(e)
print exception dir :, dir(e)
print exception args :, e.args
msg = e.args[0]
print msg type:, type(msg)
print msg.decode(utf8):, msg.decode('utf8')
t,v,tb = sys.exc_info()
print sys.exc_info() :, t, v
_log.exception(u'exception detected')
the following output is generated:
exception type : type 'instance'
exception dir : ['__doc__', '__getitem__', '__init__', '__module__',
'__str__', 'args']
exception args : ('FATAL: Passwort-Authentifizierung f\xc3\xbcr
Benutzer \xc2\xbbany-doc\xc2\xab fehlgeschlagen\n',)
msg type: type 'str'
msg.decode(utf8): FATAL: Passwort-Authentifizierung für Benutzer
»any-doc« fehlgeschlagen
sys.exc_info() : psycopg2.OperationalError FATAL:
Passwort-Authentifizierung für Benutzer »any-doc« fehlgeschlagen
Traceback (most recent call last):
File /usr/lib/python2.4/logging/__init__.py, line 739, in emit
self.stream.write(fs % msg.encode(UTF-8))
UnicodeDecodeError: 'ascii' codec can't decode byte 0xc3 in position
191: ordinal not in range(128)
Now, the string FATAL: Passwort-Auth... comes from libpq
via psycopg2. It is translated to German via gettext within
libpq (at the C level). As we can see it is of type string.
I know from the environment that it is likely encoded in
utf8 manually applying which (see the decode call) succeeds.
On _log.exception() the logging module wants to output the
message as encoded as utf8 (that's what the log file is set
up as). So it'll look at the string, decide it is of type
str and decode with the *Python default encoding* to get
to type unicode. Following which it'll re-encode with utf8
to get back to type str ready for outputting to the log
file.
However, since the Python default encoding is ascii that
conversion fails.
Changing the Python default encoding isn't really an option
as it is advocated against and would have to be made to work
reliably on other users machines.
One could, of course, write code to specifically check for
this condition and manually pre-convert the message string
to unicode but that seems not as things should be.
How can I cleanly handle this situation ?
Should the logging module internally use an encoding gotten
from the locale module rather than the default string encoding ?
Karsten
--
GPG key ID E4071346 @ wwwkeys.pgp.net
E167 67FD A291 2BEA 73BD 4537 78B9 A9F9 E407 1346
--
http://mail.python.org/mailman/listinfo/python-list
Re: problem with logging exceptions with non-ASCII __str__ result
On Jan 14, 5:46 pm, Karsten Hilbert [EMAIL PROTECTED] wrote:
Dear all,
I have a problem withloggingan exception.
environment:
Python 2.4, Debian testing
${LANGUAGE} not set
${LC_ALL} not set
${LC_CTYPE} not set
${LANG}=de_DE.UTF-8
activating user-default locale with locale.setlocale(locale.LC_ALL,
'') returns: [de_DE.UTF-8]
locale.getdefaultlocale() - default (user) locale: ('de_DE', 'utf-8')
encoding sanity check (also check locale.nl_langinfo(CODESET)
below):
sys.getdefaultencoding(): [ascii]
locale.getpreferredencoding(): [UTF-8]
locale.getlocale()[1]: [utf-8]
sys.getfilesystemencoding(): [UTF-8]
_logfile = codecs.open(filename = _logfile_name, mode = 'wb',
encoding = 'utf8', errors = 'replace')
logging.basicConfig (
format = fmt,
datefmt = '%Y-%m-%d %H:%M:%S',
level =logging.DEBUG,
stream = _logfile
)
I am using psycopg2 which in turn uses libpq. When trying to
connect to the database and providing faulty authentication
information:
try:
... try to connect ...
except StandardError, e:
_log.error(ulogin attempt %s/%s failed:, attempt+1,
max_attempts)
print exception type :, type(e)
print exception dir :, dir(e)
print exception args :, e.args
msg = e.args[0]
print msg type:, type(msg)
print msg.decode(utf8):, msg.decode('utf8')
t,v,tb = sys.exc_info()
print sys.exc_info() :, t, v
_log.exception(u'exception detected')
the following output is generated:
exception type : type 'instance'
exception dir : ['__doc__', '__getitem__', '__init__',
'__module__', '__str__', 'args']
exception args : ('FATAL: Passwort-Authentifizierung f\xc3\xbcr
Benutzer \xc2\xbbany-doc\xc2\xab fehlgeschlagen\n',)
msg type: type 'str'
msg.decode(utf8): FATAL: Passwort-Authentifizierung für Benutzer
»any-doc« fehlgeschlagen
sys.exc_info() : psycopg2.OperationalError FATAL:
Passwort-Authentifizierung für Benutzer »any-doc« fehlgeschlagen
Traceback (most recent call last):
File /usr/lib/python2.4/logging/__init__.py, line 739, in emit
self.stream.write(fs % msg.encode(UTF-8))
UnicodeDecodeError: 'ascii' codec can't decode byte 0xc3 in position
191: ordinal not in range(128)
Now, the string FATAL: Passwort-Auth... comes from libpq
via psycopg2. It is translated to German via gettext within
libpq (at the C level). As we can see it is of type string.
I know from the environment that it is likely encoded in
utf8 manually applying which (see the decode call) succeeds.
On _log.exception() theloggingmodule wants to output the
message as encoded as utf8 (that's what the log file is set
up as). So it'll look at the string, decide it is of type
str and decode with the *Python default encoding* to get
to type unicode. Following which it'll re-encode with utf8
to get back to type str ready for outputting to the log
file.
However, since the Python default encoding is ascii that
conversion fails.
Changing the Python default encoding isn't really an option
as it is advocated against and would have to be made to work
reliably on other users machines.
One could, of course, write code to specifically check for
this condition and manually pre-convert the message string
to unicode but that seems not as things should be.
How can I cleanly handle this situation ?
Should theloggingmodule internally use an encoding gotten
from the locale module rather than the default string encoding ?
Karsten
--
GPG key ID E4071346 @ wwwkeys.pgp.net
E167 67FD A291 2BEA 73BD 4537 78B9 A9F9 E407 1346
Please reduce to a minimal program which demonstrates the issue and
log an issue on bugs.python.org.
Best regards,
Vinay Sajip
--
http://mail.python.org/mailman/listinfo/python-list
