[issue37111] Logging - Inconsistent behaviour when handling unicode

Change by Vinay Sajip : -- resolution: -> fixed stage: patch review -> resolved status: open -> closed versions: +Python 3.9 -Python 3.7, Python 3.8 ___ Python tracker ___

[issue37111] Logging - Inconsistent behaviour when handling unicode

Vinay Sajip added the comment: New changeset ca7b504a4d4c3a5fde1ee4607b9501c2bab6e743 by Vinay Sajip in branch 'master': bpo-37111: Add 'encoding' and 'errors' parameters to logging.basicCon… (GH-14008) https://github.com/python/cpython/commit/ca7b504a4d4c3a5fde1ee4607b9501c2bab6e743

[issue37111] Logging - Inconsistent behaviour when handling unicode

Steve Dower added the comment: I think Vinay gets to make the call, but I also agree that some sort of error handling by default makes sense for logging. Especially now that it's (about to be) easier to override the default. FWIW, I'd leave the default arguments as None and just coerce it

[issue37111] Logging - Inconsistent behaviour when handling unicode

Inada Naoki added the comment: > What logging does when UnicodeEncodeError is happened? Hm, it seems logging does best job, show enough information to stderr. But sometime, Python is embedded in web servers and there is no stderr. pythonw.exe may not have stderr too I still feel

[issue37111] Logging - Inconsistent behaviour when handling unicode

Inada Naoki added the comment: > I've left it as is using the same rationale as I guess open() has at the > moment - "Errors should never pass silently. Unless explicitly silenced". But open() is general purpose, and logging is for logging. Note that stderr uses 'backslashreplace', even

[issue37111] Logging - Inconsistent behaviour when handling unicode

Vinay Sajip added the comment: > On the other hand, couldn't we use different default error handler? "replace" or "backslashescape" seems better default error handler for the logging. I've left it as is using the same rationale as I guess open() has at the moment - "Errors should never pass

[issue37111] Logging - Inconsistent behaviour when handling unicode

Inada Naoki added the comment: Hmm,, about encoding, I agree that default encoding of open() should be used. If we change it, encoding of log files are changed unexpectedly after upgrading Python. On the other hand, couldn't we use different default error handler? "replace" or

[issue37111] Logging - Inconsistent behaviour when handling unicode

Change by Vinay Sajip : -- pull_requests: +13871 pull_request: https://github.com/python/cpython/pull/14008 ___ Python tracker ___

[issue37111] Logging - Inconsistent behaviour when handling unicode

Change by Vinay Sajip : -- keywords: +patch pull_requests: +13798 stage: test needed -> patch review pull_request: https://github.com/python/cpython/pull/13932 ___ Python tracker

[issue37111] Logging - Inconsistent behaviour when handling unicode

Steve Dower added the comment: I agree with Paul. Consistency with open() is the most important aspect here, and when we investigated changing that to UTF-8 it was decided to be too disruptive. That said, I'm not opposed to adding encoding/errors to basicConfig if someone wants to do that

[issue37111] Logging - Inconsistent behaviour when handling unicode

Paul Moore added the comment: > User would like Python logging of Unicode characters to be consistent It is consistent. The encoding of logging.basicConfig(filename='c:\\my_log.log') is consistent with the encoding of open('c:\\my_log.log') Both use the system default encoding,

[issue37111] Logging - Inconsistent behaviour when handling unicode

Vinay Sajip added the comment: Different people have different ideas of what a default should be. You can pretty much guarantee that whatever a default is, someone will think it should be something else. The basicConfig functionality has been around in its present form since 2003, and

[issue37111] Logging - Inconsistent behaviour when handling unicode

Jonathan added the comment: > Learning is not a waste of time. You're entitled to your opinion, but this is > not a bug in logging. We'll have to agree to disagree. I agree and value learning a great deal. However learning should happen on your own time, NOT when a program crashes randomly

[issue37111] Logging - Inconsistent behaviour when handling unicode

Vinay Sajip added the comment: Learning is not a waste of time. You're entitled to your opinion, but this is not a bug in logging. We'll have to agree to disagree. -- ___ Python tracker

[issue37111] Logging - Inconsistent behaviour when handling unicode

Jonathan added the comment: > I have no idea what you mean by this. I don't see how I can be clearer. What are the reasons for NOT implementing logging to file be unicode as a default? Logging to screen is unicode as a default. What are the reasons for not wanting consistency? > A simple

[issue37111] Logging - Inconsistent behaviour when handling unicode

Vinay Sajip added the comment: > This seems like an attempt at victim blaming. Er, no, it was a straight question about whether you'd read the documentation. > I'm afraid this line of reasoning is suffering from selection bias, cherry > picking, confirmation bias, and probably some others

[issue37111] Logging - Inconsistent behaviour when handling unicode

Jonathan added the comment: > Did you look at the basicConfig documentation before raising this issue? This seems like an attempt at victim blaming. But yes, I did. In fact, this is now the third time I've looked at that page - once before raising this issue, once before my previous reply,

[issue37111] Logging - Inconsistent behaviour when handling unicode

Vinay Sajip added the comment: > Doing something as basic as logging unicode shouldn't require knowledge of > "handlers" - that's failing "simple is better than complex". I reiterate my statement that it appears that you aren't sufficiently familiar with how logging is designed to work.

[issue37111] Logging - Inconsistent behaviour when handling unicode

Jonathan added the comment: Thank you for your comments but this wasn't a question and I maintain this is a bug, or at least undesirable behaviour. I'd consider it a bug in my own software. Reasoning: * It's an inconsistent default with the logging to screen. This causes more complexity

[issue37111] Logging - Inconsistent behaviour when handling unicode

Vinay Sajip added the comment: > Given that the file created by the logger is utf-8, it's unclear why it > doesn't work ... I found a workaround by using a Handler Loggers don't create files - handlers do. The file that you attached seems to be just a text file containing ASCII text. Any

[issue37111] Logging - Inconsistent behaviour when handling unicode

Change by SilentGhost : -- components: +Windows nosy: +paul.moore, steve.dower, tim.golden, zach.ware type: -> behavior ___ Python tracker ___

[issue37111] Logging - Inconsistent behaviour when handling unicode

Jonathan added the comment: It definitely claims to be "utf-8" in NotePad++. I've attached it if you want to double-check. (Windows 7) -- Added file: https://bugs.python.org/file48380/my_log.log ___ Python tracker

[issue37111] Logging - Inconsistent behaviour when handling unicode

Change by SilentGhost : -- versions: +Python 3.7, Python 3.8 -Python 3.6 ___ Python tracker ___ ___ Python-bugs-list mailing list

[issue37111] Logging - Inconsistent behaviour when handling unicode

SilentGhost added the comment: > Given that the file created by the logger is utf-8 I don't think this is true. Default file handler created using filename argument is opened with encoding None (meaning that it relies on system-default encoding, which is on Windows is not utf-8). And it

[issue37111] Logging - Inconsistent behaviour when handling unicode

New submission from Jonathan : Python is inconsistent in how it handles errors that have some unicode characters. It works to screen but fails to log This works: ``` >>> import logging >>> logging.error('จุด1') ERROR:root:จุด1 ``` The following breaks: ``` >>> import logging >>>