Re: [Python-Dev] Using logging in the stdlib and its unit tests
Am 08.12.2010 01:09, schrieb Antoine Pitrou: > On Tue, 7 Dec 2010 23:45:39 + (UTC) > Vinay Sajip wrote: >> Antoine Pitrou pitrou.net> writes: >> >> > >> > I thought "error" and "critical" messages were logged to stderr by >> > default? Isn't it the case? >> > >> >> Only if you call basicConfig() or use the logging.debug(), logging.info(), >> etc. >> module-level convenience functions (which call basicConfig under the hood). > > Why wouldn't it be the default for all logging calls ? Such special > cases don't really make things easy to remember. > >> When is the NullHandler needed? Only for cases where an application developer >> uses a library which does logging under the covers (for those users who >> might be >> interested in logging its operations), but where that application developer >> doesn't use logging themselves for that application. > > You seem pretty tied up to the "application developer" situation. There > are cases (scripts, prototyping, etc.) where you certainly want to see > error messages (errors should not pass silently) but don't want to > configure logging for each of the libraries you use. But errors don't pass silently, do they? The usual way to present errors is still by raising exceptions. Georg ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] [RELEASED] Python 3.2 beta 1
Am 07.12.2010 09:24, schrieb Łukasz Langa: > Wiadomość napisana przez Georg Brandl w dniu 2010-12-06, o godz. 22:46: > >> -BEGIN PGP SIGNED MESSAGE- >> Hash: SHA1 >> >> On behalf of the Python development team, I'm happy to announce the >> first of two beta preview releases of Python 3.2. >> >> Python 3.2 is a continuation of the efforts to improve and stabilize the >> Python 3.x line. Since the final release of Python 2.7, the 2.x line >> will only receive bugfixes, and new features are developed for 3.x only. >> >> Since PEP 3003, the Moratorium on Language Changes, is in effect, there >> are no changes in Python's syntax and built-in types in Python 3.2. >> Development efforts concentrated on the standard library and support for >> porting code to Python 3. Highlights are: >> >> [snip] > > * configparser 1.1: new API using the mapping protocol access, support for > pluggable interpolation handlers, additional interpolation handler > (ExtendedInterpolation) which supports the zc.buildout syntax, support for > alternative option/value delimiters, support for customization of accepted INI > file structure (e.g. comment prefixes, name of the DEFAULT section, > indentation, > empty lines in multiline values, etc.), support for specifying encoding for > read > operations, ConfigParser class deprecated in favor of SafeConfigParser, lots > of > other small changes. Thanks, I've added this and the ssl module for the next announcement. Georg ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Using logging in the stdlib and its unit tests
Nick Coghlan gmail.com> writes: > The surprise came from not realising there was a basicConfig() call > hidden inside the convenience APIs, a fact which is *not* mentioned in > the docstrings. It may be mentioned in the main documentation, but I > didn't look at that at the time - there was nothing to suggest the > docstrings were lacking pertinent information: > You're right that this is missing from the docstrings; I'll rectify that. > I'm not proposing that the standard library be special-cased, I'm > proposing that the default behaviour of an unconfigured logging module > in general be changed to something more useful (i.e. warnings and > errors printed to stderr, everything else suppressed), rather than > unhelpfully suppressing *all* output except for an "Oh, I'm throwing > output away" message the first time it happens. I don't have a philosophical problem with this, but there might be differing opinions about this not just on python-dev but also in the wider community. I'd definitely want logging to do the right thing and minimize inconvenience as much as possible. There's also possibly a backwards compatibility dimension to this, but I'm not sure to what extent a change like this would really affect people in practice (as nothing will change if logging is configured). > The specific use case that triggered my interest in this default > behaviour is one where concurrent.futures *can't* raise an exception > because it knows nothing is going to be in a position to handle that > exception (due to the way callbacks are handled, c.f knows it provides > the outermost exception handler in the affected call stack). Instead > it has to catch the error and display it *somewhere* (i.e. it's very > similar to the use cases for PyErr_WriteUnraisable at the C level). > > The options for handling this are: > > 1. Write the error detail directly to stderr. (Unless the default > behaviour of logging changes, that is what I am going to suggest Brian > do, as it exactly mimics the behaviour of the PyErr_WriteUnraisable > API). > 2. Write it to the root logger with the convenience APIs (Possible > option, but I don't like the global state impact of implicitly calling > basicConfig() from library code) > 3. Add a StdErr handler for the logger (this is what is currently > implemented, and again, I don't like it because of the global state > impact on something that should be completely under an application's > control) > > Basically, the current behaviour of logging is such that libraries > *cannot* use it for unraisable warnings and error messages, as the > messages will be suppressed unless the application takes steps to see > them. That is OK for debug and info messages, but unacceptable for > warnings and errors. A throwaway script using concurrent.futures needs > to know if callbacks are failing, and that needs to happen without any > logging related boilerplate in the script itself. > > If, however, an application completely lacking even a call to > logging.basicConfig() would still see warnings and errors, then > libraries could safely use the module without needing to worry about > applications needing an particular boilerplate in order to see the > unraisable errors and warnings that are emitted. > Thanks for the detailed explanation. I agree that unraisable warnings and errors need to be handled somehow. There is a way in which this can be done without affecting a logging configuration, viz. logging can define a "handler of last resort" (not attached to any logger) which is invoked when there are no user-specified handlers. This would by default be a StreamHandler writing to sys.stderr with a threshold of WARNING (or perhaps ERROR). Thus sounds like a better option than a direct write to sys.stderr, since you can't change the latter behaviour easily if you want to do something else instead. This is of course a backwards-incompatible change to logging semantics: instead of saying that logging will be silent unless explicitly asked to produce output, we're saying that logging will always produce output for warnings and errors (or perhaps just errors), unless explicitly silenced. This is of course in line with the Zen of Python; the present behaviour, which is not so aligned, is based on the idea that logging should not affect program behaviour if it's not wanted by the program developer (as opposed to library developer). It would also mean changing the documentation about NullHandler to say: "If you have messages which must get out when you can't raise an exception, then don't add a NullHandler to your top-level loggers." Regards, Vinay Sajip ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Using logging in the stdlib and its unit tests
Robert Kern gmail.com> writes: > > I really don't understand how this view can be consistent with the > practice of adding NullHandler to loggers. If this message is so important > to prevent misconfiguration, then why should a library author decide to > silence it for his users? Because the application developer knows more about the end-user audience for their application, they are better placed to know how logging should work for their application. It's not an error for a particular application developer to decide that nothing should be produced by logging for a particular application; they (particularly when casual users) would be confused by the misconfiguration message due to logging by a library they're using. The library author's users are the application developers who use the library, not the end users who use their applications. Sometimes they're the same people, I know, but I just think of them as wearing different hats :-) > [...] I strongly suspect that almost all configurations include a > catch-all root logger and that most of those *only* consist of that > root logger. That doesn't seem right: your comment might be conflating loggers with handlers. The common pattern would be (or should be) to name loggers according to __name__ in the modules which use logging, but only configure *handlers* for the root logger. That way, logging messages indicate their origin (because of the __name__ convention) but you only need to add handlers at the root logger to capture all the logging information. > Same mistake. I intended the correction to apply to all such statements in > my post. Oh, right. I thought this was a different case. > I think that boilerplate should be minimized. If using getLogger() should > almost always be followed by adding a NullHandler, then it should be the > default behavior. The easiest way to achieve this effect is to simply not > issue the warning message. getLogger() should NOT "almost always be followed by adding a NullHandler". For example, in Django, only the logger named "django" would have that handler added; no other Django logger (e.g. "django.db.models") would need to have that handler added. Regards, Vinay Sajip ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Using logging in the stdlib and its unit tests
On Wed, 8 Dec 2010 01:51:44 + (UTC) Vinay Sajip wrote: > > > Adding a NullHandler isn't the right thing to do - the behaviour I > > would want for any standard library logging that hasn't been > > explicitly configured otherwise is to do what the root logger does > > under basicConfig(): debug() and info() get suppressed, warn(), > > error(), critical() and exception() go to stderr. It seems to me like > > that would be more useful default behaviour in general than emitting a > > warning about not finding a handler. > > > > So my suggestion would be: > > 1. In the absence of a "config" call, make the "no handler" path in > > loggers emit messages *as if* basicConfig() has been called (without > > actually calling it) > > 2. Remove the implicit calls to basicConfig() from the module level > > convenience function > > > > How *feasible* that idea is to implement, I don't know. > > > > What about "Special cases aren't special enough to break the rules."? Well, it seems we are asking to remove the special cases, since we are asking for the special case to become the norm. Antoine. ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Using logging in the stdlib and its unit tests
On Wed, 08 Dec 2010 09:09:45 +0100 Georg Brandl wrote: > Am 08.12.2010 01:09, schrieb Antoine Pitrou: > > On Tue, 7 Dec 2010 23:45:39 + (UTC) > > Vinay Sajip wrote: > >> Antoine Pitrou pitrou.net> writes: > >> > >> > > >> > I thought "error" and "critical" messages were logged to stderr by > >> > default? Isn't it the case? > >> > > >> > >> Only if you call basicConfig() or use the logging.debug(), logging.info(), > >> etc. > >> module-level convenience functions (which call basicConfig under the hood). > > > > Why wouldn't it be the default for all logging calls ? Such special > > cases don't really make things easy to remember. > > > >> When is the NullHandler needed? Only for cases where an application > >> developer > >> uses a library which does logging under the covers (for those users who > >> might be > >> interested in logging its operations), but where that application developer > >> doesn't use logging themselves for that application. > > > > You seem pretty tied up to the "application developer" situation. There > > are cases (scripts, prototyping, etc.) where you certainly want to see > > error messages (errors should not pass silently) but don't want to > > configure logging for each of the libraries you use. > > But errors don't pass silently, do they? The usual way to present errors > is still by raising exceptions. Or logging them. http://docs.python.org/dev/library/logging.html#logging.Logger.exception ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Using logging in the stdlib and its unit tests
On Wed, 8 Dec 2010 01:19:45 + (UTC) Vinay Sajip wrote: > Antoine Pitrou pitrou.net> writes: > > > > > Why wouldn't it be the default for all logging calls ? Such special > > cases don't really make things easy to remember. > > > > One size doesn't fit all. Everything's documented reasonably well. If you use > it > often, you remember it. If you use it seldom, you might need to look things > up. My point is that the default behaviour should be helpful. I would point out that logging is really the kind of thing people don't want to spend time with. I know it's not very gratifying for the module author (you!), but nobody expects (*) to have to first configure a logging library when they want to use a third-party library. They expect it to work *and* give useful error report by default. (*) (not in the Python land anyway; Java might be different) > The defaults are I believe reasonably convenient, or at least not > inconvenient: > > 1. If writing a library, add a NullHandler to the top-level logger of your > library, don't add any other handlers by default (i.e. on import), document > the > names of the loggers you use. Otherwise, see the following. First, you cannot call it a "default", since the library writer has to make it explicit. Second, I don't find that convenient at all. When I use a third-party lib I want the errors to be displayed, not silenced. I'm willing to bet that most people have the same expectation. *Especially* when prototyping, since you're not sure you've got everything right. So, that's two problems: - the library author has to explicit activate what you argue is a "default" behaviour (otherwise the library user gets an unhelpful message that handlers are not configured) - the library user still has to override that "default" since it's not satisfying for most uses There's a third problem with it: - all this is not easily discoverable. You cannot expect each user of a third-party lib to *also* read the logging docs in addition to the third-party lib docs. On my Web browser, http://docs.python.org/dev/library/logging.html seems to be at least 50 pages long. For a comparison, the warnings module logs all warnings by default on stderr without requiring any a priori configuration. And that's what stderr is for (reporting errors). > 2. If writing a simple script, prototyping etc. i.e. for casual use, use > logging.debug(), logging.info() etc. That's easy enough and convenient enough > for casual users to remember, I hope. All WARNING, ERROR and CRITICAL events > in > your code and in that of any libraries you use will be printed to sys.stderr. Indeed, this is convenient. And that's what I'm arguing should be the default for *all* loggers, not just the top-level one. There's no reason that what is convenient for the root logger isn't for other loggers. > 3. If slightly less than very casual use (e.g. you want to log to file, or > want > a specific message format), call basicConfig(). Just the one extra line to > remember to do, hopefully not too inconvenient. I know that one line sounds "not too inconvenient" since you are the library author and you are so used to it (and certainly like using it and spend a lot of time improving its features, which is good). I think most people, however, have different expectations (see beginning of this message). > 4. For more advanced use, use programmatic configuration, fileConfig or > dictConfig to specify a configuration. Convenience is inversely proportional > to > how involved the configuration is. That I have already noticed :) Regards Antoine. ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Using logging in the stdlib and its unit tests
On Wed, Dec 8, 2010 at 6:32 PM, Vinay Sajip wrote: > Thanks for the detailed explanation. I agree that unraisable warnings and > errors > need to be handled somehow. There is a way in which this can be done without > affecting a logging configuration, viz. logging can define a "handler of last > resort" (not attached to any logger) which is invoked when there are no > user-specified handlers. This would by default be a StreamHandler writing to > sys.stderr with a threshold of WARNING (or perhaps ERROR). A "handler of last resort" is exactly the kind of concept I had in mind. Just not expressed quite so succinctly :) > Thus sounds like a > better option than a direct write to sys.stderr, since you can't change the > latter behaviour easily if you want to do something else instead. Yeah, this is why I can understand Brian's desire to use the logging module rather than a direct write to stderr in the futures implementation. > This is of course a backwards-incompatible change to logging semantics: > instead > of saying that logging will be silent unless explicitly asked to produce > output, > we're saying that logging will always produce output for warnings and errors > (or > perhaps just errors), unless explicitly silenced. This is of course in line > with > the Zen of Python; the present behaviour, which is not so aligned, is based on > the idea that logging should not affect program behaviour if it's not wanted > by > the program developer (as opposed to library developer). Libraries that write directly to stderr affect program behaviour as well - at least with logging, the application developer will have an easier time of silencing (or redirecting) them. Also, in situations where this default handler will be used, logging currently produces output at least once (the warning that it couldn't find a handler to use). So while it is a behavioural change, it is one that should be acceptable from a compatibility point of view. > It would also mean changing the documentation about NullHandler to say: "If > you > have messages which must get out when you can't raise an exception, then don't > add a NullHandler to your top-level loggers." Agreed. We could also make it easy to replace the handler of last resort so people can easily recover the old behaviour (by inserting NullHandler in place of the default last resort handler). Cheers, Nick. -- Nick Coghlan | [email protected] | Brisbane, Australia ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Using logging in the stdlib and its unit tests
Am 08.12.2010 10:42, schrieb Antoine Pitrou: >> But errors don't pass silently, do they? The usual way to present errors >> is still by raising exceptions. > > Or logging them. > http://docs.python.org/dev/library/logging.html#logging.Logger.exception Yes, thank you I'm aware of the exception() method. But hopefully standard library modules don't use it to report exceptions to code that uses them? Georg ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Using logging in the stdlib and its unit tests
On Wed, 08 Dec 2010 11:48:16 +0100 Georg Brandl wrote: > Am 08.12.2010 10:42, schrieb Antoine Pitrou: > > >> But errors don't pass silently, do they? The usual way to present errors > >> is still by raising exceptions. > > > > Or logging them. > > http://docs.python.org/dev/library/logging.html#logging.Logger.exception > > Yes, thank you I'm aware of the exception() method. But hopefully standard > library modules don't use it to report exceptions to code that uses them? I'm not aware of that, but there are certainly third-party libs using it (think an HTTP server that wants to log an error in one of its request handlers without the error taking the whole server down). Antoine. ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Using logging in the stdlib and its unit tests
Antoine Pitrou pitrou.net> writes: > > My point is that the default behaviour should be helpful. > I can't disagree with that. Now if only we could agree what we mean by "default" and "helpful" ;-) > I would point out that logging is really the kind of thing people don't > want to spend time with. I know it's not very gratifying for the module > author (you!), but nobody expects (*) to have to first configure a Logging's just a tool - I completely understand that it's just an adjunct to the interesting stuff. Doesn't bother me a bit :-) > logging library when they want to use a third-party library. They > expect it to work *and* give useful error report by default. > > (*) (not in the Python land anyway; Java might be different) The Java thing is a red herring, I believe. It's more akin to the Unix idea of minimum verbosity as a default. In general, you don't need to "configure a library" except when you want specific behaviour from that library's loggers. > First, you cannot call it a "default", since the library writer has to > make it explicit. > Second, I don't find that convenient at all. When I use a third-party > lib I want the errors to be displayed, not silenced. I'm willing to bet > that most people have the same expectation. *Especially* when > prototyping, since you're not sure you've got everything right. > > So, that's two problems: > - the library author has to explicit activate what you argue is a > "default" behaviour (otherwise the library user gets an unhelpful > message that handlers are not configured) > - the library user still has to override that "default" since it's not > satisfying for most uses > See my comments to Nick Coghlan's post about getting messages out when you can't raise an exception. I think the case is not as common as you suggest (because in many instances, you would raise an exception to signal an error). I'm not opposed to changing the way things work, but I think we need more of a consensus because some backwards incompatibility will result, albeit not likely to affect too many users of logging. > There's a third problem with it: > - all this is not easily discoverable. You cannot expect each user of a > third-party lib to *also* read the logging docs in addition to the > third-party lib docs. On my Web browser, > http://docs.python.org/dev/library/logging.html seems to be at least > 50 pages long. You're complaining about too much documentation?! Don't measure it by weight! Seriously, I don't see what's wrong with a library developer reading the documentation for any module they're using, logging included. "Configuring logging for a library" is easily visible in the sidebar on that page. So I wouldn't agree that "it's not easily discoverable". It's not exactly buried. And it's the library developer who has to read it, not the user of the library. The user of the library only needs to know what logger names are used by that library, and that too only if they want to configure those loggers. > For a comparison, the warnings module logs all warnings by default on > stderr without requiring any a priori configuration. And that's what > stderr is for (reporting errors). > But that behaviour is *not* always wanted. That's why I was asked to add warnings integration to logging - to redirect warning messages to logging. Things are seldom as black and white as you're making them out to be here. > Indeed, this is convenient. And that's what I'm arguing should be the > default for *all* loggers, not just the top-level one. There's no > reason that what is convenient for the root logger isn't for other > loggers. Loggers work in a hierarchy, and the package is specifically designed so that you can attach handlers just to the root logger, and get messages from all code in your application + its libraries to destinations that you configure there. This is not special case behaviour for root logger versus other loggers - it's just that the root logger is a convenient, logical place to attach handlers. > I know that one line sounds "not too inconvenient" since you are the > library author and you are so used to it (and certainly like using it > and spend a lot of time improving its features, which is good). > I think most people, however, have different expectations (see > beginning of this message). Hey, that's not why I think "not too inconvenient"! It's because (a) It's no less inconvenient than "import foo", and it's hard to argue that one line is really inconvenient for this case - remember, for the simplest uses you can omit the basicConfig() call, since you can use logging.info() etc. (b) For this case, since there's no telepathy in this world, you need some way for the programmer to communicate their intent about the logging level, or format string, or log file that they want to use. I stand by the statement that one line (or more precisely, one function call) is not too much to ask here, so yes - "not too inconvenient". That's emp
Re: [Python-Dev] Using logging in the stdlib and its unit tests
Antoine Pitrou pitrou.net> writes: > > On Wed, 08 Dec 2010 11:48:16 +0100 > Georg Brandl gmx.net> wrote: But hopefully standard > > library modules don't use it to report exceptions to code that uses them? > > I'm not aware of that, but there are certainly third-party libs using > it (think an HTTP server that wants to log an error in one of its > request handlers without the error taking the whole server down). > That's not the same thing as Georg is talking about, IIUC. The exception() method is used in exception *handler* code to record that the exception occurred, but the correct thing for any code to do when an error condition is detected (in most situations at least) is to raise an exception. It would be quite wrong for code to e.g. call logger.error(...) instead of raise ... Regards, Vinay Sajip ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Using logging in the stdlib and its unit tests
On 8 December 2010 08:32, Vinay Sajip wrote: > Nick Coghlan gmail.com> writes: >> I'm not proposing that the standard library be special-cased, I'm >> proposing that the default behaviour of an unconfigured logging module >> in general be changed to something more useful (i.e. warnings and >> errors printed to stderr, everything else suppressed), rather than >> unhelpfully suppressing *all* output except for an "Oh, I'm throwing >> output away" message the first time it happens. > > I don't have a philosophical problem with this, but there might be differing > opinions about this not just on python-dev but also in the wider community. > I'd > definitely want logging to do the right thing and minimize inconvenience as > much > as possible. There's also possibly a backwards compatibility dimension to > this, > but I'm not sure to what extent a change like this would really affect people > in > practice (as nothing will change if logging is configured). To provide an application developer point of view, I'm +1 on this change. I want library developers to be able to report errors/warnings and have them appear by default, without me doing anything (or even needing to be aware that the logging module is involved). The benefit of the library developer using the logging module to me is that if I deem it appropriate, I can direct the warnings/errors elsewhere simply by configuring logging. So, I'd like the library developer to feel that they can use logging in precisely the same ways as they would write to sys.stderr, and with the same expectations (output will be seen by default), but with less reluctance because they know that I can easily adapt their output according to my needs. >> The options for handling this are: >> >> 1. Write the error detail directly to stderr. (Unless the default >> behaviour of logging changes, that is what I am going to suggest Brian >> do, as it exactly mimics the behaviour of the PyErr_WriteUnraisable >> API). I find library code that does this annoying. It's the right effect, but leaves me with little or no control should I want to do something like log to a file. >> 2. Write it to the root logger with the convenience APIs (Possible >> option, but I don't like the global state impact of implicitly calling >> basicConfig() from library code) Library code shouldn't use the root logger, and shouldn't call basicConfig - I agree with Nick again. >> 3. Add a StdErr handler for the logger (this is what is currently >> implemented, and again, I don't like it because of the global state >> impact on something that should be completely under an application's >> control) That seems like a best of a bad job answer. As an application user, I'd expect to find it awkward to modify this default, purely because I have to undo the configuration that the library had to do to work around the logging library's defaults. > Thanks for the detailed explanation. I agree that unraisable warnings and > errors > need to be handled somehow. There is a way in which this can be done without > affecting a logging configuration, viz. logging can define a "handler of last > resort" (not attached to any logger) which is invoked when there are no > user-specified handlers. This would by default be a StreamHandler writing to > sys.stderr with a threshold of WARNING (or perhaps ERROR). Thus sounds like a > better option than a direct write to sys.stderr, since you can't change the > latter behaviour easily if you want to do something else instead. >From my POV as an application writer, I don't care how you implement it (but I agree that your suggestion sounds more flexible). As long as errors and warnings are printed by default, and I can set my own configuration without having to undo the existing configuration, then I'm happy. > This is of course a backwards-incompatible change to logging semantics: > instead > of saying that logging will be silent unless explicitly asked to produce > output, > we're saying that logging will always produce output for warnings and errors > (or > perhaps just errors), unless explicitly silenced. This is of course in line > with > the Zen of Python; the present behaviour, which is not so aligned, is based on > the idea that logging should not affect program behaviour if it's not wanted > by > the program developer (as opposed to library developer). I can see that it's an incompatible change. But I'd still be in favour of it for 3.2. In my view, it needs to be done, and it'll only get harder to do as time goes on. Paul. ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Using logging in the stdlib and its unit tests
On 8 December 2010 12:15, Vinay Sajip wrote:
> The Java thing is a red herring, I believe. It's more akin to the Unix idea of
> minimum verbosity as a default.
And yet Unix invented the concept of stderr, precisely to ensure that
there's a route for things the program wants to inform the user about
to get out.
The NullHandler approach could be seen as the equivalent of adding
2>/dev/null to every command by default.
> In general, you don't need to "configure a
> library" except when you want specific behaviour from that library's loggers.
The "specific behaviour" I want is for errors and warnings to go to
stderr. Every time. (Unless I'm writing a service that wants to
redirect all output to a log file, or something like that - and that's
where I'm really glad to have the logging module! And the fact that
the defaults tend to encourage library authors not to use it really
frustrate me as an applicatioo developer.)
> I'm not opposed to changing the way things work, but I think we need more of a
> consensus because some backwards incompatibility will result, albeit not
> likely
> to affect too many users of logging.
Hmm. As far as I can tell from reading the thread, everyone but you is
in favour of errors and warnings going to stderr by default. I may
have missed some dissenting views, but it certainly feels like a
consensus to me!
If you feel that the proposal needs to be presented to a wider group
than just python-dev then that's fair. But personally, I think that
python-dev will be a reasonably good indicator in this case.
>> For a comparison, the warnings module logs all warnings by default on
>> stderr without requiring any a priori configuration. And that's what
>> stderr is for (reporting errors).
>>
>
> But that behaviour is *not* always wanted. That's why I was asked to add
> warnings integration to logging - to redirect warning messages to logging.
> Things are seldom as black and white as you're making them out to be here.
Exactly Logging should be a BETTER option than direct writes to
stderr. At the moment, though, it's a CHOICE - writes to stderr, which
are visible by default but (very) inflexible, or logging which is
flexible but suppressed by default (OK, "if you follow the recommended
practice to avoid a user-hostile warning"). The problem is that some
people are voting with their feet that visibility matters more than
flexibility.
>> Indeed, this is convenient. And that's what I'm arguing should be the
>> default for *all* loggers, not just the top-level one. There's no
>> reason that what is convenient for the root logger isn't for other
>> loggers.
>
> Loggers work in a hierarchy, and the package is specifically designed so that
> you can attach handlers just to the root logger, and get messages from all
> code
> in your application + its libraries to destinations that you configure there.
> This is not special case behaviour for root logger versus other loggers - it's
> just that the root logger is a convenient, logical place to attach handlers.
But you don't because the library developer added a NullHandler which
you have to switch off!!!
> Hey, that's not why I think "not too inconvenient"! It's because
>
> (a) It's no less inconvenient than "import foo", and it's hard to argue that
> one
> line is really inconvenient for this case - remember, for the simplest uses
> you
> can omit the basicConfig() call, since you can use logging.info() etc.
>
> (b) For this case, since there's no telepathy in this world, you need some way
> for the programmer to communicate their intent about the logging level, or
> format string, or log file that they want to use. I stand by the statement
> that
> one line (or more precisely, one function call) is not too much to ask here,
> so
> yes - "not too inconvenient". That's emphatically not because I'm the module's
> author and "used to it".
FWIW, basicConfig has *never* been what I want. Except in the sense
that I have to use it to switch on logging to stderr (if only it
worked, but see the comment above about NullHandler) because that's
not the default. In all other cases, I've wanted a little bit more
(which may not be hard to achieve, but that's not the point here).
Saying that's convenient is like saying that adding
sys.stderr=open("/dev/tty", "w") is a convenient way of getting errors
displayed when you don't like the default destination of /dev/null.
Paul.
___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Using logging in the stdlib and its unit tests
Am 08.12.2010 13:22, schrieb Vinay Sajip: > Antoine Pitrou pitrou.net> writes: > >> >> On Wed, 08 Dec 2010 11:48:16 +0100 >> Georg Brandl gmx.net> wrote: > But hopefully standard >> > library modules don't use it to report exceptions to code that uses them? >> >> I'm not aware of that, but there are certainly third-party libs using >> it (think an HTTP server that wants to log an error in one of its >> request handlers without the error taking the whole server down). >> > > That's not the same thing as Georg is talking about, IIUC. The exception() > method is used in exception *handler* code to record that the exception > occurred, but the correct thing for any code to do when an error condition is > detected (in most situations at least) is to raise an exception. It would be > quite wrong for code to e.g. call logger.error(...) instead of raise ... Exactly. The HTTP server is of course a good example of an application, and one of the most obvious candidates in the stdilb to use the logging module instead of sys.stderr.write(). Georg ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Using logging in the stdlib and its unit tests
Paul Moore gmail.com> writes: > And yet Unix invented the concept of stderr, precisely to ensure that > there's a route for things the program wants to inform the user about > to get out. > > The NullHandler approach could be seen as the equivalent of adding > 2>/dev/null to every command by default. > Not quite true. The program can (as of right now) ensure that all warnings and messages get directed to stderr, simply by calling basicConfig(). That'll work even if every library has added a NullHandler. Although you can argue that the call to basicConfig() shouldn't be necessary, you tone comes across as a bit of an overreaction. > The "specific behaviour" I want is for errors and warnings to go to > stderr. Every time. (Unless I'm writing a service that wants to > redirect all output to a log file, or something like that - and that's > where I'm really glad to have the logging module! And the fact that > the defaults tend to encourage library authors not to use it really > frustrate me as an applicatioo developer.) > I don't see where your assertion that "the defaults tend to encourage library developers not to use it" comes from. If it's Antoine's post in this thread, then I believe he was referring to the stdlib rather than libraries in general, and I would also say that I don't believe it's a major factor; I would say that inertia was a more pertinent factor. Of course I understand I could be wrong about this, but I don't recall when a stdlib maintainer has said to me, "I want to start using logging in stdlib module XXX, but can't justify it because ..." I doubt whether making this change will result in a flurry of stdlib modules which suddenly adopt logging, but stdlib maintainers can certainly prove me wrong easily enough by stating otherwise in response to this. It's not as if I'm not approachable - you will see that many improvements to the logging package exist because of user feedback, both from python-dev and elsewhere. > Hmm. As far as I can tell from reading the thread, everyone but you is > in favour of errors and warnings going to stderr by default. I may > have missed some dissenting views, but it certainly feels like a > consensus to me! "Everyone but you" - I think you mean "everyone on this thread but you", and "everyone else on this thread" is not remotely close to "everyone else, period". > If you feel that the proposal needs to be presented to a wider group > than just python-dev then that's fair. But personally, I think that > python-dev will be a reasonably good indicator in this case. I don't dispute that, but there's not really been enough time for more than a few people on python-dev to respond. > > But that behaviour is *not* always wanted. That's why I was asked to add > > warnings integration to logging - to redirect warning messages to logging. > > Things are seldom as black and white as you're making them out to be here. > > Exactly Logging should be a BETTER option than direct writes to > stderr. At the moment, though, it's a CHOICE - writes to stderr, which > are visible by default but (very) inflexible, or logging which is > flexible but suppressed by default (OK, "if you follow the recommended > practice to avoid a user-hostile warning"). The problem is that some > people are voting with their feet that visibility matters more than > flexibility. > I don't think it's unreasonable to ask which people are voting with their feet, that you know of. This discussion was initiated by me after I got added to the nosy list of an issue. It's not as if there has been a host of voters who have asked for change and are walking away because of some intransigence on my part. As you can see in my response to Nick's post, I am making concrete suggestions on how the situation could be improved. It's not helping to talk about people voting with their feet, especially as no serious attempt to engage with me on this issue has been made in the past. Because of backward-compatibility considerations, I would consider it premature to make this kind of change just based on a request by one or two python-dev members. I have the highest level of respect for people who have responded on this thread, but how can I be sure that they're representing everyone else on this list? It's a great community, with a lot of cohesion, but people do have divergent opinions. (I'm not talking about me here. If there's a consensus about this, I've no problem implementing a "handler of last resort".) > But you don't because the library developer added a NullHandler which > you have to switch off!!! > Where are you getting this "switch off" from? There's no switching off that happens. The NullHandler remains in place. What you are doing is "switching on" output to some specific place you want - console, file etc. - with a level of verbosity and format that you choose. > FWIW, basicConfig has *never* been what I want. Except in the sense > that I have to use it to switch on logging to stderr (if on
Re: [Python-Dev] Using logging in the stdlib and its unit tests
On Wed, Dec 8, 2010 at 8:19 AM, Paul Moore wrote: > But you don't because the library developer added a NullHandler which > you have to switch off!!! I'm suspecting there's misunderstanding on this point. If I have a logger "myns.mypackage" for my library, and register a NullHandler for that, that doesn't need to be turned off for applications. What it accomplishes is that for all messages sent to the "myns.mypackage" logger (and descendants) are handled by at least one handler. Log records are still propagated toward the root for outer loggers to handle. An application can also still choose to install additional handlers for "myns.mypackage" if desired, and do whatever output handling is appropriate. The library-installed NullHandler on a library-scoped logger simply ensures that the library doesn't trigger the message about un-handled log records. We seem to have consensus on not wanting that message triggered for library logging is desirable, and this NullHandler trick provides a way to handle that independent from other possible changes. -Fred -- Fred L. Drake, Jr. "A storm broke loose in my mind." --Albert Einstein ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Using logging in the stdlib and its unit tests
On Wed, 8 Dec 2010 12:15:53 + (UTC) Vinay Sajip wrote: > > First, you cannot call it a "default", since the library writer has to > > make it explicit. > > Second, I don't find that convenient at all. When I use a third-party > > lib I want the errors to be displayed, not silenced. I'm willing to bet > > that most people have the same expectation. *Especially* when > > prototyping, since you're not sure you've got everything right. > > > > So, that's two problems: > > - the library author has to explicit activate what you argue is a > > "default" behaviour (otherwise the library user gets an unhelpful > > message that handlers are not configured) > > - the library user still has to override that "default" since it's not > > satisfying for most uses > > > > See my comments to Nick Coghlan's post about getting messages out when you > can't > raise an exception. I think the case is not as common as you suggest (because > in > many instances, you would raise an exception to signal an error). I'm not talking specifically about exceptions, but about errors in general. If the case wasn't common, I'm not sure why the error() and critical() methods would exist at all. (of course I'm assuming error() is meant to output error messages. If that's a wrong interpretation then I'm a bit puzzled :-)) > > There's a third problem with it: > > - all this is not easily discoverable. You cannot expect each user of a > > third-party lib to *also* read the logging docs in addition to the > > third-party lib docs. On my Web browser, > > http://docs.python.org/dev/library/logging.html seems to be at least > > 50 pages long. > > You're complaining about too much documentation?! Don't measure it by weight! > Seriously, I don't see what's wrong with a library developer reading the > documentation for any module they're using, logging included. I was talking about the user of a library, not its developer (see the snippet quoted above). > The > user of the library only needs to know what logger names are used by that > library, and that too only if they want to configure those loggers. The thing is, they don't *want* to configure them, but you force them to do some configuration if they don't want error messages to be silenced. > > For a comparison, the warnings module logs all warnings by default on > > stderr without requiring any a priori configuration. And that's what > > stderr is for (reporting errors). > > > > But that behaviour is *not* always wanted. Sure. What I'm arguing is that the warnings module's default behaviour is much more useful than the logging module's default behaviour. > That's why I was asked to add > warnings integration to logging - to redirect warning messages to logging. That's a separate topic :) > (b) For this case, since there's no telepathy in this world, you need some way > for the programmer to communicate their intent about the logging level, or > format string, or log file that they want to use. Well, the programmer doesn't have to communicate their intent about what and how warnings get displayed by default, and I'm not sure the warnings module has a telepathy routine inside (although you never know, since IIRC Brett did some philosophy studies). Regards Antoine. ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Using logging in the stdlib and its unit tests
On Wed, Dec 8, 2010 at 9:27 AM, Antoine Pitrou wrote: > The thing is, they don't *want* to configure them, but you force them > to do some configuration if they don't want error messages to be > silenced. As I tried to explain earlier, a NullHandler doesn't silence anything except the message about logging not being configured. Propagation is not controlled by the handlers, but by the loggers. -Fred -- Fred L. Drake, Jr. "A storm broke loose in my mind." --Albert Einstein ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Using logging in the stdlib and its unit tests
Antoine Pitrou pitrou.net> writes: > I'm not talking specifically about exceptions, but about errors in > general. If the case wasn't common, I'm not sure why the error() and > critical() methods would exist at all. > > (of course I'm assuming error() is meant to output error messages. If > that's a wrong interpretation then I'm a bit puzzled ) > Then again, it's not always helpful when *library* code prints out messages to stderr. A user of the library might prefer that an error code be returned or an exception raised, so they can deal with the condition most appropriately. > I was talking about the user of a library, not its developer (see the > snippet quoted above). Yes, but see my response to that. A user of a library doesn't need to know the specifics of what loggers that library uses, they just need to make a positive statement of where they want warning and error messages go, and how they want them formatted. BTW we haven't discussed this yet - but these enabled-by-default messages, what format should they have? (e.g. just message, logger name + message, logger name + severity + message etc.) It might sound like a trivial point, but it also seems like whatever you pick, some bikeshed discussions would ensue. At present, the developer is encouraged to choose (as they have to call basicConfig anyway), avoiding that bikeshedding altogether. > The thing is, they don't *want* to configure them, but you force them > to do some configuration if they don't want error messages to be > silenced. > True, but it need be no more than a call to basicConfig(), so "not too inconvenient" ;-) > Sure. What I'm arguing is that the warnings module's default behaviour > is much more useful than the logging module's default behaviour. > Sure, that's what we're discussing, all right. >> That's why I was asked to add >> warnings integration to logging - to redirect warning messages to logging. > > That's a separate topic :) > If you say so ;-) >> (b) For this case, since there's no telepathy in this world, you need some >> way for the programmer to communicate their intent about the logging level, >> or format string, or log file that they want to use. > > Well, the programmer doesn't have to communicate their intent about > what and how warnings get displayed by default, and I'm not sure the > warnings module has a telepathy routine inside (although you never > know, since IIRC Brett did some philosophy studies). See my earlier comment about formatting and the potential for bikeshedding ;-) Regards, Vinay Sajip ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Using logging in the stdlib and its unit tests
On Thu, Dec 9, 2010 at 12:43 AM, Fred Drake wrote: > On Wed, Dec 8, 2010 at 9:27 AM, Antoine Pitrou wrote: >> The thing is, they don't *want* to configure them, but you force them >> to do some configuration if they don't want error messages to be >> silenced. > > As I tried to explain earlier, a NullHandler doesn't silence anything > except the message about logging not being configured. Propagation is > not controlled by the handlers, but by the loggers. If the only handler is NullHandler though, (or if there are no handlers configured at all) then output does get silenced by default, even errors and warning messages. The application has to (directly or indirectly) configure logging with additional handlers in order for the output to appear somewhere. That's the behaviour that is inappropriate for use cases like the concurrent.futures handling of unraisable errors that started this discussion over on the bug tracker. Cheers, Nick. -- Nick Coghlan | [email protected] | Brisbane, Australia ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Using logging in the stdlib and its unit tests
On Thu, Dec 9, 2010 at 12:27 AM, Antoine Pitrou wrote: > On Wed, 8 Dec 2010 12:15:53 + (UTC) > Vinay Sajip wrote: >> See my comments to Nick Coghlan's post about getting messages out when you >> can't >> raise an exception. I think the case is not as common as you suggest >> (because in >> many instances, you would raise an exception to signal an error). > > I'm not talking specifically about exceptions, but about errors in > general. If the case wasn't common, I'm not sure why the error() and > critical() methods would exist at all. > > (of course I'm assuming error() is meant to output error messages. If > that's a wrong interpretation then I'm a bit puzzled :-)) As I see it, there aren't many cases at the *library* level where logging errors is more appropriate than raising exceptions: - operations where there is no outer exception handler, and the library knows it (e.g. callback processing) - methods that are guaranteed not to raise exceptions (e.g. top level event handlers) Some batch processing operations may also work that way, but those are a little more questionable (it may be better to report a result summary and leave it up to the application to decide what to do with any errors encountered along the way) However, it *is* the case that logging is currently ill-suited to tasks that are otherwise handled by writing directly to sys.stderr. I suspect that has influenced some people's views of its suitability for use in published libraries, even though it doesn't actually affect most libraries (I suspect it is a significant chunk of what underlay my reservations about using logging in library code). Since the standard library *does* include modules that currently write to stderr, then adjusting logging to make it better suited to this task will help us as well as anyone else with similar problems. That said, while I think Vinay's suggested "handler of last resort" solution is a good one and something we should be doing for 3.2, I'm also happy to let the idea bake for at least a few weeks. Cheers, Nick. P.S. On a completely unrelated note, has anyone thought about creating a write-only TextIO stream that outputs received writes via the logging module? -- Nick Coghlan | [email protected] | Brisbane, Australia ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Using logging in the stdlib and its unit tests
On Wed, Dec 8, 2010 at 9:52 AM, Nick Coghlan wrote: .. > P.S. On a completely unrelated note, has anyone thought about creating > a write-only TextIO stream that outputs received writes via the > logging module? I've done something similar for C++ iostreams many moons ago. The idea was to prepend all lines written to std::log with timestamp and other information. It worked reasonably well, but involved a lot of C++ hackery at the time. I think this is a great idea and would allow existing libraries that use sys.stderr for messages start using say the root logger instead when user redefines sys.stderr. ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Using logging in the stdlib and its unit tests
On 12/8/10 2:51 AM, Vinay Sajip wrote: Robert Kern gmail.com> writes: I really don't understand how this view can be consistent with the practice of adding NullHandler to loggers. If this message is so important to prevent misconfiguration, then why should a library author decide to silence it for his users? Because the application developer knows more about the end-user audience for their application, they are better placed to know how logging should work for their application. It's not an error for a particular application developer to decide that nothing should be produced by logging for a particular application; they (particularly when casual users) would be confused by the misconfiguration message due to logging by a library they're using. The library author's users are the application developers who use the library, not the end users who use their applications. Sometimes they're the same people, I know, but I just think of them as wearing different hats :-) I'm sorry, but it's not at all clear that you have understood my point. There is no way for me to parse your words as a sensible reply to what I said. Let's say I write a library called Foo. I want to add logging to my functions. You want to write an application called Bar that uses Foo and you want to configure logging for your application (at the very least to provide a default if not production). The warning is supposed to help you not make mistakes when configuring logging in your application. If I, library author, attach NullHandlers to all of Foo's loggers, then you will not get that warning if you forget to add handlers the Foo loggers. My adding the NullHandler to Foo prevented that warning that you consider to be so important. I don't think the warning helps much, if at all. [...] I strongly suspect that almost all configurations include a catch-all root logger and that most of those *only* consist of that root logger. That doesn't seem right: your comment might be conflating loggers with handlers. The common pattern would be (or should be) to name loggers according to __name__ in the modules which use logging, but only configure *handlers* for the root logger. That way, logging messages indicate their origin (because of the __name__ convention) but you only need to add handlers at the root logger to capture all the logging information. Yes. That's what I meant. I think that boilerplate should be minimized. If using getLogger() should almost always be followed by adding a NullHandler, then it should be the default behavior. The easiest way to achieve this effect is to simply not issue the warning message. getLogger() should NOT "almost always be followed by adding a NullHandler". For example, in Django, only the logger named "django" would have that handler added; no other Django logger (e.g. "django.db.models") would need to have that handler added. In a large package (particularly a namespace package), I can't guarantee that any particular module will get imported. I will want to be able to import just foo.bar.baz without needing to worry about whether foo.setup_logging got imported and ran the logging configuration as a side-effect. I want to be able to loosen the coupling between modules across my package, not add more coupling. But in any case, while adding a NullHandler to just the package's root logger helps you to avoid needing a NullHandler on every logger, the effect is the same. Almost all loggers effectively terminate in a NullHandler either directly or through a chain of parent loggers. So why not just make it the default? Personally, I would back the proposals being made elsewhere in this thread, that in the absence of configuration, warnings and errors should be printed to stderr no matter where they come from. This gives useful behavior out-of-the-box without configuration but remains completely configurable. Library errors don't pass silently, but logging allows people to silence them explicitly. It separates the concerns of library authors (who should never touch logging configuration and shouldn't be required to think about it) from those of the application authors and application users. -- Robert Kern "I have come to believe that the whole world is an enigma, a harmless enigma that is made terrible by our own mad attempt to interpret it as though it had an underlying truth." -- Umberto Eco ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Using logging in the stdlib and its unit tests
Nick Coghlan gmail.com> writes: > That said, while I think Vinay's suggested "handler of last resort" > solution is a good one and something we should be doing for 3.2, I'm > also happy to let the idea bake for at least a few weeks. I agree on the baking part, since it will allow time for any drawbacks to be spotted or better solutions found. There are also the questions of what level and format to use for the handler of last resort, before it can actually be implemented. > > P.S. On a completely unrelated note, has anyone thought about creating > a write-only TextIO stream that outputs received writes via the > logging module? > Is this for use at the C level? At the Python level, there's a post I wrote a while back which shows how to use a logger like an output stream: http://plumberjack.blogspot.com/2009/09/how-to-treat-logger-like-output-stream.html Regards, Vinay Sajip ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] Redirecting stdout/stderr to the logging module (was Re: Using logging in the stdlib and its unit tests)
On Thu, Dec 9, 2010 at 2:46 AM, Vinay Sajip wrote: > Nick Coghlan gmail.com> writes: >> P.S. On a completely unrelated note, has anyone thought about creating >> a write-only TextIO stream that outputs received writes via the >> logging module? >> > > Is this for use at the C level? At the Python level, there's a post I wrote a > while back which shows how to use a logger like an output stream: > > http://plumberjack.blogspot.com/2009/09/how-to-treat-logger-like-output-stream.html Similar in concept, but more full-featured (i.e. supporting the relevant IO ABCs), allowing it to be used as a replacement for stdout/stderr. Cheers, Nick. -- Nick Coghlan | [email protected] | Brisbane, Australia ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Using logging in the stdlib and its unit tests
On Thu, Dec 9, 2010 at 2:46 AM, Vinay Sajip wrote: > Nick Coghlan gmail.com> writes: > >> That said, while I think Vinay's suggested "handler of last resort" >> solution is a good one and something we should be doing for 3.2, I'm >> also happy to let the idea bake for at least a few weeks. > > I agree on the baking part, since it will allow time for any drawbacks to be > spotted or better solutions found. There are also the questions of what level > and format to use for the handler of last resort, before it can actually be > implemented. As a starting point, I'd say warnings and above, no formatting (i.e. just the message). To minimise bikeshedding, I'd like to be guided by the idea that this is a more configurable alternative to printing directly to stderr, but in the absence of application level configuration, you wouldn't be able to tell which approach the library was using just by looking at the program output. Cheers, Nick. -- Nick Coghlan | [email protected] | Brisbane, Australia ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Using logging in the stdlib and its unit tests
Robert Kern gmail.com> writes: > I'm sorry, but it's not at all clear that you have understood my point. > There is no way for me to parse your words as a sensible reply to what I > said. > > Let's say I write a library called Foo. I want to add logging to my > functions. You want to write an application called Bar that uses Foo and > you want to configure logging for your application (at the very least to > provide a default if not production). The warning is supposed to help you > not make mistakes when configuring logging in your application. If I, > library author, attach NullHandlers to all of Foo's loggers, then you will > not get that warning if you forget to add handlers the Foo loggers. > My adding the NullHandler to Foo prevented that warning that you consider > to be so important. I understand, and what you say above is correct. > I don't think the warning helps much, if at all. Perhaps not. I'm not so hung up on the importance of the message now, but it certainly *was* important when logging was first introduced, and users couldn't expect to get the same level of help on comp.lang.python that they now can. Today there are quite a few people who can help people with finger-trouble logging issues. Consider the scenarios when not having the current behaviour would bite you: 1. You're new to Python/new to logging. You write some code, perhaps across several modules, which code makes logging calls, but you omit to configure any handlers, whether through forgetting or not understanding what needs to be done. Without the message, you've no idea why no logging messages appear, no matter how much you fiddle with levels. 2. You write some code and decide you don't want to use logging, but use some third party libraries which do. You don't care about that, so it's annoying to have "no handlers could be found for logger XXX" messages printed to console. You berate the library developer for their lack of consideration. Perhaps you don't find yourself in these situations, but surely you sympathize with people who do? How would you propose to address both those scenarios? > In a large package (particularly a namespace package), I can't guarantee > that any particular module will get imported. I will want to be able to > import just foo.bar.baz without needing to worry about whether > foo.setup_logging got imported and ran the logging configuration as a > side-effect. I want to be able to loosen the coupling between modules > across my package, not add more coupling. > I'm not sure what coupling you're talking about - perhaps you can illustrate with an example. If I develop a package "foo.bar" which is part of namespace package "foo", and use loggers named __name__ in my code, and add a NullHandler to logger "foo.bar", that's all I have to do. Likewise, if another person develops "foo.baz" and they add a NullHandler to "foo.baz", then where's the coupling between the two packages? They needn't even know about each other. > But in any case, while adding a NullHandler to just the package's root logger > helps you to avoid needing a NullHandler on every logger, the effect is the > same. Almost all loggers effectively terminate in a NullHandler either > directly or through a chain of parent loggers. So why not just make it the > default? There's no "termination" when a NullHandler is encountered. Django has dozens of modules, many of which could use logging, but only one NullHandler needs to be added for the whole of Django. The effect you say is the same is not: when adding new modules to Django, for example, no additional NullHandler adding needs to be done. I don't want to appear rude, but this seems to be another mistake (or perhaps a variant of the same mistake as before) you are making about how logging works. You obviously feel strongly about it, and if you have a specific use case which is causing/has caused you pain, please spell it out for me (on comp.lang.python, say) and I will try to help sort out the problem. As am off-topic example, Armin Ronacher kept on saying in various posts and presentations that you couldn't use stdlib logging for web applications, that there were fundamental problems with it. But when he actually sent me his specific problem statement, I gave him a solution without spending too much time on it (see http://plumberjack.blogspot.com/2010/09/configuring-logging-for-web.html if you care). I'm not trying to be obstructive, honestly. > Personally, I would back the proposals being made elsewhere in this thread, > that in the absence of configuration, warnings and errors should be printed > to stderr no matter where they come from. This gives useful behavior > out-of-the-box As you might have seen in my response to Nick's post, I've made specific proposals about this myself. I agree with Nick's viee that some more time should be given for discussion, suggestions, others to express views, and just for the ideas to bake for a bit. > without configuration but r
Re: [Python-Dev] Using logging in the stdlib and its unit tests
On 12/8/2010 4:15 AM, Vinay Sajip wrote: You're complaining about too much documentation?! Don't measure it by weight! On 12/8/2010 5:57 AM, Vinay Sajip wrote: Of course I understand I could be wrong about this, but I don't recall when a stdlib maintainer has said to me, "I want to start using logging in stdlib module XXX, but can't justify it because ..." So I'm a fairly new Python user, but 30 years programming experience. When I first looked at the Python logging facility (and again today, when I looked again), I found a simple example of debugging logging. Then a bunch of stuff about how to configure rotating logs. Then something about logging levels. And then a dissertation on the theory of loggers "The logging library takes a modular approach and offers the several categories of components: loggers, handlers, filters, and formatters." And I hadn't gotten through 10% of the documentation yet, per the scrollbar. My reaction the first time was "Sounds sophisticated and complex. I think I'll do something simpler for now, and maybe someday, when I have a spare week, I'll read the documentation and see if the benefits are worth the effort." OK, so now you've discovered that too much documentation can be a turn off... at least, if it is presented from the top to describe the sophistication of the facility, rather than how easy it is to use (if it is, I still haven't gotten to 10%, and I still don't know that). From this thread, it sounds like it might be that logging could be easy to use. Although part of my turn off from too much documentation and sophistication is a concern about the overhead of reading in and compiling a module that is that sophisticated for simple programs. Now if I was convinced that (1) it always gets read in anyway as part of Python startup (8% of the documentation didn't say that), so that no matter how simple my alternate facility, I pay the cost of loading the logger anyway, and (2) that it would take less than a week to comprehend the basics of what I need to learn to use it, (in other words, if the 8% of the documentation I've read, actually gave sufficient simple examples to use applications and libraries, and left the sohistication and complexity and theory to later sections) then I might have used the logger, instead of writing, in my personal library of "useful functions": _logfh = sys.stderr def log( text=None, textpre=None, filename=None, mode=None ): # version 2010/11/11 global _logfh if filename is not None: if mode is None: mode = "a" _logfh = open( filename, mode, **_wenc ) if text is not None: if textpre is None: _logfh.write( str( text ) + "\n") else: _logfh.write( str( textpre ) + ": " + str( text ) + "\n") _logfh.flush() return _logfh You see, I can create that faster than I read 8% of the documentation for logger, which didn't convince me that using the logger was better than the above, and it has been working fine... and I can tweak it when and if I need more sophistication. I'm well aware that my function provides much less than what logger provides. But in the first 8% of the documentation, I hadn't learned how to be able to use logging from a multiple module program in an easy and consistent manner... The example shows a 4 line cost... it is not clear from the first 8% of the documentation if it requires 4 lines in every module; it is not clear if I can or should or if by default, the file I configure for one module is available to other modules, such that only two lines are necessary in those modules (import, logging.debug (or other levels of message)), etc. And I'd have to read more than 8% of the documentation to find that out. Now I'm not saying that I only read 8% of the documentation for modules that I want to use, in general, before I give up in disgust. But I start reading, and estimate the cost of learning vs the cost of rewriting as I go. logger documentation is large, so reading it all before starting to use it is unlikely; since the first part didn't get me started soon enough, I "put it on a shelf", where it still is. If the first part had shown an example of how to use logger, in simple mode, in a trivial but multi-module application (because most trivial application do involve multiple modules, even if only one is user-written), then as I went along, I'd have likely learned more about the features as needed. There are lots of interesting sounding features in the Table of Contents. "Someday" I might decide to spend the week to read about them. Hope this perspective helps, somehow. Glenn ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Using logging in the stdlib and its unit tests
On Wed, 8 Dec 2010 14:54:09 + (UTC) Vinay Sajip wrote: > Antoine Pitrou pitrou.net> writes: > > > I'm not talking specifically about exceptions, but about errors in > > general. If the case wasn't common, I'm not sure why the error() and > > critical() methods would exist at all. > > > > (of course I'm assuming error() is meant to output error messages. If > > that's a wrong interpretation then I'm a bit puzzled ) > > > > Then again, it's not always helpful when *library* code prints out messages to > stderr. A user of the library might prefer that an error code be returned or > an > exception raised, so they can deal with the condition most appropriately. Sorry, what are you arguing about and what kind of answer are you expecting? Obviously error() is preferrable in some cases and other means of communicating the error are preferrable in other cases. That's kind of stating the obvious to me. But since you are the one you wrote the library and added error() in the first place, why are you trying to convince me that error() is not useful? Perhaps you should explain how error() is supposed to be used for if it's not supposed to log errors. > > I was talking about the user of a library, not its developer (see the > > snippet quoted above). > > Yes, but see my response to that. A user of a library doesn't need to know the > specifics of what loggers that library uses, they just need to make a positive > statement of where they want warning and error messages go, and how they want > them formatted. And my point, again, is that they want it to work by default *without* making such a positive statement. I'm not sure if I'm supposed to explain this using different words since you're not appearing to get it. > BTW we haven't discussed this yet - but these enabled-by-default messages, > what > format should they have? (e.g. just message, logger name + message, logger > name > + severity + message etc.) It might sound like a trivial point, but it also > seems like whatever you pick, some bikeshed discussions would ensue. I'm sorry but your way of argumenting is really puzzling to me. You are trying to find counter-arguments against something which you *already* implemented yourself in logging.error(). If you're interested in having a theoretical argument about what a beautiful design should be (and if you want to argue about your own design decisions), I'll leave the discussion here. I'm only interested in the *practical* matter of logging having an useful behaviour by default, which apparently you don't really care about. Fair enough, but this whole thread is quite telling, IMO, about the general impedance mismatch between the logging design and the expectations of casual users (which is 99% of them, certainly). Regards Antoine. ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Redirecting stdout/stderr to the logging module (was Re: Using logging in the stdlib and its unit tests)
On Thu, 9 Dec 2010 03:28:36 +1000 Nick Coghlan wrote: > On Thu, Dec 9, 2010 at 2:46 AM, Vinay Sajip wrote: > > Nick Coghlan gmail.com> writes: > >> P.S. On a completely unrelated note, has anyone thought about creating > >> a write-only TextIO stream that outputs received writes via the > >> logging module? > >> > > > > Is this for use at the C level? At the Python level, there's a post I wrote > > a > > while back which shows how to use a logger like an output stream: > > > > http://plumberjack.blogspot.com/2009/09/how-to-treat-logger-like-output-stream.html > > Similar in concept, but more full-featured (i.e. supporting the > relevant IO ABCs), allowing it to be used as a replacement for > stdout/stderr. That would be an interesting idea. It would fit in the logging module rather than the io module IMO. However, there are some limitations due to the fact that logging is line-based while people may output arbitrary text on stderr (perhaps ASCII-formatted tables, who knows). Regards Antoine. ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Redirecting stdout/stderr to the logging module (was Re: Using logging in the stdlib and its unit tests)
Antoine Pitrou pitrou.net> writes: > However, there are some limitations due to the fact that logging is > line-based while people may output arbitrary text on stderr (perhaps > ASCII-formatted tables, who knows). True, though the wrapper could easily buffer partial output internally to support line-based output formats. Actually the line-based nature is a function of the handler and/or formatter and just reflects what people tend to want most of the time. Regards, Vinay Sajip ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Using logging in the stdlib and its unit tests
On 8 December 2010 14:52, Nick Coghlan wrote: > As I see it, there aren't many cases at the *library* level where > logging errors is more appropriate than raising exceptions: On a slightly tangential note, what do you think of the idea of library code including info or debug level logging? In effect, tracing and diagnostic code built in and available simply by changing the logging level? Paul. ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Using logging in the stdlib and its unit tests
Antoine Pitrou pitrou.net> writes: > But since you are the one you wrote the library and added error() in > the first place, why are you trying to convince me that error() is > not useful? Perhaps you should explain how error() is supposed to be > used for if it's not supposed to log errors. I didn't say error() [or actually exception(), which you mentioned] was useless, I merely pointed out what the normal pattern of usage was. Georg, at least, seemed to be agreeing with me. > And my point, again, is that they want it to work by default *without* > making such a positive statement. I'm not sure if I'm supposed to > explain this using different words since you're not appearing to get it. No, I'm getting it, I think. See my responses to Nick's posts. > I'm sorry but your way of argumenting is really puzzling to me. You are > trying to find counter-arguments against something which you *already* > implemented yourself in logging.error(). I don't think I am, so one of us is misunderstanding the other, or perhaps both of us are misunderstanding each other. > If you're interested in having a theoretical argument about what a > beautiful design should be (and if you want to argue about your own > design decisions), I'll leave the discussion here. I'm only interested No, I'm not interested in that. Like you I'm interested in practical matters, but unlike you I have to make some implementation choices, going forward - so it shouldn't be surprising that I want to talk around the issue to make sure that no wrong decision gets made. Sorry if that offends you because it sometimes seems to be stating the (to you) obvious. > in the *practical* matter of logging having an useful behaviour by > default, which apparently you don't really care about. Actually, I don't think my response to Nick's post (about concurrent.futures) could be characterized as "I don't care", as I even made a specific proposal about how a change could be implemented. Regards, Vinay Sajip ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Using logging in the stdlib and its unit tests
Paul Moore gmail.com> writes: > > On 8 December 2010 14:52, Nick Coghlan gmail.com> wrote: > > As I see it, there aren't many cases at the *library* level where > > logging errors is more appropriate than raising exceptions: > > On a slightly tangential note, what do you think of the idea of > library code including info or debug level logging? In effect, tracing > and diagnostic code built in and available simply by changing the > logging level? That's how it works right now. You get info() and debug() messages sent via calls in library code, just by changing the level of (say) the root logger. Regards, Vinay Sajip ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Using logging in the stdlib and its unit tests
Nick Coghlan gmail.com> writes: > As a starting point, I'd say warnings and above, no formatting (i.e. > just the message). To minimise bikeshedding, I'd like to be guided by > the idea that this is a more configurable alternative to printing > directly to stderr, but in the absence of application level > configuration, you wouldn't be able to tell which approach the library > was using just by looking at the program output. Makes sense. I know it's only a small change at the implementation level but the impact may be larger (due to it being a backwards-incompatible behaviour change), and the little details need to be agreed, so does it make sense to create a PEP about this? What do people think - is this bureaucratic overkill? Regards, Vinay Sajip ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Using logging in the stdlib and its unit tests
>> On a slightly tangential note, what do you think of the idea of
>> library code including info or debug level logging? In effect,
>> tracing and diagnostic code built in and available simply by changing
>> the logging level?
Vinay> That's how it works right now. You get info() and debug()
Vinay> messages sent via calls in library code, just by changing the
Vinay> level of (say) the root logger.
There can be performance implications if you log heavily. I don't know how
the code is organized, but functionally these two calls are equivalent:
>>> logging.error("error 1 2 3 %s" % "yup")
ERROR:root:error 1 2 3 yup
>>> logging.error("error 1 2 3 %s", "yup")
ERROR:root:error 1 2 3 yup
The second form should be preferred in library code as long as the format
string expansion is deferred until after the test is made to emit the
message.
Skip
___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Using logging in the stdlib and its unit tests
On Wed, 8 Dec 2010 19:00:31 + (UTC) Vinay Sajip wrote: > > > If you're interested in having a theoretical argument about what a > > beautiful design should be (and if you want to argue about your own > > design decisions), I'll leave the discussion here. I'm only interested > > No, I'm not interested in that. Like you I'm interested in practical matters, > but unlike you I have to make some implementation choices, going forward - so > it > shouldn't be surprising that I want to talk around the issue to make sure that > no wrong decision gets made. Sorry if that offends you because it sometimes > seems to be stating the (to you) obvious. Ok, I'm sorry for the harsh words. I really hope this discussions leads to somewhere. Regards Antoine. ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Using logging in the stdlib and its unit tests
Antoine Pitrou pitrou.net> writes: > Ok, I'm sorry for the harsh words. > I really hope this discussions leads to somewhere. No offence taken, and do do I :-) Regards, Vinay Sajip ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Using logging in the stdlib and its unit tests
pobox.com> writes:
> >>> logging.error("error 1 2 3 %s" % "yup")
> ERROR:root:error 1 2 3 yup
> >>> logging.error("error 1 2 3 %s", "yup")
> ERROR:root:error 1 2 3 yup
>
> The second form should be preferred in library code as long as the format
> string expansion is deferred until after the test is made to emit the
> message.
Yes, and the string expansion normally is deferred to happen as late as
possible.
Regards,
Vinay
___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Using logging in the stdlib and its unit tests
Vinay Sajip yahoo.co.uk> writes: > No offence taken, and do do I s/do do/so do/ Perhaps it was a Freudian slip admitting that I *am* a dodo! Regards, Vinay ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] Status of PEP 3143?
What is the status of PEP 3143? It appears to be aimed at 3.2, but there does not appear to be a resolution on it. ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Status of PEP 3143?
2010/12/8 Kirk McDonald : > What is the status of PEP 3143? It appears to be aimed at 3.2, but there > does not appear to be a resolution on it. It's after beta freeze now, so definitely not 3.2. AFAIK, though, it's never been discussed here. -- Regards, Benjamin ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Using logging in the stdlib and its unit tests
If we're all converging on adding a "handler of last resort" and dropping the warning message, we can just let this branch of the thread drop. But if you want to continue, I already had most of the following already written. I hope it clears some things up more than it muddies them further. :-) On 12/8/10 11:43 AM, Vinay Sajip wrote: Robert Kern gmail.com> writes: I don't think the warning helps much, if at all. Perhaps not. I'm not so hung up on the importance of the message now, but it certainly *was* important when logging was first introduced, and users couldn't expect to get the same level of help on comp.lang.python that they now can. Today there are quite a few people who can help people with finger-trouble logging issues. I am not commenting on the reasonableness of the behavior when it was introduced, just what I think is the best behavior now. Consider the scenarios when not having the current behaviour would bite you: 1. You're new to Python/new to logging. You write some code, perhaps across several modules, which code makes logging calls, but you omit to configure any handlers, whether through forgetting or not understanding what needs to be done. Without the message, you've no idea why no logging messages appear, no matter how much you fiddle with levels. 2. You write some code and decide you don't want to use logging, but use some third party libraries which do. You don't care about that, so it's annoying to have "no handlers could be found for logger XXX" messages printed to console. You berate the library developer for their lack of consideration. Perhaps you don't find yourself in these situations, but surely you sympathize with people who do? How would you propose to address both those scenarios? I am quite familiar with the latter in the third-party library author role. However, the messages I get aren't about lack of consideration but "What is wrong with my code? Are things broken?" They think the warning comes from my code. As for the former, I'm not sure how that would cause much confusion. If I'm interested in getting the logged information, wouldn't I first find out how to configure logging? It's just about the very first thing you see reading the docs. It's one of the fundamental operations using the logging package. I can't imagine that this is a common failure mode. I think that the documentation ameliorates it better than the warning message. In a large package (particularly a namespace package), I can't guarantee that any particular module will get imported. I will want to be able to import just foo.bar.baz without needing to worry about whether foo.setup_logging got imported and ran the logging configuration as a side-effect. I want to be able to loosen the coupling between modules across my package, not add more coupling. I'm not sure what coupling you're talking about - perhaps you can illustrate with an example. If I develop a package "foo.bar" which is part of namespace package "foo", and use loggers named __name__ in my code, and add a NullHandler to logger "foo.bar", that's all I have to do. Likewise, if another person develops "foo.baz" and they add a NullHandler to "foo.baz", then where's the coupling between the two packages? They needn't even know about each other. You were saying (in the part of the conversation that you snipped) that one would just add a NullHandler to the "django" logger to take care of messages sent to "django.db.models" and all of the other child loggers. *That* introduces a tighter coupling than I am comfortable with. All modules that use logging must make sure that the code that adds the NullHandler to the "django" logger gets executed. It's certainly reasonable to make that effort sometimes, but there are good reasons to avoid it, too. Just using logging should not force my hand. If I do keep things decoupled, then I am adding a NullHandler to nearly every logger anyways, and I return to my original point that this is repetitive boilerplate and should be eliminated by appropriate defaults. But in any case, while adding a NullHandler to just the package's root logger helps you to avoid needing a NullHandler on every logger, the effect is the same. Almost all loggers effectively terminate in a NullHandler either directly or through a chain of parent loggers. So why not just make it the default? There's no "termination" when a NullHandler is encountered. All I meant is that it eventually reaches a NullHandler so no warning message is issued. Whether it actually stops there or not is irrelevant to my point. My point is that it is rarely the case that you deliberately want to have a logger which will cause the “No handlers could be found" message. Whether you add the NullHandler to each logger or to one parent logger, you almost always want that behavior. That's why I think it should be the default behavior. Django has dozens of modules, many of which could use logging, but only one
Re: [Python-Dev] Status of PEP 3143?
Am 08.12.2010 21:12, schrieb Benjamin Peterson: > 2010/12/8 Kirk McDonald : >> What is the status of PEP 3143? It appears to be aimed at 3.2, but there >> does not appear to be a resolution on it. > > It's after beta freeze now, so definitely not 3.2. AFAIK, though, it's > never been discussed here. I might add that now would be a good time to put any candidate library into PyPI, in order to experience enough real-world usage in time for inclusion in Python 3.3. Georg ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Using logging in the stdlib and its unit tests
On 12/8/2010 2:00 PM, Vinay Sajip wrote: Actually, I don't think my response to Nick's post (about concurrent.futures) could be characterized as "I don't care", as I even made a specific proposal about how a change could be implemented. Your proposal struck me as probably the best way forward. Can you code it up and put a patch on the tracker that people can test before the next beta? A couple of other notes: 1. The number of participants in this thread is probably larger than average. So I think the responses so far can be taken as representative of those here who care enough to say something. 2. To me, Django is more an app framework than a mere 'library', and one with extra problems of error reporting in that it needs to send messages to both client and server and possibly elsewhere. So I as a user would expect it to embody extra thought about message disposition relative to a typical library. -- Terry Jan Reedy ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Status of PEP 3143?
Benjamin Peterson writes: > 2010/12/8 Kirk McDonald : > > What is the status of PEP 3143? It appears to be aimed at 3.2, but > > there does not appear to be a resolution on it. I am the PEP 3143 author and champion. It hasn't been presented for a resolution yet. Development of the reference implementation is complete but not yet suitable for standard library inclusion; progress in that direction has stalled due to intermittent communication between the developers. Further questions specifically about the PEP content or its implementation to me directly, if you like. Or in ‘comp.lang.python’, if you prefer. > It's after beta freeze now, so definitely not 3.2. So it's not abandoned, but I don't know which version should be the current target. What change should I make to the PEP in such a case? -- \ “I used to be an airline pilot. I got fired because I kept | `\ locking the keys in the plane. They caught me on an 80 foot | _o__)stepladder with a coathanger.” —Steven Wright | Ben Finney ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Status of PEP 3143?
2010/12/8 Ben Finney : > Benjamin Peterson writes: > >> 2010/12/8 Kirk McDonald : >> > What is the status of PEP 3143? It appears to be aimed at 3.2, but >> > there does not appear to be a resolution on it. > > I am the PEP 3143 author and champion. It hasn't been presented for a > resolution yet. > > Development of the reference implementation is complete but not yet > suitable for standard library inclusion; progress in that direction has > stalled due to intermittent communication between the developers. > > Further questions specifically about the PEP content or its > implementation to me directly, if you like. Or in ‘comp.lang.python’, if > you prefer. > >> It's after beta freeze now, so definitely not 3.2. > > So it's not abandoned, but I don't know which version should be the > current target. What change should I make to the PEP in such a case? Put 3.3 or 3.x if you're thinking really long term. :) -- Regards, Benjamin ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Using logging in the stdlib and its unit tests
On 8 December 2010 19:04, Vinay Sajip wrote: > Paul Moore gmail.com> writes: > >> >> On 8 December 2010 14:52, Nick Coghlan gmail.com> wrote: >> > As I see it, there aren't many cases at the *library* level where >> > logging errors is more appropriate than raising exceptions: >> >> On a slightly tangential note, what do you think of the idea of >> library code including info or debug level logging? In effect, tracing >> and diagnostic code built in and available simply by changing the >> logging level? > > That's how it works right now. You get info() and debug() messages sent via > calls in library code, just by changing the level of (say) the root logger. You misunderstand me. I know that's how those levels work. What I'm not sure about (and I think would be interesting and potentially useful information) is whether the individuals participating in this thread feel that liberal use of info and debug level logging in library code is useful. All the discussion thus far has been about warning and error levels (mainly because people seem less comfortable with the "suppress by default" approach for those levels, whereas it's natural for debug and info levels). I guess I'm wondering what best practices for logging might be. (Actually, that might make the subject of an interesting blog posting, if you're looking for suggestions :-)). Paul. ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Using logging in the stdlib and its unit tests
Paul Moore gmail.com> writes: > You misunderstand me. I know that's how those levels work. What I'm > not sure about (and I think would be interesting and potentially > useful information) is whether the individuals participating in this > thread feel that liberal use of info and debug level logging in > library code is useful. All the discussion thus far has been about > warning and error levels (mainly because people seem less comfortable > with the "suppress by default" approach for those levels, whereas it's > natural for debug and info levels). OK I see, sorry - I just didn't get that from your original choice of words. I'm probably not firing on all cylinders right now :-) > I guess I'm wondering what best practices for logging might be. > (Actually, that might make the subject of an interesting blog posting, > if you're looking for suggestions ). Thanks for the suggestion, I'll think about posting on this topic after ruminating on it for a bit. Regards, Vinay Sajip ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Using logging in the stdlib and its unit tests
On 12/8/2010 9:43 AM, Vinay Sajip wrote: As am off-topic example, Armin Ronacher kept on saying in various posts and presentations that you couldn't use stdlib logging for web applications, that there were fundamental problems with it. But when he actually sent me his specific problem statement, I gave him a solution without spending too much time on it (see http://plumberjack.blogspot.com/2010/09/configuring-logging-for-web.html if you care). I'm not trying to be obstructive, honestly. Since my application is also web server related, I was curious enough to look at this blog posting to see if it would be a good starter example that is missing from the manual. I don't think it is, as it is somewhat complex, although it is multi-module, and even multi-thread, so it might make a good example for other parts of the documentation (but maybe they already have good examples; I've only looked at the first 8%). In trying to understand it, I couldn't figure out why the WebApp class needs to keep track of the threads that are running the that web app. Why isn't it good enough for the thread to know the name of the app? The filter copies the appName from the thread to the record; the InjectingFilter.filter not replace record.appName = tlocal.appName tname = threading.currentThread().getName() return tname in self.app.threads by record.appName = tlocal.appName return appName == self.app.name and get the same effect, without needing to mainting the self.threads "database"? Or what am I missing? Glenn ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] Can't compile regex module with Python 3.2
The regex module calls _PyUnicode_IsWhitespace, which is mapped by unicodeobject.h to either _PyUnicodeUCS2_IsWhitespace or _PyUnicodeUCS4_IsWhitespace. From Python 2.5 to Python 3.1 the library pythonXX.lib contains either _PyUnicodeUCS2_IsWhitespace or _PyUnicodeUCS4_IsWhitespace. However, in Python 3.2b1 the library python32.lib contains only _PyUnicode_IsWhitespace, therefore breaking the build. Is this change intentional? If so, why does unicodeobject.h still do the mapping? Windows XP Pro, 32-bit. ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] [Python-checkins] r86855 - in python/branches/py3k: Doc/library/unittest.rst Lib/unittest/case.py
Hello, > Author: raymond.hettinger > Date: Mon Nov 29 02:38:25 2010 > New Revision: 86855 > Log: Do not add an obsolete unittest name to Py3.2. > Modified: python/branches/py3k/Lib/unittest/case.py > -# Old name for assertCountEqual() > -assertItemsEqual = assertCountEqual When we merge distutils2 back into the stdlib, our tests will have to work with stdlib unittest in 3.3 or unittest2 for older Pythons (we’ll still make standalone releases for them). unittest2 doesn’t have assertCountEqual, unittest in 3.2+ doesn’t have assertItemsEqual, what’s the plan for compatibility? Regards ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Can't compile regex module with Python 3.2
Am 09.12.2010 03:56, schrieb MRAB: > The regex module calls _PyUnicode_IsWhitespace, which is mapped by > unicodeobject.h to either _PyUnicodeUCS2_IsWhitespace or > _PyUnicodeUCS4_IsWhitespace. > > From Python 2.5 to Python 3.1 the library pythonXX.lib contains either > _PyUnicodeUCS2_IsWhitespace or _PyUnicodeUCS4_IsWhitespace. > > However, in Python 3.2b1 the library python32.lib contains only > _PyUnicode_IsWhitespace, therefore breaking the build. > > Is this change intentional? If so, why does unicodeobject.h still do > the mapping? Are you sure about this? It's not intentional (except in the limited ABI). Regards, Martin ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Can't compile regex module with Python 3.2
On Thu, Dec 9, 2010 at 12:47 AM, "Martin v. Löwis" wrote: .. >> However, in Python 3.2b1 the library python32.lib contains only >> _PyUnicode_IsWhitespace, therefore breaking the build. >> >> Is this change intentional? If so, why does unicodeobject.h still do >> the mapping? > > Are you sure about this? It's not intentional (except in the limited ABI). this does seem to be intentional: $ svn log -r 84177 r84177 | amaury.forgeotdarc | 2010-08-18 16:44:58 -0400 (Wed, 18 Aug 2010) | 9 lines #5127: Even on narrow unicode builds, the C functions that access the Unicode Database (Py_UNICODE_TOLOWER, Py_UNICODE_ISDECIMAL, and others) now accept and return characters from the full Unicode range (Py_UCS4). The differences from Python code are few: - unicodedata.numeric(), unicodedata.decimal() and unicodedata.digit() now return the correct value for large code points - repr() may consider more characters as printable. http://svn.python.org/view?view=rev&revision=84177 ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Using logging in the stdlib and its unit tests
On Thu, Dec 9, 2010 at 4:49 AM, Paul Moore wrote: > On 8 December 2010 14:52, Nick Coghlan wrote: >> As I see it, there aren't many cases at the *library* level where >> logging errors is more appropriate than raising exceptions: > > On a slightly tangential note, what do you think of the idea of > library code including info or debug level logging? In effect, tracing > and diagnostic code built in and available simply by changing the > logging level? It's a trade-off between performance and the ability to diagnose faults and what you want to do will vary by application and problem domain. My background is in working on distributed projects where we need enough diagnostic information to debug one-off failures of the live system, so I'm in favour of the idea in general, but doing it *can* carry a fairly hefty speed cost (you can ameliorate that to some degree by handing off the actual I/O to a separate thread, but the system would still be faster if the log messages weren't there). As Skip says, you definitely want to take advantage of the lazy string formatting when doing this with the Python logging module. Cheers, Nick. -- Nick Coghlan | [email protected] | Brisbane, Australia ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
