[issue17075] logging documentation for library cleanup
New submission from Aaron Sherman:
This documentation states that libraries can turn off logging by adding a
NullHandler:
http://docs.python.org/2/howto/logging.html#configuring-logging-for-a-library
This is not entirely true. It only holds true if the application which calls
the library has not called basicConfig on the root logger. If it has, you get
logs to both the root logger and the NullLogger, defeating the point of having
added a NullLogger in the first place.
The correct way for a library to silence log messages is to both set a
NullHandler and set propagate to false.
For an example of the behavior on the current docs, see:
import logging
import sys
# Application configures its root logger
logging.basicConfig(level=logging.DEBUG)
# Library configures a NullLogger:
logger = logging.getLogger('foo')
logger.setLevel(logging.DEBUG)
handler = logging.NullHandler()
handler.setLevel(logging.DEBUG)
logger.addHandler(handler)
# Library then logs:
logger.warning(BLAH)
This example is not terribly interesting, but the more interesting example is
when the library configures a real log handler (e.g. in order to create a
separate security log in an authorization module). In this case, the log
messages will be sent to both the file log and the root logger by default, as
long as any part of the application has configured the root logger.
IMHO, propagate should always be False for all new loggers, but at the very
least the fact that it is True should be documented in the section on library
logging...
--
assignee: docs@python
components: Documentation
messages: 180919
nosy: ajs, docs@python
priority: normal
severity: normal
status: open
title: logging documentation for library cleanup
type: behavior
versions: Python 2.7
___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue17075
___
___
Python-bugs-list mailing list
Unsubscribe:
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue11314] Subprocess suffers 40% process creation overhead penalty
Aaron Sherman a...@ajs.com added the comment: I think it's still safe to say that high performance applications which need to create many hundreds or thousands of children (e.g. large monitoring systems) will still need another solution that isn't subprocess. That being said, you're right that no one is going to care about the extra overhead of subprocess in a vacuum, and most applications fork one or a very small number of processes at a time and typically end up waiting for orders of magnitude more time for their output than they spend creating the process in the first place. As I said when I opened this issue, I wasn't terribly concerned with most applications. That being said, I can't really submit a full-blown monitoring system against this bug, so the best I could do would something that did lots of os.popens or subprocess.Popens in parallel in a contrived way and see how the performance scales as I tune lots to different values. Sadly, the time I have for doing that needs to be spent writing other code, so I'll leave this closed and let someone else raise the issue in the future if they run into it. I can always build a dispatcher in C and communicate with it via IPC to get around the immediate concern of scalability. -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue11314 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue11314] Subprocess suffers 40% process creation overhead penalty
Aaron Sherman a...@ajs.com added the comment: That's why I asked for absolute numbers for the overhead difference. Did you not follow the link in my first post? I got pretty detailed, there. os.popen just calls the popen(3) library call, which just performs a fork/execve and some dup/close in between. subprocess.Popen is implemented in Python, so it doesn't come as a surprise that it's slower in your example. Well, of course. I don't think I was ever trying to claim that os.popen vs. subprocess without a shell was going to compare favorably. I'm not advocating os.popen, here, I'm just trying to figure out where this massive overhead is coming from. I think the answer is just, pure Python is fundamentally slower, and that's not a surprise. Now, if the 3.x subprocess work that was described here, gets back-ported into 2.x and is included with future releases, that will definitely serve to improve the situation, and might well render much of this moot (testing will tell). However, I do think that doing the performance testing before deprecating the previous interface would have been a good idea... -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue11314 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue11314] Subprocess suffers 40% process creation overhead penalty
Aaron Sherman a...@ajs.com added the comment: Python 3.2 has a _posixsubprocess: some parts of subprocess are implemented in C. Can you try it? I don't have a Python 3 installation handy, but I can see what I can do tomorrow evening to get one set up and try it out. disagree with the idea that spawning exit 0 subprocesses is a performance critical operation ;) How else would you performance test process creation overhead? By introducing as little additional overhead as possible, it's possible for me to get fairly close to measuring just the subprocess module's overhead. If you stop to think about it, though, this is actually a shockingly huge percent increase. In any process creation scenario I'm familiar with, its overhead should be so small that you could bump it up several orders of magnitude and still not compete with executing a shell and asking it to do anything, even just exit. And yet, here we are. 40% I understand that most applications won't be running massive numbers of external commands in parallel, and that's the only way this overhead will really matter (at least that I can think of). But in the two scenarios I mentioned (monitoring and Web services such as CGI, neither of which is particularly rare), this is going to make quite a lot of difference, and if you're going to deprecate os.popen, I would think that making sure your proposed replacement was at least nearly as performant would be standard procedure, no? I think your analysis is wrong. These mmap() calls are for anonymous memory, most likely they are emitted by the libc's malloc() to get some memory from the kernel. In other words they will be blazingly fast. The mremap might be a bit of a performance hit, but it's true that these calls should not be substantially slowing execution... then again, they might indicate that there's substantial amounts of work being done for which memory allocation is required, and as such may simply be a symptom of the actual problem. -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue11314 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue8465] Backreferences vs. escapes: a silent failure solved
Aaron Sherman a...@ajs.com added the comment: Matthew, thank you for replying. I still think the primary issue is the potential for confusion between single digit escapes and backreferences, and the ease with which they could be addressed, but to cover what you said: Quote: the normal way to handle \41 + 1 is \0411 That might be the way dictated by the limitations of escape expansion as it is now, but it's entirely non-intuitive and seems more like the exciting edge cases (and obfuscated code opportunities) in other languages than something Python would be proud of. With \41\e1 you would actually be able to tell, visually that the 1 does not get read by the code which reads the \41. This seems to me to be a serious win for maintainability. -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue8465 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue8465] Backreferences vs. escapes: a silent failure solved
New submission from Aaron Sherman a...@ajs.com: I tested this under 2.6 and 3.1. Under both, the common mistake that I'm sure many others have made, and which cost me quite some time today was: re.sub(r'(foo)bar', '\1baz', 'foobar') It's obvious, I'm sure, to many reading this that the second r was left out before the replacement spec. It's probably obvious that this is going to happen quite a lot, and there are many edge cases which are equally baffling to the uninitiated (e.g. \8, \418 and \) In order to avoid this, I'd like to request that such usage be deprecated, leaving only numeric escapes of the form matched by r'\\[0-7][0-7][0-7]?(?!\d)' as valid, non-deprecated uses (e.g. \01 or \111 are fine). Let's look at what that would do: Right now, the standard library uses escape sequences with \n where n is a single digit in a handful of places like sndhdr.py and difflib.py. These are certainly not widespread enough to consider this a common usage, but certainly those few would have to change to add a leading zero before the digit. OK, so the specific requested feature is that \xxx produces a warning where xxx is: * any single digit or * any invalid sequence of two or three digits (e.g containing 8 or 9) or * any sequence of 4 or more digits ... guiding the user to the more explicit \01, \x01 or, if they intended a literal backslash, the r notation. If you wish to go a step further, I'd suggest adding a no-op escape \e such that: \41\e1 would print !1. Otherwise, there's no clean way to halt the interpretation of a digit-based escape sequence. -- components: Regular Expressions, Unicode messages: 103640 nosy: Aaron.Sherman severity: normal status: open title: Backreferences vs. escapes: a silent failure solved type: feature request versions: Python 2.6, Python 3.1 ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue8465 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue5555] optparse: clarify option concatenation in docs
Aaron Sherman a...@ajs.com added the comment: I'm closing this out, as the previous poster was correct: the module does the right thing, and I misread the documentation. Thanks! -- status: open - closed ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue5555] optparse
New submission from Aaron Sherman a...@ajs.com: First off, I want to be clear that this isn't a request for changes to functionality, nor for debate over decisions which have already been made. This is purely a request for correction to mis-statements about the nature and origins of optparse's handling in its documentation. This is an edited-down excerpt form the optparse documentation from: http://docs.python.org/library/optparse.html ... the traditional Unix syntax is a hyphen (“-“) followed by a single letter [...] Some other option syntaxes that the world has seen include: * a hyphen followed by a few letters, e.g. -pf [...] [...] These option syntaxes are not supported by optparse, and they never will be. This is deliberate: the first three are non-standard on any environment[...] While, obviously, optparse is free to choose whatever model of option parsing the developers like, the above text should be removed or corrected. Traditional Unix command-line usage is detailed in the POSIX specification's definition of various utilities and the optparse C function as documented here: http://www.opengroup.org/onlinepubs/009695399/functions/getopt.html which lays out this example: This code accepts any of the following as equivalent: cmd -ao arg path path cmd -a -o arg path path Note that the concatenation of single-character arguments is, in fact, in conformance to the POSIX standard, GNU coding conventions, and Unix best-practices since at least the mid-1980s. This clearly contradicts the statement from Python's documentation. For further reference, see: any Unix or Unix-like system's man 3 getopt http://www.faqs.org/docs/artu/ch10s05.html http://www.gnu.org/prep/standards/standards.html#Command_002dLine-Interfaces (which refers back to the POSIX guidelines for the command-line options of a program) any Unix or Unix-like system's man pages for a plethora of core utilities such as rm(1), ls(1), sh(1), cp(1), etc. A more accurate statement would be: optparse has chosen to implement a subset of the GNU coding standard's command line interface guidelines, allowing for both long and short options, but not the POSIX-style concatenation of short options. A rationale for that decision may or may not be included, but I won't presume to write it since I'm not actually privy to that decision-making process. -- assignee: georg.brandl components: Documentation messages: 84103 nosy: ajs, georg.brandl severity: normal status: open title: optparse type: behavior versions: Python 2.6, Python 3.0 ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
