[issue11564] pickle not 64-bit ready
Jorgen Skancke jor...@nt.ntnu.no added the comment: I recently ran into this problem when I tried to multiprocess jobs with large objects (3-4 GB). I have plenty of memory for this, but multiprocessing hangs without error, presumably because pickle hangs without error when multiprocessing tries to pickle the object. I can't offer a solution, but I can verify that the limitation in pickle is affecting Python usage. -- nosy: +jorgsk ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue11564 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue12636] IDLE ignores -*- coding -*- with -r option
ledave123 ledave...@yahoo.fr added the comment: The problem can be fixed with tokenize : I'm sorry I never submitted a path and I have no access to the source tree from here, if someone cares to do it, do not hesitate. def execfile(self, filename, source=None): Execute an existing file if source is None: import tokenize source = tokenize.open(filename).read() -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue12636 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue6721] Locks in python standard library should be sanitized on fork
Nir Aides n...@winpdb.org added the comment: Hi Gregory, Gregory P. Smith g...@krypto.org added the comment: No Python thread is ever fork safe because the Python interpreter itself can never be made fork safe. Nor should anyone try to make the interpreter itself safe. It is too complex and effectively impossible to guarantee. a) I think the term Guarantee is not meaningful here since the interpreter is probably too complex to guarantee it does not contain other serious problems. b) If no Python thread is ever fork safe, can you illustrate how a trivial Python thread spinning endlessly might deadlock a child forked by another Python thread? I was not able to find reports of deadlocks clearly related to multiprocessing worker threads so they could be practically safe already, to the point other Python-Dev developers would be inclined to bury this as a theoretical problem :) Anyway, there exists at least the problem of forking from the pool worker thread and possibly other issues, so the code should be reviewed. Another latent problem is multiprocessing logging which is disabled by default? There is no general solution to this, fork and threading is simply broken in POSIX and no amount of duct tape outside of the OS kernel can fix it. This is why we should sanitize the multithreading module and deprecate mixing of threading and multiprocessing. I bet most developers using Python are not even aware of this problem. We should make sure they are through documentation. Here is another way to look at the current situation: 1) Don't use threading for concurrency because of the GIL. 2) Don't mix threading with multiprocessing because threading and forking don't mix well. 3) Don't use multiprocessing because it can deadlock. We should make sure developers are aware of (2) and can use (3) safely***. My only desire is that we attempt to do the right thing when possible with the locks we know about within the standard library. Right, with an atfork() mechanism. -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue6721 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue12636] IDLE ignores -*- coding -*- with -r option
STINNER Victor victor.stin...@haypocalc.com added the comment: Yes, tokenize.open() should fix this issue, but you should close the file after using it. Use for example with tokenize.open(): Can you write a patch? You can download the source code using Mercurial or download it manually from http://hg.python.org/cpython/file/tip See also our http://docs.python.org/devguide/ -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue12636 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue12648] Wrong import module search order on Windows
New submission from kota nospam.kotarou.d...@gmail.com: There seems to be a wrong import module search order (http://docs.python.org/tutorial/modules.html#the-module-search-path) on Windows. Python seems to be loading the built-in module instead of the python code with the same name as the module in the current directory. This only happens on Windows as I tested on Linux and it loaded the module properly. Steps to reproduce: 1. Create a file named `parser.py' containing `print test' 2. Open a console window to the directory you created the file in and run `python2.7 -v' or `python -v' 3. Type `import parser' On Windows, I get this output: import encodings.cp437 # from c:\Python27\lib\encodings\cp437.py # wrote c:\Python27\lib\encodings\cp437.pyc import parser # builtin On Linux, I get this: import parser # from parser.py # wrote parser.pyc test `sys.path' on Windows: ['', 'C:\\WINDOWS\\system32\\python27.zip', 'c:\\Python27\\DLLs', 'c:\\Python27\\lib', 'c:\\Python27\\lib\\plat-win', 'c:\\Python27\\lib\\lib-tk', 'c:\\Python27', 'c:\\Python27\ \lib\\site-packages'] `sys.path' on Linux: ['', '/usr/lib/python2.7', '/usr/lib/python2.7/plat-linux2', '/usr/lib/python2.7/lib-tk', '/usr/lib/python2.7/lib-old', '/usr/lib/python2.7/lib-dynload', '/usr/local/lib/python2.7/dist-packages', '/usr/lib/python2.7/dist-packages', '/usr/lib/python2.7/dist-packages/PIL', '/usr/lib/pymodules/python2.7/gtk-2.0', '/usr/lib/python2.7/dist-packages/gst-0.10', '/usr/lib/python2.7/dist-packages/gtk-2.0', '/usr/lib/pymodules/python2.7'] -- components: None messages: 141288 nosy: kota priority: normal severity: normal status: open title: Wrong import module search order on Windows type: behavior versions: Python 2.7 ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue12648 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue3605] Py_FatalError causes infinite loop
Neil Aspinall m...@neilaspinall.co.uk added the comment: Would it be possible for this issue's fix (PyErr_Occurred() returning null when the thread state is null) to be applied to the 2.7 branch? -- nosy: +naspinal ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue3605 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue3605] Py_FatalError causes infinite loop
Changes by Ralf Schmitt python-b...@systemexit.de: -- nosy: +schmir ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue3605 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue12649] email.Header corupts international email header and ignores maxlinelen in some cases
New submission from dandre andr...@gmx.net: Hello there, first of all, thank you all for Python. I didn't know it was so great; otherwise I'd have checked it out before. Using 2.7.2 MSC v.1500 32 Intel bit for now. Playing with email.header, I came across an odd behaviour. Attached please find a script which demonstrates that 1) maxlinelen is ignored and 2) header fields are split in a manner not suitable for all systems involved in email processing. The script will print the headers. They're all the same and extend over two lines; both should probably not be the case, although it dosn't hurt in itself. If you uncomment the SMTP part of the script and send that email to yourself, you'll probably see that the From: and To: header will be misinterpreted by your email client; I tested this with two different email providers. Looking at the raw data which are received, it appears that at least in one case, a system along the way added a comma between the two To: lines. This is something which one should easily be able to avoid, if only the maxlinelen would be obeyed... Having taken a look at email.header.py, it appears to me that the semantics of _encode_chunks() does not exactly match its documentation due to the results of (at least some) charset.header_encode() calls. What seems to happen is that charset.header_encode() can return several lines already, and it will apparently split the line without any deeper knowledge. As a result, the Header module will not apply its sophisticated maxlinelen/splitchars logic. The header is split at some pretty arbitrary point and not all systems appear to be happy with that, although the relevant RFC apparently only reads SHOULD in this regard. -- components: Library (Lib) files: email_header_bug.py messages: 141290 nosy: dandre priority: normal severity: normal status: open title: email.Header corupts international email header and ignores maxlinelen in some cases type: behavior versions: Python 2.7 Added file: http://bugs.python.org/file22783/email_header_bug.py ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue12649 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue670664] HTMLParser.py - more robust SCRIPT tag parsing
Éric Araujo mer...@netwok.org added the comment: HTML5 being a spec that builds on HTML 4.01 and real-world ways to deal with non-compliant input, I don’t object to fixes that follow the HTML5 spec. Regarding backward compatibility, we can break it if we decide that the behavior we’re changing was a bug. I think it’s far more useful to support BeautifoulSoup than to retain a non-useful behavior forever. -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue670664 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue10639] reindent.py should not convert newlines
Changes by Éric Araujo mer...@netwok.org: -- stage: commit review - committed/rejected ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue10639 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue9968] Let cgi.FieldStorage have named uploaded file
Éric Araujo mer...@netwok.org added the comment: where does the 1ko barrier come from? Was it only chosen out of performance considerations [...] Most certainly. I’ll look at the history of the file later to try to find the developer who decided that. Guido van Rossum made the changes. Before that a temporary file was created for every form field. Do you have the changeset ID? A last question [...] For a new feature, it’s okay to change signatures. Backward compatibility is preserved by appending the new argument at the end of the arguments lists, and making it optional. Given that tempfile respects the TMP and TMPDIR environment variables, do you think it would be possible for users to control the download dir for uploaded files by editing os.environ? This would require no change to cgi. If that’s not possible, then we’ll have to change the FieldStorage class. Also, I believe one can think of other reasons to give this freedom. Can you think about some of them? def __init__(self, tempdir=tempfile.gettempdir()): or a somewhat more convoluted form that would avoid importing tempfile needlessly. You could probably have a None argument that would be passed down to tempfile. -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue9968 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue12649] email.Header ignores maxlinelen when wrapping encoded words
R. David Murray rdmur...@bitdance.com added the comment: You are using Header incorrectly. It should look more like this: th = _e_header.Header(maxlinelen=200, header_name='To') th.append(tfc[:-1]) th.append(wtc[:-1], charset='utf-8') th.append(tem) This results in: To: ABCDEFGH =?utf-8?b?0ILYgeC5hOC8kuGPiuGauw==?= abcd...@hijk-lmnopqr.st Which is valid per RFC, which encoding the address is not. A compliant mailer should be able to handle the Subject line from your version correctly, but not the To or From lines. The fact that you don't want the trailing spaces is an artifact of the API. Using this API requires more knowledge of the RFCs than anyone should want to have. In Python 3.3 we will be introducing a new API in the email package that will make all of this *much* simpler. The maxlinelen issue does appear to be a bug, though. -- keywords: +easy nosy: +r.david.murray title: email.Header corupts international email header and ignores maxlinelen in some cases - email.Header ignores maxlinelen when wrapping encoded words versions: +Python 3.2, Python 3.3 ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue12649 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue10968] threading.Timer should be a class so that it can be derived
Éric Araujo mer...@netwok.org added the comment: I’ve committed the cleanup to my 3.3 clone and will push this week. Here’s a doc patch. Before my patch, the various classes were documented in two parts: one entry with the factory function (e.g. Thread), without index reference, and one section (e.g. “Thread Objects”), which used a class directive (and thus index target) for most but not all classes. After my patch, all classes are documented with a class directive, in their section (i.e. “Thread Objects”), with a versionchanged note informing that the name used to be that of a factory function. The only remaining glitch is that the “X Objects” sections start with a description of the class’ use, which references methods with constructs like :meth:`run`, which cannot be turned into links as Sphinx lacks context: the class directive only happens after. I could move the class directives right after the heading (“X Objects”), so that the meth roles get turned into links. -- assignee: - eric.araujo Added file: http://bugs.python.org/file22784/threading-3.3-doc.diff ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue10968 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue12649] email.Header ignores maxlinelen when wrapping encoded words
Changes by R. David Murray rdmur...@bitdance.com: -- assignee: - r.david.murray ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue12649 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue12650] Subprocess leaks fd upon kill()
New submission from gabriele.trombetti g.trombe...@plasmacore.com: There seems to be a file descriptor (fd) leak in subprocess upon call to kill() and then destroying the current subprocess object (e.g. by scope exit) if the pipe functionality is being used. This is a reproducer: (Linux 2.6.25, Python 2.7.1 (r271:86832)) import subprocess, time def leaktest(): subp = subprocess.Popen(echo hi; sleep 200; echo bye, shell=True, stdout=subprocess.PIPE, stderr=subprocess.PIPE) #you can do something here subp.terminate() #subp.wait() #Fixes the bug #time.sleep(0.001) #fixes the bug #time.sleep(0.001) #doesn't fix the bug return True Launch the function multiple times interactively, each time the number of open file descriptors for the python process will increase by 2. You can see that by ls -l /proc/pid/fd This seems to be a race condition because adding a time.sleep(0.001) before the return fixes the problem. Probably some kernel delay is responsible for the race. This bug is significant for daemons because the daemon will die once the number of open file descriptors hits the ulimit, usually 1024, so please fix. Note: until the bug is fixed, a simple workaround (not documented in module docs though) is to call Popen.wait() after Popen.kill() Thank you -- components: Library (Lib) messages: 141295 nosy: gabriele.trombetti priority: normal severity: normal status: open title: Subprocess leaks fd upon kill() type: behavior versions: Python 2.7 ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue12650 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue12641] Remove -mno-cygwin from distutils
Éric Araujo mer...@netwok.org added the comment: May I ask for a reconsideration to commit a fix for this for Python 2.7 at least? With the version check it doesn't hurt anyone There’s a misunderstanding: I explained why 2.5, 2.6 and 3.1 can’t be fixed, but if you look at the versions at the top of this page, you’ll see 2.7, 3.2 and 3.3 listed. This will get fixed in these versions. Ah, our good friend grep. Thanks for the exploration. This looks complicated. should the question be what's the first mingw gcc version that -mno-cygwin usage unnecessary rather than finding the first version the option was removed? If removing the option in these versions causes no change (i.e. doesn’t break any code), I agree. and, does it matter whether you're building on win for win, or cross compiling for win from nix? I’m afraid I don’t know enough about Windows and MinGW to answer that. If we can’t be sure about versions and consequences here, I’ll go to the MinGW ML. btw, can distutils do a lightweight configure-like check for feature presence rather than going a gcc version check? There is a config command that’s not very used, but it would be IMO overkill to use it in the compiler code. -- nosy: +loewis ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue12641 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue12650] Subprocess leaks fd upon kill()
Changes by R. David Murray rdmur...@bitdance.com: -- nosy: +gregory.p.smith ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue12650 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue670664] HTMLParser.py - more robust SCRIPT tag parsing
R. David Murray rdmur...@bitdance.com added the comment: Unless someone else has picked it up, BeautifulSoup is a no longer an issue since its author has abandoned it. That doesn't change the fact that IMO it would be nice for our library to handle input generously. -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue670664 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue12649] email.Header ignores maxlinelen when wrapping encoded words
dandre andr...@gmx.net added the comment: Thank you for pointing out my wrong usage of Header. Does this mean I should call Header.append() for each token, with tokens being separated by WS, or probably rather COMMASPACE in the case of To:? Or does it mean I should call Header.append() for each logical token of From: and To:, let's say, for the two parts returned by email.utils.parseaddr()? Please excuse me if this is not the right place to discuss this, but I'm unaware of any place on the Web wehre these questions are addressed. -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue12649 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue10968] threading.Timer should be a class so that it can be derived
R. David Murray rdmur...@bitdance.com added the comment: You ought to be able to use either a context directive (I forget its name and syntax), or the full reference syntax (:meth:`~threading.Thread.run`) to make those links work without moving things around. -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue10968 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue10968] threading.Timer should be a class so that it can be derived
Éric Araujo mer...@netwok.org added the comment: You ought to be able to use either a context directive (I forget its name and syntax), Hm, do you mean something similar to currentmodule? or the full reference syntax (:meth:`~threading.Thread.run`) to make those links work without moving things around. Yes, Thread.run would work, but I find that inelegant. -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue10968 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue12641] Remove -mno-cygwin from distutils
Jon jon.for...@gmail.com added the comment: and, does it matter whether you're building on win for win, or cross compiling for win from nix? I’m afraid I don’t know enough about Windows and MinGW to answer that. If we can’t be sure about versions and consequences here, I’ll go to the MinGW ML. Ruben's on it :) http://sourceforge.net/mailarchive/message.php?msg_id=27864860 -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue12641 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue12649] email.Header ignores maxlinelen when wrapping encoded words
R. David Murray rdmur...@bitdance.com added the comment: They probably ought to be discussed in our docs :( The only thing that may be encoded in an address is the display name (the first part returned by parseaddr). (Actually the domain name could be IDNA encoded, but we don't support that directly in email. 3.3 will.) So the easiest way to code this is probably to take a list of parseaddr'ed addresses, and append display_name as utf-8, and the address formatted as '%s,' as ascii. Omitting the comma for the last one, of course. Not very elegant, but I believe it should work. If you want to get fancy you can split out the domain and run it through the IDNA codec to encode it before passing it in as part of the ASCII token. Header puts spaces between ASCII and non-ASCII tokens automatically, so you don't have to add them to either the encoded or unencoded tokens. -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue12649 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue12649] email.Header ignores maxlinelen when wrapping encoded words
dandre andr...@gmx.net added the comment: Thanks again for the clarification. I may not look like it ;), but my fanciness has to go even further. So, for the sake of completeness, it appears that the world is now up to UTF-8 local parts of email addresses, and punycode for the domain? https://bugzilla.mozilla.org/show_bug.cgi?id=614930 But then there's RFC 5335 which seems to go further, although, frankly speaking, I'd love to see examples in RFCs every now and then, and it sounds like it's not exactly supported by too many mailers along the way. http://tools.ietf.org/html/rfc5335 Either way, if the Mozilla example is something to live up to, I hope I'll be allowed to have WS between a non-UTF-8 '', the UTF-8 local part and the '@', because email.Headers will always create that, right? Is there a place I can register such a fancy email address (AND understand the website and webmailer's UI) for testing purposes? Hats off to you who is dealing with these ugly compromises, keeping an outdated underlying standard on life support. -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue12649 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue11439] subversion keyword breakage
Roundup Robot devn...@psf.upfronthosting.co.za added the comment: New changeset f15442543e24 by Senthil Kumaran in branch '2.7': Fix closes Issue11439 - Handle the SVN Keywords in 2.7 by replacing them with a high number so that code relying on them does not break. http://hg.python.org/cpython/rev/f15442543e24 New changeset 3e26c9033306 by Senthil Kumaran in branch '3.2': Fix closes Issue11439 Remove the SVN keywords from the code as it is no longer applicable in hg. Patch Contributed by Neil Muller. http://hg.python.org/cpython/rev/3e26c9033306 New changeset 6b9f0a6efaeb by Senthil Kumaran in branch 'default': merge from 3.2 - Fix closes Issue11439 Remove the SVN keywords from the code as it is no longer applicable in hg. Patch Contributed by Neil Muller. http://hg.python.org/cpython/rev/6b9f0a6efaeb -- nosy: +python-dev resolution: - fixed stage: needs patch - committed/rejected status: open - closed ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue11439 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue11439] subversion keyword breakage
Senthil Kumaran sent...@uthcode.com added the comment: Thanks for the patch, Neil. -- nosy: +orsenthil ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue11439 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue12649] email.Header ignores maxlinelen when wrapping encoded words
R. David Murray rdmur...@bitdance.com added the comment: Interesting thread. I have my eye on supporting 5335 in the revised email package, but it is secondary goal to getting an improved API for the already-accepted RFCs. You will note that the encoded word local part is *not* standard. I think the email package may decode them anyway, but just like TB it provides no mechanism for creating them in the first place since it is not RFC compliant. You could open a feature request for adding support for doing so (as an *optional* feature :), which I would then try to get in to 3.3. (It puzzles me why it *isn't* allowed by the RFC, by the way). To do it yourself now, you will probably have to create a temporary Header, pass it just the local part, call its encode or __str__ to get the encoded word (which won't have any spaces since it will be the only token), and then format that in to your rebuilt address string. -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue12649 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue12650] Subprocess leaks fd upon kill()
Charles-François Natali neolo...@free.fr added the comment: There's indeed a leak in your code, because you don't close the stdout and stderr file descriptors. Try with: subp.terminate() subp.stdout.close() subp.stderr.close() return True And you'll be just fine. The reason why this works with a short delay is subtle however, and there's actually a bug: - when a subprocess.Popen is created, its stdout and stderr are wrapped with os.fdopen() - file objects returned by fdopen() are automatically closed when the object is deallocated (i.e. when the file object goes out of scope in cPython) - so in theory, when your subprocess.Popen goes out of scope, the corresponding FDs should be closed, and you shoudn't have a leak (note that this assumes cPython, and will maybe not work with other implementations - you shouldn't rely on this) - the problem is that when subprocess.Popen's __del__ method is called before the process has exited (i.e. if you return from leaktest() before the process has exited), the Popen object is added to a list (subprocess._active), so that it can be waited (to avoid zombies) - Popen objects from this list are collected (i.e. the process is waited for, and if terminated it's removed from the list) synchronously when a new Popen() object is created (_cleanup funtion) The problem is that there's a bug in the collection code: def _cleanup(): for inst in _active[:]: res = inst._internal_poll(_deadstate=sys.maxint) if res is not None and res = 0: try: _active.remove(inst) except ValueError: # This can happen if two threads create a new Popen instance. # It's harmless that it was already removed, so ignore. pass res = inst._internal_poll(_deadstate=sys.maxint) if res is not None and res = 0: If the process exit code is negative (in your case, -SIGTERM) then the Popen object is not removed from the list. Two consequences: - the object is not deallocated, the corresponding FDs neither, and you hit RLIMIT_NOFILE - even if stdout and stderr are closed manually, the object itself is not deallocated, so you have an ever-growing _active list, and a memory leak (and walking the list takes longer and longer) Changing if res is not None and res = 0: to if res is not None: fixes this. I'll look at this in more detail and provide a patch. -- nosy: +neologix ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue12650 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue12647] Add __bool__ to None
Changes by Daniel Urban urban.dani...@gmail.com: -- nosy: +durban ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue12647 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue12650] Subprocess leaks fd upon kill()
Changes by STINNER Victor victor.stin...@haypocalc.com: -- nosy: +haypo ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue12650 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue12650] Subprocess leaks fd upon kill()
Changes by Ross Lagerwall rosslagerw...@gmail.com: -- nosy: +rosslagerwall ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue12650 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue12647] Add __bool__ to None
Roundup Robot devn...@psf.upfronthosting.co.za added the comment: New changeset ccce01988603 by Raymond Hettinger in branch 'default': Issue 12647: Add __bool__() method to the None object. http://hg.python.org/cpython/rev/ccce01988603 -- nosy: +python-dev ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue12647 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue12647] Add __bool__ to None
Changes by Raymond Hettinger raymond.hettin...@gmail.com: -- resolution: - fixed status: open - closed ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue12647 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue12595] python.h redundant redeclarations
Petri Lehtinen pe...@digip.org added the comment: Barry Warsaw wrote: I don't feel comfortable changing what is defined in a header file in a point release, so I am not going to backport the fix. Closing as wont fix. -- keywords: +needs review resolution: - wont fix stage: needs patch - patch review status: open - closed ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue12595 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue12611] 2to3 crashes when converting doctest using reduce()
Petri Lehtinen pe...@digip.org added the comment: I cannot reproduce the crash on neither 2.7 nor 3.2. Can you provide more details; attach the exact python source file that crashes 2to3 and give the complete crashing 2to3 command that you're running. -- nosy: +petri.lehtinen status: open - pending ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue12611 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue11699] Doc for optparse.OptionParser.get_option_group is wrong
Petri Lehtinen pe...@digip.org added the comment: Attached a patch for 2.7. -- keywords: +easy, needs review, patch nosy: +petri.lehtinen versions: -Python 3.1 Added file: http://bugs.python.org/file22785/issue11699.patch ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue11699 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue11699] Doc for optparse.OptionParser.get_option_group is wrong
Changes by Petri Lehtinen pe...@digip.org: -- stage: - patch review ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue11699 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue12649] email.Header ignores maxlinelen when wrapping encoded words
dandre andr...@gmx.net added the comment: I made a test and, interestingly, I /can/ send an email to myself setting up the header like this: h.append(b'My Name', charset='utf-8') h.append(b' ', charset='us-ascii') h.append(b'my', charset='utf-8') h.append(b'@email.address', charset='us-ascii') The message in my Inbox will then have a To: header along the lines of =?utf-8?q?My Name?= =?utf-8?q?my?=@email.address so the mailers are sure nice to me. The startling part of it all seems to be that such email addresses are already out there and seem to be supported by enough mailers, albeit not by enough client-side systems. With this non-standard approach and RFC 5335, I feel tempted to hope for a helper method which finds the (an) canonical form of an email address... -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue12649 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue12649] email.Header ignores maxlinelen when wrapping encoded words
dandre andr...@gmx.net added the comment: Erm, sorry. The header, of course, does not have much to do with the address the email is to be delivered to. With my provider's setup, the mailer will reply that =?utf-8?q?my?= is not a known user. Which could change, of course... -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue12649 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue12649] email.Header ignores maxlinelen when wrapping encoded words
R. David Murray rdmur...@bitdance.com added the comment: Yes, exactly. It is a valid ascii token so MTA's pass it through. It's the receiving system that needs to be willing to decode it. -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue12649 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue12619] Automatically regenerate platform-specific modules
Changes by Jesus Rivero neurog...@gentoo.org: -- nosy: +Neurogeek ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue12619 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue11657] multiprocessing_{send,recv}fd fail with fds 256
Petri Lehtinen pe...@digip.org added the comment: Attached a patch for v2.7. It changes the assignments to memcpy() calls. -- keywords: +easy, needs review, patch nosy: +petri.lehtinen stage: - patch review versions: -Python 3.1 Added file: http://bugs.python.org/file22786/issue11657.patch ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue11657 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue11657] multiprocessing_{send,recv}fd fail with fds 256
Jean-Paul Calderone invalid@example.invalid added the comment: Thanks for the patch Petri. Are you interested in writing a unit test for this as well? -- nosy: +exarkun ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue11657 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue12650] Subprocess leaks fd upon kill()
Charles-François Natali neolo...@free.fr added the comment: Alright. I tested this on default, and couldn't reproduce the FD leak. It turned out to be due to another bug, affecting only the code path which calls pure C _posixsubprocess (which is the only implementation left in 3.3, but 3.2 still has the old pure-Python version). The code just forgets to set Popen._child_created to true after fork(), so when Popen.__del__() gets called before the process has exited, the object is not added to the _active list, and gets deallocated immediately. While this accidentaly fixes the FD leak, this has another - worse - side effet: the process remains a zombie. I'm thus attaching three patches, with tests: - one for 2.7, which fixes the original problem (i.e. remove the process from _active once exited, even if it got killed by a signal) - one for default, which also sets _child_created to True after fork() - another one for 3.2, which does the same thing as the one for default (but the code is a little different because 3.2 has both pure-Python and C implementation) Reviews welcome! -- keywords: +patch Added file: http://bugs.python.org/file22787/issue_12650_default.diff Added file: http://bugs.python.org/file22788/issue_12650_27.diff Added file: http://bugs.python.org/file22789/issue_12650_32.diff ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue12650 ___diff -r ccce01988603 Lib/subprocess.py --- a/Lib/subprocess.py Thu Jul 28 09:55:13 2011 -0700 +++ b/Lib/subprocess.py Thu Jul 28 22:51:39 2011 +0200 @@ -424,12 +424,16 @@ except: MAXFD = 256 +# This lists holds Popen instances for which the underlying process had not +# exited at the time its __del__ method got called: those processes are wait()ed +# for synchronously from _cleanup() when a new Popen object is created, to avoid +# zombie processes. _active = [] def _cleanup(): for inst in _active[:]: res = inst._internal_poll(_deadstate=sys.maxsize) -if res is not None and res = 0: +if res is not None: try: _active.remove(inst) except ValueError: @@ -1272,6 +1276,7 @@ errread, errwrite, errpipe_read, errpipe_write, restore_signals, start_new_session, preexec_fn) +self._child_created = True finally: # be sure the FD is closed no matter what os.close(errpipe_write) diff -r ccce01988603 Lib/test/test_subprocess.py --- a/Lib/test/test_subprocess.py Thu Jul 28 09:55:13 2011 -0700 +++ b/Lib/test/test_subprocess.py Thu Jul 28 22:51:39 2011 +0200 @@ -1622,6 +1622,63 @@ if dir is not None: os.rmdir(dir) +def test_zombie_fast_process_del(self): +# Issue #12650: on Unix, if Popen.__del__() was called before the +# process exited, it wouldn't be added to subprocess._active, and would +# remain a zombie. +# spawn a Popen, and delete its reference before it exits +p = subprocess.Popen([sys.executable, -c, + 'import sys, time;' + 'time.sleep(0.2)'], + stdout=subprocess.PIPE, + stderr=subprocess.PIPE) +ident = id(p) +pid = p.pid +del p +# check that p is in the active processes list +self.assertIn(ident, [id(o) for o in subprocess._active]) + +# sleep a little to let the process exit, and create a new Popen: this +# should trigger the wait() of p +time.sleep(1) +with self.assertRaises(EnvironmentError) as c: +with subprocess.Popen(['nonexisting_i_hope'], + stdout=subprocess.PIPE, + stderr=subprocess.PIPE) as proc: +pass +# p should have been wait()ed on, and removed from the _active list +self.assertRaises(OSError, os.waitpid, pid, 0) +self.assertNotIn(ident, [id(o) for o in subprocess._active]) + +def test_leak_fast_process_del_killed(self): +# Issue #12650: on Unix, if Popen.__del__() was called before the +# process exited, and the process got killed by a signal, it would never +# be removed from subprocess._active, which triggered a FD and memory +# leak. +# spawn a Popen, delete its reference and kill it +p = subprocess.Popen([sys.executable, -c, + 'import time;' + 'time.sleep(3)'], + stdout=subprocess.PIPE, + stderr=subprocess.PIPE) +ident = id(p) +pid = p.pid +del p +os.kill(pid, signal.SIGKILL) +# check that p is in the active
[issue12651] -O2 or -O3? this brings excitement to computing!
New submission from Pedro Larroy pedro.lar...@gmail.com: When I build it compiles with both -O2 and -O3... this is debian testing on amd64. gcc -pthread -c -fno-strict-aliasing -g -O2 -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -I. -IInclude -I./Include -DPy_BUILD_CORE -o Python/getcompiler.o Python/getcompiler.c -- components: Build messages: 141318 nosy: Pedro.Larroy priority: normal severity: normal status: open title: -O2 or -O3? this brings excitement to computing! versions: Python 2.7 ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue12651 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue12651] -O2 or -O3? this brings excitement to computing!
Antoine Pitrou pit...@free.fr added the comment: AFAICT the last one (-O3) is used by the compiler and that's the desired effect (we want to compile with the highest optimization level). -- nosy: +pitrou ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue12651 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue12611] 2to3 crashes when converting doctest using reduce()
Aaron Meurer asmeu...@gmail.com added the comment: Vladimir will need to confirm how to reproduce this exactly, but here is corresponding SymPy issue: http://code.google.com/p/sympy/issues/detail?id=2605. The problem is with the sympy/ntheory/factor_.py file at https://github.com/sympy/sympy/blob/sympy-0.7.1.rc1/sympy/ntheory/factor_.py#L453 (linking to the file from our release candidate, as a workaround is likely to be pushed to master soon). Vladimir, can you confirm that this particular version of the file reproduces the problem? -- status: pending - open ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue12611 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue12651] -O2 or -O3? this brings excitement to computing!
Benjamin Peterson benja...@python.org added the comment: Fixed in default. It's harmless in other versions. -- nosy: +benjamin.peterson resolution: - wont fix status: open - closed ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue12651 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue1813] Codec lookup failing under turkish locale
Stefan Krah stefan-use...@bytereef.org added the comment: Fedora bug report: https://bugzilla.redhat.com/show_bug.cgi?id=726536 -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue1813 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue11472] upload command fails to read auth information from .pypirc
Catherine Devlin fredv8vi...@liquidid.net added the comment: This blog post http://justcramer.com/2009/04/02/problems-uploading-packages-with-setuptools-on-os-x/ describes working around the same problem by replacing [pypi] in .pypirc with [server-login]. Is that the problem, a change in .pypirc formats? -- nosy: +catherine, catherinedevlin ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue11472 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue11472] upload command fails to read auth information from .pypirc
Jason R. Coombs jar...@jaraco.com added the comment: I don't believe that's the issue. The [server-login] is for Python 2.5 and earlier. Also note that I already had [server-login] in the .pypirc when the error occurred, so I don't think that's a factor either. This same .pypirc works just fine on Python 2.5, 2.6, and 2.7, so I believe the regression is somewhere in the Python 3 line. -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue11472 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue10639] reindent.py should not convert newlines
Eli Bendersky eli...@gmail.com added the comment:
It appears this breaks make patchcheck in trunk:
./python ./Tools/scripts/patchcheck.py
Getting the list of files that have been added/changed ... 5 files
Fixing whitespace ... Traceback (most recent call last):
File ./Tools/scripts/patchcheck.py, line 147, in module
main()
File ./Tools/scripts/patchcheck.py, line 129, in main
normalize_whitespace(python_files)
File ./Tools/scripts/patchcheck.py, line 22, in call_fxn
result = fxn(*args, **kwargs)
File ./Tools/scripts/patchcheck.py, line 66, in normalize_whitespace
fixed = [path for path in file_paths if path.endswith('.py') and
File ./Tools/scripts/patchcheck.py, line 67, in listcomp
reindent.check(path)]
File /home/eliben/python-src/33/Tools/scripts/reindent.py, line 129, in
check
newline = spec_newline if spec_newline else r.newlines
NameError: global name 'spec_newline' is not defined
make: *** [patchcheck] Error 1
--
nosy: +eli.bendersky
status: closed - open
___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue10639
___
___
Python-bugs-list mailing list
Unsubscribe:
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue12380] bytearray methods center, ljust, rjust don't accept a bytearray as the fill character
Roundup Robot devn...@psf.upfronthosting.co.za added the comment: New changeset 536fccc75f5a by Eli Bendersky in branch 'default': Issue #12380: PyArg_ParseTuple now accepts a bytearray for the 'c' format. http://hg.python.org/cpython/rev/536fccc75f5a -- nosy: +python-dev ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue12380 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue12380] bytearray methods center, ljust, rjust don't accept a bytearray as the fill character
Eli Bendersky eli...@gmail.com added the comment: Petri, thanks for the patch. I've updated Misc/NEWS and committed it. Unless there are objections or problems, I will close this issue in a day or two. -- resolution: - fixed stage: patch review - committed/rejected ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue12380 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue10639] reindent.py should not convert newlines
Changes by Eli Bendersky eli...@gmail.com: -- priority: normal - high stage: committed/rejected - needs patch ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue10639 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue10639] reindent.py should not convert newlines
Eli Bendersky eli...@gmail.com added the comment: This is a good example of why passing parameters into functions by means of globals sucks. In reindent.py, main() sets the spec_newline global which check() uses, but this was forgotten in patchcheck.py which also uses reindent.check() IMHO reindent.py should be rewritten to ditch all globals -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue10639 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue12650] Subprocess leaks fd upon kill()
Ross Lagerwall rosslagerw...@gmail.com added the comment: The patches look good and seem to fix the issue. -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue12650 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
