[issue4561] Optimize new io library

2008-12-16 Thread Amaury Forgeot d'Arc
Amaury Forgeot d'Arc amaur...@gmail.com added the comment: The previous implementation only returns bytes and does not translate newlines. For this particular case, indeed, the plain old FILE* based object is faster. -- nosy: +amaury.forgeotdarc ___

[issue3439] create a numbits() method for int and long types

2008-12-16 Thread Raymond Hettinger
Raymond Hettinger rhettin...@users.sourceforge.net added the comment: In the whatsnew docs, add two lines showing the binary representation of 37. That will provide clarification as to why the answer is six: n = 37 + bin(37) + '0b100101' n.bit_length() 6 Also, the

[issue4674] test_normalization failures on some buildbot

2008-12-16 Thread Antoine Pitrou
New submission from Antoine Pitrou pit...@free.fr: test_normalization intermittently fails on some buildbots with the following message: Traceback (most recent call last): File ./Lib/test/regrtest.py, line 596, in runtest_inner the_package = __import__(abstest, globals(), locals(), [])

[issue3439] create a numbits() method for int and long types

2008-12-16 Thread Antoine Pitrou
Antoine Pitrou pit...@free.fr added the comment: Le mardi 16 décembre 2008 à 13:56 +, STINNER Victor a écrit : (16).numbits() is 4, not 5. Well, I do hope (16).numbits() returns 5... ___ Python tracker rep...@bugs.python.org

[issue4674] test_normalization failures on some buildbot

2008-12-16 Thread Antoine Pitrou
Antoine Pitrou pit...@free.fr added the comment: Should be fixed (r67814, r67815). -- resolution: - fixed status: open - closed ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue4674 ___

[issue4561] Optimize new io library

2008-12-16 Thread Antoine Pitrou
Antoine Pitrou pit...@free.fr added the comment: I know that as hard as it might be for everyone to believe, there are a lot of people who crank lots of non- Unicode data with Python. But cranking data implies you'll do something useful with it, and therefore spend CPU time doing those

[issue4561] Optimize new io library

2008-12-16 Thread David M. Beazley
David M. Beazley beaz...@users.sourceforge.net added the comment: I wish I shared your optimism about this, but I don't. Here's a short explanation why. The problem of I/O and the associated interface between hardware, the operating system kernel, and user applications is one of the most

[issue4561] Optimize new io library

2008-12-16 Thread Antoine Pitrou
Antoine Pitrou pit...@free.fr added the comment: I seem to recall one of the design principles of the new IO stack was to avoid relying on the C stdlib's buffered API, which has too many platform-dependant behaviours. In any case, binary reading has acceptable performance in py3k (although

[issue4653] Patch to fix typos for Py3K

2008-12-16 Thread Mark Dickinson
Mark Dickinson dicki...@gmail.com added the comment: It looks to me as though all these are valid, and the patch should be applied to py3k and the 3.0 maintenance branch. Minor nit: the sizeof(theInfo) line exceeds 79 characters, in violation of PEP 7. Is there any core developer who's in a

[issue4501] asyncore's urgent data management and connection closed events are broken when using poll()

2008-12-16 Thread Giampaolo Rodola'
Giampaolo Rodola' billiej...@users.sourceforge.net added the comment: IMHO it would be good if this could go in the latest 2.4 and 2.5 upcoming releases while we still have the chance. ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue4501

[issue4561] Optimize new io library

2008-12-16 Thread Antoine Pitrou
Antoine Pitrou pit...@free.fr added the comment: I don't agree that that was a worthy design goal. I don't necessarily agree either, but it's probably too late now. The py3k buffered IO object has additional methods (e.g. peek(), read1()) which can be used by upper layers (text IO) and so

[issue4561] Optimize new io library

2008-12-16 Thread David M. Beazley
David M. Beazley beaz...@users.sourceforge.net added the comment: Good luck with that. Most people who get bright ideas such as gee, maybe I'll write my own version of X where X is some part of the standard C library pertaining to I/O, end up fighting a losing battle. Of course, I'd love

[issue3439] create a numbits() method for int and long types

2008-12-16 Thread Fredrik Johansson
Fredrik Johansson fredrik.johans...@gmail.com added the comment: When did the name change back to numbits? Anyway, I think this reference implementation is better: def numbits(x): x = abs(x) n = 0 while x: n += 1 x //= 2 return n (//= 2 could be changed to = 1)

[issue4561] Optimize new io library

2008-12-16 Thread David M. Beazley
David M. Beazley beaz...@users.sourceforge.net added the comment: I agree with Raymond. For binary reads, I'll go farther and say that even a 10% slowdown in performance would be surprising if not unacceptable to some people. I know that as hard as it might be for everyone to believe,

[issue3439] create a numbits() method for int and long types

2008-12-16 Thread STINNER Victor
STINNER Victor victor.stin...@haypocalc.com added the comment: Please don't promote this ugly code len(bin(x).lstrip('-0b')). It's not the best way to compute the number of bits... I prefer fredrikj's proposition (with // 2, few people understand 1). ___

[issue1673409] datetime module missing some important methods

2008-12-16 Thread Marc-Andre Lemburg
Changes by Marc-Andre Lemburg m...@egenix.com: -- nosy: -lemburg ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue1673409 ___ ___ Python-bugs-list

[issue3439] create a numbits() method for int and long types

2008-12-16 Thread Raymond Hettinger
Raymond Hettinger rhettin...@users.sourceforge.net added the comment: Oops, forgot one part of the doc definition in the proposed additional tests: for x in range(-65000, 65000): k = x.numbits() assert k == len(bin(x).lstrip('-0b')) if x 0: assert 2 ** (k-1) = x 2**k

[issue3439] create a numbits() method for int and long types

2008-12-16 Thread Raymond Hettinger
Raymond Hettinger rhettin...@users.sourceforge.net added the comment: One other thought on the docs. I would like to continue the style of supplying pure python equivalents to help precisely explain what a function does (see the itertools docs for an example, also a few builtins are explained

[issue3439] create a numbits() method for int and long types

2008-12-16 Thread STINNER Victor
STINNER Victor victor.stin...@haypocalc.com added the comment: Ooops, you're right! (15).numbits() - 4 and (16).numbits() - 5. I'm tired. ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue3439 ___

[issue3439] create a numbits() method for int and long types

2008-12-16 Thread Skip Montanaro
Skip Montanaro s...@pobox.com added the comment: Regarding the last few posts: * Raymond's implementation, while ugly, provides a completely orthogonal way to test compute numbits, useful in unit tests if nothing else. * Using x 1 in a reference implementation is perfectly reasonable.

[issue4561] Optimize new io library

2008-12-16 Thread Antoine Pitrou
Antoine Pitrou pit...@free.fr added the comment: [...] Although I agree all this is important, I'd challenge the assumption it has its place in the buffered IO library rather than in lower-level layers (i.e. kernel userspace unbuffered IO). In any case, it will be difficult to undo the

[issue4561] Optimize new io library

2008-12-16 Thread Christian Heimes
Christian Heimes li...@cheimes.de added the comment: David: Amaury's work is going to be a part of the standard library as soon as his work is done. I'm confident that we can reach the old speed of the 2.x file type by carefully moving code to C modules. ___

[issue3439] create a numbits() method for int and long types

2008-12-16 Thread STINNER Victor
STINNER Victor victor.stin...@haypocalc.com added the comment: x.numbits() is: math.ceil(math.log(abs(x)) / math.log(2)) if x != 0 0 otherwise and not 1 + math.floor(math.log(x) / math.log(2)) (16).numbits() is 4, not 5. ___ Python tracker

[issue4675] urllib's splitpasswd does not accept newline chars in passwords

2008-12-16 Thread Mihai Ibanescu
New submission from Mihai Ibanescu mihai.ibane...@gmail.com: According to http://www.ietf.org/rfc/rfc2617.txt section 2, in basic HTTP authentication the password can be any character (including newline). urllib does the following: _passwdprog = re.compile('^([^:]*):(.*)$') That should be

[issue3439] create a numbits() method for int and long types

2008-12-16 Thread Mark Dickinson
Mark Dickinson dicki...@gmail.com added the comment: Thanks for all the comments. Here's an updated patch, with quite a few changes: code: - drop lookup tables in favour of while(x) {x = 1; count += 1;} - add example to docstring, and use PyDoc_STRVAR macro for docstrings docs: - add

[issue3439] create a numbits() method for int and long types

2008-12-16 Thread Mark Dickinson
Changes by Mark Dickinson dicki...@gmail.com: -- components: +Interpreter Core -Library (Lib) ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue3439 ___

[issue4676] python3 closes + home keys

2008-12-16 Thread Somelauw
New submission from Somelauw somel...@yahoo.com: I'm using python 3.0 final which was released on December the 3th. I also have python 2.5 installed. 2 bugs in python3 IDLE which might be related (but don't have to) 1. The Python3 IDLE sometimes suddenly closes. It always happens when I'm

[issue3439] create a numbits() method for int and long types

2008-12-16 Thread Raymond Hettinger
Raymond Hettinger rhettin...@users.sourceforge.net added the comment: Of course, the name should have been bit_length() instead of numbits(). For the code equivalent, I'm aiming for something less process oriented and more focused on what it does. bit_length IS the number of bits in a binary

[issue3439] create a numbits() method for int and long types

2008-12-16 Thread Mark Dickinson
Mark Dickinson dicki...@gmail.com added the comment: Okay; I don't have strong feelings about the form the Python code takes; I'll let you guys argue it out and then fix things accordingly ___ Python tracker rep...@bugs.python.org

[issue3439] create a numbits() method for int and long types

2008-12-16 Thread Raymond Hettinger
Raymond Hettinger rhettin...@users.sourceforge.net added the comment: IMO, the choices are something like my version or none at all. The repeated floor division by two of abs(x) has ZERO explanatory power and may even detract from a beginner's ability to understand what the method does. Show

[issue3439] create a numbits() method for int and long types

2008-12-16 Thread Antoine Pitrou
Antoine Pitrou pit...@free.fr added the comment: Show that code to most finance people and they will avoid the method entirely. Why would finance people be interested in bit_length()? I think this discussion begins to feel like bikeshedding. Documentation can always be improved afterwards.

[issue3439] create a numbits() method for int and long types

2008-12-16 Thread Raymond Hettinger
Raymond Hettinger rhettin...@users.sourceforge.net added the comment: Antoine, it's not bike-shedding at all. Communicative docs are important to users other than assembly language programmers. BTW, I am a finance person (a CPA). Yes, you're correct, I can fix-up the docs after the patch is

[issue4580] slicing of memoryviews when itemsize != 1 is wrong

2008-12-16 Thread Antoine Pitrou
Antoine Pitrou pit...@free.fr added the comment: Any news? ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue4580 ___ ___ Python-bugs-list mailing list

[issue3439] create a numbits() method for int and long types

2008-12-16 Thread Mark Dickinson
Mark Dickinson dicki...@gmail.com added the comment: Updated patch. Added file: http://bugs.python.org/file12363/bit_length9.patch ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue3439 ___

[issue3439] create a numbits() method for int and long types

2008-12-16 Thread Mark Dickinson
Changes by Mark Dickinson dicki...@gmail.com: Removed file: http://bugs.python.org/file12348/bit_length7.patch ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue3439 ___

[issue3439] create a numbits() method for int and long types

2008-12-16 Thread Mark Dickinson
Changes by Mark Dickinson dicki...@gmail.com: Removed file: http://bugs.python.org/file12349/bit_length7_opt2.patch ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue3439 ___

[issue3439] create a numbits() method for int and long types

2008-12-16 Thread Mark Dickinson
Changes by Mark Dickinson dicki...@gmail.com: Removed file: http://bugs.python.org/file12362/bit_length8.patch ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue3439 ___

[issue3439] create a numbits() method for int and long types

2008-12-16 Thread Mark Dickinson
Mark Dickinson dicki...@gmail.com added the comment: Bah. Fix test_int so that it actually works. Added file: http://bugs.python.org/file12364/bit_length10.patch ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue3439

[issue3439] create a numbits() method for int and long types

2008-12-16 Thread Mark Dickinson
Changes by Mark Dickinson dicki...@gmail.com: Removed file: http://bugs.python.org/file12363/bit_length9.patch ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue3439 ___

[issue3439] create a numbits() method for int and long types

2008-12-16 Thread Martin v. Löwis
Changes by Martin v. Löwis mar...@v.loewis.de: -- nosy: -loewis ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue3439 ___ ___ Python-bugs-list

[issue3439] create a numbits() method for int and long types

2008-12-16 Thread Mark Dickinson
Mark Dickinson dicki...@gmail.com added the comment: ...and use proper unittest methods instead of asserts... Added file: http://bugs.python.org/file12365/bit_length11.patch ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue3439

[issue4486] Exception traceback is incorrect for strange exception handling

2008-12-16 Thread Antoine Pitrou
Antoine Pitrou pit...@free.fr added the comment: Here is a patch. Should it go in? -- keywords: +needs review, patch stage: - patch review type: - behavior Added file: http://bugs.python.org/file12366/issue4486.patch ___ Python tracker

[issue3439] create a numbits() method for int and long types

2008-12-16 Thread Raymond Hettinger
Raymond Hettinger rhettin...@users.sourceforge.net added the comment: Looks good. Marking as accepted. Before applying, consider adding back the part of the docs with the '1 + floor(...' definition. I think it provides a useful alternative way to look at what the method does. Also, it gives

[issue4426] UTF7 decoding is far too strict

2008-12-16 Thread Antoine Pitrou
Antoine Pitrou pit...@free.fr added the comment: I'm not in a position to comment on the encoding algorithm itself but I have a couple of comments: * I get the following compilation warning: Objects/unicodeobject.c: In function ‘PyUnicode_DecodeUTF7Stateful’: Objects/unicodeobject.c:1531:

[issue4583] segfault when mutating memoryview to array.array when array is resized

2008-12-16 Thread Antoine Pitrou
Antoine Pitrou pit...@free.fr added the comment: Nothing new in this patch except that it fixes the bogus indentation of the previous patch. Added file: http://bugs.python.org/file12367/arraybuf2.patch ___ Python tracker rep...@bugs.python.org

[issue4583] segfault when mutating memoryview to array.array when array is resized

2008-12-16 Thread Antoine Pitrou
Changes by Antoine Pitrou pit...@free.fr: -- nosy: +teoliphant ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue4583 ___ ___ Python-bugs-list mailing

[issue3439] create a numbits() method for int and long types

2008-12-16 Thread Mark Dickinson
Mark Dickinson dicki...@gmail.com added the comment: Before applying, consider adding back the part of the docs with the '1 + floor(...' definition. My only (minor) objection to this definition is that a straight Python translation of it doesn't work, thanks to roundoff error and the limited

[issue477863] New gc work

2008-12-16 Thread Antoine Pitrou
Antoine Pitrou pit...@free.fr added the comment: Although this dates back to 2001, I think this might still be useful. -- components: +Interpreter Core -None nosy: +pitrou versions: +Python 2.7, Python 3.1 ___ Python tracker rep...@bugs.python.org

[issue3439] create a numbits() method for int and long types

2008-12-16 Thread Raymond Hettinger
Raymond Hettinger rhettin...@users.sourceforge.net added the comment: Also, consider writing it in the two argument form: 1 + floor(log(n, 2)) and using the word approximately or somesuch. ___ Python tracker rep...@bugs.python.org

[issue4677] a list comprehensions tests for pybench

2008-12-16 Thread Antoine Pitrou
New submission from Antoine Pitrou pit...@free.fr: This patch adds some measurements of list comprehensions performance to the standard pybench suite. Marc-André, is it ok for you? -- components: Demos and Tools files: pybench-listcomps.patch keywords: patch messages: 77938 nosy:

[issue2183] optimize list comprehensions

2008-12-16 Thread Antoine Pitrou
Antoine Pitrou pit...@free.fr added the comment: Here is a new patch against trunk and fixing the documentation for LIST_APPEND. -- nosy: +pitrou stage: - patch review versions: +Python 2.7, Python 3.1 -Python 2.6 Added file: http://bugs.python.org/file12369/list-comp-opt3.patch

[issue4678] Unicode: multiple chars for high code points

2008-12-16 Thread Eric Eisner
New submission from Eric Eisner e...@mit.edu: I discovered this when trying to splice a string containing unicode codepoints higher than U+ all examples on 32-bit Ubuntu Linux python 2.5.2 (for comparison): sys.maxunicode # 1114111 len(unichr(66674)) # 1 len(u'\U00010472') # 1

[issue4678] Unicode: multiple chars for high code points

2008-12-16 Thread Marc-Andre Lemburg
Marc-Andre Lemburg m...@egenix.com added the comment: On 2008-12-17 00:25, Eric Eisner wrote: New submission from Eric Eisner e...@mit.edu: I discovered this when trying to splice a string containing unicode codepoints higher than U+ all examples on 32-bit Ubuntu Linux python

[issue4679] Fork + shelve causes shelve corruption and backtrace

2008-12-16 Thread Alex Roper
New submission from Alex Roper al...@ugcs.caltech.edu: Hi, I wrote a simple script (attached) to do some preprocessing of MediaWiki XML dumps. When it has a 8 MB chunk ready to dump to disk, it forks, and the child writes it out and (will) compress it, then exit. The main thread continues as

[issue2183] optimize list comprehensions

2008-12-16 Thread Raymond Hettinger
Raymond Hettinger rhettin...@users.sourceforge.net added the comment: This looks like a major improvement, not just for speed, but for getting rid of the _[1] arguments setup, retrieval, and deletion. I presume it has been tested with nested and conditional variants? def f(): return [x

[issue4678] Unicode: multiple chars for high code points

2008-12-16 Thread Martin v. Löwis
Martin v. Löwis mar...@v.loewis.de added the comment: As Mark-Andre say, this is not a bug. Finding out the exact name of the configure option is left as an exercise. -- nosy: +loewis resolution: - invalid status: open - closed ___ Python tracker

[issue4678] Unicode: multiple chars for high code points

2008-12-16 Thread Marc-Andre Lemburg
Marc-Andre Lemburg m...@egenix.com added the comment: On 2008-12-17 00:53, Martin v. Löwis wrote: Martin v. Löwis mar...@v.loewis.de added the comment: As Marc-Andre say, this is not a bug. Finding out the exact name of the configure option is left as an exercise. Ah, so that changed as

[issue2183] optimize list comprehensions

2008-12-16 Thread Antoine Pitrou
Antoine Pitrou pit...@free.fr added the comment: I presume it has been tested with nested and conditional variants? The whole test suite runs fine. test_iter and test_grammar seem to cover all possible cases. As for performance, the new benches in #4677 show a moderate improvement (~ 10%).

[issue2183] optimize list comprehensions

2008-12-16 Thread Raymond Hettinger
Raymond Hettinger rhettin...@users.sourceforge.net added the comment: This is a nice cleanup. Marking as accepted. Go ahead and apply. -- assignee: nnorwitz - pitrou resolution: - accepted ___ Python tracker rep...@bugs.python.org

[issue4675] urllib's splitpasswd does not accept newline chars in passwords

2008-12-16 Thread Mihai Ibanescu
Changes by Mihai Ibanescu mihai.ibane...@gmail.com: -- keywords: +patch Added file: http://bugs.python.org/file12371/splitpasswd.patch ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue4675 ___

[issue4653] Patch to fix typos for Py3K

2008-12-16 Thread Amaury Forgeot d'Arc
Amaury Forgeot d'Arc amaur...@gmail.com added the comment: It's difficult to really test such errors. Moreover, on the top of the bdist_wininst files, a comment says loudly: IF CHANGES TO THIS FILE ARE CHECKED INTO PYTHON CVS, THE RECOMPILED BINARIES MUST BE CHECKED IN AS WELL! --

[issue2183] optimize list comprehensions

2008-12-16 Thread Jeffrey Yasskin
Changes by Jeffrey Yasskin jyass...@gmail.com: -- nosy: +jyasskin ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue2183 ___ ___ Python-bugs-list

[issue2183] optimize list comprehensions

2008-12-16 Thread Antoine Pitrou
Antoine Pitrou pit...@free.fr added the comment: I've fixed the Python compiler package and committed it all in r67818. Will port to py3k. ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue2183 ___

[issue2183] optimize list comprehensions

2008-12-16 Thread Antoine Pitrou
Antoine Pitrou pit...@free.fr added the comment: I have started porting to py3k and I've applied the same optimizations to set and dict comprehensions. I just have a question: I have created an opcode MAP_ADD which is very similar to STORE_MAP, except that it takes as argument the stack offset

[issue2183] optimize list comprehensions

2008-12-16 Thread Antoine Pitrou
Antoine Pitrou pit...@free.fr added the comment: FWIW, here's the py3k patch. -- versions: -Python 2.7 Added file: http://bugs.python.org/file12372/py3k-comps.patch ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue2183

[issue2183] optimize list comprehensions

2008-12-16 Thread Raymond Hettinger
Raymond Hettinger rhettin...@users.sourceforge.net added the comment: Will take a look in the morning and get back to you on the MAP_ADD question. -- assignee: pitrou - rhettinger ___ Python tracker rep...@bugs.python.org

[issue1524639] Fix Tkinter Tcl-commands memory-leaks

2008-12-16 Thread Guilherme Polo
Guilherme Polo ggp...@gmail.com added the comment: Something like the proposed patch is still needed. But allow me to point out my views towards your current patch: * Changes in Misc.after, Misc._bind: good * Changes in Misc.unbind can be simplified a bit: cbs =

[issue1731706] tkinter memory leak problem

2008-12-16 Thread Guilherme Polo
Guilherme Polo ggp...@gmail.com added the comment: Sample attached for demonstrating the leak by missing a call to Tkapp_CallDeallocArgs. -- nosy: +gpolo Added file: http://bugs.python.org/file12373/test_tkleak1.py ___ Python tracker

[issue4680] Queue class should include high-water mark

2008-12-16 Thread Roy Smith
New submission from Roy Smith r...@panix.com: It would be nice if Queue.Queue included a way to access the high-water mark, i.e. the largest value which qsize() has ever reached. This is often useful when assessing application performance. I am assuming this is cheap, i.e. O(1), to provide.

[issue4074] Building a list of tuples has non-linear performance

2008-12-16 Thread Collin Winter
Changes by Collin Winter coll...@gmail.com: -- nosy: +collinwinter ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue4074 ___ ___ Python-bugs-list

[issue4680] Queue class should include high-water mark

2008-12-16 Thread David W. Lambert
Changes by David W. Lambert lamber...@corning.com: -- nosy: +LambertDW ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue4680 ___ ___ Python-bugs-list

[issue4653] Patch to fix typos for Py3K

2008-12-16 Thread Johnny Lee
Changes by Johnny Lee typo...@gmail.com: Removed file: http://bugs.python.org/file12335/py30diff.txt ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue4653 ___

[issue4653] Patch to fix typos for Py3K

2008-12-16 Thread Johnny Lee
Johnny Lee typo...@gmail.com added the comment: attached modified diff patch so line length =79 chars Added file: http://bugs.python.org/file12374/py30dif2.txt ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue4653

[issue4680] Queue class should include high-water mark

2008-12-16 Thread Martin v. Löwis
Changes by Martin v. Löwis mar...@v.loewis.de: -- versions: +Python 2.7 -Python 2.5.3 ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue4680 ___ ___