[issue9859] Add tests to verify API match of modules with 2 implementations
Daniel Stutzbach added the comment: With regret, I have not had time to work on patches and am unlikely to have time in the near future. -- ___ Python tracker <http://bugs.python.org/issue9859> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue23565] local_clear walks the list of threads without holding head_lock.
New submission from Daniel Stutzbach: local_clear in _threadmodule.c walks the list of threads without holding head_mutex. Since the list of threads can change when holding only head_mutex and not on the GIL, the list can change while local_clear is executing, which may cause Bad Things to happen. -- components: Library (Lib) messages: 237078 nosy: stutzbach priority: normal severity: normal status: open title: local_clear walks the list of threads without holding head_lock. type: behavior ___ Python tracker <http://bugs.python.org/issue23565> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue19081] zipimport behaves badly when the zip file changes while the process is running
Changes by Daniel Stutzbach : -- nosy: +stutzbach ___ Python tracker <http://bugs.python.org/issue19081> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue18606] Add statistics module to standard library
Daniel Stutzbach added the comment: As the person originally trying to take the mean of timedelta objects, I'm personally fine with the workaround of: py> m = statistics.mean([x.total_seconds() for x in data]) py> td(seconds=m) datetime.timedelta(2, 43200) At the time I was trying to take the mean of timedelta objects, even the total_seconds() method did not exist in the version of Python I was using. On the flip side, wouldn't sum() work on timedelta objects if you simply removed the "isinstance(start, numbers.Number)" check? -- nosy: +stutzbach ___ Python tracker <http://bugs.python.org/issue18606> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue2226] Small _abcoll Bugs / Oddities
Changes by Daniel Stutzbach : -- assignee: stutzbach -> rhettinger ___ Python tracker <http://bugs.python.org/issue2226> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue9141] Allow objects to decide if they can be collected by GC
Daniel Stutzbach added the comment: On Tue, Apr 17, 2012 at 4:08 AM, Antoine Pitrou wrote: > has_finalizer() in gcmodule.c doesn't check for weakref callbacks, so a > weakref callback can still be invoked from tp_dealloc. Unless I'm mistaken, weakrefs will be handled and removed by handle_weakrefs() before delete_garbage() is called. -- ___ Python tracker <http://bugs.python.org/issue9141> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue14534] Add method to mark unittest.TestCases as "do not run".
Daniel Stutzbach added the comment: Wouldn't the subclass inherit the False value? Then the user would need to remember to override the value again in the subclass, which is error prone. Antoine: I've used the pattern you describe on a couple of occasions, and it routinely confuses my code reviewers. -- nosy: +stutzbach ___ Python tracker <http://bugs.python.org/issue14534> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue9141] Allow objects to decide if they can be collected by GC
Daniel Stutzbach added the comment: Are __del__ and tp_del getting conflated in this conversation? I don't see a __del__ method on generator objects: filfre:$ python3.1 Python 3.1.2 (r312:79147, Dec 9 2011, 20:47:34) [GCC 4.4.3] on linux2 >>> (i for i in range(5)).__del__ Traceback (most recent call last): File "", line 1, in AttributeError: 'generator' object has no attribute '__del__' but I may be missing something. -- ___ Python tracker <http://bugs.python.org/issue9141> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue9141] Allow objects to decide if they can be collected by GC
Daniel Stutzbach added the comment: On Fri, Apr 6, 2012 at 12:51 PM, Jim Jewett wrote: > __del__ methods do run, even if an object was collected by the cycle > detector. And they can't do any harm that couldn't also be done by a C > finalizer. > No, if an object with a __del__ method is part of a cycle, it is not collected. The objects get appended to gc.garbage instead. See: http://docs.python.org/py3k/library/gc.html#gc.garbage -- ___ Python tracker <http://bugs.python.org/issue9141> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue13804] Python library structure creates hard to read code when using higher order functions
Daniel Stutzbach added the comment: Ah - in your first example (the one with 3 lines) did you mean to use .extend instead of .append? -- ___ Python tracker <http://bugs.python.org/issue13804> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue13804] Python library structure creates hard to read code when using higher order functions
Daniel Stutzbach added the comment: If I'm understanding Martin Häcker's code correctly, the list comprehension equivalent is: self.questions = [topic.questions for topic in self.topics] The reduce() variants are not only much harder to read, but they will take O(n**2) operations because they create an O(n) list O(n) times. -- nosy: +stutzbach ___ Python tracker <http://bugs.python.org/issue13804> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue13742] Add a key parameter (like sorted) to heapq.merge
Changes by Daniel Stutzbach : -- nosy: +stutzbach ___ Python tracker <http://bugs.python.org/issue13742> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue8654] Improve ABI compatibility between UCS2 and UCS4 builds
Daniel Stutzbach added the comment: Gladly. :-) -- resolution: -> out of date stage: patch review -> committed/rejected status: open -> closed ___ Python tracker <http://bugs.python.org/issue8654> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue12682] Meaning of 'accepted' resolution as documented in devguide
Changes by Daniel Stutzbach : -- nosy: +stutzbach ___ Python tracker <http://bugs.python.org/issue12682> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue12612] Valgrind suppressions
Changes by Daniel Stutzbach : -- nosy: +stutzbach ___ Python tracker <http://bugs.python.org/issue12612> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue11640] Shelve references globals in its __del__ method
Daniel Stutzbach added the comment: Could you add a comment above the lines defining self._BytesIO, describing why they're being set? Someone else might see them as unnecessary and rip them out if there's no explanation. Can a test launch Python in a subprocess, set up the appropriate data structure, let the subprocess Python shutdown, then check the subprocess's stderr for the error? It's a little convoluted, but perhaps it could be built into a general utility function that could be used to test __del__ methods in other modules, too. Not sure that it's worth the effort though. -- ___ Python tracker <http://bugs.python.org/issue11640> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue12434] Strengthen 2.7 io types warning
Daniel Stutzbach added the comment: On Sat, Jul 16, 2011 at 2:04 AM, Eli Bendersky wrote: > Therefore, I propose to change this error message to: > "unicode argument expected, got '%s'" > as Terry suggested. > Sounds good to me. -- Added file: http://bugs.python.org/file22687/unnamed ___ Python tracker <http://bugs.python.org/issue12434> ___On Sat, Jul 16, 2011 at 2:04 AM, Eli Bendersky <mailto:rep...@bugs.python.org";>rep...@bugs.python.org> wrote: Therefore, I propose to change this error message to:  "unicode argument expected, got '%s'"  as Terry suggested.Sounds good to me. -- Daniel Stutzbach ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue12345] Add math.tau
Daniel Stutzbach added the comment: On Wed, Jun 15, 2011 at 8:10 PM, Nick Coghlan wrote: > I'd like to add a new constant to the math module: > > tau = 2*math.pi > > Rather than repeating all the reasons for why tau makes more sense than pi > as the fundamental circle constant, I'll just refer interested readers to > http://tauday.com/ (The following is best if read aloud using your best pirate impression) I'd rather name the new constant "matey". Then we have: The circumference of a circle is: r matey The area of a circle is: one-quarter r r matey etc. -- nosy: +stutzbach Added file: http://bugs.python.org/file22397/unnamed ___ Python tracker <http://bugs.python.org/issue12345> ___On Wed, Jun 15, 2011 at 8:10 PM, Nick Coghlan <mailto:rep...@bugs.python.org";>rep...@bugs.python.org> wrote: I'd like to add a new constant to the math module:  tau = 2*math.pi Rather than repeating all the reasons for why tau makes more sense than pi as the fundamental circle constant, I'll just refer interested readers to http://tauday.com/"; target="_blank">http://tauday.com/ (The following is best if read aloud using your best pirate impression)I'd rather name the new constant "matey".  Then we have:The circumference of a circle is: r matey The area of a circle is: one-quarter r r mateyetc.-- Daniel Stutzbach ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue12192] Doc that collection mutation methods return item or None
Daniel Stutzbach added the comment: > Comment: This rule applies to special methods like __getitem__ > and __setitem__. 'lis.append(item)' is equivalent to > lis.__setitem__(len(lis):len(lis), item), so it should not be so > surprising that it has the same return. It's not true for all special methods. __iadd__ and __imul__ return the list itself. That's a minor nitpick though. +1 on adding "and return None" to the docstrings of methods that often trip people up. Not everyone will read them, but some people will. -- nosy: +stutzbach ___ Python tracker <http://bugs.python.org/issue12192> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue8777] Add threading.Barrier
Changes by Daniel Stutzbach : -- stage: -> committed/rejected ___ Python tracker <http://bugs.python.org/issue8777> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue12128] Allow `abc.abstractproperty` to be overridden by a data attribute
Daniel Stutzbach added the comment: > Daniel, the behavior you describe is already present in Python 3.2. Awesome. :) Do you have a compelling use-case for making "self.x = 5" satisfy an abstractproperty requirement? One of the problems with that approach is that the instance does not satisfy the requirements until the line in __init__ is called that sets self.x. -- ___ Python tracker <http://bugs.python.org/issue12128> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue12128] Allow `abc.abstractproperty` to be overridden by a data attribute
Daniel Stutzbach added the comment: I misread the original request. I'm +1 on making the following work, if it doesn't work already: class MySubClass(MyAbstractClass): SOME_LIMIT = 5 # Implements abstract property with fixed value We should be able to check that at instance creation time. I think abstract methods and properties are to force the class to define methods/properties. I don't think we should be checking if the instance adds things that are not defined at the class level (i.e., don't try to detect "self.SOME_LIMT = 5"). -- ___ Python tracker <http://bugs.python.org/issue12128> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue12127] Inconsistent leading zero treatment
Daniel Stutzbach added the comment: -0 on making it valid ever, due to the different meaning in many other languages. -1 on making it valid while many still use a version of python where it's valid with a different meaning. Maybe for Python 4. ;-) -- nosy: +stutzbach ___ Python tracker <http://bugs.python.org/issue12127> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue12128] Allow `abc.abstractproperty` to be overridden by a data attribute
Daniel Stutzbach added the comment: SGTM. I've written code where this would have been useful. Could you write a patch? -- nosy: +stutzbach stage: -> needs patch ___ Python tracker <http://bugs.python.org/issue12128> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue9971] Optimize BufferedReader.readinto
Daniel Stutzbach added the comment: Looking at this again, I agree with John. For BufferedIOBase, read() is abstract while readinto() is concrete. That seems backward, and, indeed, it's the opposite of RawIOBase, where readinto() is abstract and read() is concrete. Unfortunately, this code has shipped, so changing which methods are abstract may not be practical. On the other hand, the documentation doesn't mention which methods are abstract versus concrete. All of that said, we can freely change the C implementation of BufferedReader which is a concrete class. That would allow us to cut out the extra allocoations/deallocations, even if we can't clean up the abstract vs concrete method issue. Basically, this would require greatly expanding buffered_readinto() in bufferedio.c to use _bufferedreader_raw_read() and related functions. As I think about this more... I'm not sure how much performance there is to gain here in practice. It seems like any time I'd want to use readinto(), it's because I want to do my own buffering, in which case why would I use a BufferedReader? I'm thinking that BufferedIOBase only provides a readinto() method for completeness, so it can be used as a drop-in replacement for an unbuffered file object. -- ___ Python tracker <http://bugs.python.org/issue9971> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue11335] Memory leak after key function failure in sort
Changes by Daniel Stutzbach : -- resolution: -> fixed stage: patch review -> committed/rejected status: open -> closed ___ Python tracker <http://bugs.python.org/issue11335> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue11994] [2.7/gcc-4.4.3] Segfault under valgrind in string.split()
Changes by Daniel Stutzbach : -- nosy: +stutzbach ___ Python tracker <http://bugs.python.org/issue11994> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue11335] Memory leak after key function failure in sort
Daniel Stutzbach added the comment: I checked in a fix to 3.3 just before the transition to hg. I confess that I've been procrastinating on getting hg set up properly, which is why I haven't gotten to checking this in to 3.2. Georg, if I get this in by Wednesday, will that be soon enough for your schedule for 3.2.1? (it sounds like it will be) -- nosy: +georg.brandl priority: normal -> release blocker ___ Python tracker <http://bugs.python.org/issue11335> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue11854] __or__ et al instantiate subclass of set without calling __init__
Changes by Daniel Stutzbach : -- nosy: +stutzbach ___ Python tracker <http://bugs.python.org/issue11854> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue11845] Refcounting error in compute_slice_indices in rangeobject.c
Changes by Daniel Stutzbach : -- nosy: +stutzbach ___ Python tracker <http://bugs.python.org/issue11845> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue6634] sys.exit() called from threads other than the main one: undocumented behaviour
Changes by Daniel Stutzbach : -- nosy: +stutzbach ___ Python tracker <http://bugs.python.org/issue6634> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue11707] Create C version of functools.cmp_to_key()
Changes by Daniel Stutzbach : -- nosy: +stutzbach ___ Python tracker <http://bugs.python.org/issue11707> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue7995] On Mac / BSD sockets returned by accept inherit the parent's FD flags
Daniel Stutzbach added the comment: I'm confused by the patch (ed0259230611). The patch comment and the NEWS item state "the returned socket is now always non-blocking" but the code change adds "sock.setblocking(True)". -- nosy: +stutzbach ___ Python tracker <http://bugs.python.org/issue7995> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue11674] list(obj), tuple(obj) swallow TypeError (in _PyObject_LengthHint)
Changes by Daniel Stutzbach : -- nosy: +stutzbach ___ Python tracker <http://bugs.python.org/issue11674> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue11640] Shelve references globals in its __del__ method
Changes by Daniel Stutzbach : -- nosy: +stutzbach versions: +Python 3.3 ___ Python tracker <http://bugs.python.org/issue11640> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue11635] concurrent.futures uses polling
Changes by Daniel Stutzbach : -- nosy: +stutzbach ___ Python tracker <http://bugs.python.org/issue11635> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue11628] cmp_to_key generated class should use __slots__
Changes by Daniel Stutzbach : -- nosy: +stutzbach ___ Python tracker <http://bugs.python.org/issue11628> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue11608] GzipFile cannot be used for streaming
Changes by Daniel Stutzbach : -- nosy: +stutzbach ___ Python tracker <http://bugs.python.org/issue11608> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue11583] os.path.isdir() is slow on windows
Changes by Daniel Stutzbach : -- nosy: +stutzbach ___ Python tracker <http://bugs.python.org/issue11583> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue11562] += on list inside a tuple raises TypeError but succeds anyway
Changes by Daniel Stutzbach : -- nosy: +stutzbach stage: -> test needed versions: +Python 3.2 ___ Python tracker <http://bugs.python.org/issue11562> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue11480] Cannot copy a class with a metaclass other than type
Changes by Daniel Stutzbach : -- keywords: +needs review nosy: +alexandre.vassalotti, stutzbach stage: -> patch review ___ Python tracker <http://bugs.python.org/issue11480> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue11455] issue a warning when populating a CPython type dict with non-string keys
Daniel Stutzbach added the comment: For what it's worth, I believe this could be implemented easily by calling _PyDict_HasOnlyStringKeys at the end of the class creation process. -- nosy: +stutzbach ___ Python tracker <http://bugs.python.org/issue11455> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue11402] _PyUnicode_Init leaks a little memory once
Changes by Daniel Stutzbach : -- keywords: +needs review, patch Added file: http://bugs.python.org/file20999/unicode-leak.patch ___ Python tracker <http://bugs.python.org/issue11402> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue11402] _PyUnicode_Init leaks a little memory once
New submission from Daniel Stutzbach : By the time _PyUnicode_Init is called and does the following: /* Init the implementation */ free_list = NULL; numfree = 0; free_list is already in use. _PyUnicode_Init clobbers whatever was stored there, leaking memory. In Python 2, this was not an issue because no unicode functions were called prior to initialization. In Python 3, Unicode objects are created when initializing other types. I found this using valgrind to search for "definitely lost" memory. I'd like to fix all of those cases, to make it easier to run valgrind on a regular basis. I'll upload a patch momentarily. -- assignee: stutzbach components: Interpreter Core messages: 130090 nosy: stutzbach priority: normal severity: normal stage: patch review status: open title: _PyUnicode_Init leaks a little memory once type: resource usage versions: Python 3.1, Python 3.2, Python 3.3 ___ Python tracker <http://bugs.python.org/issue11402> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue9427] logging.error('...', exc_info=True) should display upper frames, too
Changes by Daniel Stutzbach : -- dependencies: +Add traceback.print_full_exception() nosy: +stutzbach versions: +Python 3.3 -Python 3.2 ___ Python tracker <http://bugs.python.org/issue9427> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue1553375] Add traceback.print_full_exception()
Changes by Daniel Stutzbach : -- nosy: +stutzbach versions: +Python 3.3 -Python 3.2 ___ Python tracker <http://bugs.python.org/issue1553375> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue11335] Memory leak after key function failure in sort
Daniel Stutzbach added the comment: I played around with this a little. That code path doesn't appear to be exercised during the existing unit tests. I'll add a test so the leak at least shows up when the tests are run under valgrind. -- ___ Python tracker <http://bugs.python.org/issue11335> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue11335] Memory leak after key function failure in sort
Daniel Stutzbach added the comment: I meant "to *detect* leaked memory", of course. :-) -- ___ Python tracker <http://bugs.python.org/issue11335> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue11335] Memory leak after key function failure in sort
Daniel Stutzbach added the comment: I'm not sure. What infrastructure do we have to leaked memory that was allocated with PyMem_MALLOC? -- ___ Python tracker <http://bugs.python.org/issue11335> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue4600] __class__ assignment: new-style? heap? == confusing
Changes by Daniel Stutzbach : -- nosy: +stutzbach ___ Python tracker <http://bugs.python.org/issue4600> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue11335] Memory leak after key function failure in sort
Daniel Stutzbach added the comment: Below is a link to the code where leak occurs and the patch goes: http://svn.python.org/view/python/branches/py3k/Objects/listobject.c?view=markup&pathrev=88554#l1944 -- keywords: +3.2regression, needs review, patch Added file: http://bugs.python.org/file20918/plug-leak.patch ___ Python tracker <http://bugs.python.org/issue11335> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue11335] Memory leak after key function failure in sort
New submission from Daniel Stutzbach : In list.sort, if a key function throws an exception, the memory to store the keys is never freed. I introduced the bug in r86937. I'll upload a patch for review shortly. -- assignee: stutzbach components: Interpreter Core messages: 129574 nosy: pitrou, rhettinger, stutzbach priority: normal severity: normal stage: patch review status: open title: Memory leak after key function failure in sort type: resource usage versions: Python 3.2, Python 3.3 ___ Python tracker <http://bugs.python.org/issue11335> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue10868] ABCMeta.register() should work as a decorator
Daniel Stutzbach added the comment: Sounds good to me. Thanks for the clarifications and satisfying my curiosity! :-) -- ___ Python tracker <http://bugs.python.org/issue10868> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue11333] Add empty __slots__ to collections.abc abstract base classes
Daniel Stutzbach added the comment: +1. I've bumped into exactly this problem (https://github.com/DanielStutzbach/blist/issues/closed#issue/29) I'm not intimately familiar with how __slots__ works. Are there any drawbacks to adding an empty __slots__ to the ABCs? -- ___ Python tracker <http://bugs.python.org/issue11333> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue10868] ABCMeta.register() should work as a decorator
Daniel Stutzbach added the comment: In what use-cases would you want to call MyABC.register() when defining a class instead of inheriting from MyABC? I always thought of the register() as hack to make it possible to support types written in C, which can't inherit from the ABC. -- nosy: +stutzbach ___ Python tracker <http://bugs.python.org/issue10868> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue7111] abort when stderr is closed
Daniel Stutzbach added the comment: On Thu, Feb 3, 2011 at 12:18 PM, Antoine Pitrou wrote: > I just took a look at http://pypi.python.org/pypi/python-daemon/, and it > uses dup2() to redirect standard streams, which is far nicer. I'm more worried about the case where a daemon launches python. At startup, could we check that 2 and 3 are valid file descriptors, and, if not, open /dev/null? That way, they cannot later be inadvertently assigned to some other file? -- ___ Python tracker <http://bugs.python.org/issue7111> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue7111] abort when stderr is closed
Daniel Stutzbach added the comment: On Thu, Feb 3, 2011 at 11:56 AM, Alexander Belopolsky wrote: > 3rd party extensions. What is the use case for "python >&-"? Is > it important enough to justify the risk of accidental data loss? I don't think closing stderr via the command line is an important use case, but pythonw.exe and Unix daemon processes are important use cases. -- ___ Python tracker <http://bugs.python.org/issue7111> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue7111] abort when stderr is closed
Daniel Stutzbach added the comment: That's an interesting point. Do you know of places where we use fd 2 instead of sys.stderr? -- ___ Python tracker <http://bugs.python.org/issue7111> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue8753] Py_ReprEnter and Py_ReprLeave are undocumented
Changes by Daniel Stutzbach : -- status: open -> closed ___ Python tracker <http://bugs.python.org/issue8753> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue8753] Py_ReprEnter and Py_ReprLeave are undocumented
Daniel Stutzbach added the comment: On Fri, Dec 17, 2010 at 2:17 PM, Amaury Forgeot d'Arc wrote wrote: > Err... in r87339 there is a typo in all occurrences of > 'Py_ReprEntr': Py_ReprEnter of course! Well, that's embarrassing. Fixed in r87354. -- ___ Python tracker <http://bugs.python.org/issue8753> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue2690] Precompute range length and enhance range subscript support
Daniel Stutzbach added the comment: Doc change committed to py3k in r87346. Thanks, SilentGhost! I also committed r87349 to reverse r87162 (which was in the wrong branch). -- ___ Python tracker <http://bugs.python.org/issue2690> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue2226] Small _abcoll Bugs / Oddities
Daniel Stutzbach added the comment: Raymond, Do you have around 10 minutes today to look at the patch I submitted in Issue 8743? It appears to solve both this issue and that one, which have priorities critical and high, respectively. It's a reasonably small patch: around 20 lines changed plus 20 lines added, all Python code. Here's a direct link: http://bugs.python.org/file20045/set-with-Set.patch If it looks good to you, I'd like to commit it in time for 3.2b2 so that it gets as much exercise as possible before 3.2-final. Earlier you had expressed that adding the __rand__ methods to collections.Set would be tricky issue, so I'm concerned that I have overlooked something important and obvious. (Feedback from others is, of course, welcome as well) -- ___ Python tracker <http://bugs.python.org/issue2226> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue8753] Py_ReprEnter and Py_ReprLeave are undocumented
Daniel Stutzbach added the comment: Committed in r87339. Thanks everyone for the feedback! -- assignee: eric.araujo -> stutzbach keywords: -needs review resolution: -> accepted stage: patch review -> committed/rejected status: open -> closed versions: -Python 2.7 ___ Python tracker <http://bugs.python.org/issue8753> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue8040] It would be nice if documentation pages linked to other versions of the same document
Daniel Stutzbach added the comment: On Tue, Dec 14, 2010 at 7:22 PM, Ezio Melotti wrote: > If you want to work on this you should also consider that the URLs used on > docs.python.org don't necessary match the directory structure present when > building the doc locally. For example here it would be: > .../py3k/library/fractions.html > .../release31-maint/library/fractions.html > .../release27-maint/library/fractions.html > .../release26-maint/library/fractions.html Will this directory structure change as part of the hg migration? -- ___ Python tracker <http://bugs.python.org/issue8040> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue8753] Py_ReprEnter and Py_ReprLeave are undocumented
Daniel Stutzbach added the comment: Thanks. New patch attached with a cross-reference to reprlib.recursive_repr. -- Added file: http://bugs.python.org/file20072/repr-3.patch ___ Python tracker <http://bugs.python.org/issue8753> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue8753] Py_ReprEnter and Py_ReprLeave are undocumented
Changes by Daniel Stutzbach : Added file: http://bugs.python.org/file20071/repr-2.patch ___ Python tracker <http://bugs.python.org/issue8753> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue8753] Py_ReprEnter and Py_ReprLeave are undocumented
Daniel Stutzbach added the comment: Good point. My perspective is skewed by all of the time that I spend working on container types. :-) How about if I change the first sentence to the following? Properly implementing :attr:`tp_repr` for container types requires special recursion handling. -- ___ Python tracker <http://bugs.python.org/issue8753> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue10711] Rip out HTTP 0.9 support
Daniel Stutzbach added the comment: At minimum, I think we should apply this part of Antoine's patch: -# Most web servers default to HTTP 0.9, i.e. don't send a status line. -default_request_version = "HTTP/0.9" +default_request_version = "HTTP/1.0" -- ___ Python tracker <http://bugs.python.org/issue10711> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue10711] Rip out HTTP 0.9 support
Daniel Stutzbach added the comment: Would it be worth keeping (but modifying) test_http_0_9 to verify that the server complains in the expected way? -- nosy: +stutzbach ___ Python tracker <http://bugs.python.org/issue10711> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue8753] Py_ReprEnter and Py_ReprLeave are undocumented
New submission from Daniel Stutzbach : Attached is a patch to add documentation for Py_ReprEnter and Py_ReprLeave. Assigning to d...@python for review. -- assignee: stutzbach -> d...@python keywords: +needs review, patch nosy: +d...@python stage: needs patch -> patch review Added file: http://bugs.python.org/file20054/repr.patch ___ Python tracker <http://bugs.python.org/issue8753> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue8743] set() operators don't work with collections.Set instances
Daniel Stutzbach added the comment: Would it be sufficient to: 1) Restrict collections.Set()'s operators to accept collection.Set but not arbitrary iterables, and 2) Fix Issue2226 and let set() | MySimpleSet() work via collections.Set.__ror__ Attached is a patch that implements this approach, nominally fixing both this and Issue2226. This solutions seems much too simple in light of how long I've been thinking about these bugs. I suspect there are code hobgoblins waiting to ambush me. ;) -- Added file: http://bugs.python.org/file20045/set-with-Set.patch ___ Python tracker <http://bugs.python.org/issue8743> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue2226] Small _abcoll Bugs / Oddities
Daniel Stutzbach added the comment: Minor point of clarity: you mean __rand__ not __radd__, right? Set objects do not support addition at all. Adding the __rand__ methods to collections.Set in and of itself is straightforward: def __rsub__(self, other): return self._from_iterable(other) - self __ror__ = __or__ __rand__ = __and__ __rxor__ = __xor__ I'm not sure I understand the can of worms. While replacing concrete tests with abstract tests may be worthwhile goal in its own right, why is it necessary to solve the particular shortcoming of missing __r* methods? Probably I'm missing something. With just the minimal change above, what kinds of things do you expect to fail? (assuming Issue8743 is also fixed) -- ___ Python tracker <http://bugs.python.org/issue2226> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue8040] It would be nice if documentation pages linked to other versions of the same document
Changes by Daniel Stutzbach : -- versions: -Python 3.2 ___ Python tracker <http://bugs.python.org/issue8040> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue10542] Py_UNICODE_NEXT and other macros for surrogates
Daniel Stutzbach added the comment: In bltinmodule.c, it looks like some of the indentation doesn't line up? Bikeshedding aside, it looks good to me. I agree with Eric Smith that the first part macro name usually refers to the type of the first argument (or the type the first argument points to). Examples: - Py_UNICODE_ISSPACE : Py_UNICODE -> int - Py_UNICODE_TOLOWER : Py_UNICODE -> Py_UNICODE - Py_UNICODE_strlen: Py_UNICODE * -> size_t This is true elsewhere in the code as well: - PyList_GET_SIZE : PyListObject * -> Py_ssize_t Yes, I know there are some unfortunate exceptions. ;-) I agree that it would be nice if something in the name hinted that the return type was Py_UCS4 though. Marc-Andre Lemburg wrote: > The first argument of the macro can be any array, not just > Py_UNICODE*, but also Py_UCS4* or even int*. It's true that macros in C do not have any type safety. While technically passing a Py_UCS4 * will work, on a UCS2 build it would needlessly check the Py_UCS4 data for surrogates. I think we should discourage that. You can also technically pass a PyListObject * to PyTuple_GET_SIZE, but that's also not a good idea. ;-) Alexander Belopolsky wrote: > The issue is that once in in the process of reading the codepoint, it > is determined whether the code point is BMP or non-BMP. Testing the > result again in order to write it is somewhat wasteful. I don't > think this would matter in practice, but would like to hear > alternative opinions before moving further. If the common pattern is: ch = Py_UNICODE_NEXT(rp, end); uc = Py_UNICODE_SOME_TRANSFORMATION(ch); Py_UNICODE_PUT_NEXT(wp, uc); The second check for surrogates in Py_UNICODE_PUT_NEXT is necessary, unless you can prove that Py_UNICODE_SOME_TRANSFORMATION will never transform characters < 0x1 into characters > 0x1 or vice versa. Can we prove will always be the case, for current and future versions of Unicode, for all or almost-all of the transformations we care about? Answering that question and figuring out what to do about it are probably more trouble than it's worth. If a particularly point proves to be a bottleneck, we can always specialize the code there later. -- ___ Python tracker <http://bugs.python.org/issue10542> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue10542] Py_UNICODE_NEXT and other macros for surrogates
Daniel Stutzbach added the comment: +1 on the general idea of abstracting out repeated code. I will take a closer look at the details within the next few days. -- ___ Python tracker <http://bugs.python.org/issue10542> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue2690] Precompute range length and enhance range subscript support
Daniel Stutzbach added the comment: The descriptions of range's limitations in the docs still needs an update. -- assignee: georg.brandl -> ncoghlan status: closed -> open ___ Python tracker <http://bugs.python.org/issue2690> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue2690] Precompute range length and enhance range subscript support
Daniel Stutzbach added the comment: > The descriptions of range's limitations there is no longer accurate > (slicing is supported following this patch and containment testing is > now efficient) Want to open a new issue for that? (or is there one already?) -- ___ Python tracker <http://bugs.python.org/issue2690> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue2690] Precompute range length and enhance range subscript support
Daniel Stutzbach added the comment: > (I also noticed that the new methods from issue #9213 are not mentioned > in the range() docs Wasn't that fixed in Issue9746? -- ___ Python tracker <http://bugs.python.org/issue2690> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue9915] speeding up sorting with a key
Changes by Daniel Stutzbach : -- components: +Interpreter Core -Library (Lib) resolution: -> accepted stage: patch review -> committed/rejected status: open -> closed ___ Python tracker <http://bugs.python.org/issue9915> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue9915] speeding up sorting with a key
Daniel Stutzbach added the comment: Committed as r86937. Thanks again for reviewing! Although I do not anticipate any problems, I will keep an eye on the buildbots just in case. Antoine, regarding "ms->alloced = (list_size + 1) / 2;", I ended up adding an extensive comment after all. -- ___ Python tracker <http://bugs.python.org/issue9915> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue9915] speeding up sorting with a key
Daniel Stutzbach added the comment: > Just wanted to post this so there weren't any illusions about the > patch being a big win. > When a key function is defined, this is all you can possibly shave > off the time for a comparison. I don't want to argue whether the patch is a big win or not (I recognize that it is a tradeoff), but when using a key it does shave off more than the call to sortwrapper_richcompare. Stack with sortwrapper: long_richcompare do_richcompare PyObject_RichCompare sortwrapper_richcompare do_richcompare PyObject_RichCompare PyObject_RichCompareBool count_run list_sort Stack without: long_richcompare do_richcompare PyObject_RichCompare PyObject_RichCompareBool count_run list_sort -- ___ Python tracker <http://bugs.python.org/issue9915> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue9915] speeding up sorting with a key
Daniel Stutzbach added the comment: > The use of Py_LOCAL_INLINE is new to me since we usually use #define > instead, but this has a cleaner look to it. I am unclear on whether > all the our target compilers support an inline keyword. If you're > sure it works everywhere, that's great. I fixed ./configure to properly set up Py_LOCAL_INLINE in Issue5553. :-) It will expand to "static inline" under both MSVC and gcc. On older compilers, it may expand to "static __inline__", "static __inline", or whatever else is needed to get the job done. As a last resort, it will expand to simply "static", but I don't know of any 32-bit (or 64-bit) compilers where that would actually happen. > Also note that this patch puts a lot of faith in branch prediction. > If some target processor doesn't support it, or has limited ability > to remember predictions, or mispredicts, then the code will be slower. I think even a limited amount of memory dedicated to branch prediction should be sufficient. There are two cases: 1) Sorting a simple type, like an int: the comparison is lightweight, and the CPU should have plenty of memory to remember which branch to take in the sorting code. 2) Sorting a complex type (i.e., calling a __lt__ method written in Python): the processor might not be able to remember which branch to take, but the performance impact will be small (as a percentage) since most of the CPU is being consumed by the comparisons. Thanks for taking the time to review this. -- ___ Python tracker <http://bugs.python.org/issue9915> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue9915] speeding up sorting with a key
Daniel Stutzbach added the comment: Antoine and Raymond, thank you for the valuable feedback. Attached is a revised version of the patch, which restricts changes to those directly related to moving elements in the keys and values arrays at the same time. I apologize for having gotten a little carried away with optimizing. I think I eliminated all of the significant style differences as well. If you still see glaring style mismatches, please let me know. It's possible that the differences aren't visible to my eyes. ;-) http://codereview.appspot.com/3369042/diff/1/Objects/listobject.c Please let me know what you think of the revised version. -- Added file: http://bugs.python.org/file19893/sort-faster-3.patch ___ Python tracker <http://bugs.python.org/issue9915> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue8743] set() operators don't work with collections.Set instances
Daniel Stutzbach added the comment: Yes, I can take a stab at it. -- ___ Python tracker <http://bugs.python.org/issue8743> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue10589] I/O ABC docs should specify which methods have implementations
Daniel Stutzbach added the comment: Other suggestions for a better name for that column are certainly welcome. :-) "Stub Methods"? -- ___ Python tracker <http://bugs.python.org/issue10589> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue10589] I/O ABC docs should specify which methods have implementations
Daniel Stutzbach added the comment: > What does "unsupported" mean? "Abstract" would look more exact. It means they raise io.UnsupportedOperation when called (unless the subclass overrides them to do something else). They are not marked with @abstractmethod, so "Abstract" would be wrong. It should be a bit more clear in context. Here's a link to the point in the documentation where the table would fit: http://docs.python.org/dev/py3k/library/io.html#class-hierarchy The most relevant bit is: "implementations are allowed to raise UnsupportedOperation if they do not support a given operation." > E.g. "readinto" is listed as provided by RawIOBase in the doc text. The doc text describes the API. That is, it describes what the readinto method should do if a subclass of RawIOBase decides to implement it. The existing text doesn't contain any clues as to which methods contain useful implementations versus which will raise io.UnsupportedOperation if not overridden. Hence the need for the table. -- ___ Python tracker <http://bugs.python.org/issue10589> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue10589] I/O ABC docs should specify which methods have implementations
Daniel Stutzbach added the comment: +io and doc people Attached is a simple patch to add a table to the documentation summarizing the I/O ABCs. -- keywords: +patch nosy: +benjamin.peterson, ezio.melotti, georg.brandl Added file: http://bugs.python.org/file19885/io-abc.diff ___ Python tracker <http://bugs.python.org/issue10589> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue10589] I/O ABC docs should specify which methods have implementations
New submission from Daniel Stutzbach : The I/O ABC documentation has a blanket disclaimer at the top: "The abstract base classes also provide default implementations of some methods in order to help implementation of concrete stream classes. For example, BufferedIOBase provides unoptimized implementations of readinto() and readline()." Which effectively means that to subclass one of them requires digging into Lib/_pyio.py to figure out which methods provide useful implementations. It would be handy to have a table, similar to the one for the collections ABCs [1], that spells out which methods are provided. [1]: http://docs.python.org/py3k/library/collections.html#abcs-abstract-base-classes I hope to have a patch ready sometime within the next week. -- assignee: stutzbach components: Documentation messages: 122930 nosy: stutzbach priority: normal severity: normal stage: needs patch status: open title: I/O ABC docs should specify which methods have implementations type: feature request versions: Python 3.2 ___ Python tracker <http://bugs.python.org/issue10589> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue10576] Add a progress callback to gcmodule
Daniel Stutzbach added the comment: > How about a string and a dict? the string can be "start" and "stop" > and we can add interesting information to the dict as you suggest. I like where this is headed. How about putting the string in the dict, too? d['phase'] = 'start' -- ___ Python tracker <http://bugs.python.org/issue10576> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue10576] Add a progress callback to gcmodule
Daniel Stutzbach added the comment: These functions will be very useful for any long-running program. Thank you for the patch. Would you be willing to write tests and documentation? Would it make more sense for the callback to take a boolean instead of an integer as the first argument? -- nosy: +stutzbach ___ Python tracker <http://bugs.python.org/issue10576> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue9915] speeding up sorting with a key
Daniel Stutzbach added the comment: Antoine, My original patch was much more focused, but had a slightly larger performance penalty for sorting random keys (see http://bugs.python.org/msg122178). Do you think the performance tradeoff there was still worthwhile? Ihave uploaded my original patch (as one patch :) ) here: http://codereview.appspot.com/3291041 If you think the original performance looks like a worthwhile tradeoff, could you spend about 1 minute looking over that version of patch and give your general impression? -- ___ Python tracker <http://bugs.python.org/issue9915> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue9915] speeding up sorting with a key
Daniel Stutzbach added the comment: Antoine Pitrou wrote: > Next time, please upload a single patch. Really. I haven't used Rietveld that much yet, and I'm still learning best-practices. I apologize for the painful experience. For anyone else planning to take a look at this, here it is as one big patch: http://codereview.appspot.com/3290041 -- ___ Python tracker <http://bugs.python.org/issue9915> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue9915] speeding up sorting with a key
Daniel Stutzbach added the comment: > How did it get *faster* than the original (in the case with no > key-function)? I was able to shave off some instructions in countrun(), binarysort(), and the setup and cleanup code in listsort() proper. For small n, these made a difference. > Is there any chance you can persuade Uncle Timmy to review this? This > is all his code and he's likely to have some good insights. Any suggestions on how he might be persuaded? ;-) > Also, is there anyone else who is knowledgeable about Python on less > common platforms? ISTM part of the optimization is dependent on the > branch-prediction and other nuances that vary from environment to > environment. That being said, doing fewer memory allocations is > always a win. I do have a Windows machine I can test on. It's not exactly a "less common" platform, but at least it's a completely different compiler. I'll post the results once I have them. The Rietveld issue is here: http://codereview.appspot.com/3269041 I ended up loading my incremental patches in, but it's easy enough to diff the base with the last patch. If for some reasons it doesn't work as conveniently as I expect, let me know and I will upload it to Rietveld again as one big patch. If there's anything else I can do to make reviewing easier, please let me know. For that matter, if there's other code you'd like me to review in exchange or straightforward bugs you'd like to pawn off, I would be happy to help out. -- ___ Python tracker <http://bugs.python.org/issue9915> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue9915] speeding up sorting with a key
Daniel Stutzbach added the comment: Raymond Hettinger added the comment: > That result is surprising though -- I thought the concept was > manipulate the key and value arrays at the same time instead of just > the keys If the "key" parameter was not used, then the values pointer is a null pointer. Each time elements must be moved, the code needs to check if the values pointer is NULL. In other words, the overhead is one conditional branch per move or group of moves (we only have to check once for memmove()). Since the branch will always be the same throughout any given call to sort(), CPU branch prediction is effective making the branches fairly inexpensive. > -- did you do more than this, perhaps changing the logic or > pattern of comparisons? In the original version of the patch, I tried hard to avoid unrelated changes. In the new version, I made many small not-strictly-related optimizations. However, I have not changed the order in which comparisons occur. > Why did the variable names change, pa/pb to ssa/ssb, etc.? I took "pa" to mean "pointer to array A", but it's now a sortslice structure instead of a pointer. > comment lines are rewrapped The comment rewrapping aren't completely spurious. The comments really have changed and in some cases the constraints on the inputs to a function have changed slightly. For what it's worth, my methodology when working on this patch went like this: 1) Make an improvement 2) Measure the performance relative to the previous reference point 3) If it was a statistically significant an improvement, commit the change to my local DVCS 4) Goto 1 I can upload the patch to Rietveld to facilitate review. If I can get Rietveld to show each of my local commits, would it be helpful to see the patch in incremental pieces? -- ___ Python tracker <http://bugs.python.org/issue9915> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue9915] speeding up sorting with a key
Daniel Stutzbach added the comment: Antoine Pitrou wrote: > Right, that wouldn't suit your present purposes. But apparently you > are proposing to add a list sorting benchmark to the Tools directory, > with lots of duplicated code from that repo... Oh, I just stuck that under Tools because it was convenient for testing. The timing script is in a separate patch because I'm -0 on committing the timing script itself. The perf.py that my timing script uses is based on the one from benchmarks/, but with 90% of the code removed, so the amount of duplicated code is less than it may at first appear. My script could go in benchmarks/ if there was a convenient way for it to import perf.py. I wouldn't want to clutter the top-level directory with my script though. Any suggestions? -- ___ Python tracker <http://bugs.python.org/issue9915> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue9915] speeding up sorting with a key
Daniel Stutzbach added the comment: Antoine Pitrou wrote: > Why don't you contribute a list sorting benchmark to the suite in > http://hg.python.org/benchmarks/? I considered that, but I want to separately benchmark sorting different kinds and quantities of data. AFAIK, there isn't an easy way to do that with perf.py. -- ___ Python tracker <http://bugs.python.org/issue9915> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue9915] speeding up sorting with a key
Changes by Daniel Stutzbach : Added file: http://bugs.python.org/file19781/detailed-speed-results.txt ___ Python tracker <http://bugs.python.org/issue9915> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue9915] speeding up sorting with a key
Changes by Daniel Stutzbach : Added file: http://bugs.python.org/file19780/sort-faster.patch ___ Python tracker <http://bugs.python.org/issue9915> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue9915] speeding up sorting with a key
Daniel Stutzbach added the comment: I'm starting to get settled in here at Google and finding time to follow up on everything that got put on hold while moving. Based on the feedback everyone gave me (thanks!), I greatly revised my script for comparing the speed of sort(). It's now compares speeds more quickly and more accurately. I discovered a few things along the way: 1) Don't try to run timings on a virtual server with crummy clock resolution. 2) Make sure that CPU frequency scaling is turned off. It increases the standard deviation of timings by an order of magnitude or two. On Linux systems, the new version of my script will check to see if CPU frequency scaling is turned on and provide suggestions on how to turn it off. On other operating systems, you are on your own. ;) Usage: filfre:~/py3k-patched$ ./python Tools/sort_speed/sort_speed.py -r 3 --maxn 10 ../py3k-pristine . sort_random Using my new-and-improved timing script and running on real hardware with CPU frequency scaling turned off, I was able to detect a slight slowdown on sorting random integers. Here's a summary for sorting random integers: Original patch (increasing execution time is bad): n in [700 .. 100,000]: 1% to 2% increase in execution time n in [70 to 700]: 2% to 3% increase n in [10 to 70]: 3% to 4% increase n in [5 to 7]: 7% to 9% decrease n in [0 to 3]: 18+% decrease I went back to the code and managed to squeeze out a bit more performance. Here's a summary of the new patch: New patch (increasing execution time is bad): n in [30,000 .. 100,000]: < 1% increase in execution time n in [35 .. 30,000]: 1% to 2% increase n in [20 .. 35]: about the same n in [10 .. 20]: 1 to 2% decrease n in [4 .. 7]: 15+% decrease n in [0 .. 3]: 22+% decrease All of the above results are for sorting without a key. For sorting *with* a key, there's a big performance boost across the board (15% to 45% decrease in execution time, when the key function is simply "int"). Overall, the patch shrinks Objects/listobject.c by 10 lines. Executive Summary - Good: - Dramatically cuts execution time of sorting with a key (15% to 45%) - Significantly cuts execution time of sorting small lists without a key (15+%) Bad: - Very slightly increases execution of sorting large lists without a key (1% to 2%; less than 1% for very large lists) Worthwhile trade? -- Added file: http://bugs.python.org/file19779/sort-speed-test.patch ___ Python tracker <http://bugs.python.org/issue9915> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue10437] ThreadPoolExecutor should accept max_workers=None
Daniel Stutzbach added the comment: Amaury Forgeot d'Arc said: > pypy does have a GIL! D'oh. That shows you how much I know about PyPy. Make that "keep in mind that IronPython doesn't have a GIL". ;) Brian Quinlan said: > I think that using the number of CPUs for max_workers makes sense > for you but won't for most users. So I wouldn't make it a default. Fair enough. Marking as invalid. -- resolution: -> invalid stage: needs patch -> committed/rejected status: open -> closed ___ Python tracker <http://bugs.python.org/issue10437> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com