[issue15079] pickle: Possibly misplaced test
Changes by Collin Winter coll...@gmail.com: -- nosy: -collinwinter ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue15079 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue13430] Add a curry function to the functools module
Collin Winter coll...@gmail.com added the comment: I assume I was added to this thread since I wrote the functional module, so I'll give my take in that capacity. IMO Python doesn't need a more general version of partial(); indeed, I question the need for partial() as it is today. Querying Google Code Search for code using partial, I haven't found any usages in the wild where partial() makes code more readable than simply defining a new function. partial() is almost always a loss for readability. -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue13430 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue9285] Add a profile decorator to profile and cProfile
Changes by Collin Winter coll...@gmail.com: -- nosy: -collinwinter ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue9285 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue2636] Regexp 2.7 (modifications to current re 2.2.2)
Changes by Collin Winter coll...@gmail.com: -- nosy: -collinwinter ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue2636 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue5572] distutils ignores the LIBS configure env var
Collin Winter coll...@gmail.com added the comment: I have no interest in forward-porting the patch, closing accordingly. At least there will be a record of this patch for anyone interested in getting a free 10% performance boost for their Python 2.x or 3.[012] systems. -- assignee: tarek - eric.araujo resolution: - wont fix status: open - closed ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue5572 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue5572] distutils ignores the LIBS configure env var
Collin Winter coll...@gmail.com added the comment: I have provided justification in the original patch submission. Without this patch, we were unable to cleanly apply gcc's feedback-directed optimization system to Python. FDO yields significant performance improvements. -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue5572 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue5572] distutils ignores the LIBS configure env var
Collin Winter coll...@gmail.com added the comment: This wasn't so much a feature request as a request for review. I had already implemented the necessary changes and provided a patch. We found this change necessary when working on Unladen Swallow. -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue5572 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue5575] Add env vars for controlling building sqlite, hashlib and ssl
Collin Winter coll...@gmail.com added the comment: I don't know that the variables are Python-specific. We used these variables to build various Python modules statically against the versions of openssl and sqlite maintained in Google's internal third-party repository. -- assignee: collinwinter - ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue5575 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue1625576] add ability to specify name to os.fdopen
Collin Winter coll...@gmail.com added the comment: Christian: yes, that sounds fine. -- assignee: collinwinter - ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue1625576 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue1731717] race condition in subprocess module
Changes by Collin Winter coll...@gmail.com: -- nosy: +collinwinter, jyasskin ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue1731717 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue1107887] Speed up function calls/can add more introspection info
Collin Winter coll...@gmail.com added the comment: I tried making this work early last year as part of Unladen Swallow, and even though I got it working and it does speed up certain builtin calls, it didn't move overall application performance at all. I believe this was due to cache effects, branch mispredicts or something else. Based on that experience, I don't believe this is really worth pursuing on its own. That said, something like this is included in Unladen Swallow, which uses it to make direct calls to certain C functions. I'm going to close this accordingly. -- resolution: - later status: open - closed type: feature request - performance ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue1107887 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue8566] 2to3 should run under python 2.5
Changes by Collin Winter coll...@gmail.com: -- nosy: +collinwinter ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue8566 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue1518] Fast globals/builtins access (patch)
Collin Winter coll...@gmail.com added the comment: Antoine: yes, this optimization is already implemented in the Unladen JIT. -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue1518 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue6713] Integer Long types: Performance improvement of 1.6x to 2x for base 10 conversions
Collin Winter coll...@gmail.com added the comment: I ran this patch against Unladen Swallow's slowspitfire template benchmark, which does more int-string conversions than any of our other benchmarks. When run against Python trunk r74737, I get these results: slowspitfire: Min: 0.888772 - 0.867427: 2.46% faster Avg: 0.891857 - 0.872461: 2.22% faster Significant (t=45.532127, a=0.95) (./perf.py -r -b slowspitfire ../a/python.exe ../b/python.exe) This was an idle MacBook Pro, OS X 10.5.8, Apple gcc 4.0.1, 2.4 GHz Core Duo. Other benchmarks benefit, but are only barely statistically significant. -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue6713 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue2636] Regexp 2.7 (modifications to current re 2.2.2)
Collin Winter coll...@gmail.com added the comment: FYI, Unladen Swallow includes several regex benchmark suites: a port of V8's regex benchmarks (regex_v8); some of the regexes used when tuning the existing sre engine 7-8 years ago (regex_effbot); and a regex_compile benchmark that tests regex compilation time. See http://code.google.com/p/unladen-swallow/wiki/Benchmarks for more details, including how to check out and run the benchmark suite. You'll need to modify your experimental Python build to have import re import the proposed regex engine, rather than _sre. The benchmark command would look something like `./perf.py -r -b regex /control/python /experiment/python`, which will run all the regex benchmarks in rigorous mode. I'll be happy to answer any questions you have about our benchmarks. I'd be very interested to see how the proposed regex engine performs on these tests. -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue2636 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue6643] joining a child that forks can deadlock in the forked child process
Changes by Collin Winter coll...@gmail.com: -- components: +Interpreter Core nosy: +collinwinter, jyasskin ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue6643 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue6292] Fix tests to work with -OO
New submission from Collin Winter coll...@gmail.com: The attached patch fixes a number of tests to work when -OO is given to Python. The majority of these tests are docstring-related, either doctests or making assertions about __doc__, with a handful of tests testing that assert statements will trigger and another handful of distutils tests that are looking for .pyc files. With this applied, PYTHONOPTIMIZE=2 ./python.exe -OO Lib/test/regrtest.py now passes for trunk. -- components: Tests files: oo_tests.patch keywords: easy, patch messages: 89434 nosy: collinwinter, jyasskin, rnk severity: normal stage: patch review status: open title: Fix tests to work with -OO type: behavior versions: Python 2.7 Added file: http://bugs.python.org/file14309/oo_tests.patch ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue6292 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue6293] Have regrtest.py echo back sys.flags
New submission from Collin Winter coll...@gmail.com: This patch makes regrtest.py echo back the contents of sys.flags at the beginning of a test run. Unladen Swallow has found this useful for verifying that the regrtest.py settings in the Makefile and in our Buildbot configs are interacting as expected. Example output: $ PYTHONOPTIMIZE=2 ./python.exe -OO Lib/test/regrtest.py Testing with flags: sys.flags(debug=0, py3k_warning=0, division_warning=0, division_new=0, inspect=0, interactive=0, optimize=2, dont_write_bytecode=0, no_user_site=0, no_site=0, ignore_environment=0, tabcheck=0, verbose=0, unicode=0, bytes_warning=0) test_grammar test_opcodes test_dict test_builtin test_exceptions test_types [snip] $ -- components: Tests files: regrtest_flags.patch keywords: easy, patch messages: 89436 nosy: collinwinter severity: normal stage: patch review status: open title: Have regrtest.py echo back sys.flags type: behavior versions: Python 2.7 Added file: http://bugs.python.org/file14310/regrtest_flags.patch ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue6293 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue6250] Python compiles dead code
Collin Winter coll...@gmail.com added the comment: Standalone bytecode-modifying tools almost never check that they're outputting correct bytecode. http://code.activestate.com/recipes/277940/ makes no attempt to check what version of Python it's running under; running it under Unladen Swallow 2009Q1 would have produced completely incorrect code because we had modified how opcode arguments were stored. I don't want to encourage people to write tools that operate over CPython bytecode. As such code proliferates, the alternate implementations have to chase it down and ask that it be removed. It complicates the process of verifying that other implementations are compatible with CPython. Like I said earlier in this issue, for me this is more a compiler cleanliness issue rather than a performance issue. -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue6250 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue6250] Python compiles dead code
Collin Winter coll...@gmail.com added the comment: As another data point, Unladen Swallow had to take explicit steps to deal with this dead code when compiling bytecode to machine code. Since Python's compiler isn't smart enough to ignore code following a return or raise in the same suite, support for that had to percolate into our compiler. For me, it's cleanliness issue, not a performance issue. That certainly lowers the priority, though. The warning James is adding for dead code detection may also be useful; it looks to have already found one bug in the stdlib. -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue6250 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue1943] improved allocation of PyUnicode objects
Collin Winter coll...@gmail.com added the comment: On Wed, Jun 3, 2009 at 2:36 AM, Marc-Andre Lemburg rep...@bugs.python.org wrote: Marc-Andre Lemburg m...@egenix.com added the comment: All this is assuming the speed-up is important enough to bother. Has anyone run a comparison benchmark using the Unladen Swallow benchmarks? I trust those much more than micro-benchmarks (including, I assume, stringbench.py). I do expect that reducing the number of allocations for short-to-medium-size strings from 2 to 1 would be a significant speed-up, but I can't guess how much. While the Unladen Swallow aims at providing high-level benchmarks, it's current state doesn't really implement that promise (yet). If you look at the list of benchmarks they use, most appear to be dealing with pickling. That doesn't strike me as particularly useful for testing real life Python usage. I would take issue with your characterization of those benchmarks. There are several benchmarks for cPickle, true, both macro and micro benchmarks, but I would not describe their number as most of [our] benchmarks. For example, slowpickle and slowunpickle both use the pure-Python pickle.py, and are testing how close we can get that implementation to the tuned cPickle version. Regardless, I don't know that any of our benchmarks really stress unicode performance. We so far haven't cared about improving the performance of unicode objects. 2to3 uses unicode internally, so that might be a good benchmark to run. If a high level benchmark is indeed what's wanted, then they should setup pre-configured Django and Zope instances and run those through a series of real-life usage scenarios to cover the web application use space. We have a benchmark for Django and Spitfire templates, both of which are heavily used in the web application use space. We focused on template languages because in talking to Google's web app teams, they found their primary CPU bottlenecks to be templating systems, not ORMs or other components. For scientific use cases, it would be good to have similar setups using BioPython, NumPy and matplotlib. And so on. Much like the high level benchmarks you have in the Windows world. We have NumPy in our correctness test suite, but no benchmarks based on it. Looking at all the packages you just named, they make heavy use of C/C++ extensions (with BioPython and matplotpib both depending on NumPy) or large C libraries (matplotlib can depend on Cairo, I see). We've been focusing on pure-Python performance, so I'm skeptical that benchmarks with such large C components would be a useful guide for our work. I'm happy to talk about this further outside of this thread, though. Collin -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue1943 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue6133] LOAD_CONST followed by LOAD_ATTR can be optimized to just be a LOAD_COST
Changes by Collin Winter coll...@gmail.com: -- nosy: +collinwinter ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue6133 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue6133] LOAD_CONST followed by LOAD_ATTR can be optimized to just be a LOAD_COST
Collin Winter coll...@gmail.com added the comment: On Fri, May 29, 2009 at 3:45 PM, Martin v. Löwis rep...@bugs.python.org wrote: py s = py s.join is s.join False Every time you read it, you get a new object. Not what I would call a constant. If you don't see how this matters, try def foo(): return .join print foo() is foo() with and without your patch. The fact that it returns a new object every time seems like an implementation detail, and one that users find confusing (I know I once filed a bug about it). One problem I recognize is that the proposed patch would presumably create an asymmetry between x.join is x.join and x = 'x'; x.join is x.join where the former would evaluate to True and the latter to False. That seems surmountable, though. -- title: LOAD_CONST followed by LOAD_ATTR can be optimized to justbe a LOAD_COST - LOAD_CONST followed by LOAD_ATTR can be optimized to just be a LOAD_COST ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue6133 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue6132] Implement the GIL with critical sections in Windows
Changes by Collin Winter coll...@gmail.com: -- nosy: +collinwinter ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue6132 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue5794] pickle/cPickle of recursive tuples create pickles that cPickle can't load
Collin Winter coll...@gmail.com added the comment: Fixed in r72930 (trunk), r72931 (2.6), r72942 (py3k). -- resolution: - fixed stage: patch review - committed/rejected status: open - closed ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue5794 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue5794] pickle/cPickle of recursive tuples create pickles that cPickle can't load
Collin Winter coll...@gmail.com added the comment: Bug-fix patch attached. Alexandre, can you take a look? Feel free to bounce it back if you don't have time. I'll port to 2.6 and py3k once this is reviewed for trunk. -- keywords: +26backport, easy, patch nosy: +alexandre.vassalotti stage: - patch review versions: +Python 2.7, Python 3.1 -Python 2.5 Added file: http://bugs.python.org/file14064/rec_tuple.patch ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue5794 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue5866] cPickle defect with tuples and different from pickle output
Changes by Collin Winter coll...@gmail.com: -- status: pending - closed ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue5866 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue5670] Speed up pickling of dicts in cPickle
Collin Winter coll...@gmail.com added the comment: Fixed the len(d) == 1 size regression. Final performance of the patch relative to trunk: Using Unladen Swallow's perf.py -b pickle,pickle_dict on trunk: pickle: Min: 2.238 - 1.895: 18.08% faster Avg: 2.241 - 1.898: 18.04% faster Significant (t=282.066701, a=0.95) pickle_dict: Min: 2.163 - 1.375: 57.36% faster Avg: 2.168 - 1.376: 57.50% faster Significant (t=527.668441, a=0.95) Performance for py3k: pickle: Min: 2.849 - 2.790: 2.10% faster Avg: 2.854 - 2.796: 2.09% faster Significant (t=27.624303, a=0.95) pickle_dict: Min: 2.121 - 1.512: 40.27% faster Avg: 2.128 - 1.519: 40.13% faster Significant (t=283.406572, a=0.95) regrtest.py -uall test_xpickle passes all backwards-compatibility tests for trunk, and all other tests run by regrtest.py on Linux pass. Committed as r72909 (trunk), r72910 (py3k). -- resolution: accepted - fixed status: open - closed ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue5670 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue6066] POP_MARK was not in pickle protocol 0
Collin Winter coll...@gmail.com added the comment: Applied in r72792 (trunk), r72793 (py3k), r72808 (release26-maint). -- assignee: - collinwinter resolution: - fixed status: open - closed ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue6066 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue6066] POP_MARK was not in pickle protocol 0
New submission from Collin Winter coll...@gmail.com: Lib/pickletools.py incorrectly thinks POP_MARK was part of protocol 0; POP_MARK was only added with the introduction of protocol 1 in r7753. This mistake led me down a dead end while fixing another pickling issue. Alexandre, can you double-check me on this? Feel free to bounce this back if you don't have time. I'll port this to py3k. I'm leaning toward a backport to 2.6; any thoughts on that? -- components: Library (Lib) files: pickletools.patch keywords: easy, patch messages: 88105 nosy: alexandre.vassalotti, collinwinter priority: low severity: normal stage: patch review status: open title: POP_MARK was not in pickle protocol 0 type: behavior versions: Python 2.7, Python 3.1 Added file: http://bugs.python.org/file14020/pickletools.patch ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue6066 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue6032] Fix refleaks in test_urllib2_localnet
Collin Winter coll...@gmail.com added the comment: Daniel, did you have any comments on this patch? If not, I'll go ahead and commit it and close both of these issues. -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue6032 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue6002] test_urllib2_localnet DigestAuthHandler leaks nonces
Collin Winter coll...@gmail.com added the comment: I've posted a patch in issue 6032 (didn't see this one); I'd be interested in any comments on the approach taken there. -- nosy: +collinwinter ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue6002 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue6002] test_urllib2_localnet DigestAuthHandler leaks nonces
Collin Winter coll...@gmail.com added the comment: Daniel Diniz reviewed in the other issue. Fixed in r72777 (trunk), r72778 (py3k). -- assignee: - collinwinter resolution: - fixed status: open - closed ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue6002 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue6032] Fix refleaks in test_urllib2_localnet
Collin Winter coll...@gmail.com added the comment: Applied as r72777 (trunk), r72778 (py3k). -- assignee: - collinwinter resolution: - fixed status: open - closed ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue6032 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue1943] improved allocation of PyUnicode objects
Collin Winter coll...@gmail.com added the comment: Daniel, which patch? freelists2.patch or unialloc4.patch? If these are targeted py3k (judging by the Versions selector above), none of Unladen Swallow's benchmarks work under 3k (we're focusing on 2.x). -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue1943 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue6042] Document and slightly simplify lnotab tracing
Changes by Collin Winter coll...@gmail.com: -- nosy: +collinwinter ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue6042 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue6032] Fix refleaks in test_urllib2_localnet
New submission from Collin Winter coll...@gmail.com: Currently (r72673), test_urllib2_localnet leaks references. This is due to state implicitly shared between tests. The attached patch fixes this by removing the shared state. The problem is also present in py3k. I'll port the patch once I commit to trunk. Should this be backported so we can take test_urllib2_localnet off the refleak test blacklist in Misc/build.sh in those branches? -- components: Tests files: urllib2_localnet.patch keywords: patch messages: 87846 nosy: collinwinter severity: normal stage: patch review status: open title: Fix refleaks in test_urllib2_localnet type: behavior versions: Python 2.7, Python 3.1 Added file: http://bugs.python.org/file13991/urllib2_localnet.patch ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue6032 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue6032] Fix refleaks in test_urllib2_localnet
Collin Winter coll...@gmail.com added the comment: Yes, this is a patch for issue 6002 (sorry, didn't find it). I haven't seen any refleaks from issue 1208304, but I was only looking at this one particular failure (since it was showing up in Unladen Swallow's refleak builds). -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue6032 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue6024] regrtest says refleaks are ok
New submission from Collin Winter coll...@gmail.com: Currently (r72643), regrtest.py -R:: says that a test passed even if it leaked references: trunk collinwinter$ ./python.exe Lib/test/regrtest.py -R:: test_urllib2_localnet test_urllib2_localnet beginning 9 repetitions 123456789 . test_urllib2_localnet leaked [3, 3, 3, 3] references, sum=12 1 test OK. [50020 refs] trunk collinwinter$ The attached patch turns that into trunk collinwinter$ ./python.exe Lib/test/regrtest.py -R:: test_urllib2_localnet test_urllib2_localnet beginning 9 repetitions 123456789 . test_urllib2_localnet leaked [3, 3, 3, 3] references, sum=12 1 test failed: test_urllib2_localnet [50010 refs] trunk collinwinter$ echo $? 1 trunk collinwinter$ This makes it easier to run regrtest.py -R:: as part of a buildbot installation. I'll merge to py3k once this is reviewed. Any thoughts on merging to release26-maint? -- components: Tests files: refleak.patch keywords: patch messages: 87776 nosy: collinwinter, jyasskin severity: normal stage: patch review status: open title: regrtest says refleaks are ok type: behavior versions: Python 2.7 Added file: http://bugs.python.org/file13985/refleak.patch ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue6024 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue6024] regrtest says refleaks are ok
Changes by Collin Winter coll...@gmail.com: Added file: http://bugs.python.org/file13986/refleak.patch ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue6024 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue6024] regrtest says refleaks are ok
Changes by Collin Winter coll...@gmail.com: Removed file: http://bugs.python.org/file13985/refleak.patch ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue6024 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue6024] regrtest says refleaks are ok
Collin Winter coll...@gmail.com added the comment: Applied in r72658 (trunk) and r72660 (py3k). -- resolution: - fixed status: open - closed ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue6024 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue5572] distutils ignores the LIBS configure env var
Collin Winter coll...@gmail.com added the comment: 2009/5/4 Tarek Ziadé rep...@bugs.python.org@psf.upfronthosting.co.za: About the patch: is there a particular reason why you have changed the call to 'set_library' into calls to 'add_library' in build_ext in your patch ? Yes. It seems like the proper thing to do is to merge the global LIBS and the module-specific libraries (via add_library()), rather than having one override the other (set_library()). -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue5572 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue5794] pickle/cPickle of recursive tuples create pickles that cPickle can't load
Changes by Collin Winter coll...@gmail.com: -- nosy: +collinwinter ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue5794 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue5794] pickle/cPickle of recursive tuples create pickles that cPickle can't load
Collin Winter coll...@gmail.com added the comment: Interestingly, it only fails with protocol 0: v = ([],) v[0].append(v) import pickle,cPickle cPickle.loads(pickle.dumps(v, 0)) Traceback (most recent call last): File stdin, line 1, in module cPickle.UnpicklingError: unpickling stack underflow cPickle.loads(pickle.dumps(v, 1)) ([([...],)],) cPickle.loads(pickle.dumps(v, 2)) ([([...],)],) I'll see if I can come up with a fix. -- assignee: - collinwinter ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue5794 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue5665] Add more pickling tests
Collin Winter coll...@gmail.com added the comment: Committed as r71408 (trunk) and r71638 (py3k). -- resolution: - fixed status: open - closed ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue5665 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue5665] Add more pickling tests
Collin Winter coll...@gmail.com added the comment: Yes, I'm porting them. I got started, but got distracted by other things since there were a lot of conflicts in the merge -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue5665 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue5665] Add more pickling tests
Collin Winter coll...@gmail.com added the comment: If no-one has any objections to the xpickle resource included in the latest version of the patch, I'd like to commit this soon so that we can be more confident in the other changes I have queued up. If I no-one objects, I'll commit this sometime early tomorrow morning PDT. -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue5665 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue5665] Add more pickling tests
Collin Winter coll...@gmail.com added the comment: Ok if it doesn't take too long to run the tests (which may imply implementing something like Raymond's suggestion of randomizing test order, if you haven't already done so). I did something similar: if you don't pass the -uxpickle flag to regrtest, only a small subset of the tests will be run, which takes less than half a second on my machine; if you do pass -uxpickle, the full battery of tests will run, which takes ~5 minutes. -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue5665 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue3873] Unpickling is really slow
Changes by Collin Winter coll...@gmail.com: -- nosy: +collinwinter ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue3873 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue5671] Speed up pickling of lists in cPickle
Collin Winter coll...@gmail.com added the comment: I've added a microbenchmark to perf.py called pickle_list. Running that on this change (perf.py -r -b pickle_list): pickle_list: Min: 1.126 - 0.888: 26.86% faster Avg: 1.154 - 0.906: 27.43% faster Significant (t=115.404547, a=0.95) That's probably the upper bound on the performance improvement to be realized from this patch. -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue5671 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue5665] Add more pickling tests
Collin Winter coll...@gmail.com added the comment: I've made test_xpickle support Python 2.4 because it uses Python 2.4, 2.5 and 2.6 to check that we haven't broken backwards compatibility with those versions. I made AbstractCompatTests use cPickle because that's what I've been changing :) I can add tests for the pure-Python pickle module if you want. At that point, though, test_xpickle will be very slow and we'll want to add a regrtest resource to control how much time test_xpickle spends running. Does xpickle work for the resource name (ie, regrtest.py -u xpickle)? I agree with your other points. -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue5665 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue5665] Add more pickling tests
Collin Winter coll...@gmail.com added the comment: test_xpickle currently takes over three minutes on my MacBook Pro to test compatibility with Python 2.4, 2.5 and 2.6, and adding tests for the pickle module will at least double that, if not push it well above 10 minutes. That's why I recommend using the resource flag if we want to add the same tests for the pickle module. -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue5665 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue5665] Add more pickling tests
Collin Winter coll...@gmail.com added the comment: Updated the patch to include a regrtest xpickle resource, which restricts if the backwards compat tests will be run. For normal development, the existing tests should be fine; you only need these tests if you're mucking with the pickle output. Added backwards compat tests for the pure-Python pickle module. Total run time with -uxpickle maxes out a little above five minutes on my machine, with a lot of variation between runs. Also addressed the comment about [0, 1, 2] - protocols. -- Added file: http://bugs.python.org/file13599/pickle_tests.patch ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue5665 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue5670] Speed up pickling of dicts in cPickle
Collin Winter coll...@gmail.com added the comment: FYI, I just added a pickle_dict microbenchmark to perf.py. Using this new microbenchmark, I see these results (perf.py -r -b pickle_dict): pickle_dict: Min: 2.092 - 1.341: 56.04% faster Avg: 2.126 - 1.360: 56.37% faster Significant (t=216.895643, a=0.95) I still need to address the comment about pickling empty dicts. -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue5670 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue5670] Speed up pickling of dicts in cPickle
Collin Winter coll...@gmail.com added the comment:
Amaury, I can't reproduce the issue you're seeing with empty dicts.
Here's what I'm doing:
dhcp-172-19-19-199:trunk collinwinter$ ./python.exe
Python 2.7a0 (trunk:71100M, Apr 3 2009, 14:40:49)
[GCC 4.0.1 (Apple Inc. build 5490)] on darwin
Type help, copyright, credits or license for more information.
import cPickle, pickletools
data = cPickle.dumps({}, protocol=2)
pickletools.dis(data)
0: \x80 PROTO 2
2: }EMPTY_DICT
3: .STOP
highest protocol among opcodes = 2
data
'\x80\x02}.'
What are you doing to produce the MARK SETITEMS sequence?
--
___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue5670
___
___
Python-bugs-list mailing list
Unsubscribe:
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue5681] Add a simple pickle optimizer to cPickle
Changes by Collin Winter coll...@gmail.com: -- components: Extension Modules files: cpickle_writes.patch keywords: needs review, patch nosy: alexandre.vassalotti, collinwinter severity: normal status: open title: Add a simple pickle optimizer to cPickle type: performance versions: Python 2.7, Python 3.1 Added file: http://bugs.python.org/file13601/cpickle_writes.patch ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue5681 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue5681] Add a simple pickle optimizer to cPickle
New submission from Collin Winter coll...@gmail.com: Bah, hit enter too soon. -- resolution: - invalid status: open - closed ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue5681 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue5683] Speed up cPickle's pickling generally
New submission from Collin Winter coll...@gmail.com: This patch simplifies cPickle's complicated internal buffering system. The new version uses a single buffer closer to the Pickler object, flushing to a file object only when necessary. This avoids the overhead of several indirection layers for what are frequently very small writes. Benchmarked against virgin trunk r71058 using perf.py -r -b pickle,pickle_list,pickle_dict: pickle: Min: 2.225 - 1.962: 13.37% faster Avg: 2.254 - 1.994: 13.03% faster Significant (t=70.763434, a=0.95) pickle_dict: Min: 2.097 - 1.418: 47.83% faster Avg: 2.136 - 1.446: 47.75% faster Significant (t=214.900146, a=0.95) pickle_list: Min: 1.128 - 0.777: 45.22% faster Avg: 1.152 - 0.807: 42.65% faster Significant (t=169.789433, a=0.95) A similar patch for unpickling will follow. As issue 5670 and 5671 are committed, I'll update this patch to support them. This patch passes all the tests added in issue 5665. I would recommend reviewing that patch first. I'll port to py3k once this is reviewed for trunk. This patch is available on Rietveld at http://codereview.appspot.com/33070. -- components: Extension Modules files: cpickle_writes.patch keywords: needs review, patch messages: 85349 nosy: alexandre.vassalotti, collinwinter severity: normal stage: patch review status: open title: Speed up cPickle's pickling generally type: performance versions: Python 2.7, Python 3.1 Added file: http://bugs.python.org/file13603/cpickle_writes.patch ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue5683 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue5634] cPickle error in case of recursion limit
Changes by Collin Winter coll...@gmail.com: -- nosy: +collinwinter ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue5634 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue3675] Python 2.6 can't read sets pickled with Python 3.0
Changes by Collin Winter coll...@gmail.com: -- nosy: +collinwinter ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue3675 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue2389] Array pickling exposes internal memory representation of elements
Changes by Collin Winter coll...@gmail.com: -- nosy: +collinwinter ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue2389 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue5670] Speed up pickling of dicts in cPickle
New submission from Collin Winter coll...@gmail.com: The attached patch adds another version of cPickle.c's batch_dict(), batch_dict_exact(), which is specialized for type(x) is dict. This provides a nice performance boost when pickling objects that use dictionaries: Pickle: Min: 2.216 - 1.858: 19.24% faster Avg: 2.238 - 1.889: 18.50% faster Significant (t=106.874099, a=0.95) Benchmark is at http://code.google.com/p/unladen-swallow/source/browse/tests/performance/macro_pickle.py (driver is ../perf.py; perf.py was run with --rigorous -b pickle). This patch passes all the tests added in issue 5665. I would recommend reviewing that patch first. I'll port to py3k once this is reviewed for trunk. -- components: Extension Modules files: cpickle_dict.patch keywords: needs review, patch messages: 85239 nosy: collinwinter severity: normal status: open title: Speed up pickling of dicts in cPickle type: performance versions: Python 2.7 Added file: http://bugs.python.org/file13584/cpickle_dict.patch ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue5670 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue5671] Speed up pickling of lists in cPickle
New submission from Collin Winter coll...@gmail.com: The attached patch adds another version of cPickle.c's batch_list(), batch_list_exact(), which is specialized for type(x) is list. This provides a nice performance boost when pickling objects that use lists. This is similar to the approach taken in issue 5670, though the performance boost on our benchmark is smaller: Pickle: Min: 2.231 - 2.200: 1.39% faster Avg: 2.266 - 2.227: 1.72% faster Significant (t=10.994064, a=0.95) Benchmark is at http://code.google.com/p/unladen-swallow/source/browse/tests/performance/macro_pickle.py (driver is ../perf.py; perf.py was run with --rigorous -b pickle). Workloads involving more lists will benefit more. This patch passes all the tests added in issue 5665. I would recommend reviewing that patch first. I'll port to py3k once this is reviewed for trunk. -- components: Extension Modules files: cpickle_list.patch keywords: needs review, patch messages: 85250 nosy: collinwinter severity: normal status: open title: Speed up pickling of lists in cPickle type: performance versions: Python 2.7 Added file: http://bugs.python.org/file13585/cpickle_list.patch ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue5671 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue5670] Speed up pickling of dicts in cPickle
Collin Winter coll...@gmail.com added the comment: On Thu, Apr 2, 2009 at 12:20 PM, Antoine Pitrou rep...@bugs.python.org wrote: Antoine Pitrou pit...@free.fr added the comment: By the way, could the same approach be applied to lists and sets as well? Certainly; see http://bugs.python.org/issue5671 for the list version. It doesn't make as big an impact on the benchmark, though. -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue5670 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue5671] Speed up pickling of lists in cPickle
Collin Winter coll...@gmail.com added the comment: No, we haven't started looking at other serialization formats yet. Marshal will probably be my next target, with json being a lower priority. There were enough instances of low-hanging fruit in cPickle that I didn't go looking at the other implementations for ideas. -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue5671 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue5670] Speed up pickling of dicts in cPickle
Collin Winter coll...@gmail.com added the comment: Antoine: pickletester.py:test_newobj_generic() appears to test dict subclasses, though in a roundabout-ish way. I don't know of any tests for dict subclasses in the C level sense (ie, PyDict_Check() vs PyDict_CheckExact()). I can add more explicit tests for Python-level dict subclasses, if you want. -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue5670 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue5665] Add more pickling tests
New submission from Collin Winter coll...@gmail.com: The attached patch adds more tests for pickling: - Add tests for the module-level load() and dump() functions. - Add tests for cPickle's internal data structures, stressing workloads with many gets/puts. - Add tests for the Pickler and Unpickler classes, in particular the memo attribute. - test_xpickle is extended to test backwards compatibility with Python 2.4, 2.5 and 2.6 by round-tripping pickled objects through a worker process. 2.3 was too difficult to support. I'll port to py3k after reviewed for trunk. -- components: Tests files: pickle_tests.patch keywords: needs review, patch messages: 85162 nosy: collinwinter severity: normal status: open title: Add more pickling tests type: behavior versions: Python 2.7, Python 3.1 Added file: http://bugs.python.org/file13571/pickle_tests.patch ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue5665 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue1714451] subprocess.py problems errors when calling cmd.exe
Collin Winter coll...@gmail.com added the comment: Do you know of anyone actively working on Windows support? If not, I say close it as won't fix. -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue1714451 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue5588] Add --randseed to regrtest
New submission from Collin Winter coll...@gmail.com: Add the ability to control the random seed used by regrtest.py -r. This patch adds a --randseed option, and makes regrtest.py -r indicate what random seed it's using so that that value can later be fed back to --randseed. This option is useful for tracking down test order-related issues found by make buildbottest, for example. -- components: Tests files: randseed.patch keywords: needs review, patch messages: 84322 nosy: collinwinter severity: normal stage: patch review status: open title: Add --randseed to regrtest type: feature request versions: Python 2.7 Added file: http://bugs.python.org/file13446/randseed.patch ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue5588 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue5588] Add --randseed to regrtest
Changes by Collin Winter coll...@gmail.com: Removed file: http://bugs.python.org/file13446/randseed.patch ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue5588 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue5588] Add --randseed to regrtest
Collin Winter coll...@gmail.com added the comment: The if rand_seed: bit was a relic from a previous iteration; fixed. The only reason I didn't use randrange() is that I didn't see it; fixed. -- Added file: http://bugs.python.org/file13448/randseed.patch ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue5588 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue5588] Add --randseed to regrtest
Collin Winter coll...@gmail.com added the comment: Committed as r70672 (trunk) and r70673 (py3k). Thanks for the quick review. -- resolution: - accepted status: open - closed ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue5588 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue5572] distutils ignores the LIBS configure env var
New submission from Collin Winter coll...@gmail.com: If you pass LIBS to ./configure (as in LIBS=-lgcov ./configure), distutils ignores this when building extension modules, which breaks when using certain gcc options which require certain libraries (I'm thinking of -fprofile-generate). The attached patch remedies this. -- assignee: tarek components: Distutils files: distutils_libs.patch keywords: needs review, patch, patch messages: 84199 nosy: collinwinter, jyasskin, tarek severity: normal stage: patch review status: open title: distutils ignores the LIBS configure env var type: behavior versions: Python 2.7 Added file: http://bugs.python.org/file13422/distutils_libs.patch ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue5572 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue5572] distutils ignores the LIBS configure env var
Collin Winter coll...@gmail.com added the comment: The patch attached to issue 5060 seems very tightly focused on a problem with gcc FDO. This is a more general patch that solves the problem of distutils ignoring LIBS. -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue5572 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue5575] Add env vars for controlling building sqlite, hashlib and ssl
New submission from Collin Winter coll...@gmail.com: This patch adds SSL_ROOT, SQLITE_INC and SQLITE_LIB environment variables used to inject additional libraries/headers for building the sqlite, hashlib and ssl modules. We've found this very useful for building these modules against their dependencies statically for deployment purposes. This may well not be useful for most CPython users, but I figured it would be nice to have a record here. -- components: Build files: setup_env_vars.patch keywords: needs review, patch, patch messages: 84220 nosy: collinwinter severity: normal stage: patch review status: open title: Add env vars for controlling building sqlite, hashlib and ssl versions: Python 2.7 Added file: http://bugs.python.org/file13424/setup_env_vars.patch ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue5575 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue5557] Byte-code compilation uses excessive memory
Changes by Collin Winter coll...@gmail.com: -- nosy: +collinwinter ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue5557 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue5372] Distutils inappropriately reuses .o files between extension modules
New submission from Collin Winter coll...@gmail.com:
(Tarek, I've been told you're the new distutils maintainer. Feel free to
unassign this if that isn't the case.)
The test distutils uses to decide whether it needs to recompile an
existing .o file when building extension modules is too simplistic,
merely comparing the modification time of the .o and .c files. If you
have two extension modules like so
Extension('foo', ['foo.c', 'bar.c'],
define_macros=[('NO_STATIC_MEMOTABLE', 1)])
Extension('bar', ['bar.c'])
even if defining NO_STATIC_MEMOTABLE=1 radically changes the code of
bar.c, distutils will use the same bar.o for both the foo and bar
extension modules. This was a real problem for me at work today.
The attached patch removes the modtime comparison entirely, preferring
to rebuild all .o files a given extension module needs. Note that while
_prep_compiler() isn't used anywhere in the stdlib, Google Code Search
turns up uses.
--
assignee: tarek
components: Distutils
files: ccompiler.patch
keywords: needs review, patch, patch
messages: 82734
nosy: collinwinter, jyasskin, tarek
severity: normal
stage: patch review
status: open
title: Distutils inappropriately reuses .o files between extension modules
type: behavior
versions: Python 2.7
Added file: http://bugs.python.org/file13180/ccompiler.patch
___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue5372
___
___
Python-bugs-list mailing list
Unsubscribe:
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue2636] Regexp 2.7 (modifications to current re 2.2.2)
Changes by Collin Winter coll...@gmail.com: -- nosy: +collinwinter ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue2636 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue5362] Add configure option to disable Py3k warnings
Collin Winter coll...@gmail.com added the comment: Jeffrey: updated the patch to address your concerns. Martin: I'm not sure I completely understand it either, though it seems similar to issue4477. In the course of developing this patch, I tried also #ifdef'ing out all usages of the Py_Py3kWarningFlag global. This actually made things slower by around 5% across all the benchmarks I tested. Could be pipeline stalls or a code size issue, I really don't know. I'm not 100% convinced that something like this should go into CPython, as a different compiler/hardware combination could well render it moot. I mostly wanted a record of it, in case those few Python deployments with homogeneous compilers/hardware across their machines that might care about 1% better performance are interested. Added file: http://bugs.python.org/file13183/no_py3k_warning.patch ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue5362 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue5362] Add configure option to disable Py3k warnings
Changes by Collin Winter coll...@gmail.com: Removed file: http://bugs.python.org/file13168/no_py3k_warning.patch ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue5362 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue5362] Add configure option to disable Py3k warnings
Collin Winter coll...@gmail.com added the comment: Bah, forgot to run autoreconf. Fixed. Added file: http://bugs.python.org/file13184/no_py3k_warning.patch ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue5362 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue5362] Add configure option to disable Py3k warnings
Changes by Collin Winter coll...@gmail.com: Removed file: http://bugs.python.org/file13183/no_py3k_warning.patch ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue5362 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue5362] Add configure option to disable Py3k warnings
New submission from Collin Winter coll...@gmail.com: The attached patch adds a --with-py3k-warnings option to configure. Passing --without-py3k-warnings disables all Py3k compatibility warnings (the default is to keep the warnings). For production deployments where performance is more important than warnings no-one will see, this can provide a useful performance improvement: 2to3 (translate 2to3's source tree five times): Min: 22.406 - 22.114: 1.32% faster Avg: 22.439 - 22.158: 1.27% faster Django (render a 150x150 table 100 times): Min: 0.595 - 0.586: 1.58% faster Avg: 0.598 - 0.588: 1.76% faster Spitfire (render a 1000x1000 table 100 times): Min: 0.752 - 0.743: 1.29% faster Avg: 0.754 - 0.745: 1.25% faster Unpickle (unpickling a list of 8000 dicts 100 times): Min: 0.305 - 0.301: 1.41% faster Avg: 0.307 - 0.302: 1.49% faster Build env: gcc 4.3.1 x86_64 on Linux 2.6.18 (Core2 Duo) -- assignee: jyasskin components: Interpreter Core files: no_py3k_warning.patch keywords: needs review, patch, patch messages: 82683 nosy: brett.cannon, collinwinter, jyasskin, pitrou severity: normal status: open title: Add configure option to disable Py3k warnings type: performance versions: Python 2.7 Added file: http://bugs.python.org/file13168/no_py3k_warning.patch ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue5362 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue5176] Special-case string formatting in BINARY_MODULO implementation
Collin Winter coll...@gmail.com added the comment: Committed as r69811. -- resolution: - accepted status: open - closed ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue5176 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue2459] speedup for / while / if with better bytecode
Collin Winter coll...@gmail.com added the comment: On Fri, Feb 13, 2009 at 3:23 PM, Collin Winter coll...@gmail.com wrote: On Fri, Feb 13, 2009 at 10:37 AM, Antoine Pitrou rep...@bugs.python.org wrote: Before committing I want to know what to do with the new jump opcodes, with respect to the alternative proposal I've made in #4715. Ideally, I should fold the #4715 patch back into the present patch, since I think the #4715 approach is more thought out. That sounds good, especially since Jeffrey and I have already reviewed #4715. If you don't have the bandwidth to integrate 4715 into this patch, I can do that, if you want. Collin ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue2459 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue2459] speedup for / while / if with better bytecode
Collin Winter coll...@gmail.com added the comment: I don't see the changes to the lnotab format being a roadblock; just mention it in NEWS. Likewise, the pure-Python compiler package shouldn't be a high priority; your changes to that package look good enough. I'm seeing encouraging speed-ups out of this (with gcc 4.3.1 x86_64, compiling Python as 64-bit): Django templates (render a 150x150 table 100 times): Min: 0.595 - 0.589: 0.94% faster Avg: 0.599 - 0.591: 1.30% faster Spitfire templates (render a 1000x1000 table 100 times): Min: 0.751 - 0.729: 2.98% faster Avg: 0.753 - 0.730: 3.09% faster None of the apps I've benchmarked are negatively impacted. I only have two minor comments. Please commit this. Review comments: - The changes to Python/compile.c:compiler_if(), compiler_for(), compiler_while() have some indentation issues (tabs and such). - Functions like def foo(): while True: pass have a useless JUMP_FORWARD 0 instruction, but I don't think it's worth teaching the peepholer to remove them since it doesn't happen in other circumstances (that I can tell). ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue2459 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue2459] speedup for / while / if with better bytecode
Collin Winter coll...@gmail.com added the comment: On Fri, Feb 13, 2009 at 10:37 AM, Antoine Pitrou rep...@bugs.python.org wrote: Antoine Pitrou pit...@free.fr added the comment: Hello Collin, Thanks for taking a look. I don't see the changes to the lnotab format being a roadblock; just mention it in NEWS. Likewise, the pure-Python compiler package shouldn't be a high priority; your changes to that package look good enough. Well, I have good news: the fixes to the pure Python compiler have been accepted and committed by Neil Schemenauer in r69373. Yeah, I saw that. Fantastic. I'm seeing encouraging speed-ups out of this (with gcc 4.3.1 x86_64, compiling Python as 64-bit): Django templates (render a 150x150 table 100 times): Min: 0.595 - 0.589: 0.94% faster Avg: 0.599 - 0.591: 1.30% faster Spitfire templates (render a 1000x1000 table 100 times): Min: 0.751 - 0.729: 2.98% faster Avg: 0.753 - 0.730: 3.09% faster Not a tremendous speedup but not totally insignificant either. (I see you like Spitfire :-)) Well, Spitfire and Django represent very different ways of implementing a template system, so I like to measure both when doing application benchmarks. Template systems tend to be heavy CPU consumers for webapps, which is why I include them. None of the apps I've benchmarked are negatively impacted. I only have two minor comments. Please commit this. Before committing I want to know what to do with the new jump opcodes, with respect to the alternative proposal I've made in #4715. Ideally, I should fold the #4715 patch back into the present patch, since I think the #4715 approach is more thought out. That sounds good, especially since Jeffrey and I have already reviewed #4715. ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue2459 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue5176] Special-case string formatting in BINARY_MODULO implementation
Collin Winter coll...@gmail.com added the comment: Updated the patch to use only PyString_CheckExact(); added a test for the behaviour of string subclasses wrt the % operator. There's a very slight performance hit when using % with numbers, but it's so small as to be statistically insignificant. If it turns out to be relevant in the future, it's easy enough to add a special case for ints/longs. For some reason, using PyString_CheckExact instead of PyString_CheckExact || PyString_Check actually results in slower code (still an improvement, but not as much). The weird thing is that none of the benchmarks I'm running use % on string subclasses at any point. I talked about it with some of the gcc guys, but they didn't have any immediate leads. I'm going to send them the .o files in case gcc is missing some optimization. New benchmark numbers: Spitfire: Min: 0.687 - 0.668: 2.96% faster Avg: 0.689 - 0.669: 2.96% faster 2to3: Min: 20.376 - 20.187: 0.94% faster Avg: 20.396 - 20.225: 0.84% faster Django: Min: 0.560 - 0.549: 1.94% faster Avg: 0.562 - 0.552: 1.93% faster SlowPickle: Min: 0.920 - 0.905: 1.62% faster Avg: 0.926 - 0.913: 1.38% faster Added file: http://bugs.python.org/file13077/faster_modulo.patch ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue5176 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue5176] Special-case string formatting in BINARY_MODULO implementation
Changes by Collin Winter coll...@gmail.com: Removed file: http://bugs.python.org/file12962/faster_modulo.patch ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue5176 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue4751] Patch for better thread support in hashlib
Changes by Collin Winter coll...@gmail.com: -- nosy: +collinwinter, jyasskin ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue4751 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue5176] Special-case string formatting in BINARY_MODULO implementation
New submission from Collin Winter coll...@gmail.com: This patch speeds up the string formatting % operator by avoiding the unnecessary indirection in PyNumber_Remainder(). This particularly benefits templating systems that do a lot of string formatting via %. Performance tested with gcc 4.3.1 x86_64 on Linux 2.6.18: 2to3: Min: 22.443 - 22.306: 0.61% faster Avg: 22.465 - 22.324: 0.63% faster Django: Min: 0.598 - 0.591: 1.24% faster Avg: 0.601 - 0.595: 1.03% faster Spitfire [1]: Min: 0.752 - 0.717: 4.91% faster Avg: 0.754 - 0.719: 4.83% faster The Django test renders a 150x150 cell table 100 times. The Spitfire test renders a 1000x1000 cell table 100 times. The 2to3 test translates all of 2to3 with `-f all` five times. There is no significant impact on PyBench. Less important code like the pure-Python pickle implementation also shows an improvement: Pickling: Min: 6.680 - 6.535: 2.22% faster Avg: 6.740 - 6.576: 2.50% faster This is pickling a list of 4 dicts 100 times. This does not impact unpickling. About the patch: I tested various combinations of PyString_Check and PyString_CheckExact and what you see in the patch demonstrated the best performance across all benchmarks. I also tested changing the definition of PyString_Check to include a short-circuit on PyString_CheckExact, but that did not provide a consistent benefit. [1] - http://code.google.com/p/spitfire/ -- assignee: jyasskin components: Interpreter Core files: faster_modulo.patch keywords: needs review, patch, patch messages: 81319 nosy: collinwinter, jyasskin severity: normal status: open title: Special-case string formatting in BINARY_MODULO implementation type: performance versions: Python 2.7 Added file: http://bugs.python.org/file12962/faster_modulo.patch ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue5176 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue5084] unpickling does not intern attribute names
Changes by Collin Winter coll...@gmail.com: -- nosy: +collinwinter, jyasskin ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue5084 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue2459] speedup for / while / if with better bytecode
Changes by Collin Winter coll...@gmail.com: -- nosy: +collinwinter ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue2459 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue1518] Fast globals/builtins access (patch)
Changes by Collin Winter coll...@gmail.com: -- nosy: +collinwinter, jyasskin type: feature request - performance ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue1518 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue4896] Faster why variable manipulation in ceval.c
Collin Winter coll...@gmail.com added the comment: Another data point: I've tested this patch applied to trunk on Core 2 Duo and Opteron 8214 HE machines using both gcc 4.0.3 and 4.3.1, and I'm seeing mixed results. Pybench with warp 1 is between ~1.5% slower and ~1% faster, depending on gcc version (fairly consistent across machines). 2to3 and two template systems I've tested are between 2.5% slower and 2% faster depending on workload and gcc version. ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue4896 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue1700288] Armin's method cache optimization updated for Python 2.6
Collin Winter coll...@gmail.com added the comment: Looks like this was re-applied in r59943 and r59944. Thanks for taking care of this, Amaury. -- nosy: +collinwinter, jyasskin resolution: accepted - fixed status: open - closed ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue1700288 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue4715] optimize bytecode for conditional branches
Collin Winter coll...@gmail.com added the comment:
I've backported condbranches-plus.patch to trunk, and I'm getting these
results:
PyBench: 1.84-2.21% faster
2to3: 3.83% faster
Spitfire: 6.13-6.23% faster
PyBench was run with -w=1; 2to3 is translating its entire source
directory five times; Spitfire is measured by the Spitfire -O4 line
from tests/perf/bigtable.py, run 15 times. This is on a Core 2 system.
My AMD system (otherwise identical) shows somewhat less improvement,
but still an improvement.
I've haven't tested condbranches.patch vs condbranches-plus.patch; what
difference are you seeing, Antoine?
I like these changes. Folding POP into JUMP_IF_{TRUE,FALSE} should have
been done years ago (I think I proposed it and Raymond shot me down).
I'm also in favor of making POP_JUMP_IF_* absolute jumps.
This patch mostly looks good, though you still need to change Doc/
library/dis.rst and the pure-Python compiler package. I'd get someone
else to look over the patch, too, though.
___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue4715
___
___
Python-bugs-list mailing list
Unsubscribe:
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue4715] optimize bytecode for conditional branches
Collin Winter coll...@gmail.com added the comment: On Tue, Jan 13, 2009 at 3:25 PM, Antoine Pitrou rep...@bugs.python.org wrote: Antoine Pitrou pit...@free.fr added the comment: Hello, I've backported condbranches-plus.patch to trunk, and I'm getting these results: Thanks! PyBench: 1.84-2.21% faster 2to3: 3.83% faster Spitfire: 6.13-6.23% faster What is Spitfire? http://code.google.com/p/spitfire/. It's a template system designed for performance that I have some experience with. I've haven't tested condbranches.patch vs condbranches-plus.patch; what difference are you seeing, Antoine? condbranches.patch is the earlier version (without POP_OR_JUMP and JUMP_OR_POP), it can be ignored. I was mostly curious whether the POP_OR_JUMP and JUMP_OR_POP opcodes had a noticeable performance impact, ie, do they make things fast enough to warrant their inclusion over the old JUMP_IF_FALSE implementations. This patch mostly looks good, though you still need to change Doc/ library/dis.rst and the pure-Python compiler package. Well, the pure-Python compiler package doesn't exist in py3k, for which I've initially made the patch. (and its state in 2.x is very sorry...) I'll update the compiler package in 2.x and post my patch once I have it working. There are also some test failures in 2.x that I'll take care of. ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue4715 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue1479611] speed up function calls
Changes by Collin Winter coll...@gmail.com: -- nosy: +collinwinter, jyasskin ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue1479611 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
