Re: [Python-Dev] Hg: inter-branch workflow

2011-03-17 Thread Reid Kleckner
On Thu, Mar 17, 2011 at 9:33 AM, Antoine Pitrou wrote: > On Thu, 17 Mar 2011 09:24:26 -0400 > "R. David Murray" wrote: >> >> It would be great if rebase did work with share, that would make a >> push race basically a non-issue for me. > > rebase as well as strip destroy some history, meaning some

Re: [Python-Dev] public visibility of python-dev decisions "before it's too late" (was: PyCObject_AsVoidPtr removed from python 3.2 - is this documented?)

2011-03-14 Thread Reid Kleckner
On Mon, Mar 14, 2011 at 6:30 PM, Lennart Regebro wrote: > On Wed, Mar 9, 2011 at 01:15, Stefan Behnel wrote: >> I can confirm that the Cython project was as surprised of the PyCapsule >> change in Python 3.2 as (I guess) most other users, and I would claim that >> we are a project with one of the

Re: [Python-Dev] cpython: Add a 'timeout' argument to subprocess.Popen.

2011-03-14 Thread Reid Kleckner
On Mon, Mar 14, 2011 at 12:36 PM, Antoine Pitrou wrote: > On Mon, 14 Mar 2011 17:16:11 +0100 > reid.kleckner wrote: >> @@ -265,34 +271,43 @@ >>        generates enough output to a pipe such that it blocks waiting >>        for the OS pipe buffer to accept more data. >> >> +   .. versionchanged::

Re: [Python-Dev] Python3 regret about deleting list.sort(cmp=...)

2011-03-12 Thread Reid Kleckner
On Sat, Mar 12, 2011 at 4:58 PM, Nick Coghlan wrote: > On Sat, Mar 12, 2011 at 4:50 PM, Reid Kleckner > wrote: >> They should be able to use a slotted cmp_to_key style class: >> http://docs.python.org/howto/sorting.html >> >> That will allocate 1 Python object

Re: [Python-Dev] Python3 regret about deleting list.sort(cmp=...)

2011-03-12 Thread Reid Kleckner
They should be able to use a slotted cmp_to_key style class: http://docs.python.org/howto/sorting.html That will allocate 1 Python object with no dict per key, but that might not be good enough. Reid On Sat, Mar 12, 2011 at 3:44 PM, Guido van Rossum wrote: > I was just reminded that in Python 3

Re: [Python-Dev] Const-correctness in C-API Object Protocol

2011-02-22 Thread Reid Kleckner
On Tue, Feb 22, 2011 at 2:09 PM, Eric Smith wrote: > Also changing it now would be a giant hassle, leading to so-called "const > poisoning" where many, many APIs need to be changed before everything would > again work. The poisoning will not break any users of the API, though, since they can pass

Re: [Python-Dev] News of the faulthandler project

2011-02-03 Thread Reid Kleckner
On Thu, Feb 3, 2011 at 8:05 AM, Victor Stinner wrote: >  - SIGABRT is not handled Why not? That seems useful for debugging assertion failures, although most C code in Python raises exceptions rather than asserting. I'm guessing it's because it aborts the process after printing the backtrace. Y

Re: [Python-Dev] Possible optimization for LOAD_FAST ?

2011-01-04 Thread Reid Kleckner
On Tue, Jan 4, 2011 at 8:21 AM, Alex Gaynor wrote: > Ugh, I can't be the only one who finds these special cases to be a little > nasty? > Special cases aren't special enough to break the rules. > Alex +1, I don't think nailing down a few builtins is that helpful for optimizing Python. Anyone att

Re: [Python-Dev] PyPy 1.4 released

2010-11-26 Thread Reid Kleckner
Congratulations! Excellent work. Reid On Fri, Nov 26, 2010 at 1:23 PM, Maciej Fijalkowski wrote: > === > PyPy 1.4: Ouroboros in practice > === > > We're pleased to announce the 1.4 release of PyPy. This is a major > breakthrough > in our

Re: [Python-Dev] [Python-checkins] r86467 - in python/branches/py3k: Doc/library/logging.rst Lib/logging/__init__.py Misc/NEWS

2010-11-15 Thread Reid Kleckner
On Mon, Nov 15, 2010 at 8:24 AM, Nick Coghlan wrote: > On Mon, Nov 15, 2010 at 7:33 AM, vinay.sajip > wrote: >> >> +   .. attribute:: stack_info >> + >> +      Stack frame information (where available) from the bottom of the stack >> +      in the current thread, up to and including the stack fr

Re: [Python-Dev] Locked-in defect? 32-bit hash values on 64-bit builds

2010-10-15 Thread Reid Kleckner
On Fri, Oct 15, 2010 at 4:10 PM, Raymond Hettinger wrote: > > On Oct 15, 2010, at 10:40 AM, Benjamin Peterson wrote: > >> I think the panic is a bit of an overreaction. PEP 384 has still not >> been accepted, and I haven't seen a final decision about freezing the >> ABI in 3.2. > > Not sure where

Re: [Python-Dev] View tracker patches with ViewVC?

2010-07-27 Thread Reid Kleckner
On Tue, Jul 27, 2010 at 7:56 AM, Terry Reedy wrote: >> I also suggest that, instead of uploading the patch to Rietveld >> yourself, you can ask the submitter to do it. > > That adds another step. > > Let me repeat me original question: Would it be feasible to add a [view] > button that I could cli

Re: [Python-Dev] Python 3 optimizations...

2010-07-23 Thread Reid Kleckner
On Fri, Jul 23, 2010 at 11:26 AM, stefan brunthaler wrote: >> I'm guessing from your comments below about cross-module inlining that >> you generate a separate .c file with the specialized opcode bodies and >> then call through to them via a table of function pointers indexed by >> opcode, but I c

Re: [Python-Dev] Python 3 optimizations...

2010-07-23 Thread Reid Kleckner
On Fri, Jul 23, 2010 at 1:58 AM, stefan brunthaler wrote: >> Do I understand correctly that you modify the byte code of modules/functions >> at runtime? >> > Yes. Quickening is runtime only optimization technique that rewrites > instructions from a generic instruction to an optimized derivative >

Re: [Python-Dev] Set the namespace free!

2010-07-22 Thread Reid Kleckner
On Thu, Jul 22, 2010 at 11:49 AM, Alexander Belopolsky wrote: > On Thu, Jul 22, 2010 at 12:53 PM,   wrote: >> I'm very amused by all the jokes about turning python into perl, but there's >> a good idea here that doesn't actually require that... > > No, there isn't.  And both '&' and '|' are valid

Re: [Python-Dev] New regex module for 3.2?

2010-07-22 Thread Reid Kleckner
On Thu, Jul 22, 2010 at 7:42 AM, Georg Brandl wrote: > Am 22.07.2010 14:12, schrieb Nick Coghlan: >> On Thu, Jul 22, 2010 at 9:34 PM, Georg Brandl wrote: >>> So, I thought there wasn't a difference in performance for this use case >>> (which is compiling a lot of regexes and matching most of them

[Python-Dev] Timeouts for subprocess module

2010-07-21 Thread Reid Kleckner
Hi python-dev, I've been working through a patch to add timeouts to the subprocess module: http://bugs.python.org/issue5673 It's gotten a fair amount of review, and I'm planning to commit it. Since it's my first contribution, I'm taking Georg's suggestion to send mail to python-dev to see if anyo

[Python-Dev] Unladen swallow status

2010-07-21 Thread Reid Kleckner
On Wed, Jul 21, 2010 at 8:11 AM, Tim Golden wrote: > Brett suggested that > the Unladen Swallow merge to trunk was waiting for some work to complete > on the JIT compiler and Georg, as release manager for 3.2, confirmed that > Unladen Swallow would not be merged before 3.3. Yeah, this has slipped

Re: [Python-Dev] Python-dev signal-to-noise processing question

2010-07-21 Thread Reid Kleckner
On Wed, Jul 21, 2010 at 4:43 AM, "Martin v. Löwis" wrote: > Unfortunately (?) the question also revealed a lack of understanding > of a fairly basic concept. IIUC, he wanted to know how Python > handles SIGKILL, when the hole point of SIGKILL is that you cannot > handle it. So he shouldn't have be

Re: [Python-Dev] Does trace modules have a unit test?

2010-07-20 Thread Reid Kleckner
On Tue, Jul 20, 2010 at 10:51 AM, Eli Bendersky wrote: > As Terry wrote in the beginning of this thread, Lib/test/test_trace.py > currently tests the sys.settrace module, so the tests of trace.py > should find a new home. Does Lib/test/test_trace_module.py make sense > or is something else prefera

Re: [Python-Dev] Function Operators

2010-07-18 Thread Reid Kleckner
Usual disclaimer: python-dev is for the development *of* python, not *with*. See python-list, etc. That said, def declares new functions or methods, so you can't put arbitrary expressions in there like type(f).__mul__ . You can usually assign to things like that though, but in this case you run

Re: [Python-Dev] New regex module for 3.2?

2010-07-13 Thread Reid Kleckner
On Mon, Jul 12, 2010 at 2:07 PM, Nick Coghlan wrote: > MRAB's module offers a superset of re's features rather than a subset > though, so once it has had more of a chance to bake on PyPI it may be > worth another look. I feel like the new module is designed to replace the current re module, and s

Re: [Python-Dev] commit privs

2010-07-13 Thread Reid Kleckner
Thanks for the support! Georg Brandl authorized my SSH keys for SVN access. Reid On Tue, Jul 13, 2010 at 7:29 AM, Gregory P. Smith wrote: > > On Sun, Jul 11, 2010 at 9:28 AM, Antoine Pitrou wrote: >> >> On Sun, 11 Jul 2010 13:23:13 +0000 >> Reid Kleckner wrote: >

Re: [Python-Dev] [Idle-dev] Removing IDLE from the standard library

2010-07-12 Thread Reid Kleckner
On Mon, Jul 12, 2010 at 9:20 AM, Kurt B. Kaiser wrote: > Also, the current right click edit action on Windows is to only open an > edit window; no shell.  And it uses the subprocess!  So, some of the > comments on this thread are not up to date. > > The reason that bug languished for two years was

[Python-Dev] Threading bug review + commit privs

2010-07-11 Thread Reid Kleckner
Hey all, I'm porting some fixes for threading.py that we applied to unladen-swallow: http://bugs.python.org/issue6643 We ran into these bizarre race conditions involving fork + threads while running the test suite with a background JIT compilation thread. I really wish we weren't trying to suppo

Re: [Python-Dev] More detailed build instructions for Windows

2010-07-03 Thread Reid Kleckner
On Sat, Jul 3, 2010 at 12:00 AM, "Martin v. Löwis" wrote: >> I'm trying to test out a patch to add a timeout in subprocess.py on >> Windows, so I need to build Python with Visual Studio.  The docs say >> the files in PCBuild/ work with VC 9 and newer. > > Which docs did you look at specifically th

[Python-Dev] More detailed build instructions for Windows

2010-07-02 Thread Reid Kleckner
Hey folks, I'm trying to test out a patch to add a timeout in subprocess.py on Windows, so I need to build Python with Visual Studio. The docs say the files in PCBuild/ work with VC 9 and newer. I downloaded Visual C++ 2010 Express, and it needs to convert the .vcproj files into .vcxproj files,

Re: [Python-Dev] PEP 3148 ready for pronouncement

2010-05-27 Thread Reid Kleckner
On Thu, May 27, 2010 at 4:13 AM, Brian Quinlan wrote: > Keep in mind that this library magic is consistent with the library magic > that the threading module does - unless the user sets Thread.daemon to True, > the interpreter does *not* exit until the thread does. Is there a compelling to make t

Re: [Python-Dev] variable name resolution in exec is incorrect

2010-05-27 Thread Reid Kleckner
On Thu, May 27, 2010 at 11:42 AM, Nick Coghlan wrote: > However, attaining the (sensible) behaviour Colin is requesting when such > top level variable references exist would actually be somewhat tricky. > Considering Guido's suggestion to treat two argument exec like a function > rather than a cla

Re: [Python-Dev] Tuning Python dicts

2010-04-13 Thread Reid Kleckner
On Tue, Apr 13, 2010 at 12:12 PM, Daniel Stutzbach wrote: > I don't know what benchmarks were used to write dictnotes.txt, but moving > forward I would recommend implementing your changes on trunk (i.e., Python > 2.x) and running the Unladen Swallow Benchmarks, which you can get from the > link be

Re: [Python-Dev] Tuning Python dicts

2010-04-10 Thread Reid Kleckner
On Sat, Apr 10, 2010 at 4:40 PM, Antoine Pitrou wrote: > Reid Kleckner mit.edu> writes: >> >> I think you're right about the number of collisions, though.  CPython >> dicts use a pretty low load factor (2/3) to keep collision counts >> down.  One of the major

Re: [Python-Dev] Tuning Python dicts

2010-04-10 Thread Reid Kleckner
On Sat, Apr 10, 2010 at 2:57 PM, "Martin v. Löwis" wrote: >> Any other advice would also be helpful. > > I may be missing the point, but ISTM that the assumption of this > approach is that there are often collisions in the hash table. I think > that assumption is false; at least, I recommend to va

[Python-Dev] Tuning Python dicts

2010-04-10 Thread Reid Kleckner
Hey folks, I was looking at tuning Python dicts for a data structures class final project. I've looked through Object/dictnotes.txt, and obviously there's already a large body of work on this topic. My idea was to alter dict collision resolution as described in the hopscotch hashing paper[1]. I

Re: [Python-Dev] Scope object (Re: nonlocals() function?)

2010-04-05 Thread Reid Kleckner
On Mon, Apr 5, 2010 at 7:35 PM, Antoine Pitrou wrote: > If you can prove that making locals() (or its replacement) writable doesn't > complicate the interpreter core too much, then why not. Otherwise -1 :-) I think writable locals would significantly complicate the job of people trying to optimiz

Re: [Python-Dev] python compiler

2010-04-05 Thread Reid Kleckner
On Mon, Apr 5, 2010 at 11:11 AM, Michael Foord wrote: > Python itself is a highly dynamic language and not amenable to direct > compilation. Instead modern just-in-time compiler technology is seen as the > way to improve Python performance. Projects that are doing this are PyPy and > Unladen Swall

Re: [Python-Dev] [Python-checkins] r79397 - in python/trunk: Doc/c-api/capsule.rst Doc/c-api/cobject.rst Doc/c-api/concrete.rst Doc/data/refcounts.dat Doc/extending/extending.rst Include/Python.h Incl

2010-03-28 Thread Reid Kleckner
On Sun, Mar 28, 2010 at 3:25 PM, Larry Hastings wrote: > M.-A. Lemburg wrote: > > Just as reminder of the process we have in place for such changes: > Please discuss any major breakage on python-dev before checking in > the patch. > > > I'm aware this is a good idea.  I simply didn't consider this

Re: [Python-Dev] Caching function pointers in type objects

2010-03-02 Thread Reid Kleckner
I don't think this will help you solve your problem, but one thing we've done in unladen swallow is to hack PyType_Modified to invalidate our own descriptor caches. We may eventually want to extend that into a callback interface, but it probably will never be considered an API that outside code sh

Re: [Python-Dev] PEP 3146: Merge Unladen Swallow into CPython

2010-02-03 Thread Reid Kleckner
On Wed, Feb 3, 2010 at 6:51 AM, M.-A. Lemburg wrote: > You lost me there :-) > > I am not familiar with how U-S actually implements the compilation > step and was thinking of it working at the functions/methods level > and based on input/output parameter type information. Yes, but it's more like

Re: [Python-Dev] PEP 3146: Merge Unladen Swallow into CPython

2010-02-02 Thread Reid Kleckner
On Tue, Feb 2, 2010 at 8:57 PM, Collin Winter wrote: >> Wouldn't it be possible to have the compiler approach work >> in three phases in order to reduce the memory footprint and >> startup time hit, ie. >> >>  1. run an instrumented Python interpreter to collect all >>    the needed compiler infor

Re: [Python-Dev] Forking and Multithreading - enemy brothers

2010-02-01 Thread Reid Kleckner
On Mon, Feb 1, 2010 at 5:48 PM, Jesse Noller wrote: > Your reasonable argument is making it difficult for me to be irrational > about this. No problem. :) > This begs the question - assuming a patch that clones the behavior of win32 > for multiprocessing, would the default continue to be forkin

Re: [Python-Dev] Forking and Multithreading - enemy brothers

2010-02-01 Thread Reid Kleckner
On Mon, Feb 1, 2010 at 5:20 PM, "Martin v. Löwis" wrote: > Instead, we should aim to make Python fork-safe. If the primary concern > is that locks get inherited, we should change the Python locks so that > they get auto-released on fork (unless otherwise specified on lock > creation). This may sou

Re: [Python-Dev] Forking and Multithreading - enemy brothers

2010-02-01 Thread Reid Kleckner
On Mon, Feb 1, 2010 at 5:18 PM, Jesse Noller wrote: > I don't disagree there; but then again, I haven't seen this issue > arise (in my own code)/no bug reports/no test cases that show this to > be a consistent issue. I'm perfectly OK with being wrong, I'm just > leery to tearing out the internals

Re: [Python-Dev] PEP 3147: PYC Repository Directories

2010-01-31 Thread Reid Kleckner
On Sun, Jan 31, 2010 at 8:34 AM, Nick Coghlan wrote: > That still leaves the question of what to do with __file__ (for which > even the solution in the PEP isn't particularly clean). Perhaps the > thing to do there is to have __file__ always point to the source file > and introduce a __file_cached

Re: [Python-Dev] PEP 3146: Merge Unladen Swallow into CPython

2010-01-28 Thread Reid Kleckner
On Thu, Jan 28, 2010 at 1:14 PM, Paul Moore wrote: > So, just to extend the question a little (or reiterate, it may be that > this is already covered and I didn't fully understand): > > On Windows, would a C extension author be able to distribute a single > binary (bdist_wininst/bdist_msi) which w

Re: [Python-Dev] PEP 3146: Merge Unladen Swallow into CPython

2010-01-25 Thread Reid Kleckner
On Mon, Jan 25, 2010 at 9:05 PM, Meador Inge wrote: > Also related to reduced code size with C++ I was wondering whether or not > anyone has explored using the ability of some toolchains to remove unused > code and data?  In GCC this can be enabled by compiling with > '-ffunction-sections' and '-f

Re: [Python-Dev] PEP 3146: Merge Unladen Swallow into CPython

2010-01-21 Thread Reid Kleckner
On Thu, Jan 21, 2010 at 5:07 PM, David Malcolm wrote: > To what extent would it be possible to use (conditionally) use full > ahead-of-time compilation as well as JIT? It would be possible to do this, but it doesn't have nearly the same benefits as JIT compilation, as Alex mentioned. You could d

Re: [Python-Dev] PEP 3146: Merge Unladen Swallow into CPython

2010-01-21 Thread Reid Kleckner
On Thu, Jan 21, 2010 at 4:34 PM, "Martin v. Löwis" wrote: >> How large is the LLVM shared library? One surprising data point is that the >> binary is much larger than some of the memory footprint measurements given in >> the PEP. > > Could it be that you need to strip the binary, or otherwise remo

Re: [Python-Dev] PEP 3146: Merge Unladen Swallow into CPython

2010-01-21 Thread Reid Kleckner
On Thu, Jan 21, 2010 at 3:14 PM, Collin Winter wrote: >> P.S. Is there any chance of LLVM doing something like tracing JITs? >> Those seem somewhat more promising to me (even though I understand >> they're quite hard in the face of Python features like stack frames). > > Yes, you could implement a

Re: [Python-Dev] PEP 3146: Merge Unladen Swallow into CPython

2010-01-21 Thread Reid Kleckner
On Thu, Jan 21, 2010 at 12:27 PM, Jake McGuire wrote: > On Wed, Jan 20, 2010 at 2:27 PM, Collin Winter > wrote: >> Profiling >> - >> >> Unladen Swallow integrates with oProfile 0.9.4 and newer [#oprofile]_ to >> support >> assembly-level profiling on Linux systems. This means that oProf

Re: [Python-Dev] PEP 3146: Merge Unladen Swallow into CPython

2010-01-21 Thread Reid Kleckner
On Thu, Jan 21, 2010 at 9:35 AM, Floris Bruynooghe wrote: > I just compiled with the --without-llvm option and see that the > binary, while only an acceptable 4.1M, still links with libstdc++.  Is > it possible to completely get rid of the C++ dependency if this option > is used?  Introducing a C+

Re: [Python-Dev] PEP 3146: Merge Unladen Swallow into CPython

2010-01-21 Thread Reid Kleckner
On Thu, Jan 21, 2010 at 7:25 AM, Antoine Pitrou wrote: >> 32-bit; gcc 4.0.3 >> >> +-+---+---+--+ | >> Binary size | CPython 2.6.4 | CPython 3.1.1 | Unladen Swallow r988 | >> +=+===+===+=

Re: [Python-Dev] PEP 3146: Merge Unladen Swallow into CPython

2010-01-20 Thread Reid Kleckner
On Wed, Jan 20, 2010 at 8:14 PM, Terry Reedy wrote: > If CPython development moves to distributed hg, the notion of 'blessed' > branches (other than the PSF release branch) will, as I understand it, > become somewhat obsolete. If you make a branch publicly available, anyone > can grab it and merge

Re: [Python-Dev] Whether to call Py_Finalize when exiting from the child process of a fork from a spawned thread

2009-09-01 Thread Reid Kleckner
On Tue, Sep 1, 2009 at 2:58 PM, "Martin v. Löwis" wrote: >> On one hand, you may not want to call the user's atexit handlers >> multiple times from different processes if they have externally >> visible effects.  On the other hand, people seem to assume that >> Py_Finalize will be called at process

[Python-Dev] Whether to call Py_Finalize when exiting from the child process of a fork from a spawned thread

2009-09-01 Thread Reid Kleckner
Hi all, I'm working on http://bugs.python.org/issue6642 for unladen swallow (because it happens to bite us in a weird way), and Jeff Yasskin told me to ask python-dev what the proper behavior should be at exit from the child process of a fork from a spawned thread. Right now, it seems that there