Re: [Python-Dev] Disabling cyclic GC in timeit module

2011-10-18 Thread John Arbash Meinel
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 ... If you are only measuring json encoding of a few select pieces of data then it's a microbenchmark. If you are measuring the whole application (or a significant part of it) then I'm not sure timeit is the right tool for that. Regards

Re: [Python-Dev] Test cases not garbage collected after run

2011-04-14 Thread John Arbash Meinel
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On 4/14/2011 1:23 AM, Martin (gzlist) wrote: On 07/04/2011, Michael Foord fuzzy...@voidspace.org.uk wrote: On 07/04/2011 20:18, Robert Collins wrote: Testtools did something to address this problem, but I forget what it was offhand. Some

Re: [Python-Dev] PEP 396, Module Version Numbers

2011-04-06 Thread John Arbash Meinel
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 ... #. ``__version_info__`` SHOULD be of the format returned by PEP 386's ``parse_version()`` function. The only reference to parse_version in PEP 386 I could find was the setuptools implementation which is pretty odd: In other words,

Re: [Python-Dev] I am now lost - committed, pulled, merged, what is collapse?

2011-03-21 Thread John Arbash Meinel
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On 3/21/2011 10:44 AM, Martin v. Löwis wrote: My understanding is that svn does not detect fast forwards, only lack of conflicts, and therefore in case of concurrent development it is possible that the repository contains a version that never

Re: [Python-Dev] Hg: inter-branch workflow

2011-03-20 Thread John Arbash Meinel
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On 3/20/2011 5:06 AM, R. David Murray wrote: On Thu, 17 Mar 2011 14:33:00 +0100, wrote: On Thu, 17 Mar 2011 09:24:26 -0400 R. David Murray rdmurray at bitdance.com wrote: It would be great if rebase did work with share, that would make a push

Re: [Python-Dev] Suggest reverting today's checkin (recursive constant folding in the peephole optimizer)

2011-03-12 Thread John Arbash Meinel
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 ... I have always felt uncomfortable with *any* kind of optimization -- whether AST-based or bytecode-based. I feel the cost in code complexity is pretty high and in most cases the optimization is not worth the effort. Also I don't see the point

Re: [Python-Dev] PyCObject_AsVoidPtr removed from python 3.2 - is this documented?

2011-03-07 Thread John Arbash Meinel
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On 3/7/2011 3:56 AM, Terry Reedy wrote: On 3/6/2011 6:09 PM, Barry Scott wrote: I see that PyCObject_AsVoidPtr has been removed from python 3.2. The 3.2 docs do not seem to explain this has happened and what to replace it with. I searched the

Re: [Python-Dev] Actual Mercurial Roadmap for February (Was: svn outage on Friday)

2011-02-22 Thread John Arbash Meinel
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On 2/22/2011 9:41 AM, anatoly techtonik wrote: On Fri, Feb 18, 2011 at 4:00 PM, Dirkjan Ochtman dirk...@ochtman.nl wrote: On Fri, Feb 18, 2011 at 14:41, anatoly techtonik techto...@gmail.com wrote: Do you have a public list of stuff to be done

Re: [Python-Dev] svn outage on Friday

2011-02-15 Thread John Arbash Meinel
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On 2/15/2011 8:03 AM, Benjamin Peterson wrote: 2011/2/15 Victor Stinner victor.stin...@haypocalc.com: Le mardi 15 février 2011 à 09:30 +0100, Martin v. Löwis a écrit : I'm going to perform a Debian upgrade of svn.python.org on Friday, between 9:00

Re: [Python-Dev] Mercurial Schedule

2010-11-19 Thread John Arbash Meinel
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On 11/19/2010 7:50 AM, Nick Coghlan wrote: On Fri, Nov 19, 2010 at 5:43 PM, Georg Brandl g.bra...@gmx.net wrote: Am 19.11.2010 03:23, schrieb Benjamin Peterson: 2010/11/18 Jesus Cea j...@jcea.es: -BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On

[Python-Dev] Intended behavior of backlash in raw strings

2010-07-12 Thread John Arbash Meinel
I'm trying to determine if this is intended behavior: r\ '\\' r'\'' \\' Normally, the quote would end the string, but it gets escaped by the preceding '\'. However, the preceding slash is interpreted as 'not a backslash' because of the raw indicator, so it gets left in verbatim. Note that it

Re: [Python-Dev] versioned .so files for Python 3.2

2010-07-07 Thread John Arbash Meinel
Scott Dial wrote: On 6/30/2010 2:53 PM, Barry Warsaw wrote: It might be amazing, but it's still a significant overhead. As I've described, multiply that by all the py files in all the distro packages containing Python source code, and then still try to fit it on a CDROM. I decided to prove

Re: [Python-Dev] email package status in 3.X

2010-06-21 Thread John Arbash Meinel
... IOW, if you're producing output that has to go into another system that doesn't take unicode, it doesn't matter how theoretically-correct it would be for your app to process the data in unicode form. In that case, unicode is not a feature: it's a bug. This is not always true. If you

Re: [Python-Dev] PEP 3148 ready for pronouncement

2010-05-21 Thread John Arbash Meinel
Brian Quinlan wrote: The PEP is here: http://www.python.org/dev/peps/pep-3148/ I think the PEP is ready for pronouncement, and the code is pretty much ready for submission into py3k (I will have to make some minor changes in the patch like changing the copyright assignment):

Re: [Python-Dev] PEP 3148 ready for pronouncement

2010-05-21 Thread John Arbash Meinel
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Brian Quinlan wrote: The PEP is here: http://www.python.org/dev/peps/pep-3148/ I think the PEP is ready for pronouncement, and the code is pretty much ready for submission into py3k (I will have to make some minor changes in the patch like

Re: [Python-Dev] Reasons behind misleading TypeError message when passing the wrong number of arguments to a method

2010-05-19 Thread John Arbash Meinel
Giampaolo Rodolà wrote: class A: ... def echo(self, x): ... return x ... a = A() a.echo() Traceback (most recent call last): File stdin, line 1, in module TypeError: echo() takes exactly 2 arguments (1 given) I bet my last 2 cents this has already been raised in past

Re: [Python-Dev] urlparse.urlunsplit should be smarter about +

2010-05-08 Thread John Arbash Meinel
Stephen J. Turnbull wrote: David Abrahams writes: This is a bug report. bugs.python.org seems to be down. from urlparse import * urlunsplit(urlsplit('git+file:///foo/bar/baz')) git+file:/foo/bar/baz Note the dropped slashes after the colon. That's clearly

Re: [Python-Dev] PEP 3146: Merge Unladen Swallow into CPython

2010-01-22 Thread John Arbash Meinel
sstein...@gmail.com wrote: On Jan 21, 2010, at 11:32 PM, Chris Bergstresser wrote: On Thu, Jan 21, 2010 at 9:49 PM, Tres Seaver tsea...@palladion.com wrote: Generally, that's not going to be the case. But the broader point--that you've no longer got an especially good idea of what's

Re: [Python-Dev] PEP 3146: Merge Unladen Swallow into CPython

2010-01-22 Thread John Arbash Meinel
Collin Winter wrote: Hey Jake, ... Hmm. So cProfile doesn't break, but it causes code to run under a completely different execution model so the numbers it produces are not connected to reality? We've found the call graph and associated execution time information from cProfile to be

Re: [Python-Dev] PEP 3146: Merge Unladen Swallow into CPython

2010-01-21 Thread John Arbash Meinel
Collin Winter wrote: Hi Dirkjan, On Wed, Jan 20, 2010 at 10:55 PM, Dirkjan Ochtman dirk...@ochtman.nl wrote: On Thu, Jan 21, 2010 at 02:56, Collin Winter collinwin...@google.com wrote: Agreed. We are actively working to improve the startup time penalty. We're interested in getting guidance

Re: [Python-Dev] PEP 3146: Merge Unladen Swallow into CPython

2010-01-21 Thread John Arbash Meinel
Martin v. Löwis wrote: There is freeze: http://wiki.python.org/moin/Freeze Which IIRC Robert Collins tried in the past, but didn't see a huge gain. It at least tries to compile all of your python files to C files and then build an executable out of that. to C files is a bit of an

Re: [Python-Dev] GIL required for _all_ Python calls?

2010-01-06 Thread John Arbash Meinel
MRAB wrote: Hi, I've been wondering whether it's possible to release the GIL in the regex engine during matching. I know that it needs to have the GIL during memory-management calls, but does it for calls like Py_UNICODE_TOLOWER or PyErr_SetString? Is there an easy way to find out? Or is

Re: [Python-Dev] Splitting something into two steps produces different behavior from doing it in one fell swoop in Python 2.6.2

2009-12-11 Thread John Arbash Meinel
Roy Hyunjin Han wrote: While debugging a network algorithm in Python 2.6.2, I encountered some strange behavior and was wondering whether it has to do with some sort of code optimization that Python does behind the scenes. After initialization: defaultdict(type 'set', {1:

Re: [Python-Dev] PEP 3003 - Python Language Moratorium

2009-11-07 Thread John Arbash Meinel
... A moratorium isn't cost-free. With the back-end free to change, patches will go stale over 2+ years. People will lose interest or otherwise move on. Those with good ideas but little patience will be discouraged. I fully expect that, human nature being as it is, those proposing a

Re: [Python-Dev] Retrieve an arbitrary element from a setwithoutremoving it

2009-11-05 Thread John Arbash Meinel
geremy condra wrote: On Thu, Nov 5, 2009 at 4:09 PM, Alexander Belopolsky alexander.belopol...@gmail.com wrote: On Thu, Nov 5, 2009 at 3:43 PM, Chris Bergstresser ch...@subtlety.com wrote: .. and x = iter(s).next() raises a StopIteration exception. And that's why the documented recipe

Re: [Python-Dev] Reworking the GIL

2009-11-02 Thread John Arbash Meinel
Sturla Molden wrote: Antoine Pitrou skrev: It certainly is. But once again, I'm no Windows developer and I don't have a native Windost host to test on; therefore someone else (you?) has to try. I'd love to try, but I don't have VC++ to build Python, I use GCC on Windows. Anyway, the

Re: [Python-Dev] Retrieve an arbitrary element from a set without removing it

2009-10-25 Thread John Arbash Meinel
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Martin v. Löwis wrote: Hmm, perhaps when using sets as work queues? A number of comments: - it's somewhat confusing to use a set as a *queue*, given that it won't provide FIFO semantics. - there are more appropriate and direct container

Re: [Python-Dev] Retrieve an arbitrary element from a set without removing it

2009-10-24 Thread John Arbash Meinel
Adam Olsen wrote: On Fri, Oct 23, 2009 at 11:04, Vitor Bosshard algor...@gmail.com wrote: I see this as being useful for frozensets as well, where you can't get an arbitrary element easily due to the obvious lack of .pop(). I ran into this recently, when I had a frozenset that I knew had 1

Re: [Python-Dev] Retrieve an arbitrary element from a set without removing it

2009-10-23 Thread John Arbash Meinel
Vitor Bosshard wrote: 2009/10/23 Willi Richert w.rich...@gmx.net: Hi, recently I wrote an algorithm, in which very often I had to get an arbitrary element from a set without removing it. Three possibilities came to mind: 1. x = some_set.pop() some_set.add(x) 2. for x in some_set:

Re: [Python-Dev] Retrieve an arbitrary element from a set without removing it

2009-10-23 Thread John Arbash Meinel
Terry Reedy wrote: John Arbash Meinel wrote: So 'for x in s: break' is about 2x faster than next(iter(s)) and 3x faster than (iter(s).next()). I was pretty surprised that it was 30% faster than for x in s: pass. I assume it has something to do with a potential else: statement? for x in s

Re: [Python-Dev] GIL behaviour under Windows

2009-10-21 Thread John Arbash Meinel
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Antoine Pitrou wrote: I don't really know how this test works, so I won't claim to understand the results either. However, here you go: Thanks. Interesting results. I wonder what they would be like on a multi-core machine. The GIL seems to

Re: [Python-Dev] [Python-ideas] Remove GIL with CAS instructions?

2009-10-21 Thread John Arbash Meinel
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Kristján Valur Jónsson wrote: ... This depends entirely on the platform and primitives used to implement the GIL. I'm interested in windows. There, I found this article: http://fonp.blogspot.com/2007/10/fairness-in-win32-lock-objects.html So,

Re: [Python-Dev] GIL behaviour under Windows

2009-10-21 Thread John Arbash Meinel
Antoine Pitrou wrote: Sturla Molden sturla at molden.no writes: It does not crash the interpreter, but it seems it can deadlock. Kristján sent me a patch which I applied and is supposed to fix this. Anyway, thanks for the numbers. The GIL does seem to fare a bit better (zero latency with

Re: [Python-Dev] GIL behaviour under Windows

2009-10-21 Thread John Arbash Meinel
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Antoine Pitrou wrote: Le mercredi 21 octobre 2009 à 12:42 -0500, John Arbash Meinel a écrit : You can use time.clock() instead to get 15ms resolution. Changing all instances of 'time.time' to 'time.clock' gives me this result: [snip] --- Latency

Re: [Python-Dev] PEP 385: the eol-type issue

2009-08-05 Thread John Arbash Meinel
Mark Hammond wrote: On 5/08/2009 8:14 PM, Dirkjan Ochtman wrote: endings. Typically, in my case, that was either Notepad2 (an awesomely light-weight Notepad replacement) or Komodo (Edit). That solved all of my issues, so I haven't had a need for win32text so far. FWIW, I use komodo and

Re: [Python-Dev] Easy way to detect filesystem case-sensitivity?

2009-05-07 Thread John Arbash Meinel
Andrew Bennetts wrote: Antoine Pitrou wrote: Robert Kern robert.kern at gmail.com writes: Since one may have more than one filesystem side-by-side, this can't be just be a system-wide boolean somewhere. One would have to query the target directory for this information. I am not aware of

Re: [Python-Dev] Rethinking intern() and its data structure

2009-04-09 Thread John Arbash Meinel
... Anyway, I the internals of intern() could be done a bit better. Here are some concrete things: [snip] Memory usage is definitely something we're interested in improving. Since you've already looked at this in some detail, could you try implementing one or two of your ideas and

Re: [Python-Dev] Rethinking intern() and its data structure

2009-04-09 Thread John Arbash Meinel
Christian Heimes wrote: John Arbash Meinel wrote: When I looked at the actual references from interned, I saw mostly variable names. Considering that every variable goes through the python intern dict. And when you look at the intern function, it doesn't use setdefault logic, it actually does

Re: [Python-Dev] Rethinking intern() and its data structure

2009-04-09 Thread John Arbash Meinel
Alexander Belopolsky wrote: On Thu, Apr 9, 2009 at 11:02 AM, John Arbash Meinel j...@arbash-meinel.com wrote: ... a) Don't keep a double reference to both key and value to the same object (1 pointer per entry), this could be as simple as using a Set() instead of a dict

Re: [Python-Dev] Rethinking intern() and its data structure

2009-04-09 Thread John Arbash Meinel
... I like your rationale (save memory) much more, and was asking in the tracker for specific numbers, which weren't forthcoming. ... Now that you brought up a specific numbers, I tried to verify them, and found them correct (although a bit unfortunate), please see my test script below.

Re: [Python-Dev] Rethinking intern() and its data structure

2009-04-09 Thread John Arbash Meinel
Martin v. Löwis wrote: I don't have numbers on how much that would improve CPU times, I would imagine improving 'intern()' would impact import times more than run times, simply because import time is interning a *lot* of strings. Though honestly, Bazaar would really like this, because startup

Re: [Python-Dev] Rethinking intern() and its data structure

2009-04-09 Thread John Arbash Meinel
Greg Ewing wrote: John Arbash Meinel wrote: And the way intern is currently written, there is a third cost when the item doesn't exist yet, which is another lookup to insert the object. That's even rarer still, since it only happens the first time you load a piece of code that uses a given

Re: [Python-Dev] Rethinking intern() and its data structure

2009-04-09 Thread John Arbash Meinel
... Somewhat true, though I know it happens 25k times during startup of bzr... And I would be a *lot* happier if startup time was 100ms instead of 400ms. I don't want to quash your idealism too severely, but it is extremely unlikely that you are going to get anywhere near that kind of