Re: [Python-Dev] Python 3.4, marshal dumps slower (version 3 protocol)

2014-01-27 Thread Kristján Valur Jónsson
Hi there. I think you should modify your program to marshal (and load) a compiled module. This is where the optimizations in versions 3 and 4 become important. K > -Original Message- > From: Python-Dev [mailto:python-dev- > bounces+kristjan=ccpgames@python.org] On Behalf Of Victor Stin

Re: [Python-Dev] Negative times behaviour in itertools.repeat for Python maintenance releases (2.7, 3.3 and maybe 3.4)

2014-01-27 Thread Larry Hastings
On 01/27/2014 06:26 PM, Vajrasky Kok wrote: So I believe the doc fix is required then. I propose the docstring should describe only supported behavior, and the docs in the manual should mention the unsupported behavior. However, I'm interested in Raymond's take, as he's the original author o

Re: [Python-Dev] Negative times behaviour in itertools.repeat for Python maintenance releases (2.7, 3.3 and maybe 3.4)

2014-01-27 Thread Vajrasky Kok
On Mon, Jan 27, 2014 at 9:13 PM, Larry Hastings wrote: > > > While it's a bug, it's a very minor bug. As Python 3.4 release manager, my > position is: Python 3.4 is in beta, so let's not change semantics for > purity's sakes now. I'm -0.5 on adding times=None right now, and until we > do we can'

Re: [Python-Dev] Negative times behaviour in itertools.repeat for Python maintenance releases (2.7, 3.3 and maybe 3.4)

2014-01-27 Thread Larry Hastings
On 01/27/2014 06:00 PM, Vajrasky Kok wrote: On Mon, Jan 27, 2014 at 9:13 PM, Larry Hastings wrote: I apologize for not making myself clear. But that's part of what I meant, yes: we should preserve the existing behavior of times=-1 when passed in by position or by keyword. However, we should *

Re: [Python-Dev] Negative times behaviour in itertools.repeat for Python maintenance releases (2.7, 3.3 and maybe 3.4)

2014-01-27 Thread Vajrasky Kok
On Mon, Jan 27, 2014 at 8:29 PM, Antoine Pitrou wrote: > > Sure, just adjust the number to fit the available memory (here, 2**29 > does the trick). > I get your point. But strangely enough, I can still recover from list(repeat('a', 2**29)). It only slows down my computer. I can ^Z the application

Re: [Python-Dev] Negative times behaviour in itertools.repeat for Python maintenance releases (2.7, 3.3 and maybe 3.4)

2014-01-27 Thread Vajrasky Kok
On Mon, Jan 27, 2014 at 9:13 PM, Larry Hastings wrote: > > I apologize for not making myself clear. But that's part of what I meant, > yes: we should preserve the existing behavior of times=-1 when passed in by > position or by keyword. However, we should *also* add a deprecation warning > when

Re: [Python-Dev] Python 3.4, marshal dumps slower (version 3 protocol)

2014-01-27 Thread Serhiy Storchaka
27.01.14 17:35, Victor Stinner написав(ла): Python 3.4 has two new versions: 3 and 4. The version 3 "shares common object references", the version 4 adds short tuples and short strings (produce smaller files). Why we need two new versions added in one Python release? _

Re: [Python-Dev] Python 3.4, marshal dumps slower (version 3 protocol)

2014-01-27 Thread Wolfgang
Thanks Victor for improving this. I also have to note, version 3 is only in the case of tuple in tuple slower. If you use a flat tuple it is faster than version 2. So I asked for this corner case and thought the recursion detection or something else has a huge cost. For pyc files, I think the hig

Re: [Python-Dev] News from asyncio

2014-01-27 Thread Devin Jeanpierre
On Mon, Jan 27, 2014 at 5:21 AM, Victor Stinner wrote: > - asyncio.IncompleReadError.expected is the total expected size, not > the remaining size Why not be consistent with the meaning of http.client.IncompleteRead.expected? The current meaning can be recovered via len(e.partial) + e.expected.

Re: [Python-Dev] Negative times behaviour in itertools.repeat for Python maintenance releases (2.7, 3.3 and maybe 3.4)

2014-01-27 Thread Georg Brandl
Am 27.01.2014 13:12, schrieb Antoine Pitrou: > On Mon, 27 Jan 2014 04:01:02 -0800 > Larry Hastings wrote: >> >> On 01/27/2014 01:39 AM, Antoine Pitrou wrote: >> > On Sun, 26 Jan 2014 21:01:08 -0800 >> > Larry Hastings wrote: >> >> On 01/26/2014 08:40 PM, Alexander Belopolsky wrote: >> >>> On Sun

Re: [Python-Dev] Python 3.4, marshal dumps slower (version 3 protocol)

2014-01-27 Thread Brett Cannon
On Mon, Jan 27, 2014 at 10:42 AM, Paul Moore wrote: > On 27 January 2014 15:35, Victor Stinner wrote: > > Version 2 is the fastest in Python 3.3 and 3.4, but version 4 with > > Python 3.4 produces the smallest file. > > Which version is used when creating pyc files? This benchmark might > sugges

Re: [Python-Dev] Python 3.4, marshal dumps slower (version 3 protocol)

2014-01-27 Thread Paul Moore
On 27 January 2014 15:35, Victor Stinner wrote: > Version 2 is the fastest in Python 3.3 and 3.4, but version 4 with > Python 3.4 produces the smallest file. Which version is used when creating pyc files? This benchmark might suggest that version 2 is the best... Paul ___

Re: [Python-Dev] Python 3.4, marshal dumps slower (version 3 protocol)

2014-01-27 Thread Victor Stinner
Hi, I'm surprised: marshal.dumps() doesn't raise an error if you pass an invalid version. In fact, Python 3.3 only supports versions 0, 1 and 2. If you pass 3, it will use the version 2. (Same apply for version 99.) Python 3.4 has two new versions: 3 and 4. The version 3 "shares common object ref

[Python-Dev] Python 3.4, marshal dumps slower (version 3 protocol)

2014-01-27 Thread Wolfgang
Hi, I tested the latest beta from 3.4 (b3) and noticed there is a new marshal protocol version 3. The documentation is a little silent about the new features, not going into detail. I've run a performance test with the new protocol version and noticed the new version is two times slower in serial

Re: [Python-Dev] Negative times behaviour in itertools.repeat for Python maintenance releases (2.7, 3.3 and maybe 3.4)

2014-01-27 Thread Mark Lawrence
On 27/01/2014 12:56, Steven D'Aprano wrote: As it stands now, the documentation is internally contradictory. In one part of the documentation, it gives a clear indication that "times is None" should select the repeat forever behaviour. In another part of the documentation, it fails to mention th

Re: [Python-Dev] News from asyncio

2014-01-27 Thread Victor Stinner
2014-01-27 Serhiy Storchaka : > 27.01.14 12:55, Victor Stinner написав(ла): > >> IncompleteReadError has two additionnal attributes: >> >> - partial: "incomplete" received bytes >> - expected: total number of expected bytes (n parameter of readexactly) > > This looks similar to http.client.Incomple

Re: [Python-Dev] Negative times behaviour in itertools.repeat for Python maintenance releases (2.7, 3.3 and maybe 3.4)

2014-01-27 Thread Larry Hastings
On 01/27/2014 04:56 AM, Steven D'Aprano wrote (rearranged slightly so I could make my points in order): I'm confused... you seem to be saying that you are *against* changing the behaviour of repeat so that: repeat(x, -1) and repeat(x, times=-1) behave the same. Is that actually wh

Re: [Python-Dev] Negative times behaviour in itertools.repeat for Python maintenance releases (2.7, 3.3 and maybe 3.4)

2014-01-27 Thread Steven D'Aprano
On Mon, Jan 27, 2014 at 04:29:04AM -0800, Larry Hastings wrote: > The code currently has semantics that cannot be accurately represented > in a Python signature. We could do one of three things: > > 1) Do nothing, and don't allow inspect.Signature to produce a signature > for the function. Th

Re: [Python-Dev] Negative times behaviour in itertools.repeat for Python maintenance releases (2.7, 3.3 and maybe 3.4)

2014-01-27 Thread Nick Coghlan
On 27 January 2014 22:29, Antoine Pitrou wrote: > On Mon, 27 Jan 2014 20:22:53 +0800 > Vajrasky Kok wrote: >> >> >>> from itertools import repeat >> >>> list(repeat('a', 2**31)) >> Traceback (most recent call last): >> File "", line 1, in >> MemoryError > > Sure, just adjust the number to fit

Re: [Python-Dev] Negative times behaviour in itertools.repeat for Python maintenance releases (2.7, 3.3 and maybe 3.4)

2014-01-27 Thread Antoine Pitrou
On Mon, 27 Jan 2014 20:22:53 +0800 Vajrasky Kok wrote: > > >>> from itertools import repeat > >>> list(repeat('a', 2**31)) > Traceback (most recent call last): > File "", line 1, in > MemoryError Sure, just adjust the number to fit the available memory (here, 2**29 does the trick). Regards

Re: [Python-Dev] Negative times behaviour in itertools.repeat for Python maintenance releases (2.7, 3.3 and maybe 3.4)

2014-01-27 Thread Larry Hastings
On 01/27/2014 01:47 AM, Mark Lawrence wrote: On 27/01/2014 01:52, Nick Coghlan wrote: In 3.5, that will be passing None, rather than -1. For those proposing to change the maintenance releases, how should a user relying on this misbehaviour update their code to handle it? I'm -1 on using None

Re: [Python-Dev] News from asyncio

2014-01-27 Thread Serhiy Storchaka
27.01.14 12:55, Victor Stinner написав(ла): IncompleteReadError has two additionnal attributes: - partial: "incomplete" received bytes - expected: total number of expected bytes (n parameter of readexactly) This looks similar to http.client.IncompleteRead. ___

Re: [Python-Dev] Negative times behaviour in itertools.repeat for Python maintenance releases (2.7, 3.3 and maybe 3.4)

2014-01-27 Thread Vajrasky Kok
On Mon, Jan 27, 2014 at 5:38 PM, Antoine Pitrou wrote: > > I would say not backport at all. The security threat is highly > theoretical. If someone blindly accepts user values for repeat(), the > user value can just as well be a very large positive with similar > effects (e.g. 2**31). > I can not

Re: [Python-Dev] Negative times behaviour in itertools.repeat for Python maintenance releases (2.7, 3.3 and maybe 3.4)

2014-01-27 Thread Antoine Pitrou
On Mon, 27 Jan 2014 04:01:02 -0800 Larry Hastings wrote: > > On 01/27/2014 01:39 AM, Antoine Pitrou wrote: > > On Sun, 26 Jan 2014 21:01:08 -0800 > > Larry Hastings wrote: > >> On 01/26/2014 08:40 PM, Alexander Belopolsky wrote: > >>> On Sun, Jan 26, 2014 at 11:26 PM, Vajrasky Kok > >>> mailto:s

Re: [Python-Dev] Negative times behaviour in itertools.repeat for Python maintenance releases (2.7, 3.3 and maybe 3.4)

2014-01-27 Thread Larry Hastings
On 01/27/2014 01:39 AM, Antoine Pitrou wrote: On Sun, 26 Jan 2014 21:01:08 -0800 Larry Hastings wrote: On 01/26/2014 08:40 PM, Alexander Belopolsky wrote: On Sun, Jan 26, 2014 at 11:26 PM, Vajrasky Kok mailto:sky@speaklikeaking.com>> wrote: In case we are taking "not backporting any

Re: [Python-Dev] News from asyncio

2014-01-27 Thread Victor Stinner
2014-01-27 Gustavo Carneiro : >> > Why not simply EOFError? >> >> IncompleteReadError has two additionnal attributes: >> >> - partial: "incomplete" received bytes >> - expected: total number of expected bytes (n parameter of readexactly) >> >> I prefer to use a different exception to ensure that th

Re: [Python-Dev] News from asyncio

2014-01-27 Thread Gustavo Carneiro
On 27 January 2014 10:55, Victor Stinner wrote: > 2014-01-27 Antoine Pitrou : > > On Mon, 27 Jan 2014 10:45:37 +0100 > > Victor Stinner wrote: > >> > >> - Tulip #111: StreamReader.readexactly() now raises an > >> IncompleteReadError if the > >> end of stream is reached before we received enough

Re: [Python-Dev] News from asyncio

2014-01-27 Thread Victor Stinner
2014-01-27 Antoine Pitrou : > On Mon, 27 Jan 2014 10:45:37 +0100 > Victor Stinner wrote: >> >> - Tulip #111: StreamReader.readexactly() now raises an >> IncompleteReadError if the >> end of stream is reached before we received enough bytes, instead of >> returning >> less bytes than requested. >

Re: [Python-Dev] News from asyncio

2014-01-27 Thread Antoine Pitrou
On Mon, 27 Jan 2014 10:45:37 +0100 Victor Stinner wrote: > > - Tulip #111: StreamReader.readexactly() now raises an > IncompleteReadError if the > end of stream is reached before we received enough bytes, instead of returning > less bytes than requested. Why not simply EOFError? Regards Antoin

[Python-Dev] News from asyncio

2014-01-27 Thread Victor Stinner
Hi, I'm working for eNovance on the asyncio module, the goal is to use it in the huge OpenStack project (2.5 millions line of code) which currently uses eventlet. I'm trying to fix remaining issues in the asyncio module before Python 3.4 final. The asyncio project is very active but discussions a

Re: [Python-Dev] Negative times behaviour in itertools.repeat for Python maintenance releases (2.7, 3.3 and maybe 3.4)

2014-01-27 Thread Mark Lawrence
On 27/01/2014 01:52, Nick Coghlan wrote: In 3.5, that will be passing None, rather than -1. For those proposing to change the maintenance releases, how should a user relying on this misbehaviour update their code to handle it? I'm -1 on using None. The code currently rejects anything except

Re: [Python-Dev] Negative times behaviour in itertools.repeat for Python maintenance releases (2.7, 3.3 and maybe 3.4)

2014-01-27 Thread Antoine Pitrou
On Sun, 26 Jan 2014 21:01:08 -0800 Larry Hastings wrote: > On 01/26/2014 08:40 PM, Alexander Belopolsky wrote: > > > > On Sun, Jan 26, 2014 at 11:26 PM, Vajrasky Kok > > mailto:sky@speaklikeaking.com>> wrote: > > > > In case we are taking "not backporting anything at all" road, what is >

Re: [Python-Dev] Negative times behaviour in itertools.repeat for Python maintenance releases (2.7, 3.3 and maybe 3.4)

2014-01-27 Thread Antoine Pitrou
On Mon, 27 Jan 2014 14:02:29 +1000 Nick Coghlan wrote: > > If we do go this path, then we should backport the full fix (i.e. > accepting None to indicate repeating forever), rather than just a > partial fix. > > That is, I'm OK with either not backporting anything at all, or > backporting the fu