[Python-Dev] O(1) deletes from the front of bytearray (was: Re: Adding bytes.frombuffer() constructor to PEP 467 (was: [Python-ideas] Adding bytes.frombuffer() constructor)

2016-10-11 Thread Nathaniel Smith
On Tue, Oct 11, 2016 at 10:53 PM, Serhiy Storchaka wrote: > On 12.10.16 08:03, Nathaniel Smith wrote: >> >> On Tue, Oct 11, 2016 at 9:08 PM, INADA Naoki >> wrote: >>> >>> From Python 3.4, bytearray is good solution for I/O buffer, thanks to >>> #19087 [1]. >>> Actually, asyncio uses bytearray as

Re: [Python-Dev] Adding bytes.frombuffer() constructor to PEP 467 (was: [Python-ideas] Adding bytes.frombuffer() constructor

2016-10-11 Thread Serhiy Storchaka
On 12.10.16 08:03, Nathaniel Smith wrote: On Tue, Oct 11, 2016 at 9:08 PM, INADA Naoki wrote: From Python 3.4, bytearray is good solution for I/O buffer, thanks to #19087 [1]. Actually, asyncio uses bytearray as I/O buffer often. Whoa what?! This is awesome, I had no idea that bytearray had O

Re: [Python-Dev] Adding bytes.frombuffer() constructor to PEP 467 (was: [Python-ideas] Adding bytes.frombuffer() constructor

2016-10-11 Thread Serhiy Storchaka
On 12.10.16 07:08, INADA Naoki wrote: Sample code: def read_line(buf: bytearray) -> bytes: try: n = buf.index(b'\r\n') except ValueError: return b'' line = bytes(buf)[:n] # bytearray -> bytes -> bytes Wouldn't be more correct to write this

Re: [Python-Dev] Adding bytes.frombuffer() constructor to PEP 467 (was: [Python-ideas] Adding bytes.frombuffer() constructor

2016-10-11 Thread Nick Coghlan
I don't think it makes sense to add any more ideas to PEP 467. That needed to be a PEP because it proposed breaking backwards compatibility in a couple of areas, and because of the complex history of Python 3's "bytes-as-tuple-of-ints" and Python 2's "bytes-as-str" semantics. Other enhancements to

Re: [Python-Dev] Adding bytes.frombuffer() constructor to PEP 467 (was: [Python-ideas] Adding bytes.frombuffer() constructor

2016-10-11 Thread Nathaniel Smith
On Tue, Oct 11, 2016 at 9:08 PM, INADA Naoki wrote: > From Python 3.4, bytearray is good solution for I/O buffer, thanks to > #19087 [1]. > Actually, asyncio uses bytearray as I/O buffer often. Whoa what?! This is awesome, I had no idea that bytearray had O(1) deletes at the front. I literally re

[Python-Dev] Adding bytes.frombuffer() constructor to PEP 467 (was: [Python-ideas] Adding bytes.frombuffer() constructor

2016-10-11 Thread INADA Naoki
Hi. While there were no reply to my previous post on Python-ideas ML, Now I'm sure about bytes.frombuffer() is worth enough. Let's describe why I think it's important. Background = >From Python 3.4, bytearray is good solution for I/O buffer, thanks to #19087 [1]. Actually, asyncio uses

Re: [Python-Dev] Optimizing list.sort() by checking type in advance

2016-10-11 Thread Tim Peters
[Terry Reedy] > > This seems like a generic issue with timing mutation methods > ... > But I am sure Tim worked this out in his test code, which should be > reused, perhaps updated with Viktor's perf module to get the most > stable timings possible. sortperf.py is older than me ;-) It's not at al

Re: [Python-Dev] Optimizing list.sort() by checking type in advance

2016-10-11 Thread Terry Reedy
On 10/11/2016 10:26 AM, Chris Angelico wrote: After the first call, the list will be sorted. Any subsequent attempts will use the sorted list. This seems like a generic issue with timing mutation methods. Is the mutated output suitable input for another mutation. With list.reverse, the out

Re: [Python-Dev] Optimizing list.sort() by checking type in advance

2016-10-11 Thread Paul Moore
On 11 October 2016 at 15:32, Elliot Gorokhovsky wrote: > But the sort mutates F...does the setup get executed each time? I thought > it's just at the beginning. So then F gets mutated (sorted) and subsequent > sorts time wrong. Did I not say earlier - sorry. I'm suggesting that you put each timin

Re: [Python-Dev] Optimizing list.sort() by checking type in advance

2016-10-11 Thread Elliot Gorokhovsky
But the sort mutates F...does the setup get executed each time? I thought it's just at the beginning. So then F gets mutated (sorted) and subsequent sorts time wrong. On Tue, Oct 11, 2016, 7:51 AM Paul Moore wrote: > On 11 October 2016 at 14:04, Elliot Gorokhovsky > wrote: > > Right, that sound

Re: [Python-Dev] Optimizing list.sort() by checking type in advance

2016-10-11 Thread Elliot Gorokhovsky
Right, that sounds good, but there's just one thing I don't understand that's keeping me from using it. Namely, I would define a benchmark list L in my setup, and then I would have code="F=FastList(L);F.fastsort()". The problem here is I'm measuring the constructor time along with the sort time, ri

Re: [Python-Dev] Optimizing list.sort() by checking type in advance

2016-10-11 Thread Chris Angelico
On Wed, Oct 12, 2016 at 1:24 AM, Paul Moore wrote: > On 11 October 2016 at 15:00, Chris Angelico wrote: >> On Wed, Oct 12, 2016 at 12:51 AM, Paul Moore wrote: >>> On 11 October 2016 at 14:04, Elliot Gorokhovsky >>> wrote: Right, that sounds good, but there's just one thing I don't understa

Re: [Python-Dev] Optimizing list.sort() by checking type in advance

2016-10-11 Thread Paul Moore
On 11 October 2016 at 15:00, Chris Angelico wrote: > On Wed, Oct 12, 2016 at 12:51 AM, Paul Moore wrote: >> On 11 October 2016 at 14:04, Elliot Gorokhovsky >> wrote: >>> Right, that sounds good, but there's just one thing I don't understand >>> that's keeping me from using it. Namely, I would de

Re: [Python-Dev] Optimizing list.sort() by checking type in advance

2016-10-11 Thread Chris Angelico
On Wed, Oct 12, 2016 at 12:51 AM, Paul Moore wrote: > On 11 October 2016 at 14:04, Elliot Gorokhovsky > wrote: >> Right, that sounds good, but there's just one thing I don't understand >> that's keeping me from using it. Namely, I would define a benchmark list L >> in my setup, and then I would h

Re: [Python-Dev] Optimizing list.sort() by checking type in advance

2016-10-11 Thread Paul Moore
On 11 October 2016 at 14:04, Elliot Gorokhovsky wrote: > Right, that sounds good, but there's just one thing I don't understand > that's keeping me from using it. Namely, I would define a benchmark list L > in my setup, and then I would have code="F=FastList(L);F.fastsort()". The > problem here is

Re: [Python-Dev] PyWeakref_GetObject() borrows its reference from... whom?

2016-10-11 Thread Larry Hastings
By the way, just to finish playing this new fun game "Who'd I Borrow This Reference From?": On 10/10/2016 11:45 AM, Chris Angelico wrote: On Mon, Oct 10, 2016 at 8:35 PM, Larry Hastings wrote: I bet for every other spot in the API I can tell you from whom you're borrowing the reference. O

Re: [Python-Dev] PyWeakref_GetObject() borrows its reference from... whom?

2016-10-11 Thread Devin Jeanpierre
> Huh? In all other circumstances, a "borrowed" reference is exactly that: X has a reference, and you are relying on X's reference to keep the object alive. Borrowing from a borrowed reference is simply a chain of these; Z borrows from Y, Y borrows from X, and X is the original person who did the

Re: [Python-Dev] Optimizing list.sort() by checking type in advance

2016-10-11 Thread Paul Moore
On 11 October 2016 at 03:15, Elliot Gorokhovsky wrote: > There's an option to provide setup code, of course, but I need to set up > before each trial, not just before the loop. Typically, I would just run the benchmark separately for each case, and then you'd do # Case 1 python -m perf timeit -s