Re: [Python-Dev] O(1) deletes from the front of bytearray (was: Re: Adding bytes.frombuffer() constructor to PEP 467 (was: [Python-ideas] Adding bytes.frombuffer() constructor)

2016-10-12 Thread Serhiy Storchaka
On 12.10.16 09:31, Nathaniel Smith wrote: But amortized O(1) deletes from the front of bytearray are totally different, and more like amortized O(1) appends to list: there are important use cases[1] that simply cannot be implemented without some feature like this, and putting the implementation i

Re: [Python-Dev] O(1) deletes from the front of bytearray (was: Re: Adding bytes.frombuffer() constructor to PEP 467 (was: [Python-ideas] Adding bytes.frombuffer() constructor)

2016-10-12 Thread Nathaniel Smith
On Wed, Oct 12, 2016 at 12:17 AM, Serhiy Storchaka wrote: > On 12.10.16 09:31, Nathaniel Smith wrote: >> >> But amortized O(1) deletes from the front of bytearray are totally >> different, and more like amortized O(1) appends to list: there are >> important use cases[1] that simply cannot be imple

Re: [Python-Dev] Adding bytes.frombuffer() constructor to PEP 467 (was: [Python-ideas] Adding bytes.frombuffer() constructor

2016-10-12 Thread INADA Naoki
On Wed, Oct 12, 2016 at 2:07 PM, Nick Coghlan wrote: > I don't think it makes sense to add any more ideas to PEP 467. That > needed to be a PEP because it proposed breaking backwards > compatibility in a couple of areas, and because of the complex history > of Python 3's "bytes-as-tuple-of-ints" a

Re: [Python-Dev] Adding bytes.frombuffer() constructor to PEP 467 (was: [Python-ideas] Adding bytes.frombuffer() constructor

2016-10-12 Thread INADA Naoki
On Wed, Oct 12, 2016 at 2:32 PM, Serhiy Storchaka wrote: > On 12.10.16 07:08, INADA Naoki wrote: >> >> Sample code: >> >> def read_line(buf: bytearray) -> bytes: >> try: >> n = buf.index(b'\r\n') >> except ValueError: >> return b'' >> >> line = b

Re: [Python-Dev] O(1) deletes from the front of bytearray (was: Re: Adding bytes.frombuffer() constructor to PEP 467 (was: [Python-ideas] Adding bytes.frombuffer() constructor)

2016-10-12 Thread INADA Naoki
>> >> [1] My use case is parsing HTTP out of a receive buffer. If deleting >> the first k bytes of an N byte buffer is O(N), then not only does >> parsing becomes O(N^2) in the worst case, but it's the sort of O(N^2) >> that random untrusted network clients can trigger at will to DoS your >> server

Re: [Python-Dev] Adding bytes.frombuffer() constructor to PEP 467 (was: [Python-ideas] Adding bytes.frombuffer() constructor

2016-10-12 Thread Victor Stinner
2016-10-12 11:34 GMT+02:00 INADA Naoki : > I see. My proposal should be another PEP (if PEP is required). I don't think that adding a single method deserves its own method. I like the idea with Serhiy's API (as Python 2 buffer constructor): bytes.frombuf(buffer, [offset, size]) bytearray.frombu

Re: [Python-Dev] O(1) deletes from the front of bytearray (was: Re: Adding bytes.frombuffer() constructor to PEP 467 (was: [Python-ideas] Adding bytes.frombuffer() constructor)

2016-10-12 Thread Victor Stinner
2016-10-12 10:01 GMT+02:00 Nathaniel Smith : > It's more complicated than that -- the right algorithm is the one that > Antoine implemented in 3.4. > (...) > My point is that > forcing everyone who writes network code in Python to do that is > silly, especially given that CPython's apparently been

Re: [Python-Dev] Adding bytes.frombuffer() constructor to PEP 467

2016-10-12 Thread Nikolaus Rath
On Oct 11 2016, Nathaniel Smith wrote: > On Tue, Oct 11, 2016 at 9:08 PM, INADA Naoki wrote: >> From Python 3.4, bytearray is good solution for I/O buffer, thanks to >> #19087 [1]. >> Actually, asyncio uses bytearray as I/O buffer often. > > Whoa what?! This is awesome, I had no idea that bytearr

Re: [Python-Dev] Adding bytes.frombuffer() constructor to PEP 467 (was: [Python-ideas] Adding bytes.frombuffer() constructor

2016-10-12 Thread Stephen J. Turnbull
Victor Stinner writes: > 2016-10-12 11:34 GMT+02:00 INADA Naoki : > > I see. My proposal should be another PEP (if PEP is required). > > I don't think that adding a single method deserves its own method. You mean "deserves own PEP", right? I interpreted Nick to say that "the reasons that a

Re: [Python-Dev] Adding bytes.frombuffer() constructor to PEP 467 (was: [Python-ideas] Adding bytes.frombuffer() constructor

2016-10-12 Thread Victor Stinner
Oops, right, I wanted to write "I don't think that adding a single method deserves its own PEP." Victor 2016-10-12 18:37 GMT+02:00 Stephen J. Turnbull : > Victor Stinner writes: > > 2016-10-12 11:34 GMT+02:00 INADA Naoki : > > > > I see. My proposal should be another PEP (if PEP is required).

Re: [Python-Dev] Adding bytes.frombuffer() constructor to PEP 467 (was: [Python-ideas] Adding bytes.frombuffer() constructor

2016-10-12 Thread Terry Reedy
On 10/12/2016 5:42 AM, INADA Naoki wrote: On Wed, Oct 12, 2016 at 2:32 PM, Serhiy Storchaka wrote: On 12.10.16 07:08, INADA Naoki wrote: Sample code: def read_line(buf: bytearray) -> bytes: try: n = buf.index(b'\r\n') except ValueError: return b''

Re: [Python-Dev] O(1) deletes from the front of bytearray (was: Re: Adding bytes.frombuffer() constructor to PEP 467 (was: [Python-ideas] Adding bytes.frombuffer() constructor)

2016-10-12 Thread Nathaniel Smith
On Wed, Oct 12, 2016 at 4:55 AM, Victor Stinner wrote: > 2016-10-12 10:01 GMT+02:00 Nathaniel Smith : >> It's more complicated than that -- the right algorithm is the one that >> Antoine implemented in 3.4. >> (...) >> My point is that >> forcing everyone who writes network code in Python to do th

Re: [Python-Dev] O(1) deletes from the front of bytearray (was: Re: Adding bytes.frombuffer() constructor to PEP 467 (was: [Python-ideas] Adding bytes.frombuffer() constructor)

2016-10-12 Thread Nathaniel Smith
On Wed, Oct 12, 2016 at 3:28 AM, INADA Naoki wrote: > When Tornado drop Python 2.7 support, they can use bytearray, and > iostream can be more simple and fast. FYI 2.7 does have bytearray. (You still have to implement the O(1) deletion part as a layer on top, like Victor points out, but I suspect

[Python-Dev] Python ROCKS! Thanks guys! [anecdote]

2016-10-12 Thread Chris Angelico
I work with a full-stack web development bootcamp. Most of the course focuses on JavaScript (Node.js, React, jQuery, etc), but there's a one-week period in which each student gets to pick some technology to learn, and at the end of the week, demos to the group some project s/he has mastered. Two ch

Re: [Python-Dev] Python ROCKS! Thanks guys! [anecdote]

2016-10-12 Thread Ryan Gonzalez
On Wed, Oct 12, 2016 at 7:05 PM, Chris Angelico wrote: > I work with a full-stack web development bootcamp. Most of the course > focuses on *JavaScript (Node.js, React, jQuery, etc),* Poor students... ;) > but there's a > one-week period in which each student gets to pick some technology to >

Re: [Python-Dev] Adding bytes.frombuffer() constructor to PEP 467 (was: [Python-ideas] Adding bytes.frombuffer() constructor

2016-10-12 Thread Nick Coghlan
On 13 October 2016 at 02:37, Stephen J. Turnbull wrote: > Victor Stinner writes: > > 2016-10-12 11:34 GMT+02:00 INADA Naoki : > > > > I see. My proposal should be another PEP (if PEP is required). > > > > I don't think that adding a single method deserves its own method. > > You mean "deserve

Re: [Python-Dev] Adding bytes.frombuffer() constructor to PEP 467 (was: [Python-ideas] Adding bytes.frombuffer() constructor

2016-10-12 Thread INADA Naoki
> > Also, why do the conversion from bytearray to bytes? It is definitely not > always needed. > ba = bytearray(b'abc') b = b'def' ba + b > bytearray(b'abcdef') b'%s %s' % (ba, b) > b'abc def' b + ba > b'defabc' ba.extend(b) ba > bytearray(b'abcdef') > > Even if

Re: [Python-Dev] Adding bytes.frombuffer() constructor to PEP 467 (was: [Python-ideas] Adding bytes.frombuffer() constructor

2016-10-12 Thread Nick Coghlan
On 13 October 2016 at 12:54, Nick Coghlan wrote: > Method proliferation on builtins is a Big Deal(TM) I wanted to quantify this concept, so here's a quick metric that helps convey how every time we add a new builtin method we're immediately making Python harder to comprehend: >>> def get_bui

Re: [Python-Dev] Python ROCKS! Thanks guys! [anecdote]

2016-10-12 Thread Nick Coghlan
Thanks for passing this feedback along, Chris! It's always wonderful to see developers feeling empowered by the potential that open source tools offer them. On 13 October 2016 at 11:01, Ryan Gonzalez wrote: > Poor students... ;) Folks, as tempting as it may be to make jokes at the expense of oth

Re: [Python-Dev] Python ROCKS! Thanks guys! [anecdote]

2016-10-12 Thread Chris Angelico
On Thu, Oct 13, 2016 at 12:01 PM, Ryan Gonzalez wrote: > On Wed, Oct 12, 2016 at 7:05 PM, Chris Angelico wrote: >> >> I work with a full-stack web development bootcamp. Most of the course >> focuses on JavaScript (Node.js, React, jQuery, etc), > > > Poor students... ;) The bootcamp guarantees th