Re: [Python-Dev] Adding bytes.frombuffer() constructor to PEP 467 (was: [Python-ideas] Adding bytes.frombuffer() constructor

2016-10-25 Thread Chris Barker
On Tue, Oct 25, 2016 at 12:25 AM, Nick Coghlan wrote: > The suggestion came from folks working on asyncio performance > improvements, > I'm *absolutely* suggesting > that they put some paid time and energy into a lower level buffer > manipulation library between now and

Re: [Python-Dev] Adding bytes.frombuffer() constructor to PEP 467 (was: [Python-ideas] Adding bytes.frombuffer() constructor

2016-10-25 Thread Nick Coghlan
On 25 October 2016 at 17:25, Nick Coghlan wrote: > as well as Yury's new > libuv-based ``uvloop`` asyncio event loop implementation. Oops, I phrased that badly - the default asyncio event loop implementation is still pure Python, but uvloop is a drop-in Cython based

Re: [Python-Dev] Adding bytes.frombuffer() constructor to PEP 467 (was: [Python-ideas] Adding bytes.frombuffer() constructor

2016-10-25 Thread Nick Coghlan
On 25 October 2016 at 02:53, Chris Barker wrote: > IS this na either-or? IF someone is proposing a nice lib for "low level data > buffer > manipulation", then yes, putting frombuffer() in there would be a fine idea. > > But if there is no such proposal on the table, then I

Re: [Python-Dev] Adding bytes.frombuffer() constructor to PEP 467 (was: [Python-ideas] Adding bytes.frombuffer() constructor

2016-10-24 Thread Chris Barker
Before the horse is totally dead... (maybe it already is), a couple comments: > In such cases, an extension module written in something like Cython, C > or Rust would be a better fit, well, yes, but: > From that perspective, adding "[bytes/bytearray].frombuffer" this would be used for the

Re: [Python-Dev] Adding bytes.frombuffer() constructor to PEP 467 (was: [Python-ideas] Adding bytes.frombuffer() constructor

2016-10-22 Thread Nick Coghlan
On 22 October 2016 at 07:57, Chris Barker wrote: > I'm still confused about the "io" in "iobuffers" -- I've used buffers a lot > -- for passing data around between various C libs -- numpy, image > processing, etc... I never really thought of it as IO though. which is why a

Re: [Python-Dev] Adding bytes.frombuffer() constructor to PEP 467 (was: [Python-ideas] Adding bytes.frombuffer() constructor

2016-10-21 Thread Chris Barker
On Thu, Oct 20, 2016 at 11:48 PM, Nick Coghlan wrote: > > len(get_builtin_methods()) > >>230 > > > > So what? No one looks in all the methods of builtins at once. > > Yes, Python implementation developers do, which is why it's a useful > part of defining the overall

Re: [Python-Dev] Adding bytes.frombuffer() constructor to PEP 467 (was: [Python-ideas] Adding bytes.frombuffer() constructor

2016-10-21 Thread Nick Coghlan
On 19 October 2016 at 01:28, Chris Barker - NOAA Federal wrote: > def get_builtin_methods(): >>... return [(name, method_name) for name, obj in >> get_builtin_types().items() for method_name, method in >> vars(obj).items() if not method_name.startswith("__")] >>

Re: [Python-Dev] Adding bytes.frombuffer() constructor to PEP 467 (was: [Python-ideas] Adding bytes.frombuffer() constructor

2016-10-18 Thread Chris Barker - NOAA Federal
>> >> >> Method proliferation on builtins is a Big Deal(TM) > > I wanted to quantify this concept, so here's a quick metric that helps > convey how every time we add a new builtin method we're immediately > making Python harder to comprehend: > def get_builtin_types(): >... import

Re: [Python-Dev] Adding bytes.frombuffer() constructor to PEP 467 (was: [Python-ideas] Adding bytes.frombuffer() constructor

2016-10-12 Thread Nick Coghlan
On 13 October 2016 at 12:54, Nick Coghlan wrote: > Method proliferation on builtins is a Big Deal(TM) I wanted to quantify this concept, so here's a quick metric that helps convey how every time we add a new builtin method we're immediately making Python harder to comprehend:

Re: [Python-Dev] Adding bytes.frombuffer() constructor to PEP 467 (was: [Python-ideas] Adding bytes.frombuffer() constructor

2016-10-12 Thread INADA Naoki
> > Also, why do the conversion from bytearray to bytes? It is definitely not > always needed. > ba = bytearray(b'abc') b = b'def' ba + b > bytearray(b'abcdef') b'%s %s' % (ba, b) > b'abc def' b + ba > b'defabc' ba.extend(b) ba > bytearray(b'abcdef') > > Even if

Re: [Python-Dev] Adding bytes.frombuffer() constructor to PEP 467 (was: [Python-ideas] Adding bytes.frombuffer() constructor

2016-10-12 Thread Nick Coghlan
On 13 October 2016 at 02:37, Stephen J. Turnbull wrote: > Victor Stinner writes: > > 2016-10-12 11:34 GMT+02:00 INADA Naoki : > > > > I see. My proposal should be another PEP (if PEP is required). > > > > I don't think that adding

Re: [Python-Dev] Adding bytes.frombuffer() constructor to PEP 467 (was: [Python-ideas] Adding bytes.frombuffer() constructor

2016-10-12 Thread Terry Reedy
On 10/12/2016 5:42 AM, INADA Naoki wrote: On Wed, Oct 12, 2016 at 2:32 PM, Serhiy Storchaka wrote: On 12.10.16 07:08, INADA Naoki wrote: Sample code: def read_line(buf: bytearray) -> bytes: try: n = buf.index(b'\r\n') except ValueError:

Re: [Python-Dev] Adding bytes.frombuffer() constructor to PEP 467 (was: [Python-ideas] Adding bytes.frombuffer() constructor

2016-10-12 Thread Victor Stinner
Oops, right, I wanted to write "I don't think that adding a single method deserves its own PEP." Victor 2016-10-12 18:37 GMT+02:00 Stephen J. Turnbull : > Victor Stinner writes: > > 2016-10-12 11:34 GMT+02:00 INADA Naoki : > > > > I

Re: [Python-Dev] Adding bytes.frombuffer() constructor to PEP 467 (was: [Python-ideas] Adding bytes.frombuffer() constructor

2016-10-12 Thread Stephen J. Turnbull
Victor Stinner writes: > 2016-10-12 11:34 GMT+02:00 INADA Naoki : > > I see. My proposal should be another PEP (if PEP is required). > > I don't think that adding a single method deserves its own method. You mean "deserves own PEP", right? I interpreted Nick to say

Re: [Python-Dev] Adding bytes.frombuffer() constructor to PEP 467 (was: [Python-ideas] Adding bytes.frombuffer() constructor

2016-10-12 Thread Victor Stinner
2016-10-12 11:34 GMT+02:00 INADA Naoki : > I see. My proposal should be another PEP (if PEP is required). I don't think that adding a single method deserves its own method. I like the idea with Serhiy's API (as Python 2 buffer constructor): bytes.frombuf(buffer,

Re: [Python-Dev] Adding bytes.frombuffer() constructor to PEP 467 (was: [Python-ideas] Adding bytes.frombuffer() constructor

2016-10-12 Thread INADA Naoki
On Wed, Oct 12, 2016 at 2:32 PM, Serhiy Storchaka wrote: > On 12.10.16 07:08, INADA Naoki wrote: >> >> Sample code: >> >> def read_line(buf: bytearray) -> bytes: >> try: >> n = buf.index(b'\r\n') >> except ValueError: >> return b''

Re: [Python-Dev] Adding bytes.frombuffer() constructor to PEP 467 (was: [Python-ideas] Adding bytes.frombuffer() constructor

2016-10-12 Thread INADA Naoki
On Wed, Oct 12, 2016 at 2:07 PM, Nick Coghlan wrote: > I don't think it makes sense to add any more ideas to PEP 467. That > needed to be a PEP because it proposed breaking backwards > compatibility in a couple of areas, and because of the complex history > of Python 3's

Re: [Python-Dev] Adding bytes.frombuffer() constructor to PEP 467 (was: [Python-ideas] Adding bytes.frombuffer() constructor

2016-10-11 Thread Serhiy Storchaka
On 12.10.16 08:03, Nathaniel Smith wrote: On Tue, Oct 11, 2016 at 9:08 PM, INADA Naoki wrote: From Python 3.4, bytearray is good solution for I/O buffer, thanks to #19087 [1]. Actually, asyncio uses bytearray as I/O buffer often. Whoa what?! This is awesome, I had no

Re: [Python-Dev] Adding bytes.frombuffer() constructor to PEP 467 (was: [Python-ideas] Adding bytes.frombuffer() constructor

2016-10-11 Thread Serhiy Storchaka
On 12.10.16 07:08, INADA Naoki wrote: Sample code: def read_line(buf: bytearray) -> bytes: try: n = buf.index(b'\r\n') except ValueError: return b'' line = bytes(buf)[:n] # bytearray -> bytes -> bytes Wouldn't be more correct to write this

Re: [Python-Dev] Adding bytes.frombuffer() constructor to PEP 467 (was: [Python-ideas] Adding bytes.frombuffer() constructor

2016-10-11 Thread Nick Coghlan
I don't think it makes sense to add any more ideas to PEP 467. That needed to be a PEP because it proposed breaking backwards compatibility in a couple of areas, and because of the complex history of Python 3's "bytes-as-tuple-of-ints" and Python 2's "bytes-as-str" semantics. Other enhancements

Re: [Python-Dev] Adding bytes.frombuffer() constructor to PEP 467 (was: [Python-ideas] Adding bytes.frombuffer() constructor

2016-10-11 Thread Nathaniel Smith
On Tue, Oct 11, 2016 at 9:08 PM, INADA Naoki wrote: > From Python 3.4, bytearray is good solution for I/O buffer, thanks to > #19087 [1]. > Actually, asyncio uses bytearray as I/O buffer often. Whoa what?! This is awesome, I had no idea that bytearray had O(1) deletes at