On 27Dec2018 12:59, Steven D'Aprano wrote:
On Thu, Dec 27, 2018 at 10:02:09AM +1100, Cameron Simpson wrote:
[...]
>Also I'm thinking about type annotations in typeshed.
>Now the type is Union[array[int], bytes, bytearray, memoryview]
>Should it be Union[io.BinaryIO, array[int], bytes, bytearray
On 27Dec2018 02:53, Anders Hovmöller wrote:
And this is why I, personally, think augumenting struct.unpack and json.read
and a myriad of other arbitrary methods to accept both file-like things and
bytes is an open ended can of worms.
And it is why I wrote myself my CornuCopyBuffer class (se
I'm quoting Steve's post here but am responding more broadly to the
whole thread too.
On Thu, Dec 27, 2018 at 1:00 PM Steven D'Aprano wrote:
> I assume you have no objection to the existence of json.load() and
> json.loads() functions. (If you do think they're a bad idea, I don't
> know what to s
On Thu, Dec 27, 2018 at 10:02:09AM +1100, Cameron Simpson wrote:
[...]
> >Also I'm thinking about type annotations in typeshed.
> >Now the type is Union[array[int], bytes, bytearray, memoryview]
> >Should it be Union[io.BinaryIO, array[int], bytes, bytearray,
> >memoryview] ?
>
> And this is why
> And this is why I, personally, think augumenting struct.unpack and json.read
> and a myriad of other arbitrary methods to accept both file-like things and
> bytes is an open ended can of worms.
>
> And it is why I wrote myself my CornuCopyBuffer class (see my other post in
> this thread).
On Wed, Dec 26, 2018 at 01:32:38PM +, Paul Moore wrote:
> On Wed, 26 Dec 2018 at 09:26, Steven D'Aprano wrote:
> > Regardless, my point doesn't change. That has nothing to do with the
> > behaviour of unpack. If you pass a non-blocking file-like object which
> > returns None, you get exactly t
On 26Dec2018 12:18, Andrew Svetlov wrote:
On Wed, Dec 26, 2018 at 11:26 AM Steven D'Aprano
wrote:
On Wed, Dec 26, 2018 at 09:48:15AM +0200, Andrew Svetlov wrote:
> The perfect demonstration of io objects complexity.
> `stream.read(N)` can return None by spec if the file is non-blocking
> and
On Wed, 26 Dec 2018 at 09:26, Steven D'Aprano wrote:
> Regardless, my point doesn't change. That has nothing to do with the
> behaviour of unpack. If you pass a non-blocking file-like object which
> returns None, you get exactly the same exception as if you wrote
>
> unpack(fmt, f.read(size))
On Wed, Dec 26, 2018 at 12:18:23PM +0200, Andrew Svetlov wrote:
[...]
> > json is correct: if `read()` is called without argument it reads the whole
> content until EOF.
> But with size argument the is different for interactive and non-interactive
> streams.
> RawIOBase and BufferedIOBase also hav
On Wed, Dec 26, 2018 at 03:10:05AM -0800, Nathaniel Smith wrote:
> On Wed, Dec 26, 2018, 02:19 Andrew Svetlov
> >
> > Also I'm thinking about type annotations in typeshed.
> > Now the type is Union[array[int], bytes, bytearray, memoryview]
> > Should it be Union[io.BinaryIO, array[int], bytes, by
On Wed, Dec 26, 2018, 02:19 Andrew Svetlov
> Also I'm thinking about type annotations in typeshed.
> Now the type is Union[array[int], bytes, bytearray, memoryview]
> Should it be Union[io.BinaryIO, array[int], bytes, bytearray, memoryview] ?
>
Yeah, trying to support both buffers and file-like o
On Wed, Dec 26, 2018 at 11:26 AM Steven D'Aprano
wrote:
> On Wed, Dec 26, 2018 at 09:48:15AM +0200, Andrew Svetlov wrote:
>
> > The perfect demonstration of io objects complexity.
> > `stream.read(N)` can return None by spec if the file is non-blocking
> > and have no ready data.
> >
> > Confusin
On Wed, Dec 26, 2018 at 09:48:15AM +0200, Andrew Svetlov wrote:
> The perfect demonstration of io objects complexity.
> `stream.read(N)` can return None by spec if the file is non-blocking
> and have no ready data.
>
> Confusing but still possible and documented behavior.
https://docs.python.org
On Wed, Dec 26, 2018 at 7:12 AM Steven D'Aprano wrote:
> On Tue, Dec 25, 2018 at 01:28:02AM +0200, Andrew Svetlov wrote:
>
> > The proposal can generate cryptic messages like
> > `a bytes-like object is required, not 'NoneType'`
>
> How will it generate such a message? That's not obvious to me.
>
On Tue, Dec 25, 2018 at 01:28:02AM +0200, Andrew Svetlov wrote:
> The proposal can generate cryptic messages like
> `a bytes-like object is required, not 'NoneType'`
How will it generate such a message? That's not obvious to me.
The message doesn't seem cryptic to me. It seems perfectly clear: a
On 12/25/18, Steven D'Aprano wrote:
> On Tue, Dec 25, 2018 at 04:51:18PM -0600, eryk sun wrote:
>>
>> Alternatively, we can memory-map the file via mmap. An important
>> difference is that the mmap buffer interface is low-level (e.g. no
>> file pointer and the offset has to be page aligned), so we
On 24Dec2018 10:19, James Edwards wrote:
Here's a snippet of semi-production code we use:
def read_and_unpack(handle, fmt):
size = struct.calcsize(fmt)
data = handle.read(size)
if len(data) < size: return None
return struct.unpack(fmt, data)
which was originally
On Tue, Dec 25, 2018 at 04:51:18PM -0600, eryk sun wrote:
> On 12/24/18, Drew Warwick wrote:
> > The struct unpack API is inconvenient to use with files. I must do:
> >
> > struct.unpack(fmt, file.read(struct.calcsize(fmt))
>
> Alternatively, we can memory-map the file via mmap. An important
> di
On 12/24/18, Drew Warwick wrote:
> The struct unpack API is inconvenient to use with files. I must do:
>
> struct.unpack(fmt, file.read(struct.calcsize(fmt))
Alternatively, we can memory-map the file via mmap. An important
difference is that the mmap buffer interface is low-level (e.g. no
file po
The proposal can generate cryptic messages like
`a bytes-like object is required, not 'NoneType'`
To produce more informative exception text all mentioned cases should be
handled:
> - read partial structs from non-blocking files without failing
> - deal with file system errors without failing
> -
On Mon, Dec 24, 2018 at 03:36:07PM +, Paul Moore wrote:
> > There should be no difference whether the text comes from a literal, a
> > variable, or is read from a file.
>
> One difference is that with a file, it's (as far as I can see)
> impossible to determine whether or not you're going to
On Mon, 24 Dec 2018 at 13:39, Steven D'Aprano wrote:
>
> > Files can be opened in text mode, what to do in this case? What
> > exception should be raised?
>
> That is easy to answer: the same exception you get if you pass text to
> unpack() when it is expecting bytes:
>
> py> struct.unpack(fmt, "a
Here's a snippet of semi-production code we use:
def read_and_unpack(handle, fmt):
size = struct.calcsize(fmt)
data = handle.read(size)
if len(data) < size: return None
return struct.unpack(fmt, data)
which was originally something like:
def read_and_unpac
On 12/24/18 7:33 AM, Steven D'Aprano wrote:
> On Mon, Dec 24, 2018 at 03:01:07PM +0200, Andrew Svetlov wrote:
>> Handling files overcomplicates both implementation and mental space
>> for API saving.
> I haven't thought about this very deeply, but at first glance, I like
> Drew's idea of being a
On Mon, Dec 24, 2018 at 03:01:07PM +0200, Andrew Svetlov wrote:
> Handling files overcomplicates both implementation and mental space for API
> saving.
Perhaps. Although the implementation doesn't seem that complicated, and
the mental space for the API not that much more difficult:
unpack f
Handling files overcomplicates both implementation and mental space for API
saving.
Files can be opened in text mode, what to do in this case? What
exception should be raised?
How to handle OS errors?
On Mon, Dec 24, 2018 at 1:11 PM Drew Warwick wrote:
> The struct unpack API is inconvenient to
The struct unpack API is inconvenient to use with files. I must do:
struct.unpack(fmt, file.read(struct.calcsize(fmt))
every time I want to read a struct from the file. I ended up having to
create a utility function for this due to how frequently I was using
struct.unpack with files:
def unpackS
27 matches
Mail list logo
