On 05Sep2016 0941, Paul Moore wrote:
On 5 September 2016 at 14:36, Steve Dower <steve.do...@python.org> wrote:
The best fix is to use a buffered reader, which will read all the available
bytes and then let you .read(1), even if it happens to be an incomplete
character.
But this is sys.stdin.buffer.raw, we're talking about. People can't
really layer anything on top of that, it's precisely because they are
trying to *bypass* the existing layering (that doesn't work the way
that they need it to, because it blocks) that is the problem here.
This layer also blocks, and always has. You need to go to platform
specific functions anyway to get non-blocking functionality (which is
also wrapped up in getc I believe, but that isn't used by FileIO or the
new WinConsoleIO classes).
We could theoretically add buffering to the raw reader to handle one character,
which would allow very small reads from raw, but that severely complicates
things and the advice to use a buffered reader is good advice anyway.
Can you provide an example of how I'd rewrite the code that I quoted
previously to follow this advice? Note - this is not theoretical, I
expect to have to provide a PR to fix exactly this code should this
change go in. At the moment I can't find a way that doesn't impact the
(currently working and not expected to need any change) Unix version
of the code, most likely I'll have to add buffering of 4-byte reads
(which as you say is complex).
The easiest way to follow it is to use "sys.stdin.buffer.read(1)" rather
than "sys.stdin.buffer.raw.read(1)".
PS I'm not 100% sure that under POSIX read() will return partial UTF-8
byte sequences. I think it must, because otherwise a lot of code I've
seen would be broken, but if a POSIX expert can confirm or deny my
assumption, that would be great.
I just tested, and yes it returns partial characters. That's a good
reason to do the single character buffering ourselves. Shouldn't be too
hard to deal with.
Cheers,
Steve
_______________________________________________
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
