Roundup Robot added the comment:
New changeset 213490268be4 by Benjamin Peterson in branch 'default':
add BufferedIOBase.readinto1 (closes #20578)
http://hg.python.org/cpython/rev/213490268be4
--
resolution: - fixed
status: open - closed
___
Python
Nikolaus Rath added the comment:
As discussed on python-devel, I'm attaching a new patch that uses
memoryview.cast to ensure that the pure-Python readinto() now works with any
object implementing the buffer protocol.
--
Added file: http://bugs.python.org/file35647/issue20578_r5.diff
Nikolaus Rath added the comment:
(refreshed patch, no changes)
--
Added file: http://bugs.python.org/file35648/issue20578_r6.diff
___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue20578
___
Roundup Robot added the comment:
New changeset b1e99b4ec374 by Benjamin Peterson in branch 'default':
backout 0fb7789b5eeb for test breakage (#20578)
http://hg.python.org/cpython/rev/b1e99b4ec374
--
___
Python tracker rep...@bugs.python.org
Changes by Benjamin Peterson benja...@python.org:
--
resolution: fixed -
status: closed - open
___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue20578
___
Nikolaus Rath added the comment:
Thanks for taking the time, and apologies about the test failure. I was
probably too eager and ran only the test_io suite instead of everything.
I looked at the failure, and the problem is that the default Python
BufferedIOBase.readinto implementation is
Nikolaus Rath added the comment:
I used the wrong interpreter when cutting and pasting the example above, here's
the correct version to avoid confusion with the traceback:
import _pyio
from array import array
buf = array('b', b'x' * 10)
_pyio.open('/dev/zero', 'rb').readinto(buf)
10
buf
Roundup Robot added the comment:
New changeset 0fb7789b5eeb by Benjamin Peterson in branch 'default':
add BufferedIOBase.readinto1 (closes #20578)
http://hg.python.org/cpython/rev/0fb7789b5eeb
--
nosy: +python-dev
resolution: - fixed
stage: - resolved
status: open - closed
Benjamin Peterson added the comment:
Looks like test_file is unhappy:
http://buildbot.python.org/all/builders/AMD64%20FreeBSD%209.x%203.x/builds/1758/steps/test/logs/stdio
--
___
Python tracker rep...@bugs.python.org
Nikolaus Rath added the comment:
Attached is an updated patch that
- removes the code duplication in _pyio.BufferedIOBase
- adds an internal _readinto helper method to _pyio.BufferedReader that makes
the implementation similar to io.BufferedReader.
- implements
Changes by Nikolaus Rath nikol...@rath.org:
Added file: http://bugs.python.org/file34864/benchmark_r3.py
___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue20578
___
Nikolaus Rath added the comment:
Here's a little script to estimate the performance difference between using
read1 and readinto1 to read large amounts of data. On my system, I get:
C readinto1: 4.960e-01 seconds
C read1: 4.055e-01 seconds
Python readinto1: 1.066e+00 seconds
Python read1:
Martin v. Löwis added the comment:
Can you please extend your benchmark to also measure read and readinto?
I'm puzzled why you are treating readinto1 differently from readinto.
--
___
Python tracker rep...@bugs.python.org
Nikolaus Rath added the comment:
(Rietveld is giving me errors, so I'm replying here)
On 2014/04/13 02:22:23, loewis wrote:
Again, why a separate implementation here?
For performance reasons. Relying on the default implementation
would fall back to using read1(), which means a new bytes
Nikolaus Rath added the comment:
Can you please extend your benchmark to also measure read and readinto?
Yes - but I don't quite understand why it matters (if you need read1/readinto1,
you cannot just use read/readinto instead).
C readinto1: 4.638e-01 seconds
C read1: 4.026e-01 seconds
C
Martin v. Löwis added the comment:
I didn't even notice the readinto implementation was missing. But I
agree, if we keep readinto1(), we should also add readinto().
[...]
Maybe this is why we seem to be talking past each other :-). I did not
look or work on readinto at all. All I noticed is
Nikolaus Rath added the comment:
Seems as if no one has an opinion on this at all:
https://mail.python.org/pipermail/python-dev/2014-April/133739.html
What next?
--
___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue20578
Martin v. Löwis added the comment:
I put some review into rietveld. In addition, please avoid code duplication of
more than three subsequent lines of code.
--
nosy: +loewis
___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue20578
Nikolaus Rath added the comment:
Thanks for the review! Attached is a new patch. I was actually pretty careful
to avoid any code duplication.. are you refering to the readinto1()
implementations for BytesIO and BufferedReader in Lib/_pyio.py?
--
Added file:
Martin v. Löwis added the comment:
Put up a new review.
--
___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue20578
___
___
Python-bugs-list mailing
Antoine Pitrou added the comment:
The IO APIs are a visible part of the stdlib, so I'd suggest asking on the
python-dev mailing-list first.
--
___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue20578
Changes by STINNER Victor victor.stin...@gmail.com:
--
nosy: +haypo
___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue20578
___
___
Python-bugs-list
Nikolaus Rath added the comment:
I have attached a patch that adds readinto1() to BufferedReader and
BufferedRWPair.
An example use case for this method is receiving a large stream over a protocol
like HTTP. You want to use a buffered reader so you can efficiently parse the
header, but after
Changes by Nikolaus Rath nikol...@rath.org:
--
nosy: +benjamin.peterson, hynek, stutzbach
___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue20578
___
New submission from Nikolaus Rath:
It would be nice to have a readinto1 method to complement the existing read,
readinto, and read1 methods of io.BufferedIOBase.
--
components: Library (Lib)
messages: 210794
nosy: nikratio
priority: normal
severity: normal
status: open
title:
Changes by STINNER Victor victor.stin...@gmail.com:
--
nosy: +pitrou
___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue20578
___
___
Python-bugs-list
Nikolaus Rath added the comment:
(I'll work on a patch for this myself, this bug is just to prevent duplicate
work)
--
___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue20578
___
27 matches
Mail list logo
