[
https://issues.apache.org/jira/browse/ARROW-3098?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16587658#comment-16587658
]
Antoine Pitrou commented on ARROW-3098:
---------------------------------------
Too late :-) I've posted a PR. Thanks [~gaqzi] for reporting this issue!
> [Python] BufferReader doesn't adhere to the seek protocol
> ---------------------------------------------------------
>
> Key: ARROW-3098
> URL: https://issues.apache.org/jira/browse/ARROW-3098
> Project: Apache Arrow
> Issue Type: Bug
> Components: Python
> Affects Versions: 0.10.0
> Reporter: Björn Andersson
> Assignee: Antoine Pitrou
> Priority: Major
> Labels: pull-request-available
> Fix For: 0.11.0
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
> I have a script that creates a Parquet file and then writes it out to a
> {{BufferOutputStream}} and then into a {{BufferReader}} with the intention of
> passing it to a place that takes a file-like object to upload it somewhere
> else. But the other location relies on being able to seek to the end of the
> file to figure out how big the file is, e.g.
> {code:python}
> reader.seek(0, 2)
> size = reader.tell()
> reader.seek(0)
> {code}
>
> But when I do that the following exception is raised:
>
> {code}
> pyarrow/io.pxi:209: in pyarrow.lib.NativeFile.seek
> ???
> _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
> _
> > ???
> E pyarrow.lib.ArrowIOError: position out of bounds
> {code}
> I compared it to casting to an {{io.BytesIO}} instead which works:
> {code:python}
> import io
> import pyarrow as pa
> def test_arrow_output_stream():
> output = pa.BufferOutputStream()
> output.write(b'hello')
> reader = pa.BufferReader(output.getvalue())
> reader.seek(0, 2)
> assert reader.tell() == 5
> def test_python_io_stream():
> output = pa.BufferOutputStream()
> output.write(b'hello')
> buffer = io.BytesIO(output.getvalue().to_pybytes())
> reader = io.BufferedRandom(buffer)
> reader.seek(0, 2)
> assert reader.tell() == 5
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
