[
https://issues.apache.org/jira/browse/ARROW-4283?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16745184#comment-16745184
]
Antoine Pitrou commented on ARROW-4283:
---------------------------------------
There are two cases here:
* RecordBatchStreamReader: this is too high-level; you need to read from your
data source in Python (using `await something.read()`) then construct a record
batch out of the data (perhaps with a BufferReader)
* RecordBatchStreamWriter: conversely, you probably want to write your record
batch first into a BufferOutputStream, then write the resulting data in Python
(using `await something.write()`)
> Should RecordBatchStreamReader/Writer be AsyncIteraable?
> --------------------------------------------------------
>
> Key: ARROW-4283
> URL: https://issues.apache.org/jira/browse/ARROW-4283
> Project: Apache Arrow
> Issue Type: Improvement
> Components: Python
> Reporter: Paul Taylor
> Priority: Minor
> Fix For: 0.13.0
>
>
> Filing this issue after a discussion today with [~xhochy] about how to
> implement streaming pyarrow http services. I had attempted to use both Flask
> and [aiohttp|https://aiohttp.readthedocs.io/en/stable/streams.html]'s
> streaming interfaces because they seemed familiar, but no dice. I have no
> idea how hard this would be to add -- supporting all the asynciterable
> primitives in JS was non-trivial.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
