[
https://issues.apache.org/jira/browse/ARROW-3777?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16931607#comment-16931607
]
Antoine Pitrou commented on ARROW-3777:
---------------------------------------
First thing would be to create slow input and output stream wrappers (though
I'm not sure it makes sense for output streams: writes are normally issued in
the background).
> [Python] Implement a mock "high latency" filesystem
> ---------------------------------------------------
>
> Key: ARROW-3777
> URL: https://issues.apache.org/jira/browse/ARROW-3777
> Project: Apache Arrow
> Issue Type: New Feature
> Components: Python
> Reporter: Wes McKinney
> Priority: Major
> Labels: parquet
> Fix For: 1.0.0
>
>
> Some of our tools don't perform well out of the box for filesystems with high
> latency reads, like cloud blob stores. In such cases, it may be better to use
> buffered reads with a larger read ahead window. Having a mock filesystem to
> introduce latency into reads will help with testing / developing APIs for this
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
