[
https://issues.apache.org/jira/browse/ARROW-14191?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Weston Pace resolved ARROW-14191.
---------------------------------
Resolution: Fixed
Issue resolved by pull request 11286
[https://github.com/apache/arrow/pull/11286]
> [C++][Dataset] Dataset writes should respect backpressure
> ---------------------------------------------------------
>
> Key: ARROW-14191
> URL: https://issues.apache.org/jira/browse/ARROW-14191
> Project: Apache Arrow
> Issue Type: Improvement
> Components: C++
> Reporter: Weston Pace
> Assignee: Weston Pace
> Priority: Major
> Labels: pull-request-available, query-engine
> Fix For: 6.0.0
>
> Time Spent: 1h 20m
> Remaining Estimate: 0h
>
> If the write destination is slow then the dataset writer should back off and
> apply backpressure to pause the reader. This will allow simple dataset API
> scans to operate on large out of core datasets.
> This is dependent on ARROW-13611 which adds a backpressure feature for
> regular scanning (but not writing data) and on ARROW-13542 which moves the
> dataset write to a node in the exec plan.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
