[
https://issues.apache.org/jira/browse/ARROW-13590?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17477181#comment-17477181
]
Krisztian Szucs commented on ARROW-13590:
-----------------------------------------
[~westonpace] is this already resolved? If not can we move it to version 8.0?
> [C++] Ensure dataset writing applies back pressure
> --------------------------------------------------
>
> Key: ARROW-13590
> URL: https://issues.apache.org/jira/browse/ARROW-13590
> Project: Apache Arrow
> Issue Type: Improvement
> Components: C++
> Reporter: Weston Pace
> Assignee: Weston Pace
> Priority: Critical
> Labels: query-engine
> Fix For: 7.0.0
>
>
> Dataset writing via exec plan (ARROW-13542) does not apply back pressure
> currently and will take up far more RAM than it should when writing a large
> dataset. The node should be applying back pressure. However, the preferred
> back pressure method (via scheduling) will need to wait for ARROW-13576.
> Once those two are finished this can be studied in more detail. Also, the
> vm.dirty_ratio might be experimented with. In theory we should be applying
> our own back pressure and have no need of dirty pages. In practice, it may
> be more work than we want to tackle right now and we just let it do its thing.
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
