Weston Pace created ARROW-14192:
-----------------------------------
Summary: [C++][Dataset] Backpressure broken on ordered scans
Key: ARROW-14192
URL: https://issues.apache.org/jira/browse/ARROW-14192
Project: Apache Arrow
Issue Type: Bug
Components: C++
Reporter: Weston Pace
Assignee: Weston Pace
ARROW-13611 adds a backpressure mechanism that works for unordered scans.
However, this backpressure is not properly applied on ordered (i.e. ScanBatches
and not ScanBatchedUnordered) scans.
The fix will be to modify the merge generator used on ordered scans so that,
while it still will read ahead somewhat on several files, it will never deliver
batches except from the currently read file.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
