[
https://issues.apache.org/jira/browse/ARROW-12412?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Weston Pace closed ARROW-12412.
-------------------------------
Resolution: Invalid
Nevermind. Turns out this idea can't work. Background tasks and I/O tasks
will complete after one call to the generator and before the next call to the
generator and they will have no choice but to spawn a CPU thread.
So running a generator serially only works via visit or to-vector. It cannot
be converted into a safe synchronous iterator.
> [C++] Add ability to convert an AsyncGenerator to an Iterator by wrapping
> each run with RunInSerialExecutor
> ------------------------------------------------------------------------------------------------------------
>
> Key: ARROW-12412
> URL: https://issues.apache.org/jira/browse/ARROW-12412
> Project: Apache Arrow
> Issue Type: Improvement
> Components: C++
> Reporter: Weston Pace
> Assignee: Weston Pace
> Priority: Major
> Labels: pull-request-available
> Time Spent: 20m
> Remaining Estimate: 0h
>
> This will be used in ARROW-12392/ARROW-12355 which convert the CSV reader to
> async. This causes the CsvScanTask to deadlock due to nested parallelism.
> This utility will make it possible to avoid that deadlock by using
> RunInSerialExecutor to read the CSV blocks.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
