Weston Pace created ARROW-15968:
-----------------------------------
Summary: [C++] Update AsyncGenerator semantics to emit a terminal
item only after all outstanding futures have completed
Key: ARROW-15968
URL: https://issues.apache.org/jira/browse/ARROW-15968
Project: Apache Arrow
Issue Type: Improvement
Components: C++
Reporter: Weston Pace
Currently, when an error occurs, we emit the error immediately. Once an error
is emitted from the generator we typically discard it and all related state.
Unfortunately, in readahead situations, there may be outstanding tasks, and
these may be referencing that related state.
Currently, we avoid errors in this situation through the copious use of
shared_ptr. This has led to excess complexity and potential performance issues.
Instead, we can update the rules for async generators to only emit a terminal
item once all outstanding tasks have completed. At this point we should be
able to safely tear down all of the state. Furthermore, if we can ensure
consumers drain an async generator before they destroy it (this can be done in
a follow-up issue), we can get rid of a lot of our shared_ptr and simplify some
of the complexity of the async reader code.
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
