[
https://issues.apache.org/jira/browse/ARROW-16259?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17525429#comment-17525429
]
Eduardo Ponce commented on ARROW-16259:
---------------------------------------
[~westonpace] I am not able to find {{GatingFs::open_output_stream}} in the C++
code.
Is there a proposed solution for this?
> [C++] If an I/O error is encountered while the dataset writer is applying
> backpressure there could be deadlock in rare (artifical?) situations
> ----------------------------------------------------------------------------------------------------------------------------------------------
>
> Key: ARROW-16259
> URL: https://issues.apache.org/jira/browse/ARROW-16259
> Project: Apache Arrow
> Issue Type: Improvement
> Components: C++
> Reporter: Weston Pace
> Priority: Minor
>
> When I/O errors occur the dataset writer handles them when the next batch
> would be written (or at teardown time if it was the last batch).
> If the dataset writer has applied backpressure then this doesn't work because
> there is no "next batch".
> This was encountered in the python backpressure test which is a bit
> artificial. To recreate this change the GatingFs::open_output_stream method
> to return None after its unlocked.
> This is fairly minor because it doesn't really happen in practice. It takes
> many concurrent writes to get into backpressure and unless all of the fail
> there will be some that succeed and eventually get the writer out of
> backpressure enough to discover the failure.
> Still, it would be nice to fix this to clean things up.
--
This message was sent by Atlassian Jira
(v8.20.7#820007)
