Weston Pace created ARROW-16259:
-----------------------------------
Summary: [C++] If an I/O error is encountered while the dataset
writer is applying backpressure there could be deadlock in rare (artifical?)
situations
Key: ARROW-16259
URL: https://issues.apache.org/jira/browse/ARROW-16259
Project: Apache Arrow
Issue Type: Improvement
Components: C++
Reporter: Weston Pace
When I/O errors occur the dataset writer handles them when the next batch would
be written (or at teardown time if it was the last batch).
If the dataset writer has applied backpressure then this doesn't work because
there is no "next batch".
This was encountered in the python backpressure test which is a bit artificial.
To recreate this change the GatingFs::open_output_stream method to return None
after its unlocked.
This is fairly minor because it doesn't really happen in practice. It takes
many concurrent writes to get into backpressure and unless all of the fail
there will be some that succeed and eventually get the writer out of
backpressure enough to discover the failure.
Still, it would be nice to fix this to clean things up.
--
This message was sent by Atlassian Jira
(v8.20.7#820007)
