andygrove opened a new pull request, #2812: URL: https://github.com/apache/datafusion-ballista/pull/2812
# Which issue does this PR close? Closes #2811. # Rationale for this change When the executor serving a shuffle partition dies while a block-transport fetch is waiting for the first block, the stream error reaches the client as a plain `IpcError` (for example `h2 protocol error ... BrokenPipe`). `BallistaClient::as_fetch_failed` only converts `GrpcActionError`, so the task fails with a runtime error and the job fails, instead of the scheduler re-running the stage that produced the input. This makes `stage_losing_input_after_partial_success_is_recovered` fail intermittently in CI. Failures after the first block were already reported as `FetchFailed` by the shuffle reader, so this was the only gap on the block path. The Flight path already reports its equivalent failure as `GrpcActionError`. # What changes are included in this PR? `execute_do_action_with_header` now maps a failure from `BlockDataStream::try_new` to `GrpcActionError`, which `fetch_partition_proxied` and `fetch_byte_ranges` already turn into `FetchFailed`. A test serves a `do_action` body that fails before its first block and checks that `fetch_partition` returns `FetchFailed`. It failed with the `IpcError` from the CI log before the change. # Are there any user-facing changes? No. A fetch that loses its connection at this point now recovers instead of failing the query. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
