akshaychitneni opened a new pull request, #2425:
URL: https://github.com/apache/datafusion-ballista/pull/2425
Add the two failure scenarios with kind cluster backend:
* G (#2029): every executor lost mid-query via scale-to-0 (a total loss the
Deployment will not undo, unlike deleting a pod); the job must fail with an
executor-loss error, not hang. Run under both AQE settings.
* F: a force-deleted executor pod is rescheduled with a new id and the
cluster reabsorbs it and keeps serving queries — genuine rescheduling the
process backend cannot express. AQE-off only (planner-independent).
Dedupe the scheduler REST polling shared by both backends into src/rest.rs;
TestCluster and K8sCluster now delegate to it
# Which issue does this PR close?
<!--
We generally require a GitHub issue to be filed for all bug fixes and
enhancements and this helps us generate change logs for our releases. You can
link an issue to this PR using the GitHub syntax. For example `Closes #123`
indicates that this PR will close issue #123.
-->
Closes #. N/A - follow-up test coverage for #2029
# Rationale for this change
<!--
Why are you proposing this change? If this is already explained clearly in
the issue then this section is not needed.
Explaining clearly why changes are proposed helps reviewers understand your
changes and offer better suggestions for fixes.
-->
The #2029 scheduler fix (#2212) was only covered by the process-based chaos
harness. The two failure modes that matter can't be reproduced without a real
cluster: total executor loss (a Deployment scaled to zero, not a transient
process/pod kill) and pod rescheduling-with-new-id.
# What changes are included in this PR?
<!--
There is no need to duplicate the description in the issue here but it is
sometimes worth providing a summary of the individual changes in this PR.
-->
# Are there any user-facing changes?
<!--
If there are user-facing changes then we may require documentation to be
updated before approving the PR.
-->
No — test/harness code only, no public API or runtime changes.
<!--
If there are any breaking changes to public APIs, please add the `api
change` label.
-->
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
