HyukjinKwon opened a new pull request, #56728:
URL: https://github.com/apache/spark/pull/56728
### What changes were proposed in this pull request?
Make the two `SparkSessionE2ESuite` "interrupt all" tests robust against two
flakiness sources:
1. **Class-fetch race.** Run each long-running typed `map` query through a
single call site and warm
it up once (`sleep=0`) before any interrupt, so the closure/`TypeTag`
artifact classes are already
loaded on the executor. Otherwise an `interruptAll()` landing during the
executor's first-time
remote fetch of `$$typecreatorNN` surfaces as `RemoteClassLoaderError`
instead of
`OPERATION_CANCELED`.
2. **Leaked interruptor / cascade.** Wrap the foreground-interrupt test body
in
`try/finally { finished = true }`. Previously, if an assertion failed,
the background `interruptor`
Future kept calling `interruptAll()` and canceled the operations of
*subsequent* tests — turning
one failure into a cascade of `OPERATION_CANCELED` failures across the
whole suite.
### Why are the changes needed?
`SparkSessionE2ESuite` intermittently fails in CI. Confirmed flaky (the same
module group passes on
other runs of the same commit).
**Before (failing in apache/spark CI):** one `RemoteClassLoaderError`
cascading into 7 failures —
https://github.com/apache/spark/actions/runs/28037082466/job/82997066297
**After (this change, validated on a fork):** full connect module `Total
1614, Failed 0`, and
`SparkSessionE2ESuite` re-run 8x with 0 failures —
https://github.com/HyukjinKwon/spark/actions/runs/28066097371 and
https://github.com/HyukjinKwon/spark/actions/runs/28074772169
### Does this PR introduce any user-facing change?
No, test-only.
### How was this patch tested?
Re-ran `SparkSessionE2ESuite` 8x plus the full connect module on CI (links
above); all green.
### Was this patch authored or co-authored using generative AI tooling?
Yes, drafted with Claude Code.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
