HyukjinKwon opened a new pull request, #57146:
URL: https://github.com/apache/spark/pull/57146

   [DO-NOT-MERGE] This is a flaky-test fix under CI verification; the title 
will be updated with the JIRA id once the fork CI run confirms the fix. Please 
do not merge yet.
   
   ### What changes were proposed in this pull request?
   
   Make the Kubernetes integration test `VolumeSuite` more robust to slow 
driver-container startup, and stop the failure reporter from masking the real 
error:
   
   1. `checkDisk()` (`VolumeSuite.scala`) runs `df <path>` **inside** the 
driver container via `Utils.executeCommand`, wrapped in `eventually(...)`. It 
used a short fixed timeout of `Span(10, Seconds)`. For the `OnDemand` PVC 
cases, dynamic PVC provisioning and mounting can keep the driver container in 
`ContainerCreating` for longer than that, so the in-pod command is not yet 
runnable and the assertion never passes within 10s. This changes `checkDisk` to 
use the shared `TIMEOUT` (3 minutes), matching the other in-pod readiness waits 
in the suite.
   2. `logForFailedTest()` (`KubernetesSuite.scala`) fetches the driver pod log 
with `.getLog` **without** a guard. When a test fails while the driver pod is 
still in `ContainerCreating`, that `.getLog` throws `KubernetesClientException` 
(HTTP 400, "container ... is waiting to start: ContainerCreating"), which then 
surfaces as the reported test failure and hides the actual cause. The 
executor-pod log fetch right below already wraps `.getLog` in a `try/catch`; 
this applies the same guard to the driver-pod fetch.
   
   ### Why are the changes needed?
   
   `build_java21.yml` on `branch-4.0` intermittently fails the K8s integration 
test "A driver-only Spark job with an OnDemand PVC volume". In the failing run 
the test failed after ~15s while the same test passed on the in-run retry 
(~13s), which is characteristic of a too-tight readiness timeout rather than a 
product bug. The reported exception was the masking `KubernetesClientException` 
from `logForFailedTest`, not the underlying timeout, making triage harder.
   
   ### Does this PR introduce _any_ user-facing change?
   
   No. Test-only change in the Kubernetes integration tests.
   
   ### How was this patch tested?
   
   Existing K8s integration test `VolumeSuite`. The change widens an existing 
timeout and adds an already-used try/catch guard; verified via a Kubernetes 
integration test run on a fork.
   
   ### Was this patch authored or co-authored using generative AI tooling?
   
   No.
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]


---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]

Reply via email to