DanielLeens commented on PR #11307: URL: https://github.com/apache/seatunnel/pull/11307#issuecomment-6411606173
Quick follow-up on my review of `380764cb` (the one I posted while the apache Build check was still queued). Thanks for your patience, @goutamadwant. That statement is now stale: the check finished with **failure** on this same head, so the "only the CI gate is left" line needs a correction. Review is by log reading only; nothing was executed locally. # What the CI says (head `380764cb`, unchanged since my review) The apache `Build` check is a pointer to the fork run: [run 40159933537](https://github.com/goutamadwant/seatunnel/actions/runs/40159933537). Failing jobs: | Job | Failure | Related to this PR? | |---|---|---| | [`unit-test (11, windows-latest)`](https://github.com/goutamadwant/seatunnel/actions/runs/40159933537/job/121089058141) and [`unit-test (17, windows-latest)`](https://github.com/goutamadwant/seatunnel/actions/runs/40159933537/job/121089058148) | `Spark35BatchWriteTest.boundedMicroBatchCommitsRowsThroughStreamingAdapter:103` -> `UnsatisfiedLinkError: 'boolean org.apache.hadoop.io.nativeio.NativeIO$Windows.access0(java.lang.String, int)'` | **Yes** - the test was added by this PR (commit `d8045875a07`) | | [`engine-v2-it (17, ubuntu-latest)`](https://github.com/goutamadwant/seatunnel/actions/runs/40159933537/job/121089058423) | `SplitClusterFaultToleranceIT.testStreamJobCancelResolvesWhenWorkerCrashesBeforeCancelAck` -> `ConditionTimeout` (1 error of 215 tests) | **No** - Zeta cancel-after-worker-loss flake, unrelated to the Spark starter | | `connector-file-local-it` (11 and 17) | cancelled | No - cancelled by the failing jobs, no test result | Both Ubuntu `unit-test` lanes are green, so the new test passes on Linux; it only breaks on the Windows runners. ## 1. Windows unit-test failure (needs a small change in this PR) The new test starts a real local `SparkSession` and a file-based `Trigger.Once` streaming query with a checkpoint directory. On Windows, Hadoop's local file system needs the native `winutils`/`hadoop.dll` layer for the permission check, which the runner does not provide, hence the `UnsatisfiedLinkError` at `Spark35BatchWriteTest.java:103` (the `.start()` path). This is an environment limitation of the Windows runner, not a bug in the production code. Minimal next step: skip this one test on Windows, exactly as you already did for the launcher test in `Spark35LauncherTest.java:48`: ```java @Test public void boundedMicroBatchCommitsRowsThroughStreamingAdapter() throws Exception { Assumptions.assumeFalse(System.getProperty("os.name").startsWith("Windows")); ... } ``` (plus `import org.junit.jupiter.api.Assumptions;`). The Linux lanes keep running it, so the streaming-adapter regression guard stays in CI. Alternatively, provide `winutils` on the Windows lane, but that is a much bigger CI change for one test. ## 2. `SplitClusterFaultToleranceIT` is a known engine flake That test fails when a worker is lost while a cancel is in flight (the job ends up in the wrong terminal state). It is unrelated to anything in `seatunnel-spark-3.5-starter`. dev has a fix, #12377 "Preserve cancellation after worker failure", merged on 2026-10-26 and not yet in your branch (your branch is 11 commits behind dev). Once you update from dev, a re-run of that job should pass; there is no need to change anything in this PR for it. # Source conclusion (unchanged) There is still no source-level blocker in the production code; my earlier review of `380764cb` stands (Fully compatible, only the Low carryovers). One addition to Issue 7 of my last review (the `Spark35BatchWriteTest` nits): the Windows guard above is now the one concrete item for that test. ### Conclusion: Ready to merge after fixes - **Blocker (CI only):** add the Windows `Assumptions` guard in `Spark35BatchWriteTest`, then update the branch from dev and let the Build check re-run (job-level re-run of `SplitClusterFaultToleranceIT` is fine if it recurs). - No other changes are requested. Thanks again for the careful work on this one! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
