Eliaaazzz opened a new pull request, #3317: URL: https://github.com/apache/sedona/pull/3317
## Did you read the Contributor Guide? - Yes, I have read the [Contributor Rules](https://sedona.apache.org/latest/community/rule/) and [Contributor Development Guide](https://sedona.apache.org/latest/community/develop/) ## Is this PR related to a ticket? - Yes, and the PR name follows the format `[GH-XXX] my subject`. Closes #3316 ## What changes were proposed in this PR? Makes the `generate data with diagonal distribution` test in `SpiderTests` deterministic, as suggested in the issue. The test asked for 1000 records with `percentage=0.7` but never set the Spider `seed` option, so `SpiderDataSource` fell back to `System.currentTimeMillis()` and the on-diagonal count was a binomial random variable. A legitimate sample could land outside the asserted `[650, 750]` window (a CI run recently produced 649), and the spurious failure also fail-fasted the JVM compatibility matrix. This PR sets `seed=42` and asserts the exact deterministic count for that seed: 676 of the 1000 points on the diagonal, which keeps the `percentage` behavior covered (676/1000 is consistent with `percentage=0.7`) while removing the randomness. A comment in the test explains why the seed is fixed. The expected count is fully determined by the seed regardless of Spark/Scala version: `SpiderScanBuilder` gives partition `i` the seed `seed + i`, `SpiderPartitionReader` draws from `java.util.Random(seed + i)`, and `java.util.Random` is specified platform-independently. I cross-checked the value by replicating the partition-range and generator logic (`computePartitionRanges(4, 1000)` + `GeneratorFactory.create("diagonal", ...)`) in a standalone driver against `sedona-common`, which also yields 676 for seed 42. ## How was this patch tested? Ran the focused suite with the default profile (Spark 3.5.0, Java 11): ``` mvn -pl spark/common test -DwildcardSuites=org.apache.sedona.sql.SpiderTests -Dtest=none ``` All `SpiderTests` cases pass, including the now-deterministic diagonal test. Repeated runs produce the same result by construction since no time-based seed remains. ## Did this PR include necessary documentation updates? - No, this PR does not affect any public API so no need to change the documentation. 🤖 Generated with [Claude Code](https://claude.com/claude-code) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
