HyukjinKwon opened a new pull request, #57032:
URL: https://github.com/apache/spark/pull/57032
### What changes were proposed in this pull request?
`MLTest.testClassificationModelSingleRawPrediction` and
`testProbClassificationModelSingleProbPrediction` compared the DataFrame
`transform`
output against the scalar `predictRaw` / `predictProbability` output with
exact `===`
equality. This PR compares the two prediction vectors with a tight absolute
tolerance
(`1e-9`) via a small `assertVectorsAlmostEqual` helper instead.
### Why are the changes needed?
On arm64 macOS the DataFrame path and the scalar path can round differently
in the last
ULP (e.g. `1.543502002724983` vs `1.5435020027249835`), so the exact `===`
check fails:
```
[1.543502002724983,-1.543502002724983] did not equal
[1.5435020027249835,-1.5435020027249835] (MLTest.scala:199)
```
This fails suites such as `MultilayerPerceptronClassifierSuite` on the
scheduled
`Build / Maven (Scala 2.13, JDK 21, MacOS-26)` lane, while the same tests
are bit-identical
on Linux. A `1e-9` absolute tolerance absorbs last-ULP platform rounding
while still catching
any real discrepancy between the two code paths.
### Does this PR introduce _any_ user-facing change?
No. Test-only.
### How was this patch tested?
Ran the affected classifier suites on a `macos-15` GitHub Actions runner
(arm64, same class as
`macos-26`):
- `MultilayerPerceptronClassifierSuite`, `LogisticRegressionSuite`,
`LinearSVCSuite`,
`NaiveBayesSuite` — **113 tests, 0 failures** (the `prediction on single
instance` cases that
previously failed now pass).
Passed: https://github.com/HyukjinKwon/spark/actions/runs/28766568832
Before this change the same suites fail on `macos-26`:
https://github.com/apache/spark/actions/runs/28753698265
### Was this patch authored or co-authored using generative AI tooling?
Generated-by: Claude Code
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
