zhengruifeng opened a new pull request, #59501: URL: https://github.com/apache/spark/pull/59501
### What changes were proposed in this pull request? Removes the stale `// TODO: Test BLAS.axpy` comment in `DTStatsAggregator.merge`. The TODO asked whether replacing the element-wise loop over `allStats` with `BLAS.getBLAS(n).daxpy` would be faster. The merge loop itself is unchanged. ### Why are the changes needed? The TODO's open question is answered. I benchmarked the `daxpy` version against the current loop and saw no measurable difference, so there is no optimization to pursue. Benchmark (JDK 17, real `DecisionTreeMetadata` and `DTStatsAggregator` objects, `merge()` timed on aggregates of several sizes, median of 4 JVM invocations per variant): | `allStats` length | Current loop | `daxpy` | |---|---|---| | 96,000 | 33.5 us | 33.6 us | | 352,000 | 171.9 us | 169.9 us | | 3,520,000 | 2.44 ms | 2.39 ms | Both variants produce identical results on a fixed-input checksum. The benchmark ran on a machine with no native BLAS, so `daxpy` used the pure-Java netlib fallback. The merge is memory-bound at roughly 35-50 GB/s, so a native BLAS is unlikely to change the result. The merge runs once per node in the `reduceByKey` combine (`RandomForest.scala`), not per training sample, so the end-to-end effect is smaller still. I did not benchmark on a cluster or in a full random forest training run. ### Does this PR introduce _any_ user-facing change? No. ### How was this patch tested? Comment-only change; no tests added. The microbenchmark described above was run locally and is not part of this PR. ### Was this patch authored or co-authored using generative AI tooling? Generated-by: Claude (system.ai.claude-haiku-5-5) via Claude Code This pull request and its description were written by Isaac. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
