zhengruifeng opened a new pull request, #57750: URL: https://github.com/apache/spark/pull/57750
### What changes were proposed in this pull request? This PR delays allocation of `ExpectationSum` in `GaussianMixture` until an aggregation partition receives its first point. Empty partitions and intermediate tree branches use `null` as the aggregation identity. ### Why are the changes needed? The previous `treeAggregate` zero value eagerly allocated the mixture weights, means, and one dense covariance matrix per cluster. That can be substantial for large models and is also serialized to every partition, even empty ones. ### Does this PR introduce _any_ user-facing change? No. ### How was this patch tested? Not run (not requested). Added coverage using an RDD with an empty partition. Static checks: `git diff --check`, ASCII scan, and changed-line length scan. ### Was this patch authored or co-authored using generative AI tooling? Generated-by: Codex (GPT-5) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
