luka-erceg-data-oss opened a new pull request, #59539: URL: https://github.com/apache/spark/pull/59539
### What changes were proposed in this pull request? Add built-in Kryo registrations for classification tree node arrays, internal and leaf nodes, continuous and categorical splits, and Gini and entropy impurity calculators. Add `NodeKryoSuite` to round-trip the complete node graphs with strict registration enabled. The registrations use the existing optional-class loading mechanism, so Spark Core does not acquire a dependency on MLlib. ### Why are the changes needed? These built-in Spark types are missing from the registration list. Serializing an `Array[Node]` with `spark.kryo.registrationRequired=true` fails unless users register the graph's classes themselves. Without strict registration, Kryo writes class-name metadata for the unregistered types. ### Does this PR introduce _any_ user-facing change? Yes. Classification tree node graphs can be serialized with Spark's Kryo serializer and strict registration enabled without manually registering these built-in classes. No new API or configuration is introduced. ### How was this patch tested? Added `NodeKryoSuite`, covering both classification impurity types, continuous split boundaries, and categorical sets of different sizes. It checks predictions, leaf indices, and class-count statistics after deserialization. Passed `NodeKryoSuite` on Java 17: 2 tests (Gini and entropy), no failures. Static whitespace, added-line length, and ASCII checks also pass. ```shell build/sbt 'mllib/testOnly org.apache.spark.ml.tree.NodeKryoSuite' ``` ### Was this patch authored or co-authored using generative AI tooling? Generated-by: OpenAI Codex CLI 0.154.0 (GPT-5.2) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
