zhengruifeng opened a new pull request, #57665: URL: https://github.com/apache/spark/pull/57665
### What changes were proposed in this pull request? This patch adds an exclusion list to `Params.estimateMatadataSize` and uses it from `CrossValidatorModel.estimatedSize` to omit the estimator, evaluator, and estimator parameter maps. The model estimate instead includes the learned best model, collected submodels, metrics, and remaining metadata. ### Why are the changes needed? CrossValidatorModel retains tuning configuration in its parameter maps. Estimators and their parameter validators can retain logger or runtime state that reaches SparkSession and SparkContext. That shared state is not learned model data and can inflate ML Connect cache accounting. ### Does this PR introduce _any_ user-facing change? Yes. CrossValidatorModel size estimates no longer include shared runtime state retained by tuning configuration. Model behavior is unchanged. ### How was this patch tested? Added a CrossValidatorModel estimated-size regression test with a logger-initialized estimator. `JAVA_HOME=/usr/lib/jvm/java-17-openjdk-amd64 build/sbt 'mllib/testOnly org.apache.spark.ml.tuning.CrossValidatorSuite'` ### Was this patch authored or co-authored using generative AI tooling? Generated-by: Codex (GPT-5) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
