zhengruifeng commented on PR #56584: URL: https://github.com/apache/spark/pull/56584#issuecomment-5053623963
One limitation of this fix is that it only clears the parent of the top-level model being estimated. For composite models, the default `SizeEstimator` traversal still walks nested model fields directly; it does not invoke each nested model's `estimatedSize`. As a result, it cannot remove parent-reachable shared state for cases such as `OneVsRestModel` and `PipelineModel`. Those composite models need their own `estimatedSize` overrides that account for their nested models by calling each nested model's `estimatedSize`. I am addressing these remaining model-specific overrides in [SPARK-58181](https://issues.apache.org/jira/browse/SPARK-58181). -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
