HyukjinKwon commented on PR #58595: URL: https://github.com/apache/spark/pull/58595#issuecomment-5609874432
@zhengruifeng you're right, thanks. A model trained by `spark.mllib` can't hit this: decision trees are depth-capped (`maxDepth <= 30`) and acyclic, and a bisecting k-means tree reaches each node id once, so the cycle/shared-child case only arises from a corrupted or externally/hand-crafted data file -- which isn't a failure mode `spark.mllib` persistence otherwise defends against. So the check is unreachable on any real load path, and the bisecting k-means part also changed behavior (replacing the memoization that normalized a shared-child DAG with a hard failure) for that same non-real case. I've reverted it from `master`, `branch-4.3`, and `branch-4.x`. Sorry for the noise, and thanks @zhengruifeng @dongjoon-hyun for the review. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
