Github user sujithjay commented on a diff in the pull request: https://github.com/apache/spark/pull/20956#discussion_r178518257 --- Diff: mllib/src/main/scala/org/apache/spark/ml/tree/impl/NodeIdCache.scala --- @@ -166,9 +166,13 @@ private[spark] class NodeIdCache( } } } + if (nodeIdsForInstances != null) { + // Unpersist current one if one exists. + nodeIdsForInstances.unpersist(false) + } if (prevNodeIdsForInstances != null) { // Unpersist the previous one if one exists. - prevNodeIdsForInstances.unpersist() + prevNodeIdsForInstances.unpersist(false) --- End diff -- Same question as above. `deleteAllCheckpoints` is blocking because it involves calls to `FileSystem.delete`. So, does it make sense to make the call to `unpersist` non-blocking? Am I missing something here?
--- --------------------------------------------------------------------- To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org