Github user srowen commented on a diff in the pull request:
https://github.com/apache/spark/pull/5669#discussion_r28995411
--- Diff:
mllib/src/main/scala/org/apache/spark/mllib/tree/GradientBoostedTrees.scala ---
@@ -177,102 +177,108 @@ object GradientBoostedTrees extends Logging {
treeStrategy.assertValid()
// Cache input
- if (input.getStorageLevel == StorageLevel.NONE) {
+ val persistedInput = if (input.getStorageLevel == StorageLevel.NONE) {
input.persist(StorageLevel.MEMORY_AND_DISK)
- }
+ true
+ } else false
- timer.stop("init")
+ try {
--- End diff --
@Jkbradley it's an interesting question -- it makes sense to ensure the RDD
is unpersisted with `try-finally` but I don't know if any other code does it. I
think the assumption has been that the app will soon die anyway if something
unexpected is going wrong like an exception, so RDD cleanup isn't that
important. Should we stick to that imperfect reasoning here and keep the code
simpler?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
