Github user dding3 commented on a diff in the pull request:
https://github.com/apache/spark/pull/15125#discussion_r102065803
--- Diff: graphx/src/main/scala/org/apache/spark/graphx/Pregel.scala ---
@@ -154,7 +169,9 @@ object Pregel extends Logging {
// count the iteration
i += 1
}
- messages.unpersist(blocking = false)
+ messageCheckpointer.unpersistDataSet()
--- End diff --
I think the thing is we use messageCheckpointer.update to do the cache, to
make a pair, we can use it to unpersist data. Please correct me if I understand
wrong.
I think it's fine to add this new method as there is already a public
method to cache data in PersistQueue, we should provide a public method to
clean the queue.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
