Github user maropu commented on a diff in the pull request:
https://github.com/apache/spark/pull/21018#discussion_r181122693
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/execution/CacheManager.scala ---
@@ -119,26 +119,60 @@ class CacheManager extends Logging {
while (it.hasNext) {
val cd = it.next()
if (cd.plan.find(_.sameResult(plan)).isDefined) {
- cd.cachedRepresentation.cachedColumnBuffers.unpersist(blocking)
+ cd.cachedRepresentation.clearCache(blocking)
it.remove()
}
}
}
+ /**
+ * Materialize the cache that refers to the given physical plan.
--- End diff --
I thought, since `InMemoryRelation` was copied in a tree sometimes, the
lazy update of `_cachedColumnBuffers` always didn't lead to the
materialization of the corresponding cache entry in `CacheManager` (maybe...).
If so, following queries might have unnecessary matiralization repeatedly?
Therefore, I though we needed to directly update the entry in `CacheManager`.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
