Github user yhuai commented on a diff in the pull request:
https://github.com/apache/spark/pull/4689#discussion_r25000852
--- Diff:
sql/core/src/test/scala/org/apache/spark/sql/CachedTableSuite.scala ---
@@ -280,4 +280,43 @@ class CachedTableSuite extends QueryTest {
assert(intercept[RuntimeException](table("t1")).getMessage.startsWith("Table
Not Found"))
assert(!isCached("t2"))
}
+
+ test("Drop cached temporary table when the table gets overwritten") {
+ val query1 = "SELECT key FROM testData LIMIT 10"
+ val df1 = sql(query1)
+ df1.registerTempTable("t1")
+ sql(s"CACHE TABLE t2 AS $query1")
+ assert(isCached("t2"))
+ // t1 will be cached too because it has the same plan as t2.
+ assert(isCached("t1"))
+ assert(cacheManager.lookupCachedData(df1).isDefined)
+
+ val query2 = "SELECT key FROM testData LIMIT 5"
+ val df2 = sql(query2)
+ sql(s"CACHE TABLE t2 AS $query2")
+ // t1 will not be cached because it has not been explicitly cached.
+ assert(!isCached("t1"))
+ assert(isCached("t2"))
+ assert(cacheManager.lookupCachedData(df2).isDefined)
+ dropTempTable("t2")
+
+ assert(cacheManager.lookupCachedData(df1).isEmpty)
+ assert(cacheManager.lookupCachedData(df2).isEmpty)
+ }
--- End diff --
Right, `t1` will be uncached. We need to also track the reference of a
logical plan to make it correct. Let me just use this PR to add clearCache
command.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
