Github user maryannxue-databricks commented on a diff in the pull request:
https://github.com/apache/spark/pull/21531#discussion_r195236991
--- Diff:
sql/core/src/test/scala/org/apache/spark/sql/DatasetCacheSuite.scala ---
@@ -96,4 +99,38 @@ class DatasetCacheSuite extends QueryTest with
SharedSQLContext {
agged.unpersist()
assert(agged.storageLevel == StorageLevel.NONE, "The Dataset agged
should not be cached.")
}
+
+ test("persist and then withColumn") {
+ val df = Seq(("test", 1)).toDF("s", "i")
+ // We should not invalidate the cached DataFrame
--- End diff --
Remove this comment? Or what is it were you trying to say?
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
