Takeshi Yamamuro created SPARK-33228:
----------------------------------------
Summary: Don't uncache data when replacing an existing view having
the same plan
Key: SPARK-33228
URL: https://issues.apache.org/jira/browse/SPARK-33228
Project: Spark
Issue Type: Improvement
Components: SQL
Affects Versions: 2.4.8, 3.0.2, 3.1.0
Reporter: Takeshi Yamamuro
SPARK-30494's updated the `CreateViewCommand` code to implicitly drop cache
when replacing an existing view. But, this change drops cache even when
replacing a view having the same logical plan. A sequence of queries to
reproduce this as follows;
{code}
scala> val df = spark.range(1).selectExpr("id a", "id b")
scala> df.cache()
scala> df.explain()
== Physical Plan ==
*(1) ColumnarToRow
+- InMemoryTableScan [a#2L, b#3L]
+- InMemoryRelation [a#2L, b#3L], StorageLevel(disk, memory, deserialized, 1
replicas)
+- *(1) Project [id#0L AS a#2L, id#0L AS b#3L]
+- *(1) Range (0, 1, step=1, splits=4)
scala> df.createOrReplaceTempView("t")
scala> sql("select * from t").explain()
== Physical Plan ==
*(1) ColumnarToRow
+- InMemoryTableScan [a#2L, b#3L]
+- InMemoryRelation [a#2L, b#3L], StorageLevel(disk, memory, deserialized, 1
replicas)
+- *(1) Project [id#0L AS a#2L, id#0L AS b#3L]
+- *(1) Range (0, 1, step=1, splits=4)
// If one re-runs the same query `df.createOrReplaceTempView("t")`, the cache's
swept away
scala> df.createOrReplaceTempView("t")
scala> sql("select * from t").explain()
== Physical Plan ==
*(1) Project [id#0L AS a#2L, id#0L AS b#3L]
+- *(1) Range (0, 1, step=1, splits=4)
{code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
