[
https://issues.apache.org/jira/browse/SPARK-18549?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15699163#comment-15699163
]
Jiang Xingbo commented on SPARK-18549:
--------------------------------------
Currently we barely don't manage a view's references, when a table/view is
changed, all views that reference(directly or in-directly) the table/view are
not notified.
Perhaps a better approach is to keep the references of a view in a variable,
say 'referenceTables'. On DDL commands, like Create/Drop/Change tables, we
search for the views that references the current table, and examine whether
they are valid after the operation, if not, we should invalidate them as soon
as possible.
I haven't generate a detailed plan to this, but I'm sure it will be useful. Any
further disscussion is welcomed. Thank you!
> Failed to Uncache a View that References a Dropped Table.
> ---------------------------------------------------------
>
> Key: SPARK-18549
> URL: https://issues.apache.org/jira/browse/SPARK-18549
> Project: Spark
> Issue Type: Bug
> Components: SQL
> Affects Versions: 2.0.2
> Reporter: Xiao Li
> Priority: Critical
>
> {code}
> spark.range(1, 10).toDF("id1").write.format("json").saveAsTable("jt1")
> spark.range(1, 10).toDF("id2").write.format("json").saveAsTable("jt2")
> sql("CREATE VIEW testView AS SELECT * FROM jt1 JOIN jt2 ON id1 == id2")
> // Cache is empty at the beginning
> assert(spark.sharedState.cacheManager.isEmpty)
> sql("CACHE TABLE testView")
> assert(spark.catalog.isCached("testView"))
> // Cache is not empty
> assert(!spark.sharedState.cacheManager.isEmpty)
> {code}
> {code}
> // drop a table referenced by a cached view
> sql("DROP TABLE jt1")
> -- So far everything is fine
> // Failed to unache the view
> val e = intercept[AnalysisException] {
> sql("UNCACHE TABLE testView")
> }.getMessage
> assert(e.contains("Table or view not found: `default`.`jt1`"))
> // We are unable to drop it from the cache
> assert(!spark.sharedState.cacheManager.isEmpty)
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
