[
https://issues.apache.org/jira/browse/SPARK-27439?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16821824#comment-16821824
]
Liang-Chi Hsieh edited comment on SPARK-27439 at 4/19/19 10:36 AM:
-------------------------------------------------------------------
The review is resolved during analysis stage when we create the dataset. That
said, it is replaced with {{select * from default.t1}} in the described
example. Even you replace the temp view {{tmp001}} after that, the analyzed
plan won't be changed in the dataset.
Although this is marked as bug, currently I'd think it is just the way Dataset
works. The content of a view in a dataset, is determined when the dataset is
created, not when it is evaluated.
was (Author: viirya):
The review is resolved during analysis stage when we create the dataset. That
said, it is replaced with {{select * from default.t1}} in the described
example. Even you replace the temp view {{tmp001}} after that, the analyzed
plan won't be changed in the dataset.
Currently I'd think it is the way Dataset works. The content of a view in a
dataset, is determined when the dataset is created, not when it is evaluated.
> createOrReplaceTempView cannot update old dataset
> -------------------------------------------------
>
> Key: SPARK-27439
> URL: https://issues.apache.org/jira/browse/SPARK-27439
> Project: Spark
> Issue Type: Bug
> Components: SQL
> Affects Versions: 2.4.1
> Reporter: xjl
> Priority: Major
>
> {code:java}
> SparkSession spark = SparkSession
> .builder()
> .appName("app").enableHiveSupport().master("local[4]")
> .getOrCreate();
> spark.sql("select * from default.t1").createOrReplaceTempView("tmp001");
> Dataset<Row> hiveTable = spark.sql("select * from tmp001");
> spark.sql("select * from default.t2").createOrReplaceTempView("tmp001");
> hiveTable.show();
>
> }
> {code}
> hiveTable show the value of t1 but not t2
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
