Ruifeng Zheng created SPARK-44406:
-------------------------------------
Summary: DataFrame depending on temp view fail after the view is
drop
Key: SPARK-44406
URL: https://issues.apache.org/jira/browse/SPARK-44406
Project: Spark
Issue Type: Bug
Components: Connect
Affects Versions: 3.5.0
Reporter: Ruifeng Zheng
In vanilla Spark:
{code:java}
In [1]: df = spark.createDataFrame([(1, 4), (2, 4), (3, 6)], ["A", "B"])
In [2]: df.createOrReplaceTempView("t")
In [3]: df2 = spark.sql("select * from t")
In [4]: df2.show()
+---+---+
| A| B|
+---+---+
| 1| 4|
| 2| 4|
| 3| 6|
+---+---+
In [5]: spark.catalog.dropTempView("t")
Out[5]: True
In [6]: df2.show()
+---+---+
| A| B|
+---+---+
| 1| 4|
| 2| 4|
| 3| 6|
+---+---+
{code}
In Spark Connect:
{code:java}
In [1]: df = spark.createDataFrame([(1, 4), (2, 4), (3, 6)], ["A", "B"])
In [2]: df.createOrReplaceTempView("t")
In [3]: df2 = spark.sql("select * from t")
In [4]: df2.show()
+---+---+
| A| B|
+---+---+
| 1| 4|
| 2| 4|
| 3| 6|
+---+---+
In [5]: spark.catalog.dropTempView("t")
Out[5]: True
In [6]: df2.show()
23/07/13 11:57:18 ERROR SparkConnectService: Error during: execute. UserId:
ruifeng.zheng. SessionId: 1fc234fd-07da-4ad0-9ec5-2d818cef6033.
org.apache.spark.sql.AnalysisException: [TABLE_OR_VIEW_NOT_FOUND] The table or
view `t` cannot be found. Verify the spelling and correctness of the schema and
catalog.
If you did not qualify the name with a schema, verify the current_schema()
output, or qualify the name with the correct schema and catalog.
To tolerate the error on drop use DROP VIEW IF EXISTS or DROP TABLE IF EXISTS.;
line 1 pos 14;
'Project [*]
+- 'UnresolvedRelation [t], [], false
{code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
