Shekhar Prasad Rajak created SPARK-55534:
--------------------------------------------
Summary: executedPlan for Insert/Update query
Key: SPARK-55534
URL: https://issues.apache.org/jira/browse/SPARK-55534
Project: Spark
Issue Type: New Feature
Components: Spark Core
Affects Versions: 4.1.1
Reporter: Shekhar Prasad Rajak
We should get consistent result for any kind of SQL query for API
df.queryExecution.executedPlan
// For SELECT queries
val df = spark.sql("SELECT * FROM t")
val plan = df.queryExecution.executedPlan // Works - returns DataFrame with
plan
// For INSERT/WRITE commands
val df = spark.sql("INSERT INTO t VALUES (1)")
// df is an EMPTY DataFrame with just row count metrics
// The actual write plan is ALREADY EXECUTED and gone, so we can not find the
executed plan
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
