davidm-db commented on code in PR #48794:
URL: https://github.com/apache/spark/pull/48794#discussion_r1850826652
##########
sql/core/src/main/scala/org/apache/spark/sql/scripting/SqlScriptingExecutionNode.scala:
##########
@@ -122,6 +124,17 @@ class SingleStatementExec(
*/
var isExecuted = false
+ /**
+ * Builds a DataFrame from the parsedPlan of this SingleStatementExec.
+ * @param session The SparkSession used.
+ * @return
+ * The DataFrame.
+ */
+ def buildDataFrame(session: SparkSession): DataFrame = {
+ isExecuted = true
Review Comment:
this doesn't seem right, and definitely won't be right in the future (i.e.
not all data frames will have/need to be executed immediately).
with execution v1 (that @miland-db) is working on, we will figure out the
proper set of flags needed and where/when to update them.
for now, I would say to do something like:
```
execNode.buildDataFrame(session).collect()
execNode.isExecuted = true
```
instead of setting the `isExecuted` in the function here
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
