imback82 commented on issue #26048: [SPARK-29373][SQL] DataSourceV2: Commands
should not submit a spark job
URL: https://github.com/apache/spark/pull/26048#issuecomment-539793532
I double-checked this. `V2TableWriteExec.writeWithV2` returns
`sparkContext.emptyRDD`. In this case, `DAGScheduler.submitJob` will return
without actually submitting a job. So there will be one job for `CREATE TABLE
AS SELECT`.
So theoretically, we could have just returned `sparkContext.emptyRDD` for
some of the commands that don't return results (such as `USE`, etc.), but I
think the new approach is cleaner (and we still need this new physical operator
for `SHOW DATABASE`, etc.)
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
