rdblue commented on a change in pull request #24798: [SPARK-27724][SQL]
Implement REPLACE TABLE and REPLACE TABLE AS SELECT with V2
URL: https://github.com/apache/spark/pull/24798#discussion_r293156610
##########
File path:
sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/v2/DataSourceV2Strategy.scala
##########
@@ -161,12 +162,42 @@ object DataSourceV2Strategy extends Strategy with
PredicateHelper {
WriteToDataSourceV2Exec(writer, planLater(query)) :: Nil
case CreateV2Table(catalog, ident, schema, parts, props, ifNotExists) =>
- CreateTableExec(catalog, ident, schema, parts, props, ifNotExists) :: Nil
+ catalog match {
+ case staging: StagingTableCatalog =>
+ CreateTableStagingExec(staging, ident, schema, parts, props,
ifNotExists) :: Nil
+ case _ =>
+ CreateTableExec(catalog, ident, schema, parts, props, ifNotExists)
:: Nil
+ }
case CreateTableAsSelect(catalog, ident, parts, query, props, options,
ifNotExists) =>
val writeOptions = new CaseInsensitiveStringMap(options.asJava)
- CreateTableAsSelectExec(
- catalog, ident, parts, planLater(query), props, writeOptions,
ifNotExists) :: Nil
+ catalog match {
+ case staging: StagingTableCatalog =>
+ CreateTableAsSelectStagingExec(
Review comment:
I don't think this name make it obvious what is different between this and
`CreateTableAsSelectExec`. Using "staging" makes me think that it isn't
committed, which is why we use that word to describe the intermediate state --
a staged create. I think it would be better to have a name that makes it clear
to a user that the table is created atomically, like
`AtomicCreateTableAsSelect`, since the CTAS is atomic when it uses a staged
table.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
