adrians commented on code in PR #44622:
URL: https://github.com/apache/spark/pull/44622#discussion_r1483012515
##########
sql/core/src/main/scala/org/apache/spark/sql/execution/command/DataWritingCommand.scala:
##########
@@ -102,14 +102,17 @@ object DataWritingCommand {
}
/**
* When execute CTAS operators, and the location is not empty, throw
[[AnalysisException]].
- * For CTAS, the SaveMode is always [[ErrorIfExists]]
+ * For CTAS, the SaveMode is always [[ErrorIfExists]].
+ * For Create-Table-If-Not-Exists, the SaveMode is [[Ignore]].
*
* @param tablePath Table location.
* @param saveMode Save mode of the table.
* @param hadoopConf Configuration.
*/
def assertEmptyRootPath(tablePath: URI, saveMode: SaveMode, hadoopConf:
Configuration): Unit = {
Review Comment:
This merge-request changes the `CreateDataSourceTableAsSelectCommand`
function (which in turn does a check using
`DataWritingCommand.assertEmptyRootPath`). That function is called as
implementation for both SQL & Scala APIs.
For SQL behavior, I tested it manually.
For the Scala API behavior, I added the 3 unit-tests, with all the tests
having the same structure:
* Phase 1 - create a simple external table
* Phase 2 - delete that table (metadata is cleaned-up, data-files remain in
place)
* Phase 3 - create another table in the same path, using
`df.write.[...].option("path",
path).mode(SaveMode.[...]).saveAsTable("test_table")` where the "mode" covers
the 3 situations , check that semantics are respected as per the previous
tables.
Without my code-changes, the test for `mode(SaveMode.ErrorIfExists)` behaves
identically to `mode(SaveMode.Override)`.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
