rdblue commented on a change in pull request #24233: [SPARK-26356][SQL] remove
SaveMode from data source v2
URL: https://github.com/apache/spark/pull/24233#discussion_r285269320
##########
File path: sql/core/src/main/scala/org/apache/spark/sql/DataFrameWriter.scala
##########
@@ -56,13 +55,16 @@ final class DataFrameWriter[T] private[sql](ds:
Dataset[T]) {
* <li>`SaveMode.Overwrite`: overwrite the existing data.</li>
* <li>`SaveMode.Append`: append the data.</li>
* <li>`SaveMode.Ignore`: ignore the operation (i.e. no-op).</li>
- * <li>`SaveMode.ErrorIfExists`: default option, throw an exception at
runtime.</li>
+ * <li>`SaveMode.ErrorIfExists`: throw an exception at runtime.</li>
* </ul>
+ * <p>
+ * When writing to data source v1, the default option is `ErrorIfExist`.
When writing to data
+ * source v2, the default option is `Append`.
Review comment:
I think @rxin objected to changing the default mode for DSv2. Has that
changed?
I think that this is a good idea because the most sensible default for v2 is
to append if no mode is set, instead of failing because the mode is not
supported. This doesn't change v1 behavior, so I think it is okay.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
