Github user gatorsmile commented on a diff in the pull request:
https://github.com/apache/spark/pull/21818#discussion_r204969835
--- Diff:
sql/catalyst/src/main/scala/org/apache/spark/sql/internal/SQLConf.scala ---
@@ -1335,7 +1335,9 @@ object SQLConf {
"overwriting. In dynamic mode, Spark doesn't delete partitions
ahead, and only overwrite " +
"those partitions that have data written into it at runtime. By
default we use static " +
"mode to keep the same behavior of Spark prior to 2.3. Note that
this config doesn't " +
- "affect Hive serde tables, as they are always overwritten with
dynamic mode.")
+ "affect Hive serde tables, as they are always overwritten with
dynamic mode. This can " +
+ "also be set as an output option for a data source using key
partitionOverwriteMode, " +
--- End diff --
Also need to explain the precedence between the option and sqlconf.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
