itskals commented on a change in pull request #25840: [SPARK-29166][SQL] Add
parameters to limit the number of dynamic partitions for data source table
URL: https://github.com/apache/spark/pull/25840#discussion_r327420033
##########
File path:
sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/SQLHadoopMapReduceCommitProtocol.scala
##########
@@ -32,10 +35,14 @@ import org.apache.spark.sql.internal.SQLConf
class SQLHadoopMapReduceCommitProtocol(
jobId: String,
path: String,
- dynamicPartitionOverwrite: Boolean = false)
+ dynamicPartitionOverwrite: Boolean = false,
Review comment:
Can we make this parameter a collection/set/map. ?? making it so keeps the
interface fixed and also extendable for future.
Suppose we have more such behaviours to be added (which I am sure will
happen), we cannot create new interfaces and break the contract each
time(though we try not to).
Having a Key-Value pair of configuration will be much easier for such
extensions.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
