itskals commented on issue #21066: [SPARK-23977][CLOUD][WIP] Add commit protocol binding to Hadoop 3.1 PathOutputCommitter mechanism URL: https://github.com/apache/spark/pull/21066#issuecomment-474203129 Hi, Thanks for the elaborate response. I think I am getting it. So if I configure as DynamicPartitionOverwrite(true)+PartitionedStagingCommitter+ConflictResolutionMode(REPLACE) in this PR(after removing the check on dynamicPartitionOverwrite) parameter, will the behaviour[end result] of Spark be exactly same as to that before this PR for partition data source? Or do you see some usecase/scenario where this will differ from what was it before this PR? Note: Small update on the previous post. The dynamicPartitionOverwrite came in [SPARK-20236](https://issues.apache.org/jira/browse/SPARK-20236). Thanks.
---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected] With regards, Apache Git Services --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
