cloud-fan commented on issue #26159: [SPARK-29506][SQL] Use dynamicPartitionOverwrite in FileCommitProtocol when insert into hive table URL: https://github.com/apache/spark/pull/26159#issuecomment-544369248 It looks to me that the underlying issue is: MR committer is slow and we need a home-made custom committer. I think this is a big story and needs more discussion. This PR is just a trick to bypass the MR committer by using dynamicPartitionOverwrite when the behavior/result is not affected.
---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected] With regards, Apache Git Services --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
