WinkerDu commented on pull request #29000: URL: https://github.com/apache/spark/pull/29000#issuecomment-654632940
@turboFei Thanks for your comments. Actually I think there is no partition-explosion cost. Commit task output dir is generated by `FileFormatDataWriter.newOutputWriter`, no matter the task is static-partitioned or dynamic-partitioned, one commit task only deal with one partition dir. ---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
