turboFei edited a comment on issue #26086: [SPARK-29302] Make the file name of 
a task for dynamic partition overwrite be unique
URL: https://github.com/apache/spark/pull/26086#issuecomment-540982545
   Oh, It seems that this issue is related with 
   I just read the comments of that, it seems that there are some risk that 
cause duplicate result.
   @ajithme  @vanzin  @LantaoJin 
   How about  using this method to name a task file for the cases below?
   1. newTaskTempFile and dynamicPartitionOverwrite is true
   2. newTaskTempFileAbsPath
   Because for these cases, it will keep a filesToMove set, which would not 
cause duplicate result.

This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:

With regards,
Apache Git Services

To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

Reply via email to