seayoun opened a new pull request #34725: URL: https://github.com/apache/spark/pull/34725
### What changes were proposed in this pull request? We think it has no data when the segment file not exists when all segment files produced by `BypassMergeSortShuffleWriter` is merging; However, `file.exists()` may rerurn `false` when then the disk which segment file in on is missing and the root catagory exists; the missing disk only lead `file.exists()` return `false` but no exception. The task will run in pease and with no current segment file written. The segment data will be ignored and leading shuffle data loss. ### Why are the changes needed? Shuffle may loss partition data leading corretness problems in final data. ### Does this PR introduce _any_ user-facing change? ### How was this patch tested? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
