Github user jinxing64 commented on the issue:
https://github.com/apache/spark/pull/21286
Cool, so do I understand correctly that there are only two things need to
do:
1. create a unique jobID, permaps `timestamp+uuid`?
2. disable cleanup in FileOutputCommitter
mapreduce.fileoutputcommitter.cleanup.skipped--- --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
