RushabhK commented on PR #9993: URL: https://github.com/apache/incubator-gluten/pull/9993#issuecomment-2983141058
> @RushabhK Thank you for your fix. I still have two questions: > > 1. In addition to this PR, do we also need [PR#9808](https://github.com/apache/incubator-gluten/pull/9808) to delete the failed files in this directory? > 2. Why does Gluten need to handle the ManifestCommitter separately, whereas Spark does not seem to require this? @JkSelf 1. No we won't require to delete the failed files in the directory now, Manifest committer will ensure to only track the successfully committed files to manifest.json (From .staging temporary directory to the target location). 2. Can you point to the spark code how it handles this once? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
