mridulm commented on pull request #33556: URL: https://github.com/apache/spark/pull/33556#issuecomment-889507340
The assumption being made here is that if no objects are written, then nothing was written to the file. This is not a general assumption to make (there could be some header written for example - not just right now, but in future as we add support for other serializers/codecs/etc). I am assuming this is an effort at cleanup ? If yes, let us avoid making this change. If there is some functional reason to make this (bug/severe perf issue/etc), please do let me know. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
