mridulm commented on pull request #33556: URL: https://github.com/apache/spark/pull/33556#issuecomment-891127364
@Ngone51, @LuciferYang There are two cases here: * No writes after flush * No writes at all (Both are almost the same - just calling them out). In these, it is possible for (meta-) data to be written out during `close` even if `objectsWritten == 0`. A `close` will implicitly `flush()` pending bytes and then `close` the output stream - potentially resulting in non-zero bytes being written out. (For an example, see `DeflaterOutputStream.finish`). Currently, we do not have data in file which is not part of some segment - with the PR, we can have these suffix bytes in file. Do let me know if I am missing something here. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
