LuciferYang edited a comment on pull request #33556: URL: https://github.com/apache/spark/pull/33556#issuecomment-891569743
@mridulm thanks for your explain I test `DeflaterOutputStream`, there will be 8 bytes([120,-100,3,0,0,0,0,1]) at the end of the file even if we don't write anything and I think I basically understand what you mean. However, I have another question：If this scenario is what we need to consider, should we need call `revertPartialWritesAndClose ` when ` objectsWritten > 0` also? otherwise the meta data will also be written to the file ... Otherwise, a strange result will appear here: 1. No writes at all(`call revertPartialWritesAndClose now`): empty file 2. objectsWritten == 0(`call revertPartialWritesAndClose now`)：only data in the file 3. objectsWritten > 0(`call flush()+close() now`): both data and meta in the file -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
