squito commented on issue #24499: [SPARK-25888][Core] Serve local disk persisted blocks by the external service after releasing executor by dynamic allocation URL: https://github.com/apache/spark/pull/24499#issuecomment-490582020 moving the discussion about file deletion and null buffers to the top-level so it doesn't get folded on code updates: > what if the file existed when FileSegmentManagedBuffer is constructed with a non-zero length but when the file is about to be put on the wire or right before that it is removed by the owning executor. Just to understand what you're protecting against here -- is there an expected path where the file gets deleted? Or are you just trying to have the behavior be a little more understandable when something bad happens on the host and the file goes missing?
---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected] With regards, Apache Git Services --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
