zsxwing opened a new pull request #24211: [SPARK-27275][CORE]Fix potential corruption in EncryptedMessage.transferTo URL: https://github.com/apache/spark/pull/24211 ## What changes were proposed in this pull request? Right now there are several issues in `EncryptedMessage.transferTo`: - When the underlying buffer has more than `1024 * 32` bytes (this should be rare but it could happen in error messages that send over the wire), it may just send a partial message as `EncryptedMessage.count` becomes less than `transferred`. This will cause the client hang forever (or timeout) as it will wait until receiving expected length of bytes, or weird errors (such as corruption or silent correctness issue) if the channel is reused by other messages. - When the underlying buffer is full, it's still trying to write out bytes in a busy loop. This PR fixes the issues in `EncryptedMessage.transferTo` and also makes it follow the contract of `FileRegion`: - `count` should be a fixed value which is just the length of the whole message. - It should be non-blocking. When the underlying socket is not ready to write, it should give up and give control back. - `transferTo` should return the length of written bytes. ## How was this patch tested? The new added tests.
---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected] With regards, Apache Git Services --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
