grundprinzip commented on PR #38659: URL: https://github.com/apache/spark/pull/38659#issuecomment-1314254660
The original idea was for the local data to be sent as Arrow IPC batches as well as it follows the same direction as on the return path. In addition, we have the benefit that the Arrow IPC message actually have a schema embedded so that we can do nice validation on the receive path. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
