otterc commented on PR #41225: URL: https://github.com/apache/spark/pull/41225#issuecomment-1553952178
> I'm wondering if this can unblock [SPARK-36744](https://issues.apache.org/jira/browse/SPARK-36744) (Support IO encryption for push-based shuffle). If not, do you happen to know what are the remaining blockers? This doesn't unblock [SPARK-36744](https://issues.apache.org/jira/browse/SPARK-36744). To be able to read merged data, clients only know the map outputs belonging to that merged blocks. If the map outputs are encrypted, the client will need to know the size as well to decrypt it which is a protocol change. Here, the server supports encryption but the application which is doing push-based shuffle doesn't have encryption enabled. I will add this to the summary. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
