Github user mridulm commented on the issue:
https://github.com/apache/spark/pull/17295
> Not really sure what you mean here. But transferring encrypted data
without RPC encryption is not really secure, since the encryption key is
transferred to executors using an RPC. There's even a warning message if RPC
encryption is not on and you enable disk encryption.
Good point, I overlooked that.
So to summarize, after this change, RDD block's transferred will always be
in plain text; with an implicit requirement that rpc encryption is strongly
preferred to be enabled.
Is there any case where it is transfered in encrypted form in supported
cases ? (cases being: broadcast, rdd block transfer, replication, anything else
?)
I wanted to ensure I understand what the final expected behavior/state
would be, and how consistent we will become.
I agree about shuffle being special case'd; I was looking at only
non-shuffle blocks.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
