Github user jacek-lewandowski commented on the pull request:
https://github.com/apache/spark/pull/3571#issuecomment-67628122
Ok, I think I've found a solution, which is somehow based to what you've
shown me. I've pushed some changes for you to take a look before I continue.
The idea for `CoarseGrainedExecutorBackend` is to use SSL settings from
"executor" namespace, defined in worker's properties file - I mean, these
settings will be inherited from Spark worker configuration. They will be used
by `CoarseGrainedExecutorBackend` in order to connect to the driver and fetch
the application configuration. Once the application configuration is fetched,
the settings inherited from the worker will be used as defaults and can be
overridden by the application settings.
Similar approach for `DriverWrapper` (which is used when you run the
application in `cluster` mode). There are two differences - we use "driver"
namespace for SSL configuration, and the way how the configuration options are
applied. However the final result is similar.
Initially, I wanted to use "daemon" settings as defaults ("daemon"
namespace is used to configure SSL for all daemon processes, like Master and
Worker). However, it could introduce some vulnerability because the application
could access keystores of those daemon processes.
So, in short - the administration will have to configure SSL in at least
two namespaces on each node: daemon and executor. It is also recommended to
configure it in "driver" namespace to provide good defaults. In the
environment, where authentication is not required, all the namespaces can be
configured with the same settings.
What do you think @vanzin ?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
