[
https://issues.apache.org/jira/browse/SPARK-6017?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14338759#comment-14338759
]
Marcelo Vanzin commented on SPARK-6017:
---------------------------------------
Hey Tom,
No, I was referring to the {{spark.authenticate.secret}} stuff. Those settings
are passed to executors on the command line.
I took a look at the UGI API before writing this spec, and the problem that I
saw with it was that all methods to stash custom credentials into the UGI were
not public (in the Hadoop API sense, even if they were in the Java sense). I
think we already use a few of them here and there, but my solution avoids that
discussion altogether, so I kinda prefer it.
> Provide transparent secure communication channel on Yarn
> --------------------------------------------------------
>
> Key: SPARK-6017
> URL: https://issues.apache.org/jira/browse/SPARK-6017
> Project: Spark
> Issue Type: Umbrella
> Components: YARN
> Reporter: Marcelo Vanzin
> Attachments: secure_spark_on_yarn.pdf
>
>
> A quick description:
> Currently driver and executors communicate through an insecure channel, so
> anyone can listen on the network and see what's going on. That prevents Spark
> from adding some features securely (e.g. SPARK-5342, SPARK-5682) without
> resorting to using internal Hadoop APIs.
> Spark 1.3.0 will add SSL support, but properly configuring SSL is not a
> trivial task for operators, let alone users.
> In light of those, we should add a more transparent secure transport layer.
> I've written a short spec to identify the areas in Spark that need work to
> achieve this, and I'll attach the document to this issue shortly.
> Note I'm restricting things to Yarn currently, because as far as I know it's
> the only cluster manager that provides the needed security features to
> bootstrap the secure Spark transport. The design itself doesn't really rely
> on Yarn per se, just on a secure way to distribute the initial secret (which
> the Yarn/HDFS combo provides).
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
