[
https://issues.apache.org/jira/browse/SPARK-16742?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15963136#comment-15963136
]
Marcelo Vanzin commented on SPARK-16742:
----------------------------------------
bq. The problem is then that a kerberos-authenticated user submitting their job
would be unaware that their credentials are being leaked to other users.
That's the gist of it, yes. But note that it isn't restricted to files. If all
the user processes are running as the same user, one can just dump the other's
heap, or connect using JVMTI, and get the credentials. Same problem.
The most basic feature needed for any kerberos-related work is user isolation
(different users cannot mess with each others' processes). I was under the
impression that Mesos supported that.
bq. I'm assuming that hadoop.security.auth_to_local is what maps the Kerberos
user to the Unix user...
I'm not exactly familiar with all the YARN settings but yes, the result you get
is that the submitting user runs YARN containers as their own user (nor as some
generic, shared user). Without that, you shouldn't even bother thinking about
inserting Kerberos in the picture, IMO.
bq. We avoid the shared-file problem for keytabs entirely
See my first comment above, that's not enough.
bq. We're probably going to punt on cluster mode for now
You don't need to punt on cluster mode. I don't know where this notion that
cluster mode requires you to distribute keytabs comes from; Spark works just
fine in YARN cluster mode without distributing keytabs. All you need to
distribute are delegation tokens. Keytabs aren't even necessary to log in and
submit the app at all (you can use passwords with kinit, after all).
The only thing distributing keytabs buys you is running applications for longer
than the delegation tokens' max lifetime (normally 7 days by default).
bq. If you see any blockers
Lack of user isolation is always a blocker; without that there's no way to
prevent one user from seeing another's credentials. But I've asked this in the
past and the answer I got is that Mesos supports it...
> Kerberos support for Spark on Mesos
> -----------------------------------
>
> Key: SPARK-16742
> URL: https://issues.apache.org/jira/browse/SPARK-16742
> Project: Spark
> Issue Type: New Feature
> Components: Mesos
> Reporter: Michael Gummelt
>
> We at Mesosphere have written Kerberos support for Spark on Mesos. We'll be
> contributing it to Apache Spark soon.
> Mesosphere design doc:
> https://docs.google.com/document/d/1xyzICg7SIaugCEcB4w1vBWp24UDkyJ1Pyt2jtnREFqc/edit#heading=h.tdnq7wilqrj6
> Mesosphere code:
> https://github.com/mesosphere/spark/commit/73ba2ab8d97510d5475ef9a48c673ce34f7173fa
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
