Github user mridulm commented on the issue:
https://github.com/apache/spark/pull/17723
@vanzin Since hadoop 1 is no longer maintained, I was referring to hadoop
security as yarn security - you are right, they are not the same : though in
what I described, replacing one with another, should not change the concerns I
hope ?
UGI and associated infra in hadoop security is tied to its ecosystem.
hadoop-security is specific to how hadoop manages security (use of kerberos,
delegation tokens, how they are renewed, implicit renewals, etc) - it is not a
standalone security library. It also includes implementation hooks for hadoop
ipc, shared state for hadoop services, implementation idioms, etc.
Depending on hadoop api as a library (codec, encryption, etc) is different
from depending on it to define how spark manages security. Having said that, we
do depend on hadoop specific api's exposed in spark - but they are for exposing
corresponding hadoop functionality and is also usually historical in nature
(use in SparkContext, RDD, etc for example).
In yarn resource manager, it makes logical sense to use hadoop security -
since spark becomes a yarn service. In mesos or other non hadoop based
schedulers, I am not sure it does (other than implementation simplicity in case
they follow same security paradigm as hadoop).
Depending on hadoop security in core for spark security should be evaluated
on its merits and we should be cognizant of the implications of doing so -
implementation simplicity should not be the only driving factor.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
