Github user mridulm commented on the issue:
https://github.com/apache/spark/pull/17723
@vanzin Looking at it from view of specific API usage, while relevant for
analyzing implementation, is not the whole picture. Is hadoop-security model
sufficient for spark ? Will spark need to support others as well - and if it
does, will our design allow for it ? [1].
A specific implementation of what we expose will be dependent on
hadoop-security (existing credential providers).
In order to make progress here, let me highlight a few areas where
hadoop-security is used explicitly or implicitly in current code.
* Pre-requisites - principal/keytab : they are specific to use of kerberos
in hadoop-security.
* Environment in which credential provider's are invoked.
* Expected to be within ugi returned via
`ugi.loginUserFromKeytabAndReturnUGI`
* Current credential providers assume this as impl detail (for example
hbase - perhaps others too).
* Use of o.a.h.security.Credential in our interface.
* The api is specific to hadoop-security - and assumes a model of secret
keys and `TokenIdentifier`.
* The latter might be non trivial to adapt for non hadoop-security
implementations.
* Application of acquired tokens at executors/driver.
* For hadoop-security, this becomes
`ugi.getCurrentUser.addCredentials(Credentials)`
* For others, it could be something specific to its implementation.
I am assuming token distribution should not be an issue - though we might
have multiple implementations for it.
We are also assuming a model of periodic renewal of credentials (based on
expiry) - are there other models we would need to support ?
[1] Having said this, I am not suggesting that we must not use
hadoop-security as basis for introduction of this feature into spark core : if
we find the model suitable for spark's needs (present and future), that is
great. I did not see an analysis being done here on that regard - If I missed
it (likely), please point me to relevant discussion.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
