[
https://issues.apache.org/jira/browse/HADOOP-12563?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15221648#comment-15221648
]
Steve Loughran commented on HADOOP-12563:
-----------------------------------------
Here are the use cases I've encountered related to this
* saving tokens for a principal to a file (HDFS, RM, ATS), so that a process
can be started in an env with {{HADOOP_TOKEN_FILE_LOCATION}} pointing at the
file. This lets me test oozie deployment outside of oozie.
* spark yarn client having to pick up tokens for HBase, Hive and others. This
is done on a case-by-case basis through introspection ugliness. With a standard
interface, all you'd need to do is load the implementation and invoke.
* spark AM doing ticket-based token retrieval, for propagation to executors in
containers.
So: one similar, two within an app, all benefiting from a standard API. Use
case #1 can be handled by your CLI tool, if it does keytab and principal
1. return values: There was a comment about returning null; I want to make sure
that that is not the case, the failures -> exceptions
2. the rationale for {{isTokenRequired()}} is related to other uses. Example,
spark only needs an hbase token if (a) hbase is on the classpath, (b) the
hbase-site.xml provides the binding for hbase and indicates that authentication
is needed. You may have unauthed hbase within a kerberized cluster. Similarly
for RM web access, the implementation would look at the auth method for the web
UI; again, there may be none, even on a secure cluster.
3. RM And ATS code can be found in these classes
https://github.com/apache/incubator-slider/blob/develop/slider-core/src/main/java/org/apache/slider/core/launch/CredentialUtils.java
https://github.com/apache/incubator-slider/blob/develop/slider-core/src/main/java/org/apache/slider/client/TokensOperation.java
4. keytabs & principals
bq. I agree it would be cool to have some mechanism to let hadoop know how to
kinit for an OS user who is already authenticated and has OS perms to access a
keytab, e.g. "kinit -kt $
it's called {{UserGroupInformation.loginUserFromKeytabAndReturnUGI()}}, and is
easy to use, provided you make it the first thing you do in your code after
reading all config, and before talking to any services. Look in
{{TokensOperation}} for the code to lift
Finally, note that token acquisition on HA clusters is tricker than you' expect
... we'll all need to review that code.
> Updated utility to create/modify token files
> --------------------------------------------
>
> Key: HADOOP-12563
> URL: https://issues.apache.org/jira/browse/HADOOP-12563
> Project: Hadoop Common
> Issue Type: New Feature
> Affects Versions: 3.0.0
> Reporter: Allen Wittenauer
> Assignee: Matthew Paduano
> Attachments: HADOOP-12563.01.patch, HADOOP-12563.02.patch,
> HADOOP-12563.03.patch, HADOOP-12563.04.patch, HADOOP-12563.05.patch,
> HADOOP-12563.06.patch, HADOOP-12563.07.patch, HADOOP-12563.07.patch,
> HADOOP-12563.08.patch, dtutil-test-out, dtutil_diff_07_08,
> example_dtutil_commands_and_output.txt, generalized_token_case.pdf
>
>
> hdfs fetchdt is missing some critical features and is geared almost
> exclusively towards HDFS operations. Additionally, the token files that are
> created use Java serializations which are hard/impossible to deal with in
> other languages. It should be replaced with a better utility in common that
> can read/write protobuf-based token files, has enough flexibility to be used
> with other services, and offers key functionality such as append and rename.
> The old version file format should still be supported for backward
> compatibility, but will be effectively deprecated.
> A follow-on JIRA will deprecrate fetchdt.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
