[
https://issues.apache.org/jira/browse/HADOOP-12563?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15277321#comment-15277321
]
Li Lu commented on HADOOP-12563:
--------------------------------
I noticed the following errors when running tez on our trunk code:
{code}
2016-05-09 16:12:18,733 [ERROR] [main] |app.DAGAppMaster|: Error starting
DAGAppMaster
java.io.IOException: Exception reading
/tmp/hadoop-llu/nm-local-dir/usercache/llu/appcache/application_1462833671675_0001/container_e03_1462833671675_0001_01_000001/container_tokens
at
org.apache.hadoop.security.Credentials.readTokenStorageFile(Credentials.java:197)
at
org.apache.hadoop.security.UserGroupInformation.loginUserFromSubject(UserGroupInformation.java:789)
at
org.apache.hadoop.security.UserGroupInformation.getLoginUser(UserGroupInformation.java:748)
at
org.apache.hadoop.security.UserGroupInformation.getCurrentUser(UserGroupInformation.java:621)
at org.apache.tez.dag.app.DAGAppMaster.main(DAGAppMaster.java:2345)
Caused by: java.io.IOException: Unknown version 1 in token storage.
at
org.apache.hadoop.security.Credentials.readTokenStorageStream(Credentials.java:215)
at
org.apache.hadoop.security.Credentials.readTokenStorageFile(Credentials.java:194)
... 4 more
{code}
Both Tez and Hadoop are from the latest master/trunk.
Looks like we introduced some incompatible changes in this JIRA? I marked this
JIRA as an incompatible change for now.
> Updated utility to create/modify token files
> --------------------------------------------
>
> Key: HADOOP-12563
> URL: https://issues.apache.org/jira/browse/HADOOP-12563
> Project: Hadoop Common
> Issue Type: New Feature
> Affects Versions: 3.0.0
> Reporter: Allen Wittenauer
> Assignee: Matthew Paduano
> Fix For: 3.0.0
>
> Attachments: HADOOP-12563.01.patch, HADOOP-12563.02.patch,
> HADOOP-12563.03.patch, HADOOP-12563.04.patch, HADOOP-12563.05.patch,
> HADOOP-12563.06.patch, HADOOP-12563.07.patch, HADOOP-12563.07.patch,
> HADOOP-12563.08.patch, HADOOP-12563.09.patch, HADOOP-12563.10.patch,
> HADOOP-12563.11.patch, HADOOP-12563.12.patch, HADOOP-12563.13.patch,
> HADOOP-12563.14.patch, HADOOP-12563.15.patch, HADOOP-12563.16.patch,
> dtutil-test-out, example_dtutil_commands_and_output.txt,
> generalized_token_case.pdf
>
>
> hdfs fetchdt is missing some critical features and is geared almost
> exclusively towards HDFS operations. Additionally, the token files that are
> created use Java serializations which are hard/impossible to deal with in
> other languages. It should be replaced with a better utility in common that
> can read/write protobuf-based token files, has enough flexibility to be used
> with other services, and offers key functionality such as append and rename.
> The old version file format should still be supported for backward
> compatibility, but will be effectively deprecated.
> A follow-on JIRA will deprecrate fetchdt.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
