Junping Du created MAPREDUCE-7012:
-------------------------------------
Summary: 3.0 deployment cannot work with old version MR tar ball
which break rolling upgrade
Key: MAPREDUCE-7012
URL: https://issues.apache.org/jira/browse/MAPREDUCE-7012
Project: Hadoop Map/Reduce
Issue Type: Bug
Components: distributed-cache, mrv2
Reporter: Junping Du
Priority: Blocker
I tried to deploy 3.0 cluster with 2.9 MR tar ball. The MR job is failed
because following error:
{noformat}
2017-11-21 12:42:50,911 INFO [main]
org.apache.hadoop.mapreduce.v2.app.MRAppMaster: Created MRAppMaster for
application appattempt_1511295641738_0003_000001
2017-11-21 12:42:51,070 WARN [main] org.apache.hadoop.util.NativeCodeLoader:
Unable to load native-hadoop library for your platform... using builtin-java
classes where applicable
2017-11-21 12:42:51,118 FATAL [main]
org.apache.hadoop.mapreduce.v2.app.MRAppMaster: Error starting MRAppMaster
java.lang.RuntimeException: Unable to determine current user
at
org.apache.hadoop.conf.Configuration$Resource.getRestrictParserDefault(Configuration.java:254)
at
org.apache.hadoop.conf.Configuration$Resource.<init>(Configuration.java:220)
at
org.apache.hadoop.conf.Configuration$Resource.<init>(Configuration.java:212)
at
org.apache.hadoop.conf.Configuration.addResource(Configuration.java:888)
at
org.apache.hadoop.mapreduce.v2.app.MRAppMaster.main(MRAppMaster.java:1638)
Caused by: java.io.IOException: Exception reading
/tmp/nm-local-dir/usercache/jdu/appcache/application_1511295641738_0003/container_e03_1511295641738_0003_01_000001/container_tokens
at
org.apache.hadoop.security.Credentials.readTokenStorageFile(Credentials.java:208)
at
org.apache.hadoop.security.UserGroupInformation.loginUserFromSubject(UserGroupInformation.java:907)
at
org.apache.hadoop.security.UserGroupInformation.getLoginUser(UserGroupInformation.java:820)
at
org.apache.hadoop.security.UserGroupInformation.getCurrentUser(UserGroupInformation.java:689)
at
org.apache.hadoop.conf.Configuration$Resource.getRestrictParserDefault(Configuration.java:252)
... 4 more
Caused by: java.io.IOException: Unknown version 1 in token storage.
at
org.apache.hadoop.security.Credentials.readTokenStorageStream(Credentials.java:226)
at
org.apache.hadoop.security.Credentials.readTokenStorageFile(Credentials.java:205)
... 8 more
2017-11-21 12:42:51,122 INFO [main] org.apache.hadoop.util.ExitUtil: Exiting
with status 1: java.lang.RuntimeException: Unable to determine current user
{noformat}
I think it is due to token incompatiblity change between 2.9 and 3.0. As we
claim "rolling upgrade" is supported in Hadoop 3, we should fix this before we
ship 3.0 otherwise all MR running applications will get stuck during/after
upgrade.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
