Junping Du created HDFS-12920:
---------------------------------
Summary: HDFS default value change (with adding time unit) breaks
old version MR tarball work with new version (3.0) of hadoop
Key: HDFS-12920
URL: https://issues.apache.org/jira/browse/HDFS-12920
Project: Hadoop HDFS
Issue Type: Bug
Components: hdfs
Reporter: Junping Du
Priority: Blocker
I tried to deploy 2.9.0 tar ball with 3.0.0 RC1, and run the job with following
errors:
{noformat}
2017-12-12 13:29:06,824 INFO [main] org.apache.hadoop.service.AbstractService:
Service org.apache.hadoop.mapreduce.v2.app.MRAppMaster failed in state INITED;
cause: org.apache.hadoop.yarn.exceptions.YarnRuntimeException:
java.lang.NumberFormatException: For input string: "30s"
org.apache.hadoop.yarn.exceptions.YarnRuntimeException:
java.lang.NumberFormatException: For input string: "30s"
at
org.apache.hadoop.mapreduce.v2.app.MRAppMaster$2.call(MRAppMaster.java:542)
at
org.apache.hadoop.mapreduce.v2.app.MRAppMaster$2.call(MRAppMaster.java:522)
at
org.apache.hadoop.mapreduce.v2.app.MRAppMaster.callWithJobClassLoader(MRAppMaster.java:1764)
at
org.apache.hadoop.mapreduce.v2.app.MRAppMaster.createOutputCommitter(MRAppMaster.java:522)
at
org.apache.hadoop.mapreduce.v2.app.MRAppMaster.serviceInit(MRAppMaster.java:308)
at
org.apache.hadoop.service.AbstractService.init(AbstractService.java:164)
at
org.apache.hadoop.mapreduce.v2.app.MRAppMaster$5.run(MRAppMaster.java:1722)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:422)
at
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1886)
at
org.apache.hadoop.mapreduce.v2.app.MRAppMaster.initAndStartAppMaster(MRAppMaster.java:1719)
at
org.apache.hadoop.mapreduce.v2.app.MRAppMaster.main(MRAppMaster.java:1650)
{noformat}
This is because HDFS-9821, we are adding time unit to hdfs-default.xml but it
cannot be recognized by old version MR jars.
This break our rolling upgrade story, so should mark as blocker.
A quick workaround is to add values in hdfs-site.xml with removing all time
unit. But the right way may be to revert HDFS-9821.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
---------------------------------------------------------------------
To unsubscribe, e-mail: hdfs-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-dev-h...@hadoop.apache.org
