[
https://issues.apache.org/jira/browse/AMBARI-14779?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Alejandro Fernandez updated AMBARI-14779:
-----------------------------------------
Attachment: AMBARI-14779.trunk.patch
> Hive service check failed after EU from HDP 2.1 to 2.3 since ATS is missing
> in Kerberized cluster and configs are wrong
> -----------------------------------------------------------------------------------------------------------------------
>
> Key: AMBARI-14779
> URL: https://issues.apache.org/jira/browse/AMBARI-14779
> Project: Ambari
> Issue Type: Bug
> Components: ambari-server
> Affects Versions: 2.2.0
> Reporter: Alejandro Fernandez
> Assignee: Alejandro Fernandez
> Fix For: 2.2.1
>
> Attachments: AMBARI-14779.trunk.patch
>
>
> *Steps*
> # Install HDP 2.1 cluster with ambari 2.2.1 (HFDS, YARN, ZK, Hive)
> # Kerberize the cluster, this will remove ATS
> # Bootstrap the cluster with versions
> # Install HDP 2.3 and perform EU
> # Run Hive service check after EU is complete
> *Result:*
> Failed with below error
> {code}
> File "/usr/lib/python2.6/site-packages/resource_management/core/shell.py",
> line 140, in _call_wrapper
> result = _call(command, **kwargs_copy)
> File "/usr/lib/python2.6/site-packages/resource_management/core/shell.py",
> line 291, in _call
> raise Fail(err_msg)
> resource_management.core.exceptions.Fail: Execution of '/usr/bin/kinit -kt
> /etc/security/keytabs/smokeuser.headless.keytab [email protected]; env
> JAVA_HOME=/usr/jdk64/jdk1.7.0_67 /var/lib/ambari-agent/tmp/hcatSmoke.sh
> hcatsmokeid16acce51_date582116 prepare true' returned 12. WARNING: Use "yarn
> jar" to launch YARN applications.
> 16/01/21 16:59:51 WARN conf.HiveConf: HiveConf of name
> hive.optimize.mapjoin.mapreduce does not exist
> 16/01/21 16:59:51 WARN conf.HiveConf: HiveConf of name hive.heapsize does not
> exist
> 16/01/21 16:59:51 WARN conf.HiveConf: HiveConf of name
> hive.server2.enable.impersonation does not exist
> 16/01/21 16:59:51 WARN conf.HiveConf: HiveConf of name
> hive.auto.convert.sortmerge.join.noconditionaltask does not exist
> ivysettings.xml file not found in HIVE_HOME or
> HIVE_CONF_DIR,file:/grid/0/hdp/2.3.4.0-3485/hadoop/lib/hadoop-lzo-0.6.0.2.3.4.0-3485-sources.jar!/ivysettings.xml
> will be used
> FAILED: Hive Internal Error:
> org.apache.hadoop.service.ServiceStateException(java.io.IOException:
> /ats/active does not exist)
> org.apache.hadoop.service.ServiceStateException: java.io.IOException:
> /ats/active does not exist
> at
> org.apache.hadoop.service.ServiceStateException.convert(ServiceStateException.java:59)
> at
> org.apache.hadoop.service.AbstractService.start(AbstractService.java:204)
> at org.apache.hadoop.hive.ql.hooks.ATSHook.<init>(ATSHook.java:74)
> at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
> at
> sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:57)
> at
> sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
> at java.lang.reflect.Constructor.newInstance(Constructor.java:526)
> at java.lang.Class.newInstance(Class.java:374)
> at org.apache.hadoop.hive.ql.hooks.HookUtils.getHooks(HookUtils.java:60)
> at org.apache.hadoop.hive.ql.Driver.getHooks(Driver.java:1351)
> at org.apache.hadoop.hive.ql.Driver.getHooks(Driver.java:1335)
> at org.apache.hadoop.hive.ql.Driver.execute(Driver.java:1395)
> at org.apache.hadoop.hive.ql.Driver.runInternal(Driver.java:1237)
> at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1101)
> at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1091)
> at org.apache.hive.hcatalog.cli.HCatDriver.run(HCatDriver.java:43)
> at org.apache.hive.hcatalog.cli.HCatCli.processCmd(HCatCli.java:291)
> at org.apache.hive.hcatalog.cli.HCatCli.processLine(HCatCli.java:245)
> at org.apache.hive.hcatalog.cli.HCatCli.main(HCatCli.java:183)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:606)
> at org.apache.hadoop.util.RunJar.run(RunJar.java:221)
> at org.apache.hadoop.util.RunJar.main(RunJar.java:136)
> Caused by: java.io.IOException: /ats/active does not exist
> at
> org.apache.hadoop.yarn.client.api.impl.TimelineClientImpl.serviceStart(TimelineClientImpl.java:371)
> at
> org.apache.hadoop.service.AbstractService.start(AbstractService.java:193)
> ... 23 more
> {code}
> In HDP 2.1, YARN ATS v1.0 did not support kerberos, so kerberizing the
> cluster removed that component. This means that EU from HDP 2.1 to 2.3 cannot
> persist the ATS v1.0 configs and instead adds the ATS v1.5 configs that
> default to stack HDP 2.3. For that reason, Hive Service Check fails.
> The fix is to persist the ATS v1.0 configs even if ATS is not present. The
> service checks will still fail, but the user can then add ATS after EU is
> done.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
