----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/41152/#review109621 -----------------------------------------------------------
Ship it! Ship It! - Nate Cole On Dec. 9, 2015, 5:16 p.m., Jonathan Hurley wrote: > > ----------------------------------------------------------- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/41152/ > ----------------------------------------------------------- > > (Updated Dec. 9, 2015, 5:16 p.m.) > > > Review request for Ambari, Alejandro Fernandez, Nate Cole, and Sumit Mohanty. > > > Bugs: AMBARI-14303 > https://issues.apache.org/jira/browse/AMBARI-14303 > > > Repository: ambari > > > Description > ------- > > *Steps* > Setup HDP 2.1 cluster with Ambari 2.2.0; Enable security on the cluster > Modify the relevant DB tables > Start Express Upgrade to 2.3.4 and let it complete > Observe the status of HIve service components > > *Result:* > All Hive servers report down > When I tried to start them they failed to come up (see attached hive log) > > This one is is a permission error in a kerberized cluster. > > Configs, > hive.server2.thrift.http.port : 10001 > hive.server2.thrift.port : 10000 > hive.cluster.delegation.token.store.class : > org.apache.hadoop.hive.thrift.ZooKeeperTokenStore > > That last config should instead be org.apache.hadoop.hive.thrift.DBTokenStore > in a kerberized cluster. > After EU/RU, the config datanucleus.rdbms.datastoreAdapterClassName did not > have a value. > > Nothing was using those ports, > [root@os-r6-oncjcs-baikaltom20sechanr-1 ~]# netstat -anp | grep 10000 > [root@os-r6-oncjcs-baikaltom20sechanr-1 ~]# netstat -anp | grep 10001 > > I then restarted HiveServer2, which reported success in Ambari, but the > process then died. > > Hive Server 2 log, > {code} > 2015-12-09 18:41:21,321 FATAL [Thread-10]: thrift.ThriftCLIService > (ThriftBinaryCLIService.java:run(101)) - Error starting HiveServer2: could > not start ThriftBinaryCLIService > org.apache.hadoop.hive.thrift.DelegationTokenStore$TokenStoreException: Error > creating path /hive/cluster/delegationHIVESERVER2/keys > at > org.apache.hadoop.hive.thrift.ZooKeeperTokenStore.ensurePath(ZooKeeperTokenStore.java:166) > at > org.apache.hadoop.hive.thrift.ZooKeeperTokenStore.initClientAndPaths(ZooKeeperTokenStore.java:236) > at > org.apache.hadoop.hive.thrift.ZooKeeperTokenStore.init(ZooKeeperTokenStore.java:469) > at > org.apache.hadoop.hive.thrift.HadoopThriftAuthBridge$Server.startDelegationTokenSecretManager(HadoopThriftAuthBridge.java:444) > at > org.apache.hive.service.auth.HiveAuthFactory.<init>(HiveAuthFactory.java:124) > at > org.apache.hive.service.cli.thrift.ThriftBinaryCLIService.run(ThriftBinaryCLIService.java:57) > at java.lang.Thread.run(Thread.java:745) > Caused by: org.apache.zookeeper.KeeperException$AuthFailedException: > KeeperErrorCode = AuthFailed for /hive/cluster/delegationHIVESERVER2/keys > at > org.apache.zookeeper.KeeperException.create(KeeperException.java:123) > at > org.apache.zookeeper.KeeperException.create(KeeperException.java:51) > at org.apache.zookeeper.ZooKeeper.create(ZooKeeper.java:783) > at > org.apache.curator.framework.imps.CreateBuilderImpl$11.call(CreateBuilderImpl.java:691) > at > org.apache.curator.framework.imps.CreateBuilderImpl$11.call(CreateBuilderImpl.java:675) > at org.apache.curator.RetryLoop.callWithRetry(RetryLoop.java:107) > at > org.apache.curator.framework.imps.CreateBuilderImpl.pathInForeground(CreateBuilderImpl.java:672) > at > org.apache.curator.framework.imps.CreateBuilderImpl.protectedPathInForeground(CreateBuilderImpl.java:453) > at > org.apache.curator.framework.imps.CreateBuilderImpl.forPath(CreateBuilderImpl.java:443) > at > org.apache.curator.framework.imps.CreateBuilderImpl.forPath(CreateBuilderImpl.java:423) > at > org.apache.curator.framework.imps.CreateBuilderImpl$3.forPath(CreateBuilderImpl.java:257) > at > org.apache.curator.framework.imps.CreateBuilderImpl$3.forPath(CreateBuilderImpl.java:205) > at > org.apache.hadoop.hive.thrift.ZooKeeperTokenStore.ensurePath(ZooKeeperTokenStore.java:160) > ... 6 more > 2015-12-09 18:41:21,323 FATAL [main]: server.HiveServer2 > (HiveServer2.java:addServerInstanceToZooKeeper(203)) - Unable to create > HiveServer2 namespace: hiveserver2 on ZooKeeper > org.apache.zookeeper.KeeperException$AuthFailedException: KeeperErrorCode = > AuthFailed for /hiveserver2 > at > org.apache.zookeeper.KeeperException.create(KeeperException.java:123) > at > org.apache.zookeeper.KeeperException.create(KeeperException.java:51) > at org.apache.zookeeper.ZooKeeper.create(ZooKeeper.java:783) > at > org.apache.curator.framework.imps.CreateBuilderImpl$11.call(CreateBuilderImpl.java:691) > at > org.apache.curator.framework.imps.CreateBuilderImpl$11.call(CreateBuilderImpl.java:675) > at org.apache.curator.RetryLoop.callWithRetry(RetryLoop.java:107) > at > org.apache.curator.framework.imps.CreateBuilderImpl.pathInForeground(CreateBuilderImpl.java:672) > at > org.apache.curator.framework.imps.CreateBuilderImpl.protectedPathInForeground(CreateBuilderImpl.java:453) > at > org.apache.curator.framework.imps.CreateBuilderImpl.forPath(CreateBuilderImpl.java:443) > at > org.apache.curator.framework.imps.CreateBuilderImpl.forPath(CreateBuilderImpl.java:423) > at > org.apache.curator.framework.imps.CreateBuilderImpl.forPath(CreateBuilderImpl.java:44) > at > org.apache.hive.service.server.HiveServer2.addServerInstanceToZooKeeper(HiveServer2.java:198) > at > org.apache.hive.service.server.HiveServer2.startHiveServer2(HiveServer2.java:417) > at > org.apache.hive.service.server.HiveServer2.access$700(HiveServer2.java:78) > at > org.apache.hive.service.server.HiveServer2$StartOptionExecutor.execute(HiveServer2.java:654) > at > org.apache.hive.service.server.HiveServer2.main(HiveServer2.java:527) > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) > at > sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) > at > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) > at java.lang.reflect.Method.invoke(Method.java:606) > at org.apache.hadoop.util.RunJar.run(RunJar.java:221) > at org.apache.hadoop.util.RunJar.main(RunJar.java:136) > {code} > > > Diffs > ----- > > > ambari-server/src/main/resources/stacks/HDP/2.1/upgrades/nonrolling-upgrade-2.3.xml > 2341dcc > ambari-server/src/main/resources/stacks/HDP/2.2/upgrades/config-upgrade.xml > 8c3e7da > > Diff: https://reviews.apache.org/r/41152/diff/ > > > Testing > ------- > > Upgraded HDP 2.1 with Hive and verified that the new value is correctly set. > > > Thanks, > > Jonathan Hurley > >
