Hi Joe,
I am able to resolve this issue by changing hbase-client version to
1.1.8-mapr-1703 in the nifi-hbase_1_1_2-client-service/pom.xml.
<dependency>
<groupId>org.apache.hbase</groupId>
<artifactId>hbase-client</artifactId>
<version>1.1.8-mapr-1703</version>
<exclusions>
<exclusion>
<groupId>org.slf4j</groupId>
<artifactId>slf4j-log4j12</artifactId>
</exclusion>
<exclusion>
<groupId>com.google.code.findbugs</groupId>
<artifactId>jsr305</artifactId>
</exclusion>
</exclusions>
</dependency>
Additionally added dependency:
<dependency>
<groupId>com.mapr.fs</groupId>
<artifactId>mapr-hbase</artifactId>
<version>6.0.1-mapr</version>
</dependency>
Now below line of code in
nifi-hbase_1_1_2-client-service/src/main/java/org/apache/nifi/hbase/HBase_1_1_2_ClientService.java
is returning null pointer exception:
masterAddress = admin.getClusterStatus().getMaster().getHostAndPort();
I see same code in nifi-1.6.0, but this worked fine with writing/reading data
from maprdb. Appreciate any insights why these issues are with nifi-1.7.1 code
base.
Thanks,
Ravi Papisetti
On 25/11/18, 7:45 PM, "Ravi Papisetti (rpapiset)" <[email protected]> wrote:
Yes, if I change the version on hbase_client_service_1_1_2 to
nifi-hbase_1_1_2-client-service-nar-1.6.0-mapr.nar (nifi 1.6 code compiled with
mapr profile) it just connects fine.
If I change client service to
nifi-hbase_1_1_2-client-service-nar-1.7.1-mapr.nar (nifi 1.7.1 compiled with
mapr profile), this error is coming.
See if below strack trace is sent with proper format:
Sun Nov 25 22:36:44 UTC 2018,
RpcRetryingCaller{globalStartTime=1543185404532, pause=100, retries=1},
org.apache.hadoop.hbase.MasterNotRunningException:
org.apache.hadoop.hbase.MasterNotRunningException: Can't get connection to
ZooKeeper: KeeperErrorCode = AuthFailed for /hbase
: {}
org.apache.hadoop.hbase.client.RetriesExhaustedException: Failed after
attempts=1, exceptions:
Sun Nov 25 22:36:44 UTC 2018,
RpcRetryingCaller{globalStartTime=1543185404532, pause=100, retries=1},
org.apache.hadoop.hbase.MasterNotRunningException:
org.apache.hadoop.hbase.MasterNotRunningException: Can't get connection to
ZooKeeper: KeeperErrorCode = AuthFailed for /hbase
at
org.apache.hadoop.hbase.client.RpcRetryingCaller.callWithRetries(RpcRetryingCaller.java:147)
at
org.apache.hadoop.hbase.client.HBaseAdmin.executeCallable(HBaseAdmin.java:3917)
at
org.apache.hadoop.hbase.client.HBaseAdmin.listTableNames(HBaseAdmin.java:413)
at
org.apache.hadoop.hbase.client.HBaseAdmin.listTableNames(HBaseAdmin.java:397)
at
org.apache.nifi.hbase.HBase_1_1_2_ClientService.onEnabled(HBase_1_1_2_ClientService.java:264)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at
org.apache.nifi.util.ReflectionUtils.invokeMethodsWithAnnotations(ReflectionUtils.java:142)
at
org.apache.nifi.util.ReflectionUtils.invokeMethodsWithAnnotations(ReflectionUtils.java:130)
at
org.apache.nifi.util.ReflectionUtils.invokeMethodsWithAnnotations(ReflectionUtils.java:75)
at
org.apache.nifi.util.ReflectionUtils.invokeMethodsWithAnnotation(ReflectionUtils.java:52)
at
org.apache.nifi.controller.service.StandardControllerServiceNode$2.run(StandardControllerServiceNode.java:433)
at
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:180)
at
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
Caused by: org.apache.hadoop.hbase.MasterNotRunningException:
org.apache.hadoop.hbase.MasterNotRunningException: Can't get connection to
ZooKeeper: KeeperErrorCode = AuthFailed for /hbase
at
org.apache.hadoop.hbase.client.ConnectionManager$HConnectionImplementation$StubMaker.makeStub(ConnectionManager.java:1533)
at
org.apache.hadoop.hbase.client.ConnectionManager$HConnectionImplementation$MasterServiceStubMaker.makeStub(ConnectionManager.java:1553)
at
org.apache.hadoop.hbase.client.ConnectionManager$HConnectionImplementation.getKeepAliveMasterService(ConnectionManager.java:1704)
at
org.apache.hadoop.hbase.client.MasterCallable.prepare(MasterCallable.java:38)
at
org.apache.hadoop.hbase.client.RpcRetryingCaller.callWithRetries(RpcRetryingCaller.java:124)
... 20 common frames omitted
Caused by: org.apache.hadoop.hbase.MasterNotRunningException: Can't get
connection to ZooKeeper: KeeperErrorCode = AuthFailed for /hbase
at
org.apache.hadoop.hbase.client.ConnectionManager$HConnectionImplementation.checkIfBaseNodeAvailable(ConnectionManager.java:906)
at
org.apache.hadoop.hbase.client.ConnectionManager$HConnectionImplementation.access$400(ConnectionManager.java:545)
at
org.apache.hadoop.hbase.client.ConnectionManager$HConnectionImplementation$StubMaker.makeStubNoRetries(ConnectionManager.java:1483)
at
org.apache.hadoop.hbase.client.ConnectionManager$HConnectionImplementation$StubMaker.makeStub(ConnectionManager.java:1524)
... 24 common frames omitted
Caused by: org.apache.zookeeper.KeeperException$AuthFailedException:
KeeperErrorCode = AuthFailed for /hbase
at
org.apache.zookeeper.KeeperException.create(KeeperException.java:123)
at
org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
at org.apache.zookeeper.ZooKeeper.exists(ZooKeeper.java:1045)
at
org.apache.hadoop.hbase.zookeeper.RecoverableZooKeeper.exists(RecoverableZooKeeper.java:221)
at
org.apache.hadoop.hbase.zookeeper.ZKUtil.checkExists(ZKUtil.java:541)
at
org.apache.hadoop.hbase.client.ConnectionManager$HConnectionImplementation.checkIfBaseNodeAvailable(ConnectionManager.java:895)
... 27 common frames omitted
2018-11-25 22:36:44,535 ERROR [Timer-Driven Process Thread-11]
o.a.n.c.s.StandardControllerServiceNode Failed to invoke @OnEnabled method of
HBase_1_1_2_ClientService[id=549bd5c3-0166-1000-0000-00000546f21d] due to
org.apache.hadoop.hbase.client.RetriesExhaustedException: Failed after
attempts=1, exceptions:
Thanks,
Ravi Papisetti
On 25/11/18, 4:47 PM, "Joe Witt" <[email protected]> wrote:
Ravi
it is very tough to read that stack trace as provided but the core
piece of it seems to be that authorization is failing for "/hbase".
Have you confirmed that the proper authorization configuration is
present?
Thanks
On Sun, Nov 25, 2018 at 4:50 PM Ravi Papisetti (rpapiset)
<[email protected]> wrote:
>
> Hi,
>
>
>
> Until recently we were using nifi-1.6
nifi-hbase_1_1_2-client-service-nar compiled with mapr profile. It has been
working fine connecting to mapr database without any issues.
>
>
>
> Recently we are upgrading to nifi-1.7, hbase client service is
failing to connect with same mapr distribution.
>
>
>
> Appreciate any insights to understand what changed in nifi 1.7 that
is failing to connect with maprdb.
>
>
>
> Below is the error message, appreciate any help:
>
> Wed Nov 21 06:09:13 UTC 2018,
RpcRetryingCaller{globalStartTime=1542780553492, pause=100, retries=3},
org.apache.hadoop.hbase.MasterNotRunningException:
org.apache.hadoop.hbase.MasterNotRunningException: Can't get connection to
ZooKeeper: KeeperErrorCode = AuthFailed for /hbase Wed Nov 21 06:09:13 UTC
2018, RpcRetryingCaller{globalStartTime=1542780553492, pause=100, retries=3},
org.apache.hadoop.hbase.MasterNotRunningException:
org.apache.hadoop.hbase.MasterNotRunningException: Can't get connection to
ZooKeeper: KeeperErrorCode = AuthFailed for /hbase at
org.apache.hadoop.hbase.client.RpcRetryingCaller.callWithRetries(RpcRetryingCaller.java:147)
at
org.apache.hadoop.hbase.client.HBaseAdmin.executeCallable(HBaseAdmin.java:3917)
at
org.apache.hadoop.hbase.client.HBaseAdmin.listTableNames(HBaseAdmin.java:413)
at
org.apache.hadoop.hbase.client.HBaseAdmin.listTableNames(HBaseAdmin.java:397)
at
org.apache.nifi.hbase.HBase_1_1_2_ClientService.onEnabled(HBase_1_1_2_ClientService.java:264)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498) at
org.apache.nifi.util.ReflectionUtils.invokeMethodsWithAnnotations(ReflectionUtils.java:142)
at
org.apache.nifi.util.ReflectionUtils.invokeMethodsWithAnnotations(ReflectionUtils.java:130)
at
org.apache.nifi.util.ReflectionUtils.invokeMethodsWithAnnotations(ReflectionUtils.java:75)
at
org.apache.nifi.util.ReflectionUtils.invokeMethodsWithAnnotation(ReflectionUtils.java:52)
at
org.apache.nifi.controller.service.StandardControllerServiceNode$2.run(StandardControllerServiceNode.java:433)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) at
java.util.concurrent.FutureTask.run(FutureTask.java:266) at
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:180)
at
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748) Caused by:
org.apache.hadoop.hbase.MasterNotRunningException:
org.apache.hadoop.hbase.MasterNotRunningException: Can't get connection to
ZooKeeper: KeeperErrorCode = AuthFailed for /hbase at
org.apache.hadoop.hbase.client.ConnectionManager$HConnectionImplementation$StubMaker.makeStub(ConnectionManager.java:1533)
at
org.apache.hadoop.hbase.client.ConnectionManager$HConnectionImplementation$MasterServiceStubMaker.makeStub(ConnectionManager.java:1553)
at
org.apache.hadoop.hbase.client.ConnectionManager$HConnectionImplementation.getKeepAliveMasterService(ConnectionManager.java:1704)
at
org.apache.hadoop.hbase.client.MasterCallable.prepare(MasterCallable.java:38)
at
org.apache.hadoop.hbase.client.RpcRetryingCaller.callWithRetries(RpcRetryingCaller.java:124)
... 20 common frames omitted Caused by:
org.apache.hadoop.hbase.MasterNotRunningException: Can't get connection to
ZooKeeper: KeeperErrorCode = AuthFailed for /hbase at
org.apache.hadoop.hbase.client.ConnectionManager$HConnectionImplementation.checkIfBaseNodeAvailable(ConnectionManager.java:906)
at
org.apache.hadoop.hbase.client.ConnectionManager$HConnectionImplementation.access$400(ConnectionManager.java:545)
at
org.apache.hadoop.hbase.client.ConnectionManager$HConnectionImplementation$StubMaker.makeStubNoRetries(ConnectionManager.java:1483)
at
org.apache.hadoop.hbase.client.ConnectionManager$HConnectionImplementation$StubMaker.makeStub(ConnectionManager.java:1524)
... 24 common frames omitted Caused by:
org.apache.zookeeper.KeeperException$AuthFailedException: KeeperErrorCode =
AuthFailed for /hbase at
org.apache.zookeeper.KeeperException.create(KeeperException.java:123) at
org.apache.zookeeper.KeeperException.create(KeeperException.java:51) at
org.apache.zookeeper.ZooKeeper.exists(ZooKeeper.java:1045) at
org.apache.hadoop.hbase.zookeeper.RecoverableZooKeeper.exists(RecoverableZooKeeper.java:221)
at org.apache.hadoop.hbase.zookeeper.ZKUtil.checkExists(ZKUtil.java:541) at
org.apache.hadoop.hbase.client.ConnectionManager$HConnectionImplementation.checkIfBaseNodeAvailable(ConnectionManager.java:895)
... 27 common frames omitted
>
>
>
>
>
> Thanks,
>
> Ravi Papisetti
>
>
>
>
