Re: Failed to receive data from Hbase due to java net connection exception :Connection Refused in Nifi
Hi Bryan, I have been working with poc in standalone NiFi. And zookeeper able to communicate with Hbase. Still i got the same issue " failed to receive data from Hbase due to java net connection exception :Connection Refused ". On Fri, May 6, 2016 at 8:47 PM, Bryan Bende wrote: > Ok so it is the connection to ZooKeeper that is a problem... > > Are you running standalone NiFi, or a cluster? and if clustered, did you > configure state management to use ZooKeeper? > (this would have been done by modifying the conf/state-management.xml) > > I'm just trying to narrow down to make sure the issue the HBase client > talking to ZooKeeper, and not NiFi's internal state management talking to > ZooKeeper. > > On Fri, May 6, 2016 at 11:03 AM, Venkatesh Bodapati < > venkatesh.bodap...@inndata.in> wrote: > >> 2016-05-06 20:01:33,201 INFO [pool-18-thread-1] >> o.a.n.c.r.WriteAheadFlowFileRepository Initiating checkpoint of FlowFile >> Repository >> 2016-05-06 20:01:33,473 INFO [pool-18-thread-1] >> org.wali.MinimalLockingWriteAheadLog >> org.wali.MinimalLockingWriteAheadLog@26650de4 checkpointed with 0 >> Records and 0 Swap Files in 134 milliseconds (Stop-the-world time = 26 >> milliseconds, Clear Edit Logs time = 37 millis), max Transaction ID 268027 >> 2016-05-06 20:01:33,473 INFO [pool-18-thread-1] >> o.a.n.c.r.WriteAheadFlowFileRepository Successfully checkpointed FlowFile >> Repository with 0 records in 135 milliseconds >> 2016-05-06 20:02:30,775 INFO [Flow Service Tasks Thread-1] >> o.a.nifi.controller.StandardFlowService Saved flow controller >> org.apache.nifi.controller.FlowController@2c4e3ffb // Another save >> pending = false >> 2016-05-06 20:02:30,958 INFO [pool-26-thread-18] >> o.a.h.h.zookeeper.RecoverableZooKeeper Process >> identifier=hconnection-0xaa14407 connecting to ZooKeeper >> ensemble=localhost:2181 >> 2016-05-06 20:02:30,958 INFO [pool-26-thread-18] >> org.apache.zookeeper.ZooKeeper Initiating client connection, >> connectString=localhost:2181 sessionTimeout=9 >> watcher=hconnection-0xaa144070x0, quorum=localhost:2181, baseZNode=/hbase >> 2016-05-06 20:02:30,975 INFO >> [pool-26-thread-18-SendThread(localhost:2181)] >> org.apache.zookeeper.ClientCnxn Opening socket connection to server >> localhost/127.0.0.1:2181. Will not attempt to authenticate using SASL >> (unknown error) >> 2016-05-06 20:02:30,977 INFO >> [pool-26-thread-18-SendThread(localhost:2181)] >> org.apache.zookeeper.ClientCnxn Socket connection established to localhost/ >> 127.0.0.1:2181, initiating session >> 2016-05-06 20:02:31,081 INFO >> [pool-26-thread-18-SendThread(localhost:2181)] >> org.apache.zookeeper.ClientCnxn Session establishment complete on server >> localhost/127.0.0.1:2181, sessionid = 0x154865d23ca000c, negotiated >> timeout = 6 >> 2016-05-06 20:02:32,657 INFO [pool-26-thread-19] >> o.a.n.c.s.TimerDrivenSchedulingAgent Scheduled >> GetHBase[id=89f3c893-f14f-4520-8d56-b27a431a7d41] to run with 1 threads >> 2016-05-06 20:02:32,658 INFO [pool-26-thread-17] >> o.a.n.c.s.TimerDrivenSchedulingAgent Scheduled >> PutHBaseCell[id=04a3e270-375c-4f2d-9e93-ac53a70765d8] to run with 1 threads >> 2016-05-06 20:02:32,659 INFO [pool-26-thread-20] >> o.a.n.c.s.TimerDrivenSchedulingAgent Scheduled >> PutHBaseJSON[id=dc18a7a7-dd2f-484c-aa27-4a6f0de65920] to run with 1 threads >> 2016-05-06 20:02:32,676 ERROR [Timer-Driven Process Thread-6] >> org.apache.nifi.hbase.GetHBase >> GetHBase[id=89f3c893-f14f-4520-8d56-b27a431a7d41] Failed to receive data >> from HBase due to java.net.ConnectException: Connection refused >> 2016-05-06 20:02:33,678 ERROR [Timer-Driven Process Thread-4] >> org.apache.nifi.hbase.GetHBase >> GetHBase[id=89f3c893-f14f-4520-8d56-b27a431a7d41] Failed to receive data >> from HBase due to java.net.ConnectException: Connection refused >> 2016-05-06 20:02:34,679 ERROR [Timer-Driven Process Thread-10] >> org.apache.nifi.hbase.GetHBase >> GetHBase[id=89f3c893-f14f-4520-8d56-b27a431a7d41] Failed to receive data >> from HBase due to java.net.ConnectException: Connection refused >> 2016-05-06 20:02:35,680 ERROR [Timer-Driven Process Thread-2] >> org.apache.nifi.hbase.GetHBase >> GetHBase[id=89f3c893-f14f-4520-8d56-b27a431a7d41] Failed to receive data >> from HBase due to java.net.ConnectException: Connection refused >> 2016-05-06 20:02:36,681 ERROR [Timer-Driven Process Thread-10] >> org.apache.nifi.hbase.GetHBase >> GetHBase[id=89f3c893-f14f-4520-8d56-b27a431a7d41] Failed to receive data >> from HBase due to java.net.ConnectException: Connection refused >> 2016-05-06 20:02:37,683 ERROR [Timer-Driven Process Thread-10] >> org.apache.nifi.hbase.GetHBase >> GetHBase[id=89f3c893-f14f-4520-8d56-b27a431a7d41] Failed to receive data >> from HBase due to java.net.ConnectException: Connection refused >> 2016-05-06 20:02:38,684 ERROR [Timer-Driven Process Thread-3] >> org.apache.nifi.hbase.GetHBase >> GetHBase[id=89f3c893-f14f-4520-8d56-b27a431a7d41] Failed to receive data >> from HBase due to java.net.ConnectException: Co
Re: Failed to receive data from Hbase due to java net connection exception :Connection Refused in Nifi
Ok so it is the connection to ZooKeeper that is a problem... Are you running standalone NiFi, or a cluster? and if clustered, did you configure state management to use ZooKeeper? (this would have been done by modifying the conf/state-management.xml) I'm just trying to narrow down to make sure the issue the HBase client talking to ZooKeeper, and not NiFi's internal state management talking to ZooKeeper. On Fri, May 6, 2016 at 11:03 AM, Venkatesh Bodapati < venkatesh.bodap...@inndata.in> wrote: > 2016-05-06 20:01:33,201 INFO [pool-18-thread-1] > o.a.n.c.r.WriteAheadFlowFileRepository Initiating checkpoint of FlowFile > Repository > 2016-05-06 20:01:33,473 INFO [pool-18-thread-1] > org.wali.MinimalLockingWriteAheadLog > org.wali.MinimalLockingWriteAheadLog@26650de4 checkpointed with 0 Records > and 0 Swap Files in 134 milliseconds (Stop-the-world time = 26 > milliseconds, Clear Edit Logs time = 37 millis), max Transaction ID 268027 > 2016-05-06 20:01:33,473 INFO [pool-18-thread-1] > o.a.n.c.r.WriteAheadFlowFileRepository Successfully checkpointed FlowFile > Repository with 0 records in 135 milliseconds > 2016-05-06 20:02:30,775 INFO [Flow Service Tasks Thread-1] > o.a.nifi.controller.StandardFlowService Saved flow controller > org.apache.nifi.controller.FlowController@2c4e3ffb // Another save > pending = false > 2016-05-06 20:02:30,958 INFO [pool-26-thread-18] > o.a.h.h.zookeeper.RecoverableZooKeeper Process > identifier=hconnection-0xaa14407 connecting to ZooKeeper > ensemble=localhost:2181 > 2016-05-06 20:02:30,958 INFO [pool-26-thread-18] > org.apache.zookeeper.ZooKeeper Initiating client connection, > connectString=localhost:2181 sessionTimeout=9 > watcher=hconnection-0xaa144070x0, quorum=localhost:2181, baseZNode=/hbase > 2016-05-06 20:02:30,975 INFO > [pool-26-thread-18-SendThread(localhost:2181)] > org.apache.zookeeper.ClientCnxn Opening socket connection to server > localhost/127.0.0.1:2181. Will not attempt to authenticate using SASL > (unknown error) > 2016-05-06 20:02:30,977 INFO > [pool-26-thread-18-SendThread(localhost:2181)] > org.apache.zookeeper.ClientCnxn Socket connection established to localhost/ > 127.0.0.1:2181, initiating session > 2016-05-06 20:02:31,081 INFO > [pool-26-thread-18-SendThread(localhost:2181)] > org.apache.zookeeper.ClientCnxn Session establishment complete on server > localhost/127.0.0.1:2181, sessionid = 0x154865d23ca000c, negotiated > timeout = 6 > 2016-05-06 20:02:32,657 INFO [pool-26-thread-19] > o.a.n.c.s.TimerDrivenSchedulingAgent Scheduled > GetHBase[id=89f3c893-f14f-4520-8d56-b27a431a7d41] to run with 1 threads > 2016-05-06 20:02:32,658 INFO [pool-26-thread-17] > o.a.n.c.s.TimerDrivenSchedulingAgent Scheduled > PutHBaseCell[id=04a3e270-375c-4f2d-9e93-ac53a70765d8] to run with 1 threads > 2016-05-06 20:02:32,659 INFO [pool-26-thread-20] > o.a.n.c.s.TimerDrivenSchedulingAgent Scheduled > PutHBaseJSON[id=dc18a7a7-dd2f-484c-aa27-4a6f0de65920] to run with 1 threads > 2016-05-06 20:02:32,676 ERROR [Timer-Driven Process Thread-6] > org.apache.nifi.hbase.GetHBase > GetHBase[id=89f3c893-f14f-4520-8d56-b27a431a7d41] Failed to receive data > from HBase due to java.net.ConnectException: Connection refused > 2016-05-06 20:02:33,678 ERROR [Timer-Driven Process Thread-4] > org.apache.nifi.hbase.GetHBase > GetHBase[id=89f3c893-f14f-4520-8d56-b27a431a7d41] Failed to receive data > from HBase due to java.net.ConnectException: Connection refused > 2016-05-06 20:02:34,679 ERROR [Timer-Driven Process Thread-10] > org.apache.nifi.hbase.GetHBase > GetHBase[id=89f3c893-f14f-4520-8d56-b27a431a7d41] Failed to receive data > from HBase due to java.net.ConnectException: Connection refused > 2016-05-06 20:02:35,680 ERROR [Timer-Driven Process Thread-2] > org.apache.nifi.hbase.GetHBase > GetHBase[id=89f3c893-f14f-4520-8d56-b27a431a7d41] Failed to receive data > from HBase due to java.net.ConnectException: Connection refused > 2016-05-06 20:02:36,681 ERROR [Timer-Driven Process Thread-10] > org.apache.nifi.hbase.GetHBase > GetHBase[id=89f3c893-f14f-4520-8d56-b27a431a7d41] Failed to receive data > from HBase due to java.net.ConnectException: Connection refused > 2016-05-06 20:02:37,683 ERROR [Timer-Driven Process Thread-10] > org.apache.nifi.hbase.GetHBase > GetHBase[id=89f3c893-f14f-4520-8d56-b27a431a7d41] Failed to receive data > from HBase due to java.net.ConnectException: Connection refused > 2016-05-06 20:02:38,684 ERROR [Timer-Driven Process Thread-3] > org.apache.nifi.hbase.GetHBase > GetHBase[id=89f3c893-f14f-4520-8d56-b27a431a7d41] Failed to receive data > from HBase due to java.net.ConnectException: Connection refused > 2016-05-06 20:02:39,686 ERROR [Timer-Driven Process Thread-4] > org.apache.nifi.hbase.GetHBase > GetHBase[id=89f3c893-f14f-4520-8d56-b27a431a7d41] Failed to receive data > from HBase due to java.net.ConnectException: Connection refused > 2016-05-06 20:02:40,690 ERROR [Timer-Driven Process Thread-7] > org.apache.nifi.hbase.GetHBase > GetHBase[id=89f3c893-f14f
Re: Failed to receive data from Hbase due to java net connection exception :Connection Refused in Nifi
2016-05-06 20:01:33,201 INFO [pool-18-thread-1] o.a.n.c.r.WriteAheadFlowFileRepository Initiating checkpoint of FlowFile Repository 2016-05-06 20:01:33,473 INFO [pool-18-thread-1] org.wali.MinimalLockingWriteAheadLog org.wali.MinimalLockingWriteAheadLog@26650de4 checkpointed with 0 Records and 0 Swap Files in 134 milliseconds (Stop-the-world time = 26 milliseconds, Clear Edit Logs time = 37 millis), max Transaction ID 268027 2016-05-06 20:01:33,473 INFO [pool-18-thread-1] o.a.n.c.r.WriteAheadFlowFileRepository Successfully checkpointed FlowFile Repository with 0 records in 135 milliseconds 2016-05-06 20:02:30,775 INFO [Flow Service Tasks Thread-1] o.a.nifi.controller.StandardFlowService Saved flow controller org.apache.nifi.controller.FlowController@2c4e3ffb // Another save pending = false 2016-05-06 20:02:30,958 INFO [pool-26-thread-18] o.a.h.h.zookeeper.RecoverableZooKeeper Process identifier=hconnection-0xaa14407 connecting to ZooKeeper ensemble=localhost:2181 2016-05-06 20:02:30,958 INFO [pool-26-thread-18] org.apache.zookeeper.ZooKeeper Initiating client connection, connectString=localhost:2181 sessionTimeout=9 watcher=hconnection-0xaa144070x0, quorum=localhost:2181, baseZNode=/hbase 2016-05-06 20:02:30,975 INFO [pool-26-thread-18-SendThread(localhost:2181)] org.apache.zookeeper.ClientCnxn Opening socket connection to server localhost/127.0.0.1:2181. Will not attempt to authenticate using SASL (unknown error) 2016-05-06 20:02:30,977 INFO [pool-26-thread-18-SendThread(localhost:2181)] org.apache.zookeeper.ClientCnxn Socket connection established to localhost/ 127.0.0.1:2181, initiating session 2016-05-06 20:02:31,081 INFO [pool-26-thread-18-SendThread(localhost:2181)] org.apache.zookeeper.ClientCnxn Session establishment complete on server localhost/127.0.0.1:2181, sessionid = 0x154865d23ca000c, negotiated timeout = 6 2016-05-06 20:02:32,657 INFO [pool-26-thread-19] o.a.n.c.s.TimerDrivenSchedulingAgent Scheduled GetHBase[id=89f3c893-f14f-4520-8d56-b27a431a7d41] to run with 1 threads 2016-05-06 20:02:32,658 INFO [pool-26-thread-17] o.a.n.c.s.TimerDrivenSchedulingAgent Scheduled PutHBaseCell[id=04a3e270-375c-4f2d-9e93-ac53a70765d8] to run with 1 threads 2016-05-06 20:02:32,659 INFO [pool-26-thread-20] o.a.n.c.s.TimerDrivenSchedulingAgent Scheduled PutHBaseJSON[id=dc18a7a7-dd2f-484c-aa27-4a6f0de65920] to run with 1 threads 2016-05-06 20:02:32,676 ERROR [Timer-Driven Process Thread-6] org.apache.nifi.hbase.GetHBase GetHBase[id=89f3c893-f14f-4520-8d56-b27a431a7d41] Failed to receive data from HBase due to java.net.ConnectException: Connection refused 2016-05-06 20:02:33,678 ERROR [Timer-Driven Process Thread-4] org.apache.nifi.hbase.GetHBase GetHBase[id=89f3c893-f14f-4520-8d56-b27a431a7d41] Failed to receive data from HBase due to java.net.ConnectException: Connection refused 2016-05-06 20:02:34,679 ERROR [Timer-Driven Process Thread-10] org.apache.nifi.hbase.GetHBase GetHBase[id=89f3c893-f14f-4520-8d56-b27a431a7d41] Failed to receive data from HBase due to java.net.ConnectException: Connection refused 2016-05-06 20:02:35,680 ERROR [Timer-Driven Process Thread-2] org.apache.nifi.hbase.GetHBase GetHBase[id=89f3c893-f14f-4520-8d56-b27a431a7d41] Failed to receive data from HBase due to java.net.ConnectException: Connection refused 2016-05-06 20:02:36,681 ERROR [Timer-Driven Process Thread-10] org.apache.nifi.hbase.GetHBase GetHBase[id=89f3c893-f14f-4520-8d56-b27a431a7d41] Failed to receive data from HBase due to java.net.ConnectException: Connection refused 2016-05-06 20:02:37,683 ERROR [Timer-Driven Process Thread-10] org.apache.nifi.hbase.GetHBase GetHBase[id=89f3c893-f14f-4520-8d56-b27a431a7d41] Failed to receive data from HBase due to java.net.ConnectException: Connection refused 2016-05-06 20:02:38,684 ERROR [Timer-Driven Process Thread-3] org.apache.nifi.hbase.GetHBase GetHBase[id=89f3c893-f14f-4520-8d56-b27a431a7d41] Failed to receive data from HBase due to java.net.ConnectException: Connection refused 2016-05-06 20:02:39,686 ERROR [Timer-Driven Process Thread-4] org.apache.nifi.hbase.GetHBase GetHBase[id=89f3c893-f14f-4520-8d56-b27a431a7d41] Failed to receive data from HBase due to java.net.ConnectException: Connection refused 2016-05-06 20:02:40,690 ERROR [Timer-Driven Process Thread-7] org.apache.nifi.hbase.GetHBase GetHBase[id=89f3c893-f14f-4520-8d56-b27a431a7d41] Failed to receive data from HBase due to java.net.ConnectException: Connection refused 2016-05-06 20:02:41,691 ERROR [Timer-Driven Process Thread-5] org.apache.nifi.hbase.GetHBase GetHBase[id=89f3c893-f14f-4520-8d56-b27a431a7d41] Failed to receive data from HBase due to java.net.ConnectException: Connection refused 2016-05-06 20:02:42,693 ERROR [Timer-Driven Process Thread-9] org.apache.nifi.hbase.GetHBase GetHBase[id=89f3c893-f14f-4520-8d56-b27a431a7d41] Failed to receive data from HBase due to java.net.ConnectException: Connection refused 2016-05-06 20:02:42,703 INFO [pool-26-thread-21] o.a.n.c.s.TimerDrivenSchedulingAgent Schedu
Re: Failed to receive data from Hbase due to java net connection exception :Connection Refused in Nifi
Looks like it is not logging the full stacktrace by default... Can you edit NIFI_HOME/conf/logback.xml and add a line like this: Wait about 15-30 seconds and then see what is in nifi-app.log On Fri, May 6, 2016 at 10:37 AM, Venkatesh Bodapati < venkatesh.bodap...@inndata.in> wrote: > This is whole stacktrace in nifi-app.log. > > 2016-05-06 20:01:33,201 INFO [pool-18-thread-1] > o.a.n.c.r.WriteAheadFlowFileRepository Initiating checkpoint of FlowFile > Repository > 2016-05-06 20:01:33,473 INFO [pool-18-thread-1] > org.wali.MinimalLockingWriteAheadLog > org.wali.MinimalLockingWriteAheadLog@26650de4 checkpointed with 0 Records > and 0 Swap Files in 134 milliseconds (Stop-the-world time = 26 > milliseconds, Clear Edit Logs time = 37 millis), max Transaction ID 268027 > 2016-05-06 20:01:33,473 INFO [pool-18-thread-1] > o.a.n.c.r.WriteAheadFlowFileRepository Successfully checkpointed FlowFile > Repository with 0 records in 135 milliseconds > 2016-05-06 20:02:30,775 INFO [Flow Service Tasks Thread-1] > o.a.nifi.controller.StandardFlowService Saved flow controller > org.apache.nifi.controller.FlowController@2c4e3ffb // Another save > pending = false > 2016-05-06 20:02:30,958 INFO [pool-26-thread-18] > o.a.h.h.zookeeper.RecoverableZooKeeper Process > identifier=hconnection-0xaa14407 connecting to ZooKeeper > ensemble=localhost:2181 > 2016-05-06 20:02:30,958 INFO [pool-26-thread-18] > org.apache.zookeeper.ZooKeeper Initiating client connection, > connectString=localhost:2181 sessionTimeout=9 > watcher=hconnection-0xaa144070x0, quorum=localhost:2181, baseZNode=/hbase > 2016-05-06 20:02:30,975 INFO > [pool-26-thread-18-SendThread(localhost:2181)] > org.apache.zookeeper.ClientCnxn Opening socket connection to server > localhost/127.0.0.1:2181. Will not attempt to authenticate using SASL > (unknown error) > 2016-05-06 20:02:30,977 INFO > [pool-26-thread-18-SendThread(localhost:2181)] > org.apache.zookeeper.ClientCnxn Socket connection established to localhost/ > 127.0.0.1:2181, initiating session > 2016-05-06 20:02:31,081 INFO > [pool-26-thread-18-SendThread(localhost:2181)] > org.apache.zookeeper.ClientCnxn Session establishment complete on server > localhost/127.0.0.1:2181, sessionid = 0x154865d23ca000c, negotiated > timeout = 6 > 2016-05-06 20:02:32,657 INFO [pool-26-thread-19] > o.a.n.c.s.TimerDrivenSchedulingAgent Scheduled > GetHBase[id=89f3c893-f14f-4520-8d56-b27a431a7d41] to run with 1 threads > 2016-05-06 20:02:32,658 INFO [pool-26-thread-17] > o.a.n.c.s.TimerDrivenSchedulingAgent Scheduled > PutHBaseCell[id=04a3e270-375c-4f2d-9e93-ac53a70765d8] to run with 1 threads > 2016-05-06 20:02:32,659 INFO [pool-26-thread-20] > o.a.n.c.s.TimerDrivenSchedulingAgent Scheduled > PutHBaseJSON[id=dc18a7a7-dd2f-484c-aa27-4a6f0de65920] to run with 1 threads > 2016-05-06 20:02:32,676 ERROR [Timer-Driven Process Thread-6] > org.apache.nifi.hbase.GetHBase > GetHBase[id=89f3c893-f14f-4520-8d56-b27a431a7d41] Failed to receive data > from HBase due to java.net.ConnectException: Connection refused > 2016-05-06 20:02:33,678 ERROR [Timer-Driven Process Thread-4] > org.apache.nifi.hbase.GetHBase > GetHBase[id=89f3c893-f14f-4520-8d56-b27a431a7d41] Failed to receive data > from HBase due to java.net.ConnectException: Connection refused > 2016-05-06 20:02:34,679 ERROR [Timer-Driven Process Thread-10] > org.apache.nifi.hbase.GetHBase > GetHBase[id=89f3c893-f14f-4520-8d56-b27a431a7d41] Failed to receive data > from HBase due to java.net.ConnectException: Connection refused > 2016-05-06 20:02:35,680 ERROR [Timer-Driven Process Thread-2] > org.apache.nifi.hbase.GetHBase > GetHBase[id=89f3c893-f14f-4520-8d56-b27a431a7d41] Failed to receive data > from HBase due to java.net.ConnectException: Connection refused > 2016-05-06 20:02:36,681 ERROR [Timer-Driven Process Thread-10] > org.apache.nifi.hbase.GetHBase > GetHBase[id=89f3c893-f14f-4520-8d56-b27a431a7d41] Failed to receive data > from HBase due to java.net.ConnectException: Connection refused > 2016-05-06 20:02:37,683 ERROR [Timer-Driven Process Thread-10] > org.apache.nifi.hbase.GetHBase > GetHBase[id=89f3c893-f14f-4520-8d56-b27a431a7d41] Failed to receive data > from HBase due to java.net.ConnectException: Connection refused > 2016-05-06 20:02:38,684 ERROR [Timer-Driven Process Thread-3] > org.apache.nifi.hbase.GetHBase > GetHBase[id=89f3c893-f14f-4520-8d56-b27a431a7d41] Failed to receive data > from HBase due to java.net.ConnectException: Connection refused > 2016-05-06 20:02:39,686 ERROR [Timer-Driven Process Thread-4] > org.apache.nifi.hbase.GetHBase > GetHBase[id=89f3c893-f14f-4520-8d56-b27a431a7d41] Failed to receive data > from HBase due to java.net.ConnectException: Connection refused > 2016-05-06 20:02:40,690 ERROR [Timer-Driven Process Thread-7] > org.apache.nifi.hbase.GetHBase > GetHBase[id=89f3c893-f14f-4520-8d56-b27a431a7d41] Failed to receive data > from HBase due to java.net.ConnectException: Connection refused > 2016-05-06 20:02:41,691 ERROR [Timer-Driven Proces
Re: Failed to receive data from Hbase due to java net connection exception :Connection Refused in Nifi
This is whole stacktrace in nifi-app.log. 2016-05-06 20:01:33,201 INFO [pool-18-thread-1] o.a.n.c.r.WriteAheadFlowFileRepository Initiating checkpoint of FlowFile Repository 2016-05-06 20:01:33,473 INFO [pool-18-thread-1] org.wali.MinimalLockingWriteAheadLog org.wali.MinimalLockingWriteAheadLog@26650de4 checkpointed with 0 Records and 0 Swap Files in 134 milliseconds (Stop-the-world time = 26 milliseconds, Clear Edit Logs time = 37 millis), max Transaction ID 268027 2016-05-06 20:01:33,473 INFO [pool-18-thread-1] o.a.n.c.r.WriteAheadFlowFileRepository Successfully checkpointed FlowFile Repository with 0 records in 135 milliseconds 2016-05-06 20:02:30,775 INFO [Flow Service Tasks Thread-1] o.a.nifi.controller.StandardFlowService Saved flow controller org.apache.nifi.controller.FlowController@2c4e3ffb // Another save pending = false 2016-05-06 20:02:30,958 INFO [pool-26-thread-18] o.a.h.h.zookeeper.RecoverableZooKeeper Process identifier=hconnection-0xaa14407 connecting to ZooKeeper ensemble=localhost:2181 2016-05-06 20:02:30,958 INFO [pool-26-thread-18] org.apache.zookeeper.ZooKeeper Initiating client connection, connectString=localhost:2181 sessionTimeout=9 watcher=hconnection-0xaa144070x0, quorum=localhost:2181, baseZNode=/hbase 2016-05-06 20:02:30,975 INFO [pool-26-thread-18-SendThread(localhost:2181)] org.apache.zookeeper.ClientCnxn Opening socket connection to server localhost/127.0.0.1:2181. Will not attempt to authenticate using SASL (unknown error) 2016-05-06 20:02:30,977 INFO [pool-26-thread-18-SendThread(localhost:2181)] org.apache.zookeeper.ClientCnxn Socket connection established to localhost/ 127.0.0.1:2181, initiating session 2016-05-06 20:02:31,081 INFO [pool-26-thread-18-SendThread(localhost:2181)] org.apache.zookeeper.ClientCnxn Session establishment complete on server localhost/127.0.0.1:2181, sessionid = 0x154865d23ca000c, negotiated timeout = 6 2016-05-06 20:02:32,657 INFO [pool-26-thread-19] o.a.n.c.s.TimerDrivenSchedulingAgent Scheduled GetHBase[id=89f3c893-f14f-4520-8d56-b27a431a7d41] to run with 1 threads 2016-05-06 20:02:32,658 INFO [pool-26-thread-17] o.a.n.c.s.TimerDrivenSchedulingAgent Scheduled PutHBaseCell[id=04a3e270-375c-4f2d-9e93-ac53a70765d8] to run with 1 threads 2016-05-06 20:02:32,659 INFO [pool-26-thread-20] o.a.n.c.s.TimerDrivenSchedulingAgent Scheduled PutHBaseJSON[id=dc18a7a7-dd2f-484c-aa27-4a6f0de65920] to run with 1 threads 2016-05-06 20:02:32,676 ERROR [Timer-Driven Process Thread-6] org.apache.nifi.hbase.GetHBase GetHBase[id=89f3c893-f14f-4520-8d56-b27a431a7d41] Failed to receive data from HBase due to java.net.ConnectException: Connection refused 2016-05-06 20:02:33,678 ERROR [Timer-Driven Process Thread-4] org.apache.nifi.hbase.GetHBase GetHBase[id=89f3c893-f14f-4520-8d56-b27a431a7d41] Failed to receive data from HBase due to java.net.ConnectException: Connection refused 2016-05-06 20:02:34,679 ERROR [Timer-Driven Process Thread-10] org.apache.nifi.hbase.GetHBase GetHBase[id=89f3c893-f14f-4520-8d56-b27a431a7d41] Failed to receive data from HBase due to java.net.ConnectException: Connection refused 2016-05-06 20:02:35,680 ERROR [Timer-Driven Process Thread-2] org.apache.nifi.hbase.GetHBase GetHBase[id=89f3c893-f14f-4520-8d56-b27a431a7d41] Failed to receive data from HBase due to java.net.ConnectException: Connection refused 2016-05-06 20:02:36,681 ERROR [Timer-Driven Process Thread-10] org.apache.nifi.hbase.GetHBase GetHBase[id=89f3c893-f14f-4520-8d56-b27a431a7d41] Failed to receive data from HBase due to java.net.ConnectException: Connection refused 2016-05-06 20:02:37,683 ERROR [Timer-Driven Process Thread-10] org.apache.nifi.hbase.GetHBase GetHBase[id=89f3c893-f14f-4520-8d56-b27a431a7d41] Failed to receive data from HBase due to java.net.ConnectException: Connection refused 2016-05-06 20:02:38,684 ERROR [Timer-Driven Process Thread-3] org.apache.nifi.hbase.GetHBase GetHBase[id=89f3c893-f14f-4520-8d56-b27a431a7d41] Failed to receive data from HBase due to java.net.ConnectException: Connection refused 2016-05-06 20:02:39,686 ERROR [Timer-Driven Process Thread-4] org.apache.nifi.hbase.GetHBase GetHBase[id=89f3c893-f14f-4520-8d56-b27a431a7d41] Failed to receive data from HBase due to java.net.ConnectException: Connection refused 2016-05-06 20:02:40,690 ERROR [Timer-Driven Process Thread-7] org.apache.nifi.hbase.GetHBase GetHBase[id=89f3c893-f14f-4520-8d56-b27a431a7d41] Failed to receive data from HBase due to java.net.ConnectException: Connection refused 2016-05-06 20:02:41,691 ERROR [Timer-Driven Process Thread-5] org.apache.nifi.hbase.GetHBase GetHBase[id=89f3c893-f14f-4520-8d56-b27a431a7d41] Failed to receive data from HBase due to java.net.ConnectException: Connection refused 2016-05-06 20:02:42,693 ERROR [Timer-Driven Process Thread-9] org.apache.nifi.hbase.GetHBase GetHBase[id=89f3c893-f14f-4520-8d56-b27a431a7d41] Failed to receive data from HBase due to java.net.ConnectException: Connection refused 2016-05-06 20:02:42,703 INFO [pool-26-thread-21]
Re: Failed to receive data from Hbase due to java net connection exception :Connection Refused in Nifi
Ok, thanks for sharing that...
Can you look in NIFI_HOME/logs/nifi-app.log and see if you can find that
error message that says "Failed to receive data from HBase due to",
and then there should be a whole stack trace that goes with that.
If you could paste that whole stacktrace here that would be helpful.
Thanks,
Bryan
On Fri, May 6, 2016 at 10:14 AM, Venkatesh Bodapati <
venkatesh.bodap...@inndata.in> wrote:
> I am running NiFi, Hbase and Zookeeper in same machine.
>
> This my Hbase-site.xml :
>
>
> hbase.master
> localhost:6
>
>
> hbase.rootdir
> hdfs://localhost:8020/hbase
>
>
> hbase.cluster.distributed
> true
>
>
> hbase.zookeeper.quorum
> localhost
>
>
> dfs.replication
> 1
>
>
> hbase.zookeeper.property.clientPort
> 2181
>
>
> hbase.zookeeper.property.dataDir
> /usr/local/hadoop/hbase-1.1.2/zookeeper
>
>
> zookeeper.znode.parent
> /hbase
>
>
>
> and my Hbase-client Service properties like this :
> Hadoop Configuration FilesInfo :
> /usr/local/hadoop/hadoop-2.6.0/etc/hadoop/core-site.xml,/usr/local/hadoop/hbase-1.1.2/conf/hbase-site.xml
> ZooKeeper QuorumInfo: No value set
> ZooKeeper Client PortInfo: No value set
> ZooKeeper ZNode ParentInfo : /hbase
> HBase Client RetriesInfo : 1
>
> On Fri, May 6, 2016 at 7:13 PM, Bryan Bende wrote:
>
>> Can you describe your setup a little bit more, things like...
>>
>> Is NiFi running on the same machine that HBase and ZooKeeper are running
>> on (it doesn't have to be, just want to understand the setup) ?
>>
>> What values do you have in hbase-site.xml for properties like...
>> - hbase.zookeeper.quorum
>> - hbase.zookeeper.property.clientPort
>>
>> Here is a template that is working for me:
>>
>> https://gist.githubusercontent.com/bbende/9457b5ed261e6eeb0f98995a5a2699e0/raw/38e050828c2545cb50e623d6dcf45dbbc9ad1d9d/FunWithHBaseUpdated.xml
>>
>> I am running NiFi on the HDP Sandbox so everything is local for me.
>>
>>
>> On Fri, May 6, 2016 at 8:54 AM, Venkatesh Bodapati <
>> venkatesh.bodap...@inndata.in> wrote:
>>
>>> " Missing Row id failure, routing to failure " issue solved . Still i
>>> got the "failed to receive data from Hbase due to java net connection
>>> exception :Connection Refused" error in GetHbase Processor. I update
>>> Hbase-Site.xml and remove remaining properties like " Zookeeper Quorum
>>> Info,Zookeeper Client Port Info . still i got the same error in GetHbase
>>> processor.
>>>
>>> On Thu, May 5, 2016 at 9:07 PM, Bryan Bende wrote:
>>>
I was looking at this a little more and instead of needing the
RouteOnAttribute, I think you can just update EvaluateJsonPath
properties...
Based on the JSON I see coming in, they should be:
email = $.email
firstName = $.name.first
lastName = $.name.last
ssn = $.id.value
On Thu, May 5, 2016 at 11:30 AM, Bryan Bende wrote:
> Hello,
>
> For HBaseClient, are you sure your ZooKeeper being used by HBase is
> running on localhost:2181?
>
> Typically you don't really need to set the three ZooKeeper properties,
> and you can instead just set the hbase-site.xml in the config resources.
>
> For example, my Hadoop Configuration Resources is set as:
> /etc/hadoop/conf/core-site.xml,/etc/hbase/conf/hbase-site.xml
>
> And then I don't have anything specified for ZooKeeper QuorumInfo,
> etc., because hbase-site.xml is providing all that information.
>
> The example template requires a Users table, so from Hbase shell:
> create 'Users', {NAME => 'cf'}
>
> Then the template has a slight problem where it is using
> EvaluteJsonPath to extract an attribute called "ssn" and using that for
> the
> row id... the problem is a lot of the ssn values are empty so they are
> failing.
>
> If you stick a RouteOnAttribute right after EvaluateJsonPath, and add
> a property "matched" = ${ssn:isEmpty():not()} that will filter out all the
> empty SSNs, then connect RouteOnAttribute's matched relationship to the
> HBase processors.
>
> Lets get the inserting side working and then I would expect the
> GetHBase to work since they use the same underlying connection.
>
> Thanks,
>
> Bryan
>
>
> On Thu, May 5, 2016 at 10:59 AM, Venkatesh Bodapati <
> venkatesh.bodap...@inndata.in> wrote:
>
>> I am working on "Fun_With_Hbase.xml" Template. In this i will send
>> data to hbase table, but i will not get data from hbase. I will get the
>> "failed to receive data from hbase due t
Re: Failed to receive data from Hbase due to java net connection exception :Connection Refused in Nifi
I am running NiFi, Hbase and Zookeeper in same machine.
This my Hbase-site.xml :
hbase.master
localhost:6
hbase.rootdir
hdfs://localhost:8020/hbase
hbase.cluster.distributed
true
hbase.zookeeper.quorum
localhost
dfs.replication
1
hbase.zookeeper.property.clientPort
2181
hbase.zookeeper.property.dataDir
/usr/local/hadoop/hbase-1.1.2/zookeeper
zookeeper.znode.parent
/hbase
and my Hbase-client Service properties like this :
Hadoop Configuration FilesInfo :
/usr/local/hadoop/hadoop-2.6.0/etc/hadoop/core-site.xml,/usr/local/hadoop/hbase-1.1.2/conf/hbase-site.xml
ZooKeeper QuorumInfo: No value set
ZooKeeper Client PortInfo: No value set
ZooKeeper ZNode ParentInfo : /hbase
HBase Client RetriesInfo : 1
On Fri, May 6, 2016 at 7:13 PM, Bryan Bende wrote:
> Can you describe your setup a little bit more, things like...
>
> Is NiFi running on the same machine that HBase and ZooKeeper are running
> on (it doesn't have to be, just want to understand the setup) ?
>
> What values do you have in hbase-site.xml for properties like...
> - hbase.zookeeper.quorum
> - hbase.zookeeper.property.clientPort
>
> Here is a template that is working for me:
>
> https://gist.githubusercontent.com/bbende/9457b5ed261e6eeb0f98995a5a2699e0/raw/38e050828c2545cb50e623d6dcf45dbbc9ad1d9d/FunWithHBaseUpdated.xml
>
> I am running NiFi on the HDP Sandbox so everything is local for me.
>
>
> On Fri, May 6, 2016 at 8:54 AM, Venkatesh Bodapati <
> venkatesh.bodap...@inndata.in> wrote:
>
>> " Missing Row id failure, routing to failure " issue solved . Still i
>> got the "failed to receive data from Hbase due to java net connection
>> exception :Connection Refused" error in GetHbase Processor. I update
>> Hbase-Site.xml and remove remaining properties like " Zookeeper Quorum
>> Info,Zookeeper Client Port Info . still i got the same error in GetHbase
>> processor.
>>
>> On Thu, May 5, 2016 at 9:07 PM, Bryan Bende wrote:
>>
>>> I was looking at this a little more and instead of needing the
>>> RouteOnAttribute, I think you can just update EvaluateJsonPath properties...
>>>
>>> Based on the JSON I see coming in, they should be:
>>>
>>> email = $.email
>>> firstName = $.name.first
>>> lastName = $.name.last
>>> ssn = $.id.value
>>>
>>>
>>>
>>> On Thu, May 5, 2016 at 11:30 AM, Bryan Bende wrote:
>>>
Hello,
For HBaseClient, are you sure your ZooKeeper being used by HBase is
running on localhost:2181?
Typically you don't really need to set the three ZooKeeper properties,
and you can instead just set the hbase-site.xml in the config resources.
For example, my Hadoop Configuration Resources is set as:
/etc/hadoop/conf/core-site.xml,/etc/hbase/conf/hbase-site.xml
And then I don't have anything specified for ZooKeeper QuorumInfo,
etc., because hbase-site.xml is providing all that information.
The example template requires a Users table, so from Hbase shell:
create 'Users', {NAME => 'cf'}
Then the template has a slight problem where it is using
EvaluteJsonPath to extract an attribute called "ssn" and using that for the
row id... the problem is a lot of the ssn values are empty so they are
failing.
If you stick a RouteOnAttribute right after EvaluateJsonPath, and add a
property "matched" = ${ssn:isEmpty():not()} that will filter out all the
empty SSNs, then connect RouteOnAttribute's matched relationship to the
HBase processors.
Lets get the inserting side working and then I would expect the
GetHBase to work since they use the same underlying connection.
Thanks,
Bryan
On Thu, May 5, 2016 at 10:59 AM, Venkatesh Bodapati <
venkatesh.bodap...@inndata.in> wrote:
> I am working on "Fun_With_Hbase.xml" Template. In this i will send
> data to hbase table, but i will not get data from hbase. I will get the
> "failed to receive data from hbase due to java net connection exception
> :Connection Refused" error in GetHbase Processor and In
> PutHbaseCell,PutHbaseJson Processor i got error Like This " Missing Row id
> failure, routing to failure ".
>
> This is my HbaseClient Properties :
>
> Hadoop Configuration FilesInfo :
> /usr/local/hadoop/hadoop-2.6.0/etc/hadoop/core-site.xml,/usr/local/hadoop/hadoop-2.6.0/etc/hadoop/hdfs-site.xml
> ZooKeeper QuorumInfo: localhost:2181
> ZooKeeper Client PortInfo: 2181
> ZooKeeper ZNode ParentInfo : /hbase
> HBase Client RetriesInfo
Re: Failed to receive data from Hbase due to java net connection exception :Connection Refused in Nifi
Can you describe your setup a little bit more, things like...
Is NiFi running on the same machine that HBase and ZooKeeper are running on
(it doesn't have to be, just want to understand the setup) ?
What values do you have in hbase-site.xml for properties like...
- hbase.zookeeper.quorum
- hbase.zookeeper.property.clientPort
Here is a template that is working for me:
https://gist.githubusercontent.com/bbende/9457b5ed261e6eeb0f98995a5a2699e0/raw/38e050828c2545cb50e623d6dcf45dbbc9ad1d9d/FunWithHBaseUpdated.xml
I am running NiFi on the HDP Sandbox so everything is local for me.
On Fri, May 6, 2016 at 8:54 AM, Venkatesh Bodapati <
venkatesh.bodap...@inndata.in> wrote:
> " Missing Row id failure, routing to failure " issue solved . Still i
> got the "failed to receive data from Hbase due to java net connection
> exception :Connection Refused" error in GetHbase Processor. I update
> Hbase-Site.xml and remove remaining properties like " Zookeeper Quorum
> Info,Zookeeper Client Port Info . still i got the same error in GetHbase
> processor.
>
> On Thu, May 5, 2016 at 9:07 PM, Bryan Bende wrote:
>
>> I was looking at this a little more and instead of needing the
>> RouteOnAttribute, I think you can just update EvaluateJsonPath properties...
>>
>> Based on the JSON I see coming in, they should be:
>>
>> email = $.email
>> firstName = $.name.first
>> lastName = $.name.last
>> ssn = $.id.value
>>
>>
>>
>> On Thu, May 5, 2016 at 11:30 AM, Bryan Bende wrote:
>>
>>> Hello,
>>>
>>> For HBaseClient, are you sure your ZooKeeper being used by HBase is
>>> running on localhost:2181?
>>>
>>> Typically you don't really need to set the three ZooKeeper properties,
>>> and you can instead just set the hbase-site.xml in the config resources.
>>>
>>> For example, my Hadoop Configuration Resources is set as:
>>> /etc/hadoop/conf/core-site.xml,/etc/hbase/conf/hbase-site.xml
>>>
>>> And then I don't have anything specified for ZooKeeper QuorumInfo, etc.,
>>> because hbase-site.xml is providing all that information.
>>>
>>> The example template requires a Users table, so from Hbase shell:
>>> create 'Users', {NAME => 'cf'}
>>>
>>> Then the template has a slight problem where it is using EvaluteJsonPath
>>> to extract an attribute called "ssn" and using that for the row id... the
>>> problem is a lot of the ssn values are empty so they are failing.
>>>
>>> If you stick a RouteOnAttribute right after EvaluateJsonPath, and add a
>>> property "matched" = ${ssn:isEmpty():not()} that will filter out all the
>>> empty SSNs, then connect RouteOnAttribute's matched relationship to the
>>> HBase processors.
>>>
>>> Lets get the inserting side working and then I would expect the GetHBase
>>> to work since they use the same underlying connection.
>>>
>>> Thanks,
>>>
>>> Bryan
>>>
>>>
>>> On Thu, May 5, 2016 at 10:59 AM, Venkatesh Bodapati <
>>> venkatesh.bodap...@inndata.in> wrote:
>>>
I am working on "Fun_With_Hbase.xml" Template. In this i will send data
to hbase table, but i will not get data from hbase. I will get the "failed
to receive data from hbase due to java net connection exception :Connection
Refused" error in GetHbase Processor and In PutHbaseCell,PutHbaseJson
Processor i got error Like This " Missing Row id failure, routing to
failure ".
This is my HbaseClient Properties :
Hadoop Configuration FilesInfo :
/usr/local/hadoop/hadoop-2.6.0/etc/hadoop/core-site.xml,/usr/local/hadoop/hadoop-2.6.0/etc/hadoop/hdfs-site.xml
ZooKeeper QuorumInfo: localhost:2181
ZooKeeper Client PortInfo: 2181
ZooKeeper ZNode ParentInfo : /hbase
HBase Client RetriesInfo : 1
How do i get data from Hbase Table in Nifi,Any suggesions please.
>>>
>>>
>>
>
Re: Failed to receive data from Hbase due to java net connection exception :Connection Refused in Nifi
" Missing Row id failure, routing to failure " issue solved . Still i got
the "failed to receive data from Hbase due to java net connection exception
:Connection Refused" error in GetHbase Processor. I update Hbase-Site.xml
and remove remaining properties like " Zookeeper Quorum
Info,Zookeeper Client Port Info . still i got the same error in GetHbase
processor.
On Thu, May 5, 2016 at 9:07 PM, Bryan Bende wrote:
> I was looking at this a little more and instead of needing the
> RouteOnAttribute, I think you can just update EvaluateJsonPath properties...
>
> Based on the JSON I see coming in, they should be:
>
> email = $.email
> firstName = $.name.first
> lastName = $.name.last
> ssn = $.id.value
>
>
>
> On Thu, May 5, 2016 at 11:30 AM, Bryan Bende wrote:
>
>> Hello,
>>
>> For HBaseClient, are you sure your ZooKeeper being used by HBase is
>> running on localhost:2181?
>>
>> Typically you don't really need to set the three ZooKeeper properties,
>> and you can instead just set the hbase-site.xml in the config resources.
>>
>> For example, my Hadoop Configuration Resources is set as:
>> /etc/hadoop/conf/core-site.xml,/etc/hbase/conf/hbase-site.xml
>>
>> And then I don't have anything specified for ZooKeeper QuorumInfo, etc.,
>> because hbase-site.xml is providing all that information.
>>
>> The example template requires a Users table, so from Hbase shell:
>> create 'Users', {NAME => 'cf'}
>>
>> Then the template has a slight problem where it is using EvaluteJsonPath
>> to extract an attribute called "ssn" and using that for the row id... the
>> problem is a lot of the ssn values are empty so they are failing.
>>
>> If you stick a RouteOnAttribute right after EvaluateJsonPath, and add a
>> property "matched" = ${ssn:isEmpty():not()} that will filter out all the
>> empty SSNs, then connect RouteOnAttribute's matched relationship to the
>> HBase processors.
>>
>> Lets get the inserting side working and then I would expect the GetHBase
>> to work since they use the same underlying connection.
>>
>> Thanks,
>>
>> Bryan
>>
>>
>> On Thu, May 5, 2016 at 10:59 AM, Venkatesh Bodapati <
>> venkatesh.bodap...@inndata.in> wrote:
>>
>>> I am working on "Fun_With_Hbase.xml" Template. In this i will send data
>>> to hbase table, but i will not get data from hbase. I will get the "failed
>>> to receive data from hbase due to java net connection exception :Connection
>>> Refused" error in GetHbase Processor and In PutHbaseCell,PutHbaseJson
>>> Processor i got error Like This " Missing Row id failure, routing to
>>> failure ".
>>>
>>> This is my HbaseClient Properties :
>>>
>>> Hadoop Configuration FilesInfo :
>>> /usr/local/hadoop/hadoop-2.6.0/etc/hadoop/core-site.xml,/usr/local/hadoop/hadoop-2.6.0/etc/hadoop/hdfs-site.xml
>>> ZooKeeper QuorumInfo: localhost:2181
>>> ZooKeeper Client PortInfo: 2181
>>> ZooKeeper ZNode ParentInfo : /hbase
>>> HBase Client RetriesInfo : 1
>>>
>>>
>>> How do i get data from Hbase Table in Nifi,Any suggesions please.
>>>
>>
>>
>
Re: Failed to receive data from Hbase due to java net connection exception :Connection Refused in Nifi
I fallow the above properties in EvaluateJsonPath processor , " Missing
Row id failure, routing to failure " issue solved . Still i got the "failed
to receive data from Hbase due to java net connection exception :Connection
Refused" error in GetHbase Processor. I update Hbase-Site.xml and remove
remaining properties like " Zookeeper Quorum Info,Zookeeper Client Port
Info . still i got the same error in GetHbase processor.
On Thu, May 5, 2016 at 9:07 PM, Bryan Bende wrote:
> I was looking at this a little more and instead of needing the
> RouteOnAttribute, I think you can just update EvaluateJsonPath properties...
>
> Based on the JSON I see coming in, they should be:
>
> email = $.email
> firstName = $.name.first
> lastName = $.name.last
> ssn = $.id.value
>
>
>
> On Thu, May 5, 2016 at 11:30 AM, Bryan Bende wrote:
>
>> Hello,
>>
>> For HBaseClient, are you sure your ZooKeeper being used by HBase is
>> running on localhost:2181?
>>
>> Typically you don't really need to set the three ZooKeeper properties,
>> and you can instead just set the hbase-site.xml in the config resources.
>>
>> For example, my Hadoop Configuration Resources is set as:
>> /etc/hadoop/conf/core-site.xml,/etc/hbase/conf/hbase-site.xml
>>
>> And then I don't have anything specified for ZooKeeper QuorumInfo, etc.,
>> because hbase-site.xml is providing all that information.
>>
>> The example template requires a Users table, so from Hbase shell:
>> create 'Users', {NAME => 'cf'}
>>
>> Then the template has a slight problem where it is using EvaluteJsonPath
>> to extract an attribute called "ssn" and using that for the row id... the
>> problem is a lot of the ssn values are empty so they are failing.
>>
>> If you stick a RouteOnAttribute right after EvaluateJsonPath, and add a
>> property "matched" = ${ssn:isEmpty():not()} that will filter out all the
>> empty SSNs, then connect RouteOnAttribute's matched relationship to the
>> HBase processors.
>>
>> Lets get the inserting side working and then I would expect the GetHBase
>> to work since they use the same underlying connection.
>>
>> Thanks,
>>
>> Bryan
>>
>>
>> On Thu, May 5, 2016 at 10:59 AM, Venkatesh Bodapati <
>> venkatesh.bodap...@inndata.in> wrote:
>>
>>> I am working on "Fun_With_Hbase.xml" Template. In this i will send data
>>> to hbase table, but i will not get data from hbase. I will get the "failed
>>> to receive data from hbase due to java net connection exception :Connection
>>> Refused" error in GetHbase Processor and In PutHbaseCell,PutHbaseJson
>>> Processor i got error Like This " Missing Row id failure, routing to
>>> failure ".
>>>
>>> This is my HbaseClient Properties :
>>>
>>> Hadoop Configuration FilesInfo :
>>> /usr/local/hadoop/hadoop-2.6.0/etc/hadoop/core-site.xml,/usr/local/hadoop/hadoop-2.6.0/etc/hadoop/hdfs-site.xml
>>> ZooKeeper QuorumInfo: localhost:2181
>>> ZooKeeper Client PortInfo: 2181
>>> ZooKeeper ZNode ParentInfo : /hbase
>>> HBase Client RetriesInfo : 1
>>>
>>>
>>> How do i get data from Hbase Table in Nifi,Any suggesions please.
>>>
>>
>>
>
Re: Failed to receive data from Hbase due to java net connection exception :Connection Refused in Nifi
I was looking at this a little more and instead of needing the
RouteOnAttribute, I think you can just update EvaluateJsonPath properties...
Based on the JSON I see coming in, they should be:
email = $.email
firstName = $.name.first
lastName = $.name.last
ssn = $.id.value
On Thu, May 5, 2016 at 11:30 AM, Bryan Bende wrote:
> Hello,
>
> For HBaseClient, are you sure your ZooKeeper being used by HBase is
> running on localhost:2181?
>
> Typically you don't really need to set the three ZooKeeper properties, and
> you can instead just set the hbase-site.xml in the config resources.
>
> For example, my Hadoop Configuration Resources is set as:
> /etc/hadoop/conf/core-site.xml,/etc/hbase/conf/hbase-site.xml
>
> And then I don't have anything specified for ZooKeeper QuorumInfo, etc.,
> because hbase-site.xml is providing all that information.
>
> The example template requires a Users table, so from Hbase shell:
> create 'Users', {NAME => 'cf'}
>
> Then the template has a slight problem where it is using EvaluteJsonPath
> to extract an attribute called "ssn" and using that for the row id... the
> problem is a lot of the ssn values are empty so they are failing.
>
> If you stick a RouteOnAttribute right after EvaluateJsonPath, and add a
> property "matched" = ${ssn:isEmpty():not()} that will filter out all the
> empty SSNs, then connect RouteOnAttribute's matched relationship to the
> HBase processors.
>
> Lets get the inserting side working and then I would expect the GetHBase
> to work since they use the same underlying connection.
>
> Thanks,
>
> Bryan
>
>
> On Thu, May 5, 2016 at 10:59 AM, Venkatesh Bodapati <
> venkatesh.bodap...@inndata.in> wrote:
>
>> I am working on "Fun_With_Hbase.xml" Template. In this i will send data
>> to hbase table, but i will not get data from hbase. I will get the "failed
>> to receive data from hbase due to java net connection exception :Connection
>> Refused" error in GetHbase Processor and In PutHbaseCell,PutHbaseJson
>> Processor i got error Like This " Missing Row id failure, routing to
>> failure ".
>>
>> This is my HbaseClient Properties :
>>
>> Hadoop Configuration FilesInfo :
>> /usr/local/hadoop/hadoop-2.6.0/etc/hadoop/core-site.xml,/usr/local/hadoop/hadoop-2.6.0/etc/hadoop/hdfs-site.xml
>> ZooKeeper QuorumInfo: localhost:2181
>> ZooKeeper Client PortInfo: 2181
>> ZooKeeper ZNode ParentInfo : /hbase
>> HBase Client RetriesInfo : 1
>>
>>
>> How do i get data from Hbase Table in Nifi,Any suggesions please.
>>
>
>
Re: Failed to receive data from Hbase due to java net connection exception :Connection Refused in Nifi
Hello,
For HBaseClient, are you sure your ZooKeeper being used by HBase is running
on localhost:2181?
Typically you don't really need to set the three ZooKeeper properties, and
you can instead just set the hbase-site.xml in the config resources.
For example, my Hadoop Configuration Resources is set as:
/etc/hadoop/conf/core-site.xml,/etc/hbase/conf/hbase-site.xml
And then I don't have anything specified for ZooKeeper QuorumInfo, etc.,
because hbase-site.xml is providing all that information.
The example template requires a Users table, so from Hbase shell:
create 'Users', {NAME => 'cf'}
Then the template has a slight problem where it is using EvaluteJsonPath to
extract an attribute called "ssn" and using that for the row id... the
problem is a lot of the ssn values are empty so they are failing.
If you stick a RouteOnAttribute right after EvaluateJsonPath, and add a
property "matched" = ${ssn:isEmpty():not()} that will filter out all the
empty SSNs, then connect RouteOnAttribute's matched relationship to the
HBase processors.
Lets get the inserting side working and then I would expect the GetHBase to
work since they use the same underlying connection.
Thanks,
Bryan
On Thu, May 5, 2016 at 10:59 AM, Venkatesh Bodapati <
venkatesh.bodap...@inndata.in> wrote:
> I am working on "Fun_With_Hbase.xml" Template. In this i will send data to
> hbase table, but i will not get data from hbase. I will get the "failed to
> receive data from hbase due to java net connection exception :Connection
> Refused" error in GetHbase Processor and In PutHbaseCell,PutHbaseJson
> Processor i got error Like This " Missing Row id failure, routing to
> failure ".
>
> This is my HbaseClient Properties :
>
> Hadoop Configuration FilesInfo :
> /usr/local/hadoop/hadoop-2.6.0/etc/hadoop/core-site.xml,/usr/local/hadoop/hadoop-2.6.0/etc/hadoop/hdfs-site.xml
> ZooKeeper QuorumInfo: localhost:2181
> ZooKeeper Client PortInfo: 2181
> ZooKeeper ZNode ParentInfo : /hbase
> HBase Client RetriesInfo : 1
>
>
> How do i get data from Hbase Table in Nifi,Any suggesions please.
>
Failed to receive data from Hbase due to java net connection exception :Connection Refused in Nifi
I am working on "Fun_With_Hbase.xml" Template. In this i will send data to hbase table, but i will not get data from hbase. I will get the "failed to receive data from hbase due to java net connection exception :Connection Refused" error in GetHbase Processor and In PutHbaseCell,PutHbaseJson Processor i got error Like This " Missing Row id failure, routing to failure ". This is my HbaseClient Properties : Hadoop Configuration FilesInfo : /usr/local/hadoop/hadoop-2.6.0/etc/hadoop/core-site.xml,/usr/local/hadoop/hadoop-2.6.0/etc/hadoop/hdfs-site.xml ZooKeeper QuorumInfo: localhost:2181 ZooKeeper Client PortInfo: 2181 ZooKeeper ZNode ParentInfo : /hbase HBase Client RetriesInfo : 1 How do i get data from Hbase Table in Nifi,Any suggesions please.
