Re: Failed to receive data from Hbase due to java net connection exception :Connection Refused in Nifi

[Bryan Bende]

Looks like it is not logging the full stacktrace by default...

Can you edit NIFI_HOME/conf/logback.xml and add a line like this:



Wait about 15-30 seconds and then see what is in nifi-app.log

On Fri, May 6, 2016 at 10:37 AM, Venkatesh Bodapati <
venkatesh.bodap...@inndata.in> wrote:

> This is  whole stacktrace in nifi-app.log.
>
> 2016-05-06 20:01:33,201 INFO [pool-18-thread-1]
> o.a.n.c.r.WriteAheadFlowFileRepository Initiating checkpoint of FlowFile
> Repository
> 2016-05-06 20:01:33,473 INFO [pool-18-thread-1]
> org.wali.MinimalLockingWriteAheadLog
> org.wali.MinimalLockingWriteAheadLog@26650de4 checkpointed with 0 Records
> and 0 Swap Files in 134 milliseconds (Stop-the-world time = 26
> milliseconds, Clear Edit Logs time = 37 millis), max Transaction ID 268027
> 2016-05-06 20:01:33,473 INFO [pool-18-thread-1]
> o.a.n.c.r.WriteAheadFlowFileRepository Successfully checkpointed FlowFile
> Repository with 0 records in 135 milliseconds
> 2016-05-06 20:02:30,775 INFO [Flow Service Tasks Thread-1]
> o.a.nifi.controller.StandardFlowService Saved flow controller
> org.apache.nifi.controller.FlowController@2c4e3ffb // Another save
> pending = false
> 2016-05-06 20:02:30,958 INFO [pool-26-thread-18]
> o.a.h.h.zookeeper.RecoverableZooKeeper Process
> identifier=hconnection-0xaa14407 connecting to ZooKeeper
> ensemble=localhost:2181
> 2016-05-06 20:02:30,958 INFO [pool-26-thread-18]
> org.apache.zookeeper.ZooKeeper Initiating client connection,
> connectString=localhost:2181 sessionTimeout=9
> watcher=hconnection-0xaa144070x0, quorum=localhost:2181, baseZNode=/hbase
> 2016-05-06 20:02:30,975 INFO
> [pool-26-thread-18-SendThread(localhost:2181)]
> org.apache.zookeeper.ClientCnxn Opening socket connection to server
> localhost/127.0.0.1:2181. Will not attempt to authenticate using SASL
> (unknown error)
> 2016-05-06 20:02:30,977 INFO
> [pool-26-thread-18-SendThread(localhost:2181)]
> org.apache.zookeeper.ClientCnxn Socket connection established to localhost/
> 127.0.0.1:2181, initiating session
> 2016-05-06 20:02:31,081 INFO
> [pool-26-thread-18-SendThread(localhost:2181)]
> org.apache.zookeeper.ClientCnxn Session establishment complete on server
> localhost/127.0.0.1:2181, sessionid = 0x154865d23ca000c, negotiated
> timeout = 6
> 2016-05-06 20:02:32,657 INFO [pool-26-thread-19]
> o.a.n.c.s.TimerDrivenSchedulingAgent Scheduled
> GetHBase[id=89f3c893-f14f-4520-8d56-b27a431a7d41] to run with 1 threads
> 2016-05-06 20:02:32,658 INFO [pool-26-thread-17]
> o.a.n.c.s.TimerDrivenSchedulingAgent Scheduled
> PutHBaseCell[id=04a3e270-375c-4f2d-9e93-ac53a70765d8] to run with 1 threads
> 2016-05-06 20:02:32,659 INFO [pool-26-thread-20]
> o.a.n.c.s.TimerDrivenSchedulingAgent Scheduled
> PutHBaseJSON[id=dc18a7a7-dd2f-484c-aa27-4a6f0de65920] to run with 1 threads
> 2016-05-06 20:02:32,676 ERROR [Timer-Driven Process Thread-6]
> org.apache.nifi.hbase.GetHBase
> GetHBase[id=89f3c893-f14f-4520-8d56-b27a431a7d41] Failed to receive data
> from HBase due to java.net.ConnectException: Connection refused
> 2016-05-06 20:02:33,678 ERROR [Timer-Driven Process Thread-4]
> org.apache.nifi.hbase.GetHBase
> GetHBase[id=89f3c893-f14f-4520-8d56-b27a431a7d41] Failed to receive data
> from HBase due to java.net.ConnectException: Connection refused
> 2016-05-06 20:02:34,679 ERROR [Timer-Driven Process Thread-10]
> org.apache.nifi.hbase.GetHBase
> GetHBase[id=89f3c893-f14f-4520-8d56-b27a431a7d41] Failed to receive data
> from HBase due to java.net.ConnectException: Connection refused
> 2016-05-06 20:02:35,680 ERROR [Timer-Driven Process Thread-2]
> org.apache.nifi.hbase.GetHBase
> GetHBase[id=89f3c893-f14f-4520-8d56-b27a431a7d41] Failed to receive data
> from HBase due to java.net.ConnectException: Connection refused
> 2016-05-06 20:02:36,681 ERROR [Timer-Driven Process Thread-10]
> org.apache.nifi.hbase.GetHBase
> GetHBase[id=89f3c893-f14f-4520-8d56-b27a431a7d41] Failed to receive data
> from HBase due to java.net.ConnectException: Connection refused
> 2016-05-06 20:02:37,683 ERROR [Timer-Driven Process Thread-10]
> org.apache.nifi.hbase.GetHBase
> GetHBase[id=89f3c893-f14f-4520-8d56-b27a431a7d41] Failed to receive data
> from HBase due to java.net.ConnectException: Connection refused
> 2016-05-06 20:02:38,684 ERROR [Timer-Driven Process Thread-3]
> org.apache.nifi.hbase.GetHBase
> GetHBase[id=89f3c893-f14f-4520-8d56-b27a431a7d41] Failed to receive data
> from HBase due to java.net.ConnectException: Connection refused
> 2016-05-06 20:02:39,686 ERROR [Timer-Driven Process Thread-4]
> org.apache.nifi.hbase.GetHBase
> GetHBase[id=89f3c893-f14f-4520-8d56-b27a431a7d41] Failed to receive data
> from HBase due to java.net.ConnectException: Connection refused
> 2016-05-06 20:02:40,690 ERROR [Timer-Driven Process Thread-7]
> org.apache.nifi.hbase.GetHBase
> GetHBase[id=89f3c893-f14f-4520-8d56-b27a431a7d41] Failed to receive data
> from HBase due to java.net.ConnectException: Connection refused
> 2016-05-06 20:02:41,691 ERROR [Timer-Driven 

Re: Failed to receive data from Hbase due to java net connection exception :Connection Refused in Nifi

[Venkatesh Bodapati]

This is  whole stacktrace in nifi-app.log.

2016-05-06 20:01:33,201 INFO [pool-18-thread-1]
o.a.n.c.r.WriteAheadFlowFileRepository Initiating checkpoint of FlowFile
Repository
2016-05-06 20:01:33,473 INFO [pool-18-thread-1]
org.wali.MinimalLockingWriteAheadLog
org.wali.MinimalLockingWriteAheadLog@26650de4 checkpointed with 0 Records
and 0 Swap Files in 134 milliseconds (Stop-the-world time = 26
milliseconds, Clear Edit Logs time = 37 millis), max Transaction ID 268027
2016-05-06 20:01:33,473 INFO [pool-18-thread-1]
o.a.n.c.r.WriteAheadFlowFileRepository Successfully checkpointed FlowFile
Repository with 0 records in 135 milliseconds
2016-05-06 20:02:30,775 INFO [Flow Service Tasks Thread-1]
o.a.nifi.controller.StandardFlowService Saved flow controller
org.apache.nifi.controller.FlowController@2c4e3ffb // Another save pending
= false
2016-05-06 20:02:30,958 INFO [pool-26-thread-18]
o.a.h.h.zookeeper.RecoverableZooKeeper Process
identifier=hconnection-0xaa14407 connecting to ZooKeeper
ensemble=localhost:2181
2016-05-06 20:02:30,958 INFO [pool-26-thread-18]
org.apache.zookeeper.ZooKeeper Initiating client connection,
connectString=localhost:2181 sessionTimeout=9
watcher=hconnection-0xaa144070x0, quorum=localhost:2181, baseZNode=/hbase
2016-05-06 20:02:30,975 INFO [pool-26-thread-18-SendThread(localhost:2181)]
org.apache.zookeeper.ClientCnxn Opening socket connection to server
localhost/127.0.0.1:2181. Will not attempt to authenticate using SASL
(unknown error)
2016-05-06 20:02:30,977 INFO [pool-26-thread-18-SendThread(localhost:2181)]
org.apache.zookeeper.ClientCnxn Socket connection established to localhost/
127.0.0.1:2181, initiating session
2016-05-06 20:02:31,081 INFO [pool-26-thread-18-SendThread(localhost:2181)]
org.apache.zookeeper.ClientCnxn Session establishment complete on server
localhost/127.0.0.1:2181, sessionid = 0x154865d23ca000c, negotiated timeout
= 6
2016-05-06 20:02:32,657 INFO [pool-26-thread-19]
o.a.n.c.s.TimerDrivenSchedulingAgent Scheduled
GetHBase[id=89f3c893-f14f-4520-8d56-b27a431a7d41] to run with 1 threads
2016-05-06 20:02:32,658 INFO [pool-26-thread-17]
o.a.n.c.s.TimerDrivenSchedulingAgent Scheduled
PutHBaseCell[id=04a3e270-375c-4f2d-9e93-ac53a70765d8] to run with 1 threads
2016-05-06 20:02:32,659 INFO [pool-26-thread-20]
o.a.n.c.s.TimerDrivenSchedulingAgent Scheduled
PutHBaseJSON[id=dc18a7a7-dd2f-484c-aa27-4a6f0de65920] to run with 1 threads
2016-05-06 20:02:32,676 ERROR [Timer-Driven Process Thread-6]
org.apache.nifi.hbase.GetHBase
GetHBase[id=89f3c893-f14f-4520-8d56-b27a431a7d41] Failed to receive data
from HBase due to java.net.ConnectException: Connection refused
2016-05-06 20:02:33,678 ERROR [Timer-Driven Process Thread-4]
org.apache.nifi.hbase.GetHBase
GetHBase[id=89f3c893-f14f-4520-8d56-b27a431a7d41] Failed to receive data
from HBase due to java.net.ConnectException: Connection refused
2016-05-06 20:02:34,679 ERROR [Timer-Driven Process Thread-10]
org.apache.nifi.hbase.GetHBase
GetHBase[id=89f3c893-f14f-4520-8d56-b27a431a7d41] Failed to receive data
from HBase due to java.net.ConnectException: Connection refused
2016-05-06 20:02:35,680 ERROR [Timer-Driven Process Thread-2]
org.apache.nifi.hbase.GetHBase
GetHBase[id=89f3c893-f14f-4520-8d56-b27a431a7d41] Failed to receive data
from HBase due to java.net.ConnectException: Connection refused
2016-05-06 20:02:36,681 ERROR [Timer-Driven Process Thread-10]
org.apache.nifi.hbase.GetHBase
GetHBase[id=89f3c893-f14f-4520-8d56-b27a431a7d41] Failed to receive data
from HBase due to java.net.ConnectException: Connection refused
2016-05-06 20:02:37,683 ERROR [Timer-Driven Process Thread-10]
org.apache.nifi.hbase.GetHBase
GetHBase[id=89f3c893-f14f-4520-8d56-b27a431a7d41] Failed to receive data
from HBase due to java.net.ConnectException: Connection refused
2016-05-06 20:02:38,684 ERROR [Timer-Driven Process Thread-3]
org.apache.nifi.hbase.GetHBase
GetHBase[id=89f3c893-f14f-4520-8d56-b27a431a7d41] Failed to receive data
from HBase due to java.net.ConnectException: Connection refused
2016-05-06 20:02:39,686 ERROR [Timer-Driven Process Thread-4]
org.apache.nifi.hbase.GetHBase
GetHBase[id=89f3c893-f14f-4520-8d56-b27a431a7d41] Failed to receive data
from HBase due to java.net.ConnectException: Connection refused
2016-05-06 20:02:40,690 ERROR [Timer-Driven Process Thread-7]
org.apache.nifi.hbase.GetHBase
GetHBase[id=89f3c893-f14f-4520-8d56-b27a431a7d41] Failed to receive data
from HBase due to java.net.ConnectException: Connection refused
2016-05-06 20:02:41,691 ERROR [Timer-Driven Process Thread-5]
org.apache.nifi.hbase.GetHBase
GetHBase[id=89f3c893-f14f-4520-8d56-b27a431a7d41] Failed to receive data
from HBase due to java.net.ConnectException: Connection refused
2016-05-06 20:02:42,693 ERROR [Timer-Driven Process Thread-9]
org.apache.nifi.hbase.GetHBase
GetHBase[id=89f3c893-f14f-4520-8d56-b27a431a7d41] Failed to receive data
from HBase due to java.net.ConnectException: Connection refused
2016-05-06 20:02:42,703 INFO 

Re: Failed to receive data from Hbase due to java net connection exception :Connection Refused in Nifi

[Bryan Bende]

Ok, thanks for sharing that...

Can you look in NIFI_HOME/logs/nifi-app.log and see if you can find that
error message that says "Failed to receive data from HBase due to",
and then there should be a whole stack trace that goes with that.

If you could paste that whole stacktrace here that would be helpful.

Thanks,

Bryan

On Fri, May 6, 2016 at 10:14 AM, Venkatesh Bodapati <
venkatesh.bodap...@inndata.in> wrote:

> I am running NiFi, Hbase and Zookeeper in same machine.
>
> This my Hbase-site.xml :
> 
> 
> hbase.master
> localhost:6
> 
> 
> hbase.rootdir
> hdfs://localhost:8020/hbase
> 
> 
> hbase.cluster.distributed
> true
> 
> 
> hbase.zookeeper.quorum
> localhost
> 
> 
> dfs.replication
> 1
> 
> 
> hbase.zookeeper.property.clientPort
> 2181
> 
> 
> hbase.zookeeper.property.dataDir
> /usr/local/hadoop/hbase-1.1.2/zookeeper
> 
> 
> zookeeper.znode.parent
> /hbase
> 
> 
>
> and my Hbase-client Service properties like this :
> Hadoop Configuration FilesInfo  :
> /usr/local/hadoop/hadoop-2.6.0/etc/hadoop/core-site.xml,/usr/local/hadoop/hbase-1.1.2/conf/hbase-site.xml
> ZooKeeper QuorumInfo: No value set
> ZooKeeper Client PortInfo: No value set
> ZooKeeper ZNode ParentInfo   : /hbase
> HBase Client RetriesInfo   : 1
>
> On Fri, May 6, 2016 at 7:13 PM, Bryan Bende  wrote:
>
>> Can you describe your setup a little bit more, things like...
>>
>> Is NiFi running on the same machine that HBase and ZooKeeper are running
>> on (it doesn't have to be, just want to understand the setup) ?
>>
>> What values do you have in hbase-site.xml for properties like...
>> - hbase.zookeeper.quorum
>> - hbase.zookeeper.property.clientPort
>>
>> Here is a template that is working for me:
>>
>> https://gist.githubusercontent.com/bbende/9457b5ed261e6eeb0f98995a5a2699e0/raw/38e050828c2545cb50e623d6dcf45dbbc9ad1d9d/FunWithHBaseUpdated.xml
>>
>> I am running NiFi on the HDP Sandbox so everything is local for me.
>>
>>
>> On Fri, May 6, 2016 at 8:54 AM, Venkatesh Bodapati <
>> venkatesh.bodap...@inndata.in> wrote:
>>
>>>   " Missing Row id failure, routing to failure " issue solved . Still i
>>> got the "failed to receive data from Hbase due to java net connection
>>> exception :Connection Refused" error in GetHbase Processor.  I update
>>> Hbase-Site.xml and remove remaining properties like " Zookeeper Quorum
>>> Info,Zookeeper Client Port Info . still i got the same error in GetHbase
>>> processor.
>>>
>>> On Thu, May 5, 2016 at 9:07 PM, Bryan Bende  wrote:
>>>
 I was looking at this a little more and instead of needing the
 RouteOnAttribute, I think you can just update EvaluateJsonPath 
 properties...

 Based on the JSON I see coming in, they should be:

 email = $.email
 firstName = $.name.first
 lastName = $.name.last
 ssn = $.id.value



 On Thu, May 5, 2016 at 11:30 AM, Bryan Bende  wrote:

> Hello,
>
> For HBaseClient, are you sure your ZooKeeper being used by HBase is
> running on localhost:2181?
>
> Typically you don't really need to set the three ZooKeeper properties,
> and you can instead just set the hbase-site.xml in the config resources.
>
> For example, my Hadoop Configuration Resources is set as:
> /etc/hadoop/conf/core-site.xml,/etc/hbase/conf/hbase-site.xml
>
> And then I don't have anything specified for ZooKeeper QuorumInfo,
> etc., because hbase-site.xml is providing all that information.
>
> The example template requires a Users table, so from Hbase shell:
> create 'Users', {NAME => 'cf'}
>
> Then the template has a slight problem where it is using
> EvaluteJsonPath to extract an attribute called "ssn" and using that for 
> the
> row id... the problem is a lot of the ssn values are empty so they are
> failing.
>
> If you stick a RouteOnAttribute right after EvaluateJsonPath, and add
> a property "matched" = ${ssn:isEmpty():not()} that will filter out all the
> empty SSNs, then connect RouteOnAttribute's matched relationship to the
> HBase processors.
>
> Lets get the inserting side working and then I would expect the
> GetHBase to work since they use the same underlying connection.
>
> Thanks,
>
> Bryan
>
>
> On Thu, May 5, 2016 at 10:59 AM, Venkatesh Bodapati <
> venkatesh.bodap...@inndata.in> wrote:
>
>> I am working on "Fun_With_Hbase.xml" Template. In this i will send
>> data to hbase table, but i will not get data from hbase. I will 

Re: Failed to receive data from Hbase due to java net connection exception :Connection Refused in Nifi

[Venkatesh Bodapati]

I am running NiFi, Hbase and Zookeeper in same machine.

This my Hbase-site.xml :


hbase.master
localhost:6


hbase.rootdir
hdfs://localhost:8020/hbase


hbase.cluster.distributed
true


hbase.zookeeper.quorum
localhost


dfs.replication
1


hbase.zookeeper.property.clientPort
2181


hbase.zookeeper.property.dataDir
/usr/local/hadoop/hbase-1.1.2/zookeeper


zookeeper.znode.parent
/hbase



and my Hbase-client Service properties like this :
Hadoop Configuration FilesInfo  :
/usr/local/hadoop/hadoop-2.6.0/etc/hadoop/core-site.xml,/usr/local/hadoop/hbase-1.1.2/conf/hbase-site.xml
ZooKeeper QuorumInfo: No value set
ZooKeeper Client PortInfo: No value set
ZooKeeper ZNode ParentInfo   : /hbase
HBase Client RetriesInfo   : 1

On Fri, May 6, 2016 at 7:13 PM, Bryan Bende  wrote:

> Can you describe your setup a little bit more, things like...
>
> Is NiFi running on the same machine that HBase and ZooKeeper are running
> on (it doesn't have to be, just want to understand the setup) ?
>
> What values do you have in hbase-site.xml for properties like...
> - hbase.zookeeper.quorum
> - hbase.zookeeper.property.clientPort
>
> Here is a template that is working for me:
>
> https://gist.githubusercontent.com/bbende/9457b5ed261e6eeb0f98995a5a2699e0/raw/38e050828c2545cb50e623d6dcf45dbbc9ad1d9d/FunWithHBaseUpdated.xml
>
> I am running NiFi on the HDP Sandbox so everything is local for me.
>
>
> On Fri, May 6, 2016 at 8:54 AM, Venkatesh Bodapati <
> venkatesh.bodap...@inndata.in> wrote:
>
>>   " Missing Row id failure, routing to failure " issue solved . Still i
>> got the "failed to receive data from Hbase due to java net connection
>> exception :Connection Refused" error in GetHbase Processor.  I update
>> Hbase-Site.xml and remove remaining properties like " Zookeeper Quorum
>> Info,Zookeeper Client Port Info . still i got the same error in GetHbase
>> processor.
>>
>> On Thu, May 5, 2016 at 9:07 PM, Bryan Bende  wrote:
>>
>>> I was looking at this a little more and instead of needing the
>>> RouteOnAttribute, I think you can just update EvaluateJsonPath properties...
>>>
>>> Based on the JSON I see coming in, they should be:
>>>
>>> email = $.email
>>> firstName = $.name.first
>>> lastName = $.name.last
>>> ssn = $.id.value
>>>
>>>
>>>
>>> On Thu, May 5, 2016 at 11:30 AM, Bryan Bende  wrote:
>>>
 Hello,

 For HBaseClient, are you sure your ZooKeeper being used by HBase is
 running on localhost:2181?

 Typically you don't really need to set the three ZooKeeper properties,
 and you can instead just set the hbase-site.xml in the config resources.

 For example, my Hadoop Configuration Resources is set as:
 /etc/hadoop/conf/core-site.xml,/etc/hbase/conf/hbase-site.xml

 And then I don't have anything specified for ZooKeeper QuorumInfo,
 etc., because hbase-site.xml is providing all that information.

 The example template requires a Users table, so from Hbase shell:
 create 'Users', {NAME => 'cf'}

 Then the template has a slight problem where it is using
 EvaluteJsonPath to extract an attribute called "ssn" and using that for the
 row id... the problem is a lot of the ssn values are empty so they are
 failing.

 If you stick a RouteOnAttribute right after EvaluateJsonPath, and add a
 property "matched" = ${ssn:isEmpty():not()} that will filter out all the
 empty SSNs, then connect RouteOnAttribute's matched relationship to the
 HBase processors.

 Lets get the inserting side working and then I would expect the
 GetHBase to work since they use the same underlying connection.

 Thanks,

 Bryan


 On Thu, May 5, 2016 at 10:59 AM, Venkatesh Bodapati <
 venkatesh.bodap...@inndata.in> wrote:

> I am working on "Fun_With_Hbase.xml" Template. In this i will send
> data to hbase table, but i will not get data from hbase. I will get the
> "failed to receive data from hbase due to java net connection exception
> :Connection Refused" error in GetHbase Processor and In
> PutHbaseCell,PutHbaseJson Processor i got error Like This " Missing Row id
> failure, routing to failure ".
>
> This is my HbaseClient Properties :
>
> Hadoop Configuration FilesInfo  :
> /usr/local/hadoop/hadoop-2.6.0/etc/hadoop/core-site.xml,/usr/local/hadoop/hadoop-2.6.0/etc/hadoop/hdfs-site.xml
> ZooKeeper QuorumInfo: localhost:2181
> ZooKeeper Client PortInfo: 2181
> ZooKeeper ZNode 

Re: Failed to receive data from Hbase due to java net connection exception :Connection Refused in Nifi

[Bryan Bende]

Can you describe your setup a little bit more, things like...

Is NiFi running on the same machine that HBase and ZooKeeper are running on
(it doesn't have to be, just want to understand the setup) ?

What values do you have in hbase-site.xml for properties like...
- hbase.zookeeper.quorum
- hbase.zookeeper.property.clientPort

Here is a template that is working for me:
https://gist.githubusercontent.com/bbende/9457b5ed261e6eeb0f98995a5a2699e0/raw/38e050828c2545cb50e623d6dcf45dbbc9ad1d9d/FunWithHBaseUpdated.xml

I am running NiFi on the HDP Sandbox so everything is local for me.


On Fri, May 6, 2016 at 8:54 AM, Venkatesh Bodapati <
venkatesh.bodap...@inndata.in> wrote:

>   " Missing Row id failure, routing to failure " issue solved . Still i
> got the "failed to receive data from Hbase due to java net connection
> exception :Connection Refused" error in GetHbase Processor.  I update
> Hbase-Site.xml and remove remaining properties like " Zookeeper Quorum
> Info,Zookeeper Client Port Info . still i got the same error in GetHbase
> processor.
>
> On Thu, May 5, 2016 at 9:07 PM, Bryan Bende  wrote:
>
>> I was looking at this a little more and instead of needing the
>> RouteOnAttribute, I think you can just update EvaluateJsonPath properties...
>>
>> Based on the JSON I see coming in, they should be:
>>
>> email = $.email
>> firstName = $.name.first
>> lastName = $.name.last
>> ssn = $.id.value
>>
>>
>>
>> On Thu, May 5, 2016 at 11:30 AM, Bryan Bende  wrote:
>>
>>> Hello,
>>>
>>> For HBaseClient, are you sure your ZooKeeper being used by HBase is
>>> running on localhost:2181?
>>>
>>> Typically you don't really need to set the three ZooKeeper properties,
>>> and you can instead just set the hbase-site.xml in the config resources.
>>>
>>> For example, my Hadoop Configuration Resources is set as:
>>> /etc/hadoop/conf/core-site.xml,/etc/hbase/conf/hbase-site.xml
>>>
>>> And then I don't have anything specified for ZooKeeper QuorumInfo, etc.,
>>> because hbase-site.xml is providing all that information.
>>>
>>> The example template requires a Users table, so from Hbase shell:
>>> create 'Users', {NAME => 'cf'}
>>>
>>> Then the template has a slight problem where it is using EvaluteJsonPath
>>> to extract an attribute called "ssn" and using that for the row id... the
>>> problem is a lot of the ssn values are empty so they are failing.
>>>
>>> If you stick a RouteOnAttribute right after EvaluateJsonPath, and add a
>>> property "matched" = ${ssn:isEmpty():not()} that will filter out all the
>>> empty SSNs, then connect RouteOnAttribute's matched relationship to the
>>> HBase processors.
>>>
>>> Lets get the inserting side working and then I would expect the GetHBase
>>> to work since they use the same underlying connection.
>>>
>>> Thanks,
>>>
>>> Bryan
>>>
>>>
>>> On Thu, May 5, 2016 at 10:59 AM, Venkatesh Bodapati <
>>> venkatesh.bodap...@inndata.in> wrote:
>>>
 I am working on "Fun_With_Hbase.xml" Template. In this i will send data
 to hbase table, but i will not get data from hbase. I will get the "failed
 to receive data from hbase due to java net connection exception :Connection
 Refused" error in GetHbase Processor and In PutHbaseCell,PutHbaseJson
 Processor i got error Like This " Missing Row id failure, routing to
 failure ".

 This is my HbaseClient Properties :

 Hadoop Configuration FilesInfo  :
 /usr/local/hadoop/hadoop-2.6.0/etc/hadoop/core-site.xml,/usr/local/hadoop/hadoop-2.6.0/etc/hadoop/hdfs-site.xml
 ZooKeeper QuorumInfo: localhost:2181
 ZooKeeper Client PortInfo: 2181
 ZooKeeper ZNode ParentInfo   : /hbase
 HBase Client RetriesInfo   : 1


 How do i get data from Hbase Table in Nifi,Any suggesions please.

>>>
>>>
>>
>

Re: Failed to receive data from Hbase due to java net connection exception :Connection Refused in Nifi

[Venkatesh Bodapati]

  " Missing Row id failure, routing to failure " issue solved . Still i got
the "failed to receive data from Hbase due to java net connection exception
:Connection Refused" error in GetHbase Processor.  I update Hbase-Site.xml
and remove remaining properties like " Zookeeper Quorum
Info,Zookeeper Client Port Info . still i got the same error in GetHbase
processor.

On Thu, May 5, 2016 at 9:07 PM, Bryan Bende  wrote:

> I was looking at this a little more and instead of needing the
> RouteOnAttribute, I think you can just update EvaluateJsonPath properties...
>
> Based on the JSON I see coming in, they should be:
>
> email = $.email
> firstName = $.name.first
> lastName = $.name.last
> ssn = $.id.value
>
>
>
> On Thu, May 5, 2016 at 11:30 AM, Bryan Bende  wrote:
>
>> Hello,
>>
>> For HBaseClient, are you sure your ZooKeeper being used by HBase is
>> running on localhost:2181?
>>
>> Typically you don't really need to set the three ZooKeeper properties,
>> and you can instead just set the hbase-site.xml in the config resources.
>>
>> For example, my Hadoop Configuration Resources is set as:
>> /etc/hadoop/conf/core-site.xml,/etc/hbase/conf/hbase-site.xml
>>
>> And then I don't have anything specified for ZooKeeper QuorumInfo, etc.,
>> because hbase-site.xml is providing all that information.
>>
>> The example template requires a Users table, so from Hbase shell:
>> create 'Users', {NAME => 'cf'}
>>
>> Then the template has a slight problem where it is using EvaluteJsonPath
>> to extract an attribute called "ssn" and using that for the row id... the
>> problem is a lot of the ssn values are empty so they are failing.
>>
>> If you stick a RouteOnAttribute right after EvaluateJsonPath, and add a
>> property "matched" = ${ssn:isEmpty():not()} that will filter out all the
>> empty SSNs, then connect RouteOnAttribute's matched relationship to the
>> HBase processors.
>>
>> Lets get the inserting side working and then I would expect the GetHBase
>> to work since they use the same underlying connection.
>>
>> Thanks,
>>
>> Bryan
>>
>>
>> On Thu, May 5, 2016 at 10:59 AM, Venkatesh Bodapati <
>> venkatesh.bodap...@inndata.in> wrote:
>>
>>> I am working on "Fun_With_Hbase.xml" Template. In this i will send data
>>> to hbase table, but i will not get data from hbase. I will get the "failed
>>> to receive data from hbase due to java net connection exception :Connection
>>> Refused" error in GetHbase Processor and In PutHbaseCell,PutHbaseJson
>>> Processor i got error Like This " Missing Row id failure, routing to
>>> failure ".
>>>
>>> This is my HbaseClient Properties :
>>>
>>> Hadoop Configuration FilesInfo  :
>>> /usr/local/hadoop/hadoop-2.6.0/etc/hadoop/core-site.xml,/usr/local/hadoop/hadoop-2.6.0/etc/hadoop/hdfs-site.xml
>>> ZooKeeper QuorumInfo: localhost:2181
>>> ZooKeeper Client PortInfo: 2181
>>> ZooKeeper ZNode ParentInfo   : /hbase
>>> HBase Client RetriesInfo   : 1
>>>
>>>
>>> How do i get data from Hbase Table in Nifi,Any suggesions please.
>>>
>>
>>
>

Re: Failed to receive data from Hbase due to java net connection exception :Connection Refused in Nifi

[Venkatesh Bodapati]

I fallow the above properties in EvaluateJsonPath processor ,  " Missing
Row id failure, routing to failure " issue solved . Still i got the "failed
to receive data from Hbase due to java net connection exception :Connection
Refused" error in GetHbase Processor.  I update Hbase-Site.xml and remove
remaining properties like " Zookeeper Quorum Info,Zookeeper Client Port
Info . still i got the same error in GetHbase processor.

On Thu, May 5, 2016 at 9:07 PM, Bryan Bende  wrote:

> I was looking at this a little more and instead of needing the
> RouteOnAttribute, I think you can just update EvaluateJsonPath properties...
>
> Based on the JSON I see coming in, they should be:
>
> email = $.email
> firstName = $.name.first
> lastName = $.name.last
> ssn = $.id.value
>
>
>
> On Thu, May 5, 2016 at 11:30 AM, Bryan Bende  wrote:
>
>> Hello,
>>
>> For HBaseClient, are you sure your ZooKeeper being used by HBase is
>> running on localhost:2181?
>>
>> Typically you don't really need to set the three ZooKeeper properties,
>> and you can instead just set the hbase-site.xml in the config resources.
>>
>> For example, my Hadoop Configuration Resources is set as:
>> /etc/hadoop/conf/core-site.xml,/etc/hbase/conf/hbase-site.xml
>>
>> And then I don't have anything specified for ZooKeeper QuorumInfo, etc.,
>> because hbase-site.xml is providing all that information.
>>
>> The example template requires a Users table, so from Hbase shell:
>> create 'Users', {NAME => 'cf'}
>>
>> Then the template has a slight problem where it is using EvaluteJsonPath
>> to extract an attribute called "ssn" and using that for the row id... the
>> problem is a lot of the ssn values are empty so they are failing.
>>
>> If you stick a RouteOnAttribute right after EvaluateJsonPath, and add a
>> property "matched" = ${ssn:isEmpty():not()} that will filter out all the
>> empty SSNs, then connect RouteOnAttribute's matched relationship to the
>> HBase processors.
>>
>> Lets get the inserting side working and then I would expect the GetHBase
>> to work since they use the same underlying connection.
>>
>> Thanks,
>>
>> Bryan
>>
>>
>> On Thu, May 5, 2016 at 10:59 AM, Venkatesh Bodapati <
>> venkatesh.bodap...@inndata.in> wrote:
>>
>>> I am working on "Fun_With_Hbase.xml" Template. In this i will send data
>>> to hbase table, but i will not get data from hbase. I will get the "failed
>>> to receive data from hbase due to java net connection exception :Connection
>>> Refused" error in GetHbase Processor and In PutHbaseCell,PutHbaseJson
>>> Processor i got error Like This " Missing Row id failure, routing to
>>> failure ".
>>>
>>> This is my HbaseClient Properties :
>>>
>>> Hadoop Configuration FilesInfo  :
>>> /usr/local/hadoop/hadoop-2.6.0/etc/hadoop/core-site.xml,/usr/local/hadoop/hadoop-2.6.0/etc/hadoop/hdfs-site.xml
>>> ZooKeeper QuorumInfo: localhost:2181
>>> ZooKeeper Client PortInfo: 2181
>>> ZooKeeper ZNode ParentInfo   : /hbase
>>> HBase Client RetriesInfo   : 1
>>>
>>>
>>> How do i get data from Hbase Table in Nifi,Any suggesions please.
>>>
>>
>>
>

Re: Failed to receive data from Hbase due to java net connection exception :Connection Refused in Nifi

[Bryan Bende]

I was looking at this a little more and instead of needing the
RouteOnAttribute, I think you can just update EvaluateJsonPath properties...

Based on the JSON I see coming in, they should be:

email = $.email
firstName = $.name.first
lastName = $.name.last
ssn = $.id.value



On Thu, May 5, 2016 at 11:30 AM, Bryan Bende  wrote:

> Hello,
>
> For HBaseClient, are you sure your ZooKeeper being used by HBase is
> running on localhost:2181?
>
> Typically you don't really need to set the three ZooKeeper properties, and
> you can instead just set the hbase-site.xml in the config resources.
>
> For example, my Hadoop Configuration Resources is set as:
> /etc/hadoop/conf/core-site.xml,/etc/hbase/conf/hbase-site.xml
>
> And then I don't have anything specified for ZooKeeper QuorumInfo, etc.,
> because hbase-site.xml is providing all that information.
>
> The example template requires a Users table, so from Hbase shell:
> create 'Users', {NAME => 'cf'}
>
> Then the template has a slight problem where it is using EvaluteJsonPath
> to extract an attribute called "ssn" and using that for the row id... the
> problem is a lot of the ssn values are empty so they are failing.
>
> If you stick a RouteOnAttribute right after EvaluateJsonPath, and add a
> property "matched" = ${ssn:isEmpty():not()} that will filter out all the
> empty SSNs, then connect RouteOnAttribute's matched relationship to the
> HBase processors.
>
> Lets get the inserting side working and then I would expect the GetHBase
> to work since they use the same underlying connection.
>
> Thanks,
>
> Bryan
>
>
> On Thu, May 5, 2016 at 10:59 AM, Venkatesh Bodapati <
> venkatesh.bodap...@inndata.in> wrote:
>
>> I am working on "Fun_With_Hbase.xml" Template. In this i will send data
>> to hbase table, but i will not get data from hbase. I will get the "failed
>> to receive data from hbase due to java net connection exception :Connection
>> Refused" error in GetHbase Processor and In PutHbaseCell,PutHbaseJson
>> Processor i got error Like This " Missing Row id failure, routing to
>> failure ".
>>
>> This is my HbaseClient Properties :
>>
>> Hadoop Configuration FilesInfo  :
>> /usr/local/hadoop/hadoop-2.6.0/etc/hadoop/core-site.xml,/usr/local/hadoop/hadoop-2.6.0/etc/hadoop/hdfs-site.xml
>> ZooKeeper QuorumInfo: localhost:2181
>> ZooKeeper Client PortInfo: 2181
>> ZooKeeper ZNode ParentInfo   : /hbase
>> HBase Client RetriesInfo   : 1
>>
>>
>> How do i get data from Hbase Table in Nifi,Any suggesions please.
>>
>
>

Re: Failed to receive data from Hbase due to java net connection exception :Connection Refused in Nifi

[Bryan Bende]

Hello,

For HBaseClient, are you sure your ZooKeeper being used by HBase is running
on localhost:2181?

Typically you don't really need to set the three ZooKeeper properties, and
you can instead just set the hbase-site.xml in the config resources.

For example, my Hadoop Configuration Resources is set as:
/etc/hadoop/conf/core-site.xml,/etc/hbase/conf/hbase-site.xml

And then I don't have anything specified for ZooKeeper QuorumInfo, etc.,
because hbase-site.xml is providing all that information.

The example template requires a Users table, so from Hbase shell:
create 'Users', {NAME => 'cf'}

Then the template has a slight problem where it is using EvaluteJsonPath to
extract an attribute called "ssn" and using that for the row id... the
problem is a lot of the ssn values are empty so they are failing.

If you stick a RouteOnAttribute right after EvaluateJsonPath, and add a
property "matched" = ${ssn:isEmpty():not()} that will filter out all the
empty SSNs, then connect RouteOnAttribute's matched relationship to the
HBase processors.

Lets get the inserting side working and then I would expect the GetHBase to
work since they use the same underlying connection.

Thanks,

Bryan


On Thu, May 5, 2016 at 10:59 AM, Venkatesh Bodapati <
venkatesh.bodap...@inndata.in> wrote:

> I am working on "Fun_With_Hbase.xml" Template. In this i will send data to
> hbase table, but i will not get data from hbase. I will get the "failed to
> receive data from hbase due to java net connection exception :Connection
> Refused" error in GetHbase Processor and In PutHbaseCell,PutHbaseJson
> Processor i got error Like This " Missing Row id failure, routing to
> failure ".
>
> This is my HbaseClient Properties :
>
> Hadoop Configuration FilesInfo  :
> /usr/local/hadoop/hadoop-2.6.0/etc/hadoop/core-site.xml,/usr/local/hadoop/hadoop-2.6.0/etc/hadoop/hdfs-site.xml
> ZooKeeper QuorumInfo: localhost:2181
> ZooKeeper Client PortInfo: 2181
> ZooKeeper ZNode ParentInfo   : /hbase
> HBase Client RetriesInfo   : 1
>
>
> How do i get data from Hbase Table in Nifi,Any suggesions please.
>

Failed to receive data from Hbase due to java net connection exception :Connection Refused in Nifi

[Venkatesh Bodapati]

I am working on "Fun_With_Hbase.xml" Template. In this i will send data to
hbase table, but i will not get data from hbase. I will get the "failed to
receive data from hbase due to java net connection exception :Connection
Refused" error in GetHbase Processor and In PutHbaseCell,PutHbaseJson
Processor i got error Like This " Missing Row id failure, routing to
failure ".

This is my HbaseClient Properties :

Hadoop Configuration FilesInfo  :
/usr/local/hadoop/hadoop-2.6.0/etc/hadoop/core-site.xml,/usr/local/hadoop/hadoop-2.6.0/etc/hadoop/hdfs-site.xml
ZooKeeper QuorumInfo: localhost:2181
ZooKeeper Client PortInfo: 2181
ZooKeeper ZNode ParentInfo   : /hbase
HBase Client RetriesInfo   : 1


How do i get data from Hbase Table in Nifi,Any suggesions please.