[jira] [Commented] (NIFI-4866) HBase_1_1_2_ClientService performance dropped due to accessing HBase admin upon every FlowFile transfer

[ASF GitHub Bot (JIRA)]

    [ 
https://issues.apache.org/jira/browse/NIFI-4866?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16359211#comment-16359211
 ] 

ASF GitHub Bot commented on NIFI-4866:
--------------------------------------

Github user ijokarumawak commented on the issue:

    https://github.com/apache/nifi/pull/2464
  
    Confirmed that transit URI is created correctly:
    - 1.5 
hbase://0.hdp.aws.mine:16000/nifi-4866/a20f31cb-f503-4e0c-9806-73a2ed7fd43e
    - 1.6 
hbase://0.hdp.aws.mine:16000/nifi-4866/93376ca2-ec3c-421f-8e4e-93cf28677b31
    
    Confirmed performance issue has been addressed:
    
    - Used a simple GenerateFlowFile and PutHBaseCell flow, template is 
available 
[here](https://gist.github.com/ijokarumawak/85db60ca71f1825f543c18c62bf7c3fd)
    - Putting only 1 cell with 4bytes data, batched size is 25
    - 1.5 saturated about at the rate of 14k FlowFIles/5min
    - 1.6 saturated about at the rate of 500k FlowFiles/5min
    
    NiFi 1.5.0
    
![image](https://user-images.githubusercontent.com/1107620/36057769-55896c0c-0e56-11e8-98ff-f3464f6dd736.png)
    
    NiFi 1.6.0-SNAPSHOT with this fix
    
![image](https://user-images.githubusercontent.com/1107620/36057771-5d46e32a-0e56-11e8-9f95-542e835361dc.png)



> HBase_1_1_2_ClientService performance dropped due to accessing HBase admin 
> upon every FlowFile transfer
> -------------------------------------------------------------------------------------------------------
>
>                 Key: NIFI-4866
>                 URL: https://issues.apache.org/jira/browse/NIFI-4866
>             Project: Apache NiFi
>          Issue Type: Bug
>          Components: Extensions
>            Reporter: Koji Kawamura
>            Assignee: Koji Kawamura
>            Priority: Critical
>
> NIFI-4543 added 
> "connection.getAdmin().getClusterStatus().getMaster().getHostAndPort()” call 
> from HBase_1_1_2_ClientService.toTransitUri method in order to get HBase 
> master address, which is called upon every FlowFile transfer and essentially 
> doubles the traffic through the HBase connector.
> The performance of our PutHBaseJSON processor dropped to 1/3 after deploying 
> NiFi 1.5.0.
> Those lines of code should be moved from toTransitUri to createConnection.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)