ASF GitHub Bot commented on NIFI-4866:

Github user ijokarumawak commented on a diff in the pull request:

    --- Diff: 
    @@ -549,11 +551,7 @@ public String toTransitUri(String tableName, String 
rowKey) {
                 logger.warn("Connection has not been established, could not 
create a transit URI. Returning null.");
                 return null;
    -        try {
    -            final String masterAddress = 
    -            return "hbase://" + masterAddress + "/" + tableName + (rowKey 
!= null && !rowKey.isEmpty() ? "/" + rowKey : "");
    -        } catch (IOException e) {
    -            throw new RuntimeException("Failed to get HBase Admin 
interface, due to " + e, e);
    -        }
    +        final String transitUriMasterAddress = 
StringUtils.isEmpty(masterAddress) ? "unknown" : masterAddress;
    --- End diff --
    @zenfenan Thanks for your comment. Yes, it would be a problem if this 
actually happens. But it only happens when the ControllerService was not able 
to get the master address when it was enabled. I think HBase processors will 
not be able to access HBase, too.
    It depends on the programs how critical this can be, for those use the 
returned transit URI later. For example, NiFi and Atlas integration case, 
ReportLineageToAtlas reporting task will not be able to determine the right 
cluster name in Atlas.
    Please let me know any situation that the ControllerServer can connect to a 
HBase, but can not get its master address. I write the code that way just to be 
more protective, but not sure if that actually happens.

> HBase_1_1_2_ClientService performance dropped due to accessing HBase admin 
> upon every FlowFile transfer
> -------------------------------------------------------------------------------------------------------
>                 Key: NIFI-4866
>                 URL: https://issues.apache.org/jira/browse/NIFI-4866
>             Project: Apache NiFi
>          Issue Type: Bug
>          Components: Extensions
>            Reporter: Koji Kawamura
>            Assignee: Koji Kawamura
>            Priority: Critical
> NIFI-4543 added 
> "connection.getAdmin().getClusterStatus().getMaster().getHostAndPort()” call 
> from HBase_1_1_2_ClientService.toTransitUri method in order to get HBase 
> master address, which is called upon every FlowFile transfer and essentially 
> doubles the traffic through the HBase connector.
> The performance of our PutHBaseJSON processor dropped to 1/3 after deploying 
> NiFi 1.5.0.
> Those lines of code should be moved from toTransitUri to createConnection.

This message was sent by Atlassian JIRA

Reply via email to