[
https://issues.apache.org/jira/browse/NIFI-4866?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16359693#comment-16359693
]
ASF GitHub Bot commented on NIFI-4866:
--------------------------------------
Github user ijokarumawak commented on a diff in the pull request:
https://github.com/apache/nifi/pull/2464#discussion_r167416142
--- Diff:
nifi-nar-bundles/nifi-standard-services/nifi-hbase_1_1_2-client-service-bundle/nifi-hbase_1_1_2-client-service/src/main/java/org/apache/nifi/hbase/HBase_1_1_2_ClientService.java
---
@@ -549,11 +551,7 @@ public String toTransitUri(String tableName, String
rowKey) {
logger.warn("Connection has not been established, could not
create a transit URI. Returning null.");
return null;
}
- try {
- final String masterAddress =
connection.getAdmin().getClusterStatus().getMaster().getHostAndPort();
- return "hbase://" + masterAddress + "/" + tableName + (rowKey
!= null && !rowKey.isEmpty() ? "/" + rowKey : "");
- } catch (IOException e) {
- throw new RuntimeException("Failed to get HBase Admin
interface, due to " + e, e);
- }
+ final String transitUriMasterAddress =
StringUtils.isEmpty(masterAddress) ? "unknown" : masterAddress;
--- End diff --
@MikeThomsen That would make provenance info more accurate. I think that
feature will be helpful when the HBase master is failed over.
Again, it depends on the other program those uses the provenance event how
it is important. For the particular use-case of NiFi and Atlas integration, it
is sufficient to know which HBase 'cluster' NiFi is talking to. If there are
multiple master addresses used for HA purpose, then any of those would be
enough to identify a cluster. E.g. cluster A (master1 and master2) and cluster
B (master3 and 4), then it's not significant if it is master1 or 2 to tell NiFi
is accessing cluster A.
I agree with your suggestion to make this more accurate, and good to have.
But I'd prefer to do that separately. Would you create another JIRA if you
don't mind? Thank you!
> HBase_1_1_2_ClientService performance dropped due to accessing HBase admin
> upon every FlowFile transfer
> -------------------------------------------------------------------------------------------------------
>
> Key: NIFI-4866
> URL: https://issues.apache.org/jira/browse/NIFI-4866
> Project: Apache NiFi
> Issue Type: Bug
> Components: Extensions
> Reporter: Koji Kawamura
> Assignee: Koji Kawamura
> Priority: Critical
>
> NIFI-4543 added
> "connection.getAdmin().getClusterStatus().getMaster().getHostAndPort()” call
> from HBase_1_1_2_ClientService.toTransitUri method in order to get HBase
> master address, which is called upon every FlowFile transfer and essentially
> doubles the traffic through the HBase connector.
> The performance of our PutHBaseJSON processor dropped to 1/3 after deploying
> NiFi 1.5.0.
> Those lines of code should be moved from toTransitUri to createConnection.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
