[
https://issues.apache.org/jira/browse/NIFI-4866?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16359351#comment-16359351
]
ASF GitHub Bot commented on NIFI-4866:
--------------------------------------
Github user MikeThomsen commented on a diff in the pull request:
https://github.com/apache/nifi/pull/2464#discussion_r167394630
--- Diff:
nifi-nar-bundles/nifi-standard-services/nifi-hbase_1_1_2-client-service-bundle/nifi-hbase_1_1_2-client-service/src/main/java/org/apache/nifi/hbase/HBase_1_1_2_ClientService.java
---
@@ -549,11 +551,7 @@ public String toTransitUri(String tableName, String
rowKey) {
logger.warn("Connection has not been established, could not
create a transit URI. Returning null.");
return null;
}
- try {
- final String masterAddress =
connection.getAdmin().getClusterStatus().getMaster().getHostAndPort();
- return "hbase://" + masterAddress + "/" + tableName + (rowKey
!= null && !rowKey.isEmpty() ? "/" + rowKey : "");
- } catch (IOException e) {
- throw new RuntimeException("Failed to get HBase Admin
interface, due to " + e, e);
- }
+ final String transitUriMasterAddress =
StringUtils.isEmpty(masterAddress) ? "unknown" : masterAddress;
--- End diff --
@ijokarumawak I think a timer that asks the cluster for the current master
on a configurable interval would be good to have here. It could have a default
of 5-15 minutes. I skimmed through the Java API for HBase and didn't see
anything that we can use for that, but we should periodically make sure the
current master info is correct so the provenance info is correct(ish).
> HBase_1_1_2_ClientService performance dropped due to accessing HBase admin
> upon every FlowFile transfer
> -------------------------------------------------------------------------------------------------------
>
> Key: NIFI-4866
> URL: https://issues.apache.org/jira/browse/NIFI-4866
> Project: Apache NiFi
> Issue Type: Bug
> Components: Extensions
> Reporter: Koji Kawamura
> Assignee: Koji Kawamura
> Priority: Critical
>
> NIFI-4543 added
> "connection.getAdmin().getClusterStatus().getMaster().getHostAndPort()” call
> from HBase_1_1_2_ClientService.toTransitUri method in order to get HBase
> master address, which is called upon every FlowFile transfer and essentially
> doubles the traffic through the HBase connector.
> The performance of our PutHBaseJSON processor dropped to 1/3 after deploying
> NiFi 1.5.0.
> Those lines of code should be moved from toTransitUri to createConnection.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
