Re: Re: NiFi 1.5.0 HBase_1_1_2_ClientService performance bug

2018-02-12 Thread Martini, Adam
Hi Koji and all other who responded, Thanks for getting this PR out so quickly. Your original response chain was sent to the CC address (Maxwell Eng) so I only now got this email. I made on comment on your PR, but otw it was the same fix I made to the AMI we are testing now with great performa

Re: NiFi 1.5.0 HBase_1_1_2_ClientService performance bug

2018-02-09 Thread Koji Kawamura
Hi, The PR is ready for review. I confirmed that performance issue is addressed. https://github.com/apache/nifi/pull/2464 I was also testing to see if the nifi-hbase_1_1_2-client-service-nar-1.6.0-SNAPSHOT.nar can be used in NiFi 1.5.0 env. But unfortunately it doesn't seem we can put it as it is

Re: NiFi 1.5.0 HBase_1_1_2_ClientService performance bug

2018-02-09 Thread Koji Kawamura
Hi Adam, Thank you very much for reporting the performance issue. I created NIFI-4866 and started fixing the issue by moving the problematic code block to createConnection. After confirming that addresses performance issue, I will send a PR to get it merged. Koji On Sat, Feb 10, 2018 at 9:25 AM

Re: NiFi 1.5.0 HBase_1_1_2_ClientService performance bug

2018-02-09 Thread Joe Witt
adam you should also be able to put the old hbase nar in and switch to that version. we now support multiple versions of the same component. thanks On Feb 9, 2018 7:10 PM, "Mike Thomsen" wrote: > Adam, > > If you're doing bulk ingestion of JSON, I would recommend using > PutHBaseRecord. I wro

Re: NiFi 1.5.0 HBase_1_1_2_ClientService performance bug

2018-02-09 Thread Mike Thomsen
Adam, If you're doing bulk ingestion of JSON, I would recommend using PutHBaseRecord. I wrote it/contributed it when my team ran into similar limitations doing genomic data ingestion (several 10s of billions of Puts from the 1000 genomes project). If you run into problems with it, just post them a

Re: NiFi 1.5.0 HBase_1_1_2_ClientService performance bug

2018-02-09 Thread Joe Witt
adam thanks for reporting and if you can do a contrib that would be great! thanks joe On Feb 9, 2018 6:56 PM, "Martini, Adam" wrote: > Hello NiFi Dev Community, > > This commit hash (part of the NiFi 1.5.0 release) created serious > performance issues for HBase Put operations: " > 116c8463428c

NiFi 1.5.0 HBase_1_1_2_ClientService performance bug

2018-02-09 Thread Martini, Adam
Hello NiFi Dev Community, This commit hash (part of the NiFi 1.5.0 release) created serious performance issues for HBase Put operations: "116c8463428c1fb51bfb7a8adfcf23c32fded964". The override of the “toTransitUri” method makes a call to “connection.getAdmin().getClusterStatus().getMaster().ge