Hi Adam,

Thank you very much for reporting the performance issue.
I created NIFI-4866 and started fixing the issue by moving the
problematic code block to createConnection.
After confirming that addresses performance issue, I will send a PR to
get it merged.

Koji


On Sat, Feb 10, 2018 at 9:25 AM, Joe Witt <joe.w...@gmail.com> wrote:
> adam
>
> you should also be able to put the old hbase nar in and switch to that
> version.
>
> we now support multiple versions of the same component.
>
> thanks
>
> On Feb 9, 2018 7:10 PM, "Mike Thomsen" <mikerthom...@gmail.com> wrote:
>
>> Adam,
>>
>> If you're doing bulk ingestion of JSON, I would recommend using
>> PutHBaseRecord. I wrote it/contributed it when my team ran into similar
>> limitations doing genomic data ingestion (several 10s of billions of Puts
>> from the 1000 genomes project). If you run into problems with it, just post
>> them and poke me.
>>
>> Mike
>>
>> On Fri, Feb 9, 2018 at 6:56 PM, Joe Witt <joe.w...@gmail.com> wrote:
>>
>> > adam
>> >
>> > thanks for reporting and if you can do a contrib that would be great!
>> >
>> > thanks
>> > joe
>> >
>> > On Feb 9, 2018 6:56 PM, "Martini, Adam" <adam.mart...@nike.com> wrote:
>> >
>> > > Hello NiFi Dev Community,
>> > >
>> > > This commit hash (part of the NiFi 1.5.0 release) created serious
>> > > performance issues for HBase Put operations: "
>> > > 116c8463428c1fb51bfb7a8adfcf23c32fded964".
>> > >
>> > > The override of the “toTransitUri” method makes a call to
>> > > “connection.getAdmin().getClusterStatus().getMaster()
>> .getHostAndPort()”
>> > > upon every flow file transfer, which essentially doubles the traffic
>> > > through the HBase connector.  The performance of our PutHBaseJSON
>> > processor
>> > > dropped to 1/3 after deploying NiFi 1.5.0.
>> > >
>> > > Please let us know a timeline for a fix.  We are building and testing
>> our
>> > > own tar ball in the interim to fix the issue and are happy to
>> contribute
>> > > our code back to the project if you would like.
>> > >
>> > > All the best and thank you.
>> > >
>> > > Adam Martini
>> > > Senior Developer, Nike Digital
>> > >
>> > >
>> > >
>> >
>>

Reply via email to