Implementation of ListFile's Primary Node only in a cluster

2018-02-09 Thread Sivaprasanna
I was going through ListFile processor's code and found out that in the documentation , it is mentioned that "this

Re: NiFi 1.5.0 HBase_1_1_2_ClientService performance bug

2018-02-09 Thread Koji Kawamura
Hi, The PR is ready for review. I confirmed that performance issue is addressed. https://github.com/apache/nifi/pull/2464 I was also testing to see if the nifi-hbase_1_1_2-client-service-nar-1.6.0-SNAPSHOT.nar can be used in NiFi 1.5.0 env. But unfortunately it doesn't seem we can put it as it

Re: NiFi 1.5.0 HBase_1_1_2_ClientService performance bug

2018-02-09 Thread Koji Kawamura
Hi Adam, Thank you very much for reporting the performance issue. I created NIFI-4866 and started fixing the issue by moving the problematic code block to createConnection. After confirming that addresses performance issue, I will send a PR to get it merged. Koji On Sat, Feb 10, 2018 at 9:25

Re: NiFi 1.5.0 HBase_1_1_2_ClientService performance bug

2018-02-09 Thread Joe Witt
adam you should also be able to put the old hbase nar in and switch to that version. we now support multiple versions of the same component. thanks On Feb 9, 2018 7:10 PM, "Mike Thomsen" wrote: > Adam, > > If you're doing bulk ingestion of JSON, I would recommend

Re: NiFi 1.5.0 HBase_1_1_2_ClientService performance bug

2018-02-09 Thread Mike Thomsen
Adam, If you're doing bulk ingestion of JSON, I would recommend using PutHBaseRecord. I wrote it/contributed it when my team ran into similar limitations doing genomic data ingestion (several 10s of billions of Puts from the 1000 genomes project). If you run into problems with it, just post them

Re: NiFi 1.5.0 HBase_1_1_2_ClientService performance bug

2018-02-09 Thread Joe Witt
adam thanks for reporting and if you can do a contrib that would be great! thanks joe On Feb 9, 2018 6:56 PM, "Martini, Adam" wrote: > Hello NiFi Dev Community, > > This commit hash (part of the NiFi 1.5.0 release) created serious > performance issues for HBase Put

NiFi 1.5.0 HBase_1_1_2_ClientService performance bug

2018-02-09 Thread Martini, Adam
Hello NiFi Dev Community, This commit hash (part of the NiFi 1.5.0 release) created serious performance issues for HBase Put operations: "116c8463428c1fb51bfb7a8adfcf23c32fded964". The override of the “toTransitUri” method makes a call to

Re: Trouble with setting up SSL connecting MiNiFi to NiFi

2018-02-09 Thread Marc
Jonah, Sorry for the double reply. https://issues.apache.org/jira/browse/MINIFICPP-396 is the ticket corresponding to that PR. It may address your issue depending no whether or not you are currently using the Context service to configure the RPG. It may also address another issue preventing

Re: Trouble with setting up SSL connecting MiNiFi to NiFi

2018-02-09 Thread Marc
Jonah, There is a pull request to address some of the configuration of these objects since there were two routes to configure it ( Context Service and minifi.properties ) : https://github.com/apache/nifi-minifi-cpp/pull/263 ; however, I'm not sure this is your issues since the

Trouble with setting up SSL connecting MiNiFi to NiFi

2018-02-09 Thread Jonah Husson
Hey All, Figured I'd shoot off an email before looking into issue reporting, in case this is a product of my own stupidity rather than an actual bug. I'm trying to get MiNiFi communicating with a NiFi cluster on an internal network running with SSL. I'm able to connect to NiFi from a web

Re: Will you accept contributions in Scala?

2018-02-09 Thread Andy LoPresto
Hi Adam, It’s great that your team wants to contribute to NiFi. I think there should be some discussion from other community members on this, but I’ll give my view here. At this time, I believe we should only accept core contributions in Java. It’s not because I have a predisposition against

Will you accept contributions in Scala?

2018-02-09 Thread Weiss, Adam
Devs, I have some interns starting with my team and we use Scala internally for our work. If I wanted to have them work to contribute some new processors, would they have to be Java to be included with the base distribution or could they use Scala as long as it fit within your current build