Re: Is this possible to increase speed of processors in NiFi?

2017-02-10 Thread Joe Witt
For processors that support it consider using a 'run duration of 25 milliseconds'. This allows the framework to batch many operations into fewer transactions which can dramatically increase throughput. 100% memory usage is a problem. Increase heap size or consider redesign/simplifying your flow.

Is this possible to increase speed of processors in NiFi?

2017-02-10 Thread prabhu Mahendran
Since each processor have concurrent tasks if i have set the concurrent tasks for processors then it boosts processing speed of processors.But it affects the System performance such as 100% Disk Usage,100% Memory Usage..etc Is there is any other way to speed up processors without use concurrent ta

Re: Custom Processor with Hive database connection pooling service not working

2017-02-10 Thread Nijandan D
Hi Bryan, Thanks for your update. It's working fine. Thank you so much. Regards, Nijandan D On Fri, Feb 10, 2017 at 7:21 PM, Bryan Bende wrote: > Hello, > > In general, to link a processor to a controller service, you need two > dependencies as shown in the wiki page in the previous email: >

Re: Custom Processor with Hive database connection pooling service not working

2017-02-10 Thread Bryan Bende
Hello, In general, to link a processor to a controller service, you need two dependencies as shown in the wiki page in the previous email: - The processor project needs a provided dependency on the JAR where the service interface is - The NAR project needs a NAR dependency on the NAR where the se

Re: ListHDFS ( some suggestions depending on our usage )

2017-02-10 Thread Alessio Palma
Hello Pierre, card has been opened, maybe I'll will send you the patch so it can be added in the next release. I'm working on it in local. From: Pierre Villard Sent: Friday, February 10, 2017 10:21:28 AM To: users@nifi.apache.org Subject: Re: ListHDFS ( some sugg

Re: ListHDFS ( some suggestions depending on our usage )

2017-02-10 Thread Pierre Villard
Hi Alessio, It sounds like a valid improvement on the existing processor. Please feel free to file a JIRA. Since the directory path is already allowing expression language it wouldn't be a huge change. - Pierre 2017-02-10 9:58 GMT+01:00 Alessio Palma : > Hello all, > I'm using ListHDFS but...

ListHDFS ( some suggestions depending on our usage )

2017-02-10 Thread Alessio Palma
Hello all, I'm using ListHDFS but... 1) it can start only on a schedule strategy 2) directory path to list is "hard coded" into the processor it self. I wonder if a new version, which can be started using a flowfile, accept the attributes of the incoming flow file as parameters, it is more usabl