Re: nifi help

Your error log didn't appear to go through (at least not for me), is it possible to paste the relevant log information as text, or provide a link to a page where we can view it? Regards, Matt On Mon, Jul 3, 2017 at 10:10 PM, 朱志明 wrote: > hello, > > nifi error log: > > > > > >

nifi help

hello, nifi error log: but how to solve

Pull request for PutHBaseRecord

https://github.com/apache/nifi/pull/1961 If anyone gets some time to review it, I would appreciate it. My boss wants to see this returned to the community, so I have some latitude to work on any feedback that I get during the day so we can close this out. Thanks, Mike

Re: [EXT] Re: NiFi Throughput and Slowness

Is there a reason your using a RAID-5 Array? Write speed on RAID-5 is greatly influenced by the controller itself. We typically do setups using RAID-10 today now for better read/write performance. The 'parity' bits of RAID5 cause a number of re-writes & re-reads (4 iops) before it writes out the

RE: [EXT] Re: NiFi Throughput and Slowness

Joe, Thanks for your inputs, I will for sure try all these before I start clustering the instance. I will keep the community updated on my results. Thanks Karthik -Original Message- From: Joe Witt [mailto:joe.w...@gmail.com] Sent: Monday, July 03, 2017 12:30 PM To: dev@nifi.apache.org

RE: [EXT] Re: NiFi Throughput and Slowness

Karthik It is really important to follow best practice configuration for placement of repos to your underlying storage. Configured well you can have hundreds of MB/s sustained throughput per node. Also be sure to take advantage of the record reader / writer capability if appropriate for your

Re: NiFi Throughput and Slowness

Karthik, This is my go to article for high performance tuning and a first read for most. It still applies for newer releases of Apache NIFI. [1] You should also consider looking at your disk queuing algorithm strategy depending on if you are on physical servers or not. If on physical and using

RE: [EXT] Re: NiFi Throughput and Slowness

Rick, Thanks a lot for the suggestion, clustering is something that even I was thinking of for a long time. Just wanted to see if anyone in the community have similar problems and solutions they found. -Karthik -Original Message- From: Richard St. John [mailto:rstjoh...@gmail.com]

Re: NiFi Throughput and Slowness

Hi there, In the beginning of our NiFi adoption, we faced similar issues. For us, we clustered NiFi, limited the number of concurrent tasks for each processor and added more logical partitions for content and provenance repositories. Now, we easily processor million of flow files per minute on

NiFi Throughput and Slowness

All, I am currently using NiFi 1.2.0 on a Linux (RHEL) machine. I am using a single instance without any clustering. My machine has ~800GB of RAM and 2.5 TB of disk space (SSD's with RAID5). I have set my Java heap space values to below in "bootstrap.conf" file # JVM memory settings