Another thing to look would be to check if you are creating two much flow files as this hammers the disks. If you see very high memory usage it could be you are having to much data in attributes and not in the content of the flow file.
> On 8. Feb 2021, at 07:07, [email protected] wrote: > > Hi Asmath, > > In terms of performance tuning, I’ve found a lot that needs to be considered. > > I’ve done tuning it’s mainly been caused large queues building due to the > read-write queues on the content repo disks being very high. I’ve also had > previous threads on the JOLT default transformation, which should be avoided > at all costs as it’s not performant. > > If you are confident the performance bottleneck isn’t the hardware in its > cluster itself, I would look at of tuning threads on the flow. I can > recommend looking at Mark Payne’s NiFi Notes Videos on Youtube: > https://www.youtube.com/channel/UCcNsK56MZRXzPbRfe8sODWw > <https://www.youtube.com/channel/UCcNsK56MZRXzPbRfe8sODWw> > > Bear in mind, that you may need to go round the loop of tuning the hardware > and the flow to meet your requirements. > > Kind Regards, > > Nathan > From: KhajaAsmath Mohammed [mailto:[email protected]] > Sent: 08 February 2021 08:22 > To: [email protected] > Subject: NIFI - Performance issues > > Hi, > > I am looking for some information on how to improve performance of our flows. > Any suggestions? > > 1. how to troubleshoot which processor in the nifi is causing an issue? > 2. What is the best way to increase the threads or tasks for a particular > processor? > 3. How to debug the flow when there is an issue on a particular processor > group? > > Thanks, > Asmath
