Shanker It sounds like you've gone through some changes in general and have worked through those. Now you have a flow running with a high volume of data (history load) and want to know which parts of the flow are most expensive/consuming the CPU.
You should be able to look at the statistics provided on the processors to see where the majority of CPU time is spent. You can usually very easily reason over this if it is doing compression/encryption/etc.. and determine if you want to give it more threads/less threads/batch data together better, etc.. The configuration of the VMs, the NiFi instance itself, the flow, and the nature of the data are all important to see/understand to be of much help here. THanks On Fri, Jun 7, 2019 at 7:07 AM Shanker Sneh <shanker.s...@zoomcar.com> wrote: > Hello all, > > I am facing strange issue with NiFi 1.8.0 (2 nodes) > My flows had been running fine since months. > > Yesterday I had to do some history load which filled up my both disks (I > have FlowFile repository as separate disk). > > I increased the size of the root & flowflile disk both. And 'grow' the > disk partition and 'extended' the file system (it's an EC2 linux). > But post that my CPU has been spiking to complete 100% - even at regular > load (earlier it used to be somewhere around 50%) > Also I did no change to the config values or thread count etc. > > I upgraded the 2 nodes to see if that solves the problem - from 16 Gb box > (4 core) to 64 Gb (16 core). > But even the larger box is throttling on the CPU at 100%. > > I tried clearing all repositories and restarted NiFi application and the > EC2 - but no improvement. > > Kindly point me in the right direction. I am unable to pinpoint anything. > > -- > Best, > Sneh >
