This brings up NiFi best practices question: whether a developer can use spawn threads in processors or controller services? If there are any guidelines , it would be nice to document in developer guide. Sumanth
Sent from my iPad > On Oct 26, 2015, at 8:14 AM, Oleg Zhurakousky <[email protected]> > wrote: > > Unfortunately I can’t seem to even see how it would be possible for NiFi to > tell you that since your custom Processors are running within the same JVM as > NiFi. > Having said that the 800% tells me that you probably have some processor with > custom thread pool where each thread is spinning in the loop with a lot of > misses on the functionality it expects to perform. > For example: > while (true) { > if (someCondition){ > // do something > } > } > The above will definitely eat 100% of your CPU if ’someCondition’ never > happens and if you have something like this running in multiple threads on 8 > cores there is your 800%. > That could be your code or some library you are using. > > There is also a slight chance that your the code executed by multiple threads > is actually doing something very CPU intensive > > Hope that helps > > Oleg > > >> On Oct 26, 2015, at 10:57 AM, Elli Schwarz <[email protected]> wrote: >> >> Hello, >> >> We have a nifi flow with many custom processors (and many processor groups). >> We suspect that one or more processors are eating up CPU usage, so we're >> wondering if there's an easy way to tell which processor has a heavy load on >> the CPU. There are tables to see processors in order of number of flow files >> or bytes in/out, etc, but not based on CPU usage. In fact, I can't find a >> way to see a table of all processors that have active threads. All that we >> know is that the top command has nifi running at 800%, and we're doing trial >> and error by turning off processors until we hit the one that makes CPU >> utilization go down. >> >> We did see an earlier post about processors that poll can be eating up CPU >> cycles, but that doesn't seem to be the case here. Once in the past we had a >> custom processor with a bug that caused it to eat CPU cycles, but we >> discovered the issue not through Nifi but because we happened to be >> examining the code. >> >> Thank you! >> >> -Elli >
