Pompillio Keep in mind too you can look at that summary page and it will give you a tabular listing of various types of components. You can sort processors by which is using the most threads (at that very moment anyway), or has run the most tasks over 5 minutes or taken the most time over 5 minutes. That helps you figure out the hungry CPU users.
You can also sort by bytes written/read to get a sense of hungry IO users. We don't help you figure out memory consumers yet but that is generally because that problem is much different in NiFi given that a processor has access to an underlying input stream and output stream rather than being stuck handling massive byte arrays. Knowing the top users you can then do things like decide whether you have the sufficient or optimal installation in terms of number of disks, partitions, CPU usage and total threads, etc.. Thanks for the kind comment. We all definitely appreciate those and that user experience you suggested is what we're shooting for. Thanks Joe On Wed, Dec 2, 2015 at 12:15 PM, Pompilio Ramirez <[email protected]> wrote: > Oleg, > > Thank you for reassurance, I was hoping for a magic formula but figured its > based on unique system and probably trial and error. > > The UI system setting question was more an observation rather than a > complaint. > > NIFI has rocked my world and it is greatly making our lives easier here. > > On Wed, Dec 2, 2015 at 12:07 PM, Oleg Zhurakousky > <[email protected]> wrote: >> >> Pompilio >> >> With regard to Properties vs UI, I’ll defer to others to answer as it is >> indeed debatable. >> With regard to best practices, there is no static answer, but i think you >> are on the right track when mentioning RAM,CPU etc. >> The way I would look at it in general is by looking ay what type of >> flow(s) you are running within a given instance, analyze them and see if >> higher levels of concurrency adds value since in a lot of cases adding more >> threads will actually kill the performance rather then give it a boost. For >> example, if your flow is CPU intensive then setting value higher then the >> amount of cores you have won’t produce anything good and in fact may have a >> negative impact on your performance as it may introduce unnecessary context >> switching. On the flip side if you are dealing with the flow(s) that is >> let’s say consist of a lot of IO intensive operations, then increasing >> concurrency most likely will help. >> So its a dance where the answer could also be as simple as saying “I want >> to limit NiFi to no more then 5 threads”. >> >> That would be my 2cents >> >> Cheers >> Oleg >> >> >> > On Dec 2, 2015, at 11:20 AM, Pompilio Ramirez <[email protected]> >> > wrote: >> > >> > All, >> > >> > As I was working on my flow I noticed that things worked slowly as I >> > looked around I noticed that the is a UI setting: >> > >> > "maximum timer driven thread count" >> > >> > I increased these numbers to 50 and my flows worked much better. >> > >> > I tried to look around but could not find more documentation on these >> > setting ( I could have overlooked them sorry ) >> > >> > I am wondering if there is a recommended setting? I cant remember >> > default value but increasing the setting on UI helped however not sure how >> > to set that at install .... >> > >> > I need to take in to consideration my RAM and CPU and my JVM max ram, >> > but just wondering what you all use as a benchmark? >> > >> > Also why is this setting only controlled by the UI, I assumed all >> > settings would be configured at the nifi.properties file and was surprised >> > to find this valuable nifi as a whole setting only configurable from the >> > UI. >> > >> > >> > Thanks >> >
