Re: Re: NiFi active thread count is no more than 10 ?
Thanks very much Bryan, chaning the overall timer-driven thread pool value works. wangl...@geekplus.com.cn From: Bryan Bende Date: 2019-09-18 20:53 To: users Subject: Re: NiFi active thread count is no more than 10 ? The overall timer-driven thread pool defaults to 10 (configured from the controller settings in top right menu). So even if a processor has 100 concurrent tasks, there are still only 10 threads available. On Wed, Sep 18, 2019 at 8:20 AM Joe Witt wrote: > > Hello > > The 100 threads for the controller overall is the maximum number of threads > that could run concurrently. On a 16 core system and a flow which is very I/O > bound this is definitely achievable. Generally you want to look at some > multiple of the number of physical cores such as 2,4,8, etc.. but in the end > it isn't that important. > > So why do you only see at most 10 or so threads in active use? Generally > this means your flow isn't demanding or configured to do more. > > How many processors do you have? How many tasks does each have? How much > data is flowing through the system when it reaches 10? Are backlogs growing > at that time? What are the run schedules? How does the load average on the > system look? How does IO utilization/iowait look during those times? > > Thanks > Joe > > On Wed, Sep 18, 2019 at 8:05 AM wangl...@geekplus.com.cn > wrote: >> >> >> My NiFi server is 16 cores. I also configed some processor cocurrent tasks >> to 100. >> But why the active thread count shown on the NiFi web ui is no more than 10? >> >> >> >> wangl...@geekplus.com.cn
Re: NiFi active thread count is no more than 10 ?
The overall timer-driven thread pool defaults to 10 (configured from the controller settings in top right menu). So even if a processor has 100 concurrent tasks, there are still only 10 threads available. On Wed, Sep 18, 2019 at 8:20 AM Joe Witt wrote: > > Hello > > The 100 threads for the controller overall is the maximum number of threads > that could run concurrently. On a 16 core system and a flow which is very I/O > bound this is definitely achievable. Generally you want to look at some > multiple of the number of physical cores such as 2,4,8, etc.. but in the end > it isn't that important. > > So why do you only see at most 10 or so threads in active use? Generally > this means your flow isn't demanding or configured to do more. > > How many processors do you have? How many tasks does each have? How much > data is flowing through the system when it reaches 10? Are backlogs growing > at that time? What are the run schedules? How does the load average on the > system look? How does IO utilization/iowait look during those times? > > Thanks > Joe > > On Wed, Sep 18, 2019 at 8:05 AM wangl...@geekplus.com.cn > wrote: >> >> >> My NiFi server is 16 cores. I also configed some processor cocurrent tasks >> to 100. >> But why the active thread count shown on the NiFi web ui is no more than 10? >> >> >> >> wangl...@geekplus.com.cn
Re: NiFi active thread count is no more than 10 ?
Hello The 100 threads for the controller overall is the maximum number of threads that could run concurrently. On a 16 core system and a flow which is very I/O bound this is definitely achievable. Generally you want to look at some multiple of the number of physical cores such as 2,4,8, etc.. but in the end it isn't that important. So why do you only see at most 10 or so threads in active use? Generally this means your flow isn't demanding or configured to do more. How many processors do you have? How many tasks does each have? How much data is flowing through the system when it reaches 10? Are backlogs growing at that time? What are the run schedules? How does the load average on the system look? How does IO utilization/iowait look during those times? Thanks Joe On Wed, Sep 18, 2019 at 8:05 AM wangl...@geekplus.com.cn < wangl...@geekplus.com.cn> wrote: > > My NiFi server is 16 cores. I also configed some processor cocurrent tasks > to 100. > But why the active thread count shown on the NiFi web ui is no more than > 10? > > > -- > wangl...@geekplus.com.cn >
