The actual connections themselves are managed with a selector, so if all the connections are idle there should only be one thread for the socket.
As soon as a connection has something available to read then a thread is spawned to start reading the connection until either no matter is available, or it is closed. On Fri, Aug 2, 2019 at 7:18 AM Clay Teahouse <[email protected]> wrote: > > Hello Edward, > So, if have of to listen to 32,000 tcp connections and I have only 80 cores, > and I configure each ListenSyslog instance for 4,000 connections, doesn't > each spawn 4,000 threads behind the scene? The tcp connections will be idle > most of the time. > > thanks > Clay > > > On Fri, Aug 2, 2019 at 6:10 AM Edward Armes <[email protected]> wrote: >> >> Hi Clay, >> >> Because Nifi underneath uses a thread pool for it's own threading >> underneath, and each instance processor runs does so in it's own thread, I >> don't see any reason why not. One thing to note that the way the ListenTCP >> processor appears to have been written such that it gets all the requests >> that have been received on that socket and processes them until either it >> has no more requests left or process or that instance of the processor is no >> longer scheduled to run. >> >> Hope that helps >> >> Edward >> >> On Fri, Aug 2, 2019 at 11:28 AM Clay Teahouse <[email protected]> wrote: >>> >>> Hello All, >>> >>> I need to listen to and process thousands of persistent TCP connections. I >>> have 10 nodes, each having 8 cores. >>> My understanding is that with existing NiFi listening processors, such as >>> ListnSyslog, a thread is utilized for each TCP connection. Does this scale? >>> Do I need to write a custom processor that utilizes a thread pool for >>> reading the data from the socket and processing them? >>> >>> thanks >>> Clay
