Re: Too many files open on CentOS 7

2019-04-12 Thread Mike Thomsen
Yeah. SystemD completely disregarded the limits our admins were setting and imposed some threshold probably not even 10k on us. On Fri, Apr 12, 2019 at 9:35 AM Joe Witt wrote: > so you found the prob? > > those numbers for nifi looked good. > > thanks > > On Fri, Apr 12, 2019, 9:34 AM Mike Thoms

Re: Too many files open on CentOS 7

2019-04-12 Thread Joe Witt
so you found the prob? those numbers for nifi looked good. thanks On Fri, Apr 12, 2019, 9:34 AM Mike Thomsen wrote: > And it was SystemD... > > On Fri, Apr 12, 2019 at 8:30 AM Mike Thomsen > wrote: > >> When I do lsof -u nifi, it says the nifi user only has 5761 handles >> associated with

Re: Too many files open on CentOS 7

2019-04-12 Thread Mike Thomsen
And it was SystemD... On Fri, Apr 12, 2019 at 8:30 AM Mike Thomsen wrote: > When I do lsof -u nifi, it says the nifi user only has 5761 handles > associated with it. > > One warning I saw on StackExchange said that sometimes SystemD subtly > messes with this stuff on RHEL. > > On Fri, Apr 12

Re: Too many files open on CentOS 7

2019-04-12 Thread Mike Thomsen
When I do lsof -u nifi, it says the nifi user only has 5761 handles associated with it. One warning I saw on StackExchange said that sometimes SystemD subtly messes with this stuff on RHEL. On Fri, Apr 12, 2019 at 8:14 AM Mike Thomsen wrote: > About 5600-5700 starting fresh. Got to about 6500-6

Re: Too many files open on CentOS 7

2019-04-12 Thread Mike Thomsen
About 5600-5700 starting fresh. Got to about 6500-6800 before hitting the ceiling. On Fri, Apr 12, 2019 at 7:30 AM Joe Witt wrote: > mike > > lsof -p > > with the pid of the actual nifi process is probably better to look at for > nifi resource handling observation. what is that count. yes the

Re: Too many files open on CentOS 7

2019-04-12 Thread Joe Witt
mike lsof -p with the pid of the actual nifi process is probably better to look at for nifi resource handling observation. what is that count. yes the jars and such will all be loaded. you can expect a few thousand off that. then there are sockets and content and prov and flowfilewhich

Too many files open on CentOS 7

2019-04-12 Thread Mike Thomsen
I know you can increase the file handle limit in /etc/security/limits.conf, but we're having a really weird issue where a CentOS 7.5 box can handle a massive record set just fine and another running CentOS 7.6 cannot. When I run *lsof | wc -l* on the 7.6 box after NiFi has been running for a while