I've got 2 problems :)
1.- Yesterday, In one of our rsyslog servers, we started to see massive too many open files messages: Jan 8 21:05:39 logcenter rsyslogd: file '/logs/2020/frontal/web/01/apacheaccess_08_S1.log': open error: Too many open files [v8.24.0-41.el7_7.2 try http://www.rsyslog.com/e/2433 ] Jan 8 21:05:40 logcenter rsyslogd: file '/logs/2020/munihosting/web/01/apacheaccess_08_S1.log': open error: Too many open files [v8.24.0-41.el7_7.2 try http://www.rsyslog.com/e/2433 ] ... This morning I restarted the rsyslog server and now it seems to be all ok but... How can I raise this max value up? Is the default config for the operating system (CentOs 7.6)? or there is something we have to configure in rsyslog config? The second problem is related with this one... 2.-On my rsyslog clients I have this config for the output: ruleset(name="sendToLogserver") { action( type="omfwd" target="logcenter.intranet.dtgna" port="514" protocol="tcp" queue.type="LinkedList" queue.size="500000" queue.filename="q_sendToLogserver" queue.highwatermark="290000" queue.lowwa termark="50000" queue.maxdiskspace="2g" queue.saveonshutdown="on" action.resumeRetryCount="-1" action.resumeInterval="20") action( type="omfwd" target="logcenter02.intranet.dtgna" port="514" protocol="tcp" queue.type="LinkedList" queue.size="500000" queue.filename="q_sendToLogserver02" queue.highwatermark="290000" queue.l owwatermark="50000" queue.maxdiskspace="2g" queue.saveonshutdown="on" action.resumeRetryCount="-1" action.resumeInterval="20" action.execOnlyWhenPreviousIsSuspended="on") stop } Yesterday, the problem with the open files started about 21:05, and there is a problem with this client file (server appsact01) Client config: input(type="imfile" file="/logs/wildfly10/server.log" tag="wildfly_serverlog" ruleset="sendToLogserver" reopenontruncate="on") In the rsyslog server, the last modification time and line are: [root@logcenter ~]# tail -1 /logs/2020/appsact01/appssrv/01/wildfly_serverlog_08_S1.log 2020-01-08 21:20:38,138 FINE [groovy.sql.Sql] (default task-48) select distinct * from XXX | [] But in the server, the last line for the file yesterday was: [root@appsactio01 ~]# tail -1 /logs/wildfly10/server.log.2020-01-08 2020-01-08 23:43:17,289 FINE [groovy.sql.Sql] (default task-64) select distinct * fromXXX WHERE codi = '02' | [] Is not the same :(( This line is neither in the file created after I restarted the service, which starts about 00:42 in the night... [root@logcenter ~]# more /logs/2020/appsact01/appssrv/01/wildfly_serverlog_09_S1.log 2020-01-09 00:42:34,093 FINE [groovy.sql.Sql] (sidng_scheduler_Worker-2) select view_name from user_views While in the server, the first line for today's server.log file is: [root@appsact01 ~]# more /logs/wildfly10/server.log 2020-01-09 00:13:11,796 FINE [groovy.sql.Sql] (default task-40) select distinct * from V_XXX codi = '02' | [] Also different :(( The questions are: -If the primary rsyslog server was giving those open files errors, the client shouldn't have to start send the messages to the secondary server? -What happened to the disappeared messages? how we could trace it? it's a config problem? -During the problem, the client server.log rotated, I suppose that there wouldn't have to be a problem because those lines where in the rsyslog queue, isn't it? I'm very worried about those lines lost, It happended to various log files :( PS:Rsyslog version in server and clients is the last red-hat official update, 8.24.0-41.el7_7.2 -- signatura ** _______________________________________________ rsyslog mailing list http://lists.adiscon.net/mailman/listinfo/rsyslog http://www.rsyslog.com/professional-services/ What's up with rsyslog? Follow https://twitter.com/rgerhards NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE THAT.
