On Wed, 15 Nov 2017, Войнович Андрей Александрович wrote:
We have upgraded our linux box to the latest available (Debian 9) and now
rsyslog version is 8.24 (the newest from deb repo), but we still experience
the same problems. Seems we are doing something wrong in our rsyslog.conf -
number of threads, buffers, etc.. but I can't figure out how to tune these
params better from official docs.
Current is 8.31, but you are now at least a lot closer.
My rsyslog.conf:
module(load="imudp" threads="8" timeRequery="8" batchSize="128")
input(type="imudp" port="514" ruleset="writeRemoteData")
$ActionFileDefaultTemplate RSYSLOG_TraditionalFileFormat
$RepeatedMsgReduction on
You probably don't want this, it breaks most log analysis tools
$FileOwner root
$FileGroup adm
$FileCreateMode 0644
$DirCreateMode 0755
$Umask 0022
$WorkDirectory /var/spool/rsyslog
$IncludeConfig /etc/rsyslog.d/*.conf
template (name="DynFile" type="string"
string="/var/log/remotes/LAN/%FROMHOST-IP%/%syslogfacility-text%.log")
ruleset(name="writeRemoteData"
queue.type="fixedArray"
queue.size="250000"
queue.dequeueBatchSize="4096"
queue.workerThreads="8"
this is almost always going to be a mistake. There is a lot of locking overhead
when you have multiple threads fighting for data, and it makes your batch sizes
really small. Start off with 1 worker thread and only increase it if you find
that this thread is maxing out a CPU core.
queue.workerThreadMinimumMessages="60000"
) {
if ($fromhost-ip startswith "192.168.") then {
Action (type="omfile" dynaFile="DynFile" flushOnTXEnd="off" asyncWriting="on"
flushInterval="1" ioBufferSize="64k")
You don't want to do async writing to local files at all (unless possibly you
are doing compression/encryption of the file). The overhead of moving the
messages from the ruleset queue that the action reads, to the output queue for
the async writer is probably larger than the cost of writing the data to the
fiel
And when you combine it with dynafile, you end up with a new thread for every
file that you are writing to, and your memory useage explodes.
removethe asynwriting and try reducing the thread count to 1
Also, with dynafiles, you need to set the dynafilecachesize to some number
larger than the number of files you are writing to (the default used to be 10,I
think it's 128 now)
David Lang
stop
} else if ($fromhost-ip startswith "10.") then {
Action (type="omfile" dynaFile="DynFile" flushOnTXEnd="off" asyncWriting="on"
flushInterval="1" ioBufferSize="64k")
stop
} else if ($fromhost-ip startswith "172.") then {
Action (type="omfile" dynaFile="DynFile" flushOnTXEnd="off" asyncWriting="on"
flushInterval="1" ioBufferSize="64k")
stop
}
}
auth,authpriv.* /var/log/auth.log
*.*;auth,authpriv.none -/var/log/syslog
daemon.* -/var/log/daemon.log
kern.* -/var/log/kern.log
lpr.* -/var/log/lpr.log
mail.* -/var/log/mail.log
user.* -/var/log/user.log
mail.info -/var/log/mail.info
mail.warn -/var/log/mail.warn
mail.err /var/log/mail.err
news.crit /var/log/news/news.crit
news.err /var/log/news/news.err
news.notice -/var/log/news/news.notice
*.=debug;\
auth,authpriv.none;\
news.none;mail.none
-/var/log/debug
*.=info;*.=notice;*.=warn;\
auth,authpriv.none;\
cron,daemon.none;\
mail,news.none
-/var/log/messages
*.emerg :omusrmsg:*
daemon.*;mail.*;\
news.err;\
*.=debug;*.=info;\
*.=notice;*.=warn
|/dev/xconsole
Thank you for any help.
_______________________________________________
rsyslog mailing list
http://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards NOTE
WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of sites
beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE THAT.
_______________________________________________
rsyslog mailing list
http://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards
NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of
sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE
THAT.
