[rsyslog] Missing messages in our databases
What does the following mean?
How can we correct this?
We just discovered missing rsyslog messages in our databases, going back to
February 2018.
/var/log/messages shows errors like these:
Feb 3 02:16:33 hermes rsyslogd[30458]: The error statement was: insert
into SystemEvents (Message, Facility, FromHost, Priority,
DeviceReportedTime, ReceivedAt, InfoUnitID, SysLogTag) values ('Connection
reset by 172.99.99.100 port 39596 [preauth]', 10, 'hermes', 6,
'20180203021157', '20180203021157', 1, 'sshd[17977]:') [v8.32.0 try
http://www.rsyslog.com/e/2218 ]
Nov 15 11:24:23 hermes rsyslogd[701]: The error statement was: insert into
SystemEvents (Message, Facility, FromHost, Priority, DeviceReportedTime,
ReceivedAt, InfoUnitID, SysLogTag) values ('pam_unix(sshd:session): session
opened for user ms50013 by (uid=0)', 10, 'hermes', 6, '20181115112422',
'20181115112422', 1, 'sshd[12048]:') [v8.39.0 try
http://www.rsyslog.com/e/2218 ]
Nov 15 11:39:43 hermes rsyslogd[701]: The error statement was: insert into
SystemEvents (Message, Facility, FromHost, Priority, DeviceReportedTime,
ReceivedAt, InfoUnitID, SysLogTag) values ('pam_unix(sshd:session): session
closed for user ms50013', 10, 'hermes', 6, '20181115113943',
'20181115113943', 1, 'sshd[12048]:') [v8.39.0 try
http://www.rsyslog.com/e/2218 ]
Below are ommysql configurations for two connections to same host, different
databases. We do not find any of these failures for ActionName Ftp, only
Sftp.
$ActionName Ftp
$ActionQueueDequeueSlowdown 1000 # How long (in microseconds) dequeueing
should be delayed
$ActionQueueFileName dbFtpQueue # Set file name, also enables disk mode
$ActionQueueSaveOnShutdown on # Save messages to disk on shutdown
$ActionQueueType LinkedList # Use asynchronous processing
$ActionResumeRetryCount -1# Infinite retries on insert failure
ftp.*
:ommysql:172.99.99.125,vsftplog,hermesvsftplog,_PASSWORD_
$ActionName Sftp
$ActionQueueDequeueSlowdown 1000 # How long (in microseconds) dequeueing
should be delayed
$ActionQueueFileName dbSftpQueue # Set file name, also enables disk mode
$ActionQueueSaveOnShutdown on # Save messages to disk on shutdown
$ActionQueueType LinkedList# Use asynchronous processing
$ActionResumeRetryCount -1 # Infinite retries on insert failure
authpriv.*
:ommysql:172.99.99.125,sftplogDB,hermesvsftplog,_PASSWORD_
___
rsyslog mailing list
http://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards
NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of
sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE
THAT.
Re: [rsyslog] exclude a program from syslog file
El jue., 15 nov. 2018 a las 18:14, Xavier Saint-Claude () escribió: > > wow that was that simple ? > maybe that's the point of the & stop ? stopping those logs to go anywhere > else ? i seem to remember that in the old syntax (which i still have in > rsyslog.conf) it was not that easy... it's the same no matter which config type you use. rules are executed from top to bottom and processing stop when you reach "stop" (or ~ in very old style). Rainer > > thanks Rainer :) > > On 11/15/2018 07:03 PM, Rainer Gerhards wrote: > > El jue., 15 nov. 2018 a las 17:38, Xavier Saint-Claude via rsyslog > () escribió: > > Hi > > i've created a rule like this (which can be improved i'm sure) : > > if $programname == 'autofs' then /var/log/autofs.log > & stop > if $programname == 'automount' then /var/log/autofs.log > & stop > > it's working fine, but i would like to also exclude it from logging to > /var/log/syslog (and others) because of previous rule i have up in the > conf (and i'm using debug log in autofs) > > how does one achieve that ? > > move it up before the other rule > > Rainer > > thanks in advance > Dann > ___ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > http://www.rsyslog.com/professional-services/ > What's up with rsyslog? Follow https://twitter.com/rgerhards > NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of > sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T > LIKE THAT. > > ___ rsyslog mailing list http://lists.adiscon.net/mailman/listinfo/rsyslog http://www.rsyslog.com/professional-services/ What's up with rsyslog? Follow https://twitter.com/rgerhards NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE THAT.
Re: [rsyslog] exclude a program from syslog file
wow that was that simple ? maybe that's the point of the & stop ? stopping those logs to go anywhere else ? i seem to remember that in the old syntax (which i still have in rsyslog.conf) it was not that easy... thanks Rainer :) On 11/15/2018 07:03 PM, Rainer Gerhards wrote: El jue., 15 nov. 2018 a las 17:38, Xavier Saint-Claude via rsyslog () escribió: Hi i've created a rule like this (which can be improved i'm sure) : if $programname == 'autofs' then /var/log/autofs.log & stop if $programname == 'automount' then /var/log/autofs.log & stop it's working fine, but i would like to also exclude it from logging to /var/log/syslog (and others) because of previous rule i have up in the conf (and i'm using debug log in autofs) how does one achieve that ? move it up before the other rule Rainer thanks in advance Dann ___ rsyslog mailing list http://lists.adiscon.net/mailman/listinfo/rsyslog http://www.rsyslog.com/professional-services/ What's up with rsyslog? Follow https://twitter.com/rgerhards NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE THAT. ___ rsyslog mailing list http://lists.adiscon.net/mailman/listinfo/rsyslog http://www.rsyslog.com/professional-services/ What's up with rsyslog? Follow https://twitter.com/rgerhards NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE THAT.
Re: [rsyslog] exclude a program from syslog file
El jue., 15 nov. 2018 a las 17:38, Xavier Saint-Claude via rsyslog () escribió: > > Hi > > i've created a rule like this (which can be improved i'm sure) : > > if $programname == 'autofs' then /var/log/autofs.log > & stop > if $programname == 'automount' then /var/log/autofs.log > & stop > > it's working fine, but i would like to also exclude it from logging to > /var/log/syslog (and others) because of previous rule i have up in the > conf (and i'm using debug log in autofs) > > how does one achieve that ? move it up before the other rule Rainer > > thanks in advance > Dann > ___ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > http://www.rsyslog.com/professional-services/ > What's up with rsyslog? Follow https://twitter.com/rgerhards > NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of > sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T > LIKE THAT. ___ rsyslog mailing list http://lists.adiscon.net/mailman/listinfo/rsyslog http://www.rsyslog.com/professional-services/ What's up with rsyslog? Follow https://twitter.com/rgerhards NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE THAT.
[rsyslog] exclude a program from syslog file
Hi i've created a rule like this (which can be improved i'm sure) : if $programname == 'autofs' then /var/log/autofs.log & stop if $programname == 'automount' then /var/log/autofs.log & stop it's working fine, but i would like to also exclude it from logging to /var/log/syslog (and others) because of previous rule i have up in the conf (and i'm using debug log in autofs) how does one achieve that ? thanks in advance Dann ___ rsyslog mailing list http://lists.adiscon.net/mailman/listinfo/rsyslog http://www.rsyslog.com/professional-services/ What's up with rsyslog? Follow https://twitter.com/rgerhards NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE THAT.
Re: [rsyslog] Internet drop followed by a restart - queue
> One problem with TCP logging is that rsyslog has no visibility after it > submits > the log to the OS, so what is happening here is that your "no connection" > message is going to rsyslog, it still thinks there is a connection open, so it > submits it to the OS and considers it sent. > > also see > https://rainer.gerhards.net/2008/04/on-unreliability-of-plain-tcp-syslog.html > > This is exactly the use case that RELP is designed for. > I tried implementing RELP one time... it seems to be quite a bit more expensive on CPU time than standard TCP. I saw my CPU spike between 25-50% over plain TCP when I tried it on a high volume receiver. I would advise caution before implementing it on anything with high volume. In my environment I don't need the guaranteed lossless so I've stuck with TCP. YMMV. Scot Kreienkamp |Senior Systems Engineer | La-Z-Boy Corporate One La-Z-Boy Drive| Monroe, Michigan 48162 | Office: 734-384-6403 | | Mobile: 7349151444 | Email: scot.kreienk...@la-z-boy.com This message is intended only for the individual or entity to which it is addressed. It may contain privileged, confidential information which is exempt from disclosure under applicable laws. If you are not the intended recipient, you are strictly prohibited from disseminating or distributing this information (other than to the intended recipient) or copying this information. If you have received this communication in error, please notify us immediately by e-mail or by telephone at the above number. Thank you. ___ rsyslog mailing list http://lists.adiscon.net/mailman/listinfo/rsyslog http://www.rsyslog.com/professional-services/ What's up with rsyslog? Follow https://twitter.com/rgerhards NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE THAT.
Re: [rsyslog] TLS and rsyslog
Hi Carsten, I added this but got an error in the logs. 2018-11-15T15:25:44.803514+01:00 3msl rsyslogd: imrelp: invalid auth mode 'anon' [v8.39.0 try http://www.rsyslog.com/e/2291 ] I tried this config with : tls.authMode="anon" and then with, tls.authMode=anon # Input Modules input(type="imrelp" port="10514" tls="on" tls.caCert="/etc/pki/tls/private/ca-cert.pem" tls.myCert="/etc/pki/tls/private/collector-cert.pem" tls.myPrivKey="/etc/pki/tls/private/collector-key.pem" tls.authMode="anon" tls.permittedpeer=["*.local"] ) I'm running rsyslogd 8.39.0 Best wishes, Sophie > -Original Message- > From: carsten.la...@bt.com [mailto:carsten.la...@bt.com] > Sent: Tuesday, November 13, 2018 5:08 PM > To: rsyslog@lists.adiscon.com > Cc: LOEWENTHAL Sophie > Subject: RE: [rsyslog] TLS and rsyslog > > Hi Sophie, > > To get rid of the cert verification I used the following config parameter > > $InputTCPServerStreamDriverAuthMode anon # client is NOT authenticated > > So in your case I would think it's the following: > tls.authMode=anon > > for anonymous authentication. > > This was set on the server site. > > My whole config looked like this: > > :: > /etc/rsyslog.d/tls.conf > :: > # syslog via tls configuration > $ModLoad imtcp # TCP listener > > # make gtls driver the default > $DefaultNetstreamDriver gtls > > # certificate files > $DefaultNetstreamDriverCAFile /etc/rsyslog.d/certs/ca.pem > $DefaultNetstreamDriverCertFile /etc/rsyslog.d/certs/cert.pem > $DefaultNetstreamDriverKeyFile /etc/rsyslog.d/certs/key.pem > > #Ruleset > $Ruleset TLS > *.* /apps/log/tls.log > action( > name="rsyslog-debug-local" > template="RSYSLOG_DebugFormat" > type="omfile" > file="/apps/log/rsyslog-debug-tls.log" > ) > $Ruleset RSYSLOG_DefaultRuleset > > $InputTCPServerStreamDriverAuthMode anon # client is NOT authenticated > $InputTCPServerStreamDriverMode 1 # run driver in TLS-only mode > $InputTCPServerBindRuleset TLS > $InputTCPServerRun 6514 # start up listener at port 6514 > > Regards > > carsten > > > Carsten Lange | Security Specialist | CISSP | E: carsten.la...@bt.com > > -Original Message- > From: rsyslog [mailto:rsyslog-boun...@lists.adiscon.com] On Behalf Of > sophie.loewenthal--- via rsyslog > Sent: Dienstag, 13. November 2018 16:27 > To: rsyslog-users > Cc: sophie.loewent...@externe.bnpparibas.com > Subject: Re: [rsyslog] TLS and rsyslog > > Hi, > > New error message after removing the tls.permittedpeer=["*.local"] and the > tls.authmode I saw this, > > 2018-11-13T16:19:22.691302+01:00 6 rsyslogd: imrelp[10514]: error 'TLS record > write failed [gnutls error -10: The specified session has been invalidated for > some reason.]', object 'lstn 10514: conn to clt 10.1.1.8/a2.local ' - input > may > not work as intended [v8.24.0 try http://www.rsyslog.com/e/2353 ] > > > > > -Original Message- > > From: rsyslog [mailto:rsyslog-boun...@lists.adiscon.com] On Behalf Of > > sophie.loewenthal--- via rsyslog > > Sent: Tuesday, November 13, 2018 4:03 PM > > To: rsyslog-users > > Cc: LOEWENTHAL Sophie > > Subject: Re: [rsyslog] TLS and rsyslog > > > > Thanks. I configured ca and certs for the clients and servers. I set the > > dnsName > > to a wildcard. e.g *.local. in the certificates. > > > > Both clients and servers started, and the server listens on the ports with > > TLS. > > > > However TLS connections cannot be established. This is something to do > > with a 'peer' which was specified in tls.permittedpeer= > > > > The error on the server is : > > 2018-11-13T15:50:12.783315+01:00 6 rsyslogd: imrelp[10514]: > > authentication error 'no permited name found', peer is '' [v8.24.0 try > > http://www.rsyslog.com/e/2353 ] > > > > > > I used this config: > > Server: > > input(type="imrelp" port="10514" tls="on" > > tls.caCert="/etc/pki/tls/private/ca-cert.pem" > > tls.myCert="/etc/pki/tls/private/collector-cert.pem" > > tls.myPrivKey="/etc/pki/tls/private/collector-key.pem" > > tls.authMode="name" > > tls.permittedpeer=["*.local"] > > ) > > > > Client: > > action(type="omrelp" target="5" port="10514" tls="on" > > tls.caCert="/etc/pki/tls/private/ca-cert.pem" > > tls.myCert="/etc/pki/tls/private/sender-cert.pem" > > tls.myPrivKey="/etc/pki/tls/private/sender-key.pem" > > tls.authmode="name" > > tls.permittedpeer=["*.local"] > > ) > > action(type="omrelp" target="6" port="10514" tls="on" > > tls.caCert="/etc/pki/tls/private/ca-cert.pem" > > tls.myCert="/etc/pki/tls/private/sender-cert.pem" > > tls.myPrivKey="/etc/pki/tls/private/sender-key.pem" > > tls.authmode="name" > > tls.permittedpeer=["*.local"] > > ) > > > > What should I added for the tls.permittedpeer? I don't think this is > > correct. Or my certificates are wrong. > > > > > -Original Message- > > > From: rsyslog [mailto:rsyslog-boun...@lists.adiscon.com] On Behalf > > > Of John Chivian > > > Sent: Tuesday, November 13, 2018 1:56 PM > >
