Hello Rainer, I am not terribly firm with tcpdump, how would I go about doing that?
Also I forgot to mention, the logs are always coming from the same Host with exactly the same format. ( A VmWare logInsight-Server if that’s relevant) so I doubt the messages are suddenly changing form. -----Ursprüngliche Nachricht----- Von: Rainer Gerhards <[email protected]> Gesendet: Mittwoch, 11. November 2020 09:26 An: Seifer, Florian <[email protected]> Cc: rsyslog-users <[email protected]> Betreff: Re: [rsyslog] rsyslog ignoring random incoming tcp-messages looks good. my gut feeling is that some senders do not properly terminate the syslog frame with '\n'. Can you check you captures for a case where a message is missing and post all bytes of the last seen and missing messages in the correct sequence? Rainer El mié., 11 nov. 2020 a las 9:16, Seifer, Florian (<[email protected]>) escribió: > > # rsyslog configuration file > > # For more information see /usr/share/doc/rsyslog-*/rsyslog_conf.html > # or latest version online at http://www.rsyslog.com/doc/rsyslog_conf.html > # If you experience problems, see http://www.rsyslog.com/doc/troubleshoot.html > > #### MODULES #### > > module(load="imuxsock" # provides support for local system logging (e.g. > via > > logger command) > SysSock.Use="off") # Turn off message reception via local log socket; > # local messages are retrieved through imjournal > now. > module(load="imjournal" # provides access to the systemd journal > StateFile="imjournal.state") # File to store the position in the > journal > #module(load="imklog") # reads kernel messages (the same are read from > journald) > #module(load"immark") # provides --MARK-- message capability > > # Provides UDP syslog reception > # for parameters see http://www.rsyslog.com/doc/imudp.html > module(load="imudp") # needs to be done just once > input(type="imudp" port="514") > > # Provides TCP syslog reception > # for parameters see http://www.rsyslog.com/doc/imtcp.html > module(load="imtcp") # needs to be done just once > > > > > > > > > #### GLOBAL DIRECTIVES #### > > # Where to place auxiliary files > global(workDirectory="/var/lib/rsyslog") > > # Use default timestamp format > module(load="builtin:omfile" Template="RSYSLOG_TraditionalFileFormat") > > # Include all config files in /etc/rsyslog.d/ > include(file="/etc/rsyslog.d/*.conf" mode="optional") > > #### RULES #### > > # Log all kernel messages to the console. > # Logging much else clutters up the screen. > #kern.* /dev/console > > # Log anything (except mail) of level info or higher. > # Don't log private authentication messages! > *.info;mail.none;authpriv.none;cron.none /var/log/messages > > # The authpriv file has restricted access. > authpriv.* /var/log/secure > > # Log all the mail messages in one place. > mail.* -/var/log/maillog > > > # Log cron stuff > cron.* /var/log/cron > > # Everybody gets emergency messages > *.emerg :omusrmsg:* > > # Save news errors of level crit and higher in a special file. > uucp,news.crit /var/log/spooler > > # Save boot messages also to boot.log > local7.* /var/log/boot.log > > > > > > > # ### sample forwarding rule ### > #action(type="omfwd" > # An on-disk queue is created for this action. If the remote host is > # down, messages are spooled to disk and sent when it is up again. > #queue.filename="fwdRule1" # unique name prefix for spool files > #queue.maxdiskspace="1g" # 1gb space limit (use as much as possible) > #queue.saveonshutdown="on" # save messages to disk on shutdown > #queue.type="LinkedList" # run asynchronously > #action.resumeRetryCount="-1" # infinite retries if host is down > # Remote Logging (we use TCP for reliable delivery) > # remote_host is: name/ip, e.g. 192.168.0.1, port optional e.g. 10514 > #Target="remote_host" Port="XXX" Protocol="tcp") > > > # Diese Regel erstellt pro Hostname ein eigenes Verzeichnis > $template RemoteHost,"/var/log/remote/%hostname%/%programname%.log" > > # Der Regelsatz remote wendet auf alle eingehenden Logs das Template > RemoteHost an: > > $RuleSet remote > *.* ?RemoteHost > > # Nun binden wir den Regelsatz "remote" an alle syslog Meldungen von Außen. > > input(type="imtcp" port="514" ruleset="remote" > supportoctetcountedframing="off") > > $EscapeControlCharactersOnReceive on > > $RepeatedMsgReduction off > > $SystemLogRateLimitInterval 0 > > $DebugLevel 2 > > $DebugFile /var/log/rsyslog/debug.log > > -- > Florian Seifer > Berater secunet Anwendungen, Managed Security Services > Division Operational Services > secunet Security Networks AG > > Tel.: +49 201 5454-2297, Fax: +49 201 5454-1259 > [email protected] > Kurfürstenstraße 58, 45138 Essen, Germany > www.secunet.com > > ----------------------------------------------------------------------- > Sitz: Kurfürstenstraße 58, 45138 Essen > Amtsgericht Essen HRB 13615 > Vorstand: Axel Deininger (Vors.), Torsten Henn, Dr. Kai Martius, Thomas > Pleines > Aufsichtsratsvorsitzender: Ralf Wintergerst > > -----Ursprüngliche Nachricht----- > Von: Rainer Gerhards <[email protected]> > Gesendet: Mittwoch, 11. November 2020 09:13 > An: rsyslog-users <[email protected]> > Cc: Seifer, Florian <[email protected]> > Betreff: Re: [rsyslog] rsyslog ignoring random incoming tcp-messages > > config? > > El mié., 11 nov. 2020 a las 9:11, Seifer, Florian via rsyslog > (<[email protected]>) escribió: > > > > Hello, > > > > I have a rather strange problem with an rsyslog-machine. > > > > I configured it to process incoming logs over network via tcp port 514 > > using imtcp. > > > > The incoming logs on that connection are simply stored in a specific folder > > in a log-file. > > This setup works 90% of the time but sometimes logs get "lost". > > > > I have no idea what causes it. I set up a tcpdump on the receiving machine > > and I can confirm that the packages are indeed being delivered. > > But for some reason rsyslog does not notice them. I can find all other > > log-messages mentioned in the rsyslog-debug log, where they are noticed and > > processed correctly. > > > > The ones that don't make it to the logfiles are no where to be found. I > > cannot find any difference in these files, so I would expect them to be > > handled identically. > > > > Has anyone of you ever had similar problems and found a solution? Its not > > a firewall or network problem as the packages are definitely reaching the > > client. > > > > I am grateful for any support you guys can provide > > > > With kind regards > > > > -- > > Florian Seifer > > Berater secunet Anwendungen, Managed Security Services > > Division Operational Services > > secunet Security Networks AG > > > > Tel.: +49 201 5454-2297, Fax: +49 201 5454-1259 > > [email protected] > > Kurfürstenstraße 58, 45138 Essen, Germany > > www.secunet.com > > > > ----------------------------------------------------------------------- > > Sitz: Kurfürstenstraße 58, 45138 Essen > > Amtsgericht Essen HRB 13615 > > Vorstand: Axel Deininger (Vors.), Torsten Henn, Dr. Kai Martius, Thomas > > Pleines > > Aufsichtsratsvorsitzender: Ralf Wintergerst > > > > _______________________________________________ > > rsyslog mailing list > > https://lists.adiscon.net/mailman/listinfo/rsyslog > > http://www.rsyslog.com/professional-services/ > > What's up with rsyslog? Follow https://twitter.com/rgerhards > > NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of > > sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T > > LIKE THAT. _______________________________________________ rsyslog mailing list https://lists.adiscon.net/mailman/listinfo/rsyslog http://www.rsyslog.com/professional-services/ What's up with rsyslog? Follow https://twitter.com/rgerhards NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE THAT.
