All three servers have a base config that looks like the following: http://pastebin.com/NfMzTHfp
Server 1 and 2 have an additional config: http://pastebin.com/h3uWLnCs Server 3 also had an additional simplified config for this testing, different from what server 1 and 2 were using: http://pastebin.com/9eb8dwWj I used netcat to pipe events to port 1001 on server 2 or 3 for testing using these configs and the situation I described. Thanks, Sean Frost -----Original Message----- From: [email protected] [mailto:[email protected]] On Behalf Of David Lang Sent: Tuesday, May 26, 2015 4:43 PM To: rsyslog-users Subject: Re: [rsyslog] Missed log lines upgrading rsyslog from 7 to 8 On Tue, 26 May 2015, Frost, Sean wrote: > Hello, > We were looking into upgrading our rsyslog from version 7 to 8 and ran into > an issue where some passed log events seem to be missing and could use some > advice on how to proceed. > > We currently use rsyslog to pass log event streams to various servers for > processing. For some basic smoke testing I streamed some old logs to two > servers which passed it on to a third: > * Server 1 is running our current configuration and version of rsyslog - > rsyslog-7.4.9-2 (my control). It passes all events to server three and writes > all received events to a file. > * Server 2 is running the same configs and rsyslog-8.8.0.ad1-2 It also > passes all events to server 3 and writes all received events to a file > * Server 3 is running rsyslog-7.4.9-2 and just writes what it receives to a > file using a simplified ruleset for this testing. > > I would pipe events via netcat from a test server to server 1 or 2, wait for > everything to finish writing to the files, stop rsyslog, rename the file and > then repeat. > Test source server -> Server (1 or 2) -> Server 3 At the end I > compared the number of lines written to each file. > > The source log file contained 45102253 event lines > > Server 1, still running rsyslog 7.4.9 with old config > Run 1: > 45102253 lines in file - Server 1 > 45102253 lines in file - Server 3 > Run 2: > 45202253 lines in file - Server 1 > 45102253 lines in file - Server 3 > > Server 2, rsyslog 8.8.0 with old config on event12-2-qa-sjc: > Run 1: > 45102253 lines in file - Server 2 > 45102237 lines in file - Server 3 > Run 2: > 45102253 lines in file - Server 2 > 45102250 lines in file - Server 3 > > As shown, the number of lines written out on the intermediate server (1 or 2) > is always consistent, but the lines written out on server 3 are not, except > when running things through server 1 (running rsyslog 7). > The above results are actually the last tests I ran as we were trying to > eliminate other possible sources of issues. > Previously, we also tried running rsyslog 8 on Server 3 and a re-written > configuration on server 2 (to use updated syntax). In each case, the results > were consistent and the same when running through server 1 but always missing > a seemingly random number of lines when running through server 2, from 3 to > 54 lines across 10 different runs. > > Could use some advice on how to move forward. If this is something that might > be fixed in 8.10 (which I will probably test with next) or if there are any > parameters we could check on or modify to correct this issue as we do more > testing. it would help to see your configs, we don't know if you are sending via TCP/UDP/relp/etc. There are other options that could cause issues if they are set as well. David Lang _______________________________________________ rsyslog mailing list https://urldefense.proofpoint.com/v2/url?u=http-3A__lists.adiscon.net_mailman_listinfo_rsyslog&d=AwICAg&c=MOptNlVtIETeDALC_lULrw&r=2GjZkSjP4d8SfDKENxVc2szLfTfOba9JUSBOMngKEUg&m=etif6e2W2I_n_xKtfe_3yfVcfupKRV-N0J28Qoc4H98&s=s6OvmH9g67lFEf6a5Z_wxlQgRdJ-XmzbjuS8zk7X7Yg&e= https://urldefense.proofpoint.com/v2/url?u=http-3A__www.rsyslog.com_professional-2Dservices_&d=AwICAg&c=MOptNlVtIETeDALC_lULrw&r=2GjZkSjP4d8SfDKENxVc2szLfTfOba9JUSBOMngKEUg&m=etif6e2W2I_n_xKtfe_3yfVcfupKRV-N0J28Qoc4H98&s=HmTgmEOAaFX1SYtZ5PCUzZ9eEQzP9V1rZDtp62l7txI&e= What's up with rsyslog? Follow https://urldefense.proofpoint.com/v2/url?u=https-3A__twitter.com_rgerhards&d=AwICAg&c=MOptNlVtIETeDALC_lULrw&r=2GjZkSjP4d8SfDKENxVc2szLfTfOba9JUSBOMngKEUg&m=etif6e2W2I_n_xKtfe_3yfVcfupKRV-N0J28Qoc4H98&s=Mbk8DfnomSeyST6YrUcjrlrCavlg1_OVuzbokabeLEQ&e= NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE THAT. _______________________________________________ rsyslog mailing list http://lists.adiscon.net/mailman/listinfo/rsyslog http://www.rsyslog.com/professional-services/ What's up with rsyslog? Follow https://twitter.com/rgerhards NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE THAT.
