David is bang on. This might be off topic, but i used to have to poll Checkpoint logs via their opsec API. This worked very well:
https://github.com/blackhole-em/OPSEC2Elastic While I know this might not help your exact question, it might be relevant as well. Then again... maybe not? Cheers, JB On Fri, Oct 11, 2019 at 2:37 PM David Lang via rsyslog < [email protected]> wrote: > repeated message reduction is not recommended. It is a feature that was > useful > back when it was only people looking at the logs, but when you are > analyzing the > logs with automation, the 'message repeated' logs reduce your accuracy, > not > improve it. > > that 'message repeated' is happening on the sender, so look to fix it > there (in > rsyslog we started having the message repeated log include the log being > repeated so that it was easier to track what's happening) > > David Lang > > On Fri, 11 Oct 2019, Marki via rsyslog wrote: > > > Date: Fri, 11 Oct 2019 19:14:27 +0200 > > From: Marki via rsyslog <[email protected]> > > To: [email protected] > > Cc: Marki <[email protected]> > > Subject: [rsyslog] Rsyslog seems to think there are duplicates > > > > > > Hey, > > > > We are using Checkpoint Log Exporter > > ( > https://supportcenter.checkpoint.com/supportcenter/portal?eventSubmit_doGoviewsolutiondetails=&solutionid=sk122323) > > > I don't think that it matters, except maybe for the fact that they are > > implementing RFC5424. > > > > The logs are being written to file by Rsyslog as follows: > > > > --------------------- > > module(load="imtcp") > > module(load="imudp") > > > > input(type="imtcp" port="514") > > input(type="imudp" port="514") > > > > template(name="FileNameByHost" type="string" > > string="/syslog/%fromhost-ip%.log") > > if ($fromhost-ip startswith "192.168.x.y") then { # RFC5424 > > action(type="omfile" dynaFile="FileNameByHost" > > template="RSYSLOG_SyslogProtocol23Format") <------------------- > > } else { > > action(type="omfile" dynaFile="FileNameByHost" > > template="RSYSLOG_FileFormat") > > } > > > > call asyncelk > > ... > > --------------------- > > > > Now Rsyslog thinks some messages are duplicates. And apparently it > thinks > > there are _many_ duplicates. I have sniffed the traffic, and no > duplicates > > are being transmitted AFAICS. Here is an example of what is logged > > nevertheless: > > > > <134>1 2019-10-11T16:27:51Z chkpt-mgt CheckPoint 15439 - > [action:"Reject"; > > ifdir:"inbound"; ifname:"eth2.53"; logid:"0"; > loguid:"{0x0,0x0,0x0,0x0}"; > > origin:"192.168.1.2"; originsicname:"xxxxx"; sequencenum:"32"; > > time:"1570811271"; version:"5"; dst:"224.0.0.18"; > > hll_key:"2008789500278145398"; inzone:"Internal"; layer_name:"Unified"; > > layer_uuid:"a-b-c-d-e"; match_id:"177"; parent_rule:"0"; > > rule_action:"Reject"; rule_name:"xyz"; rule_uid:"a-b-c-d-e"; > product:"VPN-1 & > > FireWall-1"; proto:"51"; service_id:"AH"; src:"1.2.3.4"; ] message > repeated > > 49 times: [] > > > > Other times it says "99 times" or "149 times", ... I've created somes > stats > > in fact :D > > > > # tail -n 10000 //syslog/1.2.3.4.log | grep -o -E 'repeated [0-9]+ > times' | > > sort | uniq -c | sort -n | tail -10 > > 23 repeated 44 times > > 26 repeated 199 times > > 35 repeated 159 times > > 37 repeated 9 times > > 38 repeated 4 times > > 101 repeated 149 times > > 155 repeated 109 times > > 408 repeated 59 times > > 417 repeated 99 times > > 2926 repeated 49 times > > > > This is SLES 15-SP1 running rsyslog-8.33.1-3.17.1.x86_64 > > > > Any idea? > > > > Thanks, > > Marki > > > > > > _______________________________________________ > > rsyslog mailing list > > http://lists.adiscon.net/mailman/listinfo/rsyslog > > http://www.rsyslog.com/professional-services/ > > What's up with rsyslog? Follow https://twitter.com/rgerhards > > NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad > of > > sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you > DON'T > > LIKE THAT. > > > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > http://www.rsyslog.com/professional-services/ > What's up with rsyslog? Follow https://twitter.com/rgerhards > NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad > of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you > DON'T LIKE THAT. > _______________________________________________ rsyslog mailing list http://lists.adiscon.net/mailman/listinfo/rsyslog http://www.rsyslog.com/professional-services/ What's up with rsyslog? Follow https://twitter.com/rgerhards NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE THAT.
