Thanks, that helped a little but I'm unfortunately still on 5.8.5 so I had some
difficulty translating that in my head to the old config format.
I was able to work around it with some hackery right now, unfortunately
involving the rival syslog-ng. This isn't a permanent solution but just until
we change the delimiters on all the Windows boxes. For future reference for
others:
Message flow:
Sending Windows server -> rsyslog -> syslog-ng -> final destination(SIEM)
On rsyslog, I turned off control characters, set up a template and forwarded
all Windows messages:
$EscapeControlCharactersOnReceive off
$template WindowsSIEMMsgFormat,"<%pri%>%timestamp% %fromhost-ip%
%syslogtag%%msg:::drop-last-lf%\n"
On syslog-ng, I created this rewrite filter to replace commas with tabs. Full
related config:
destination d_qradar{udp("1.1.1.1" port(514) template("<133>$DATE $HOST
$PROGRAM $MESSAGE\n\n"));};
#rewrite rule. replace commas with tabs
rewrite r_fixsnare{
subst(",\t", "\t", value("MESSAGE"), flags("global"));
subst(",", "\t", value("MESSAGE"), flags("global"));
subst(",", "\t", value("PROGRAM"), flags("global"));
};
#actually log it
log { source(s_network); rewrite(r_fixsnare); destination(d_qradar); };
And in the options for syslog-ng, set keep_hostnames(yes);
That seemed to do the trick. Thanks for the suggestions.
Dan
-----Original Message-----
From: [email protected]
[mailto:[email protected]] On Behalf Of Rainer Gerhards
Sent: Tuesday, February 26, 2013 1:55 AM
To: [email protected]
Subject: Re: [rsyslog] Replace a character in a message with a different one
On Mon, 2013-02-25 at 13:51 -0800, David Lang wrote:
> one problem you will have is that the text in the messages sometimes
> includes commas.
>
> how are you getting the logs to your program? could you just insert
> sed into the path? I don't know of a way to do this inside rsyslog
> without a small custom module.
I could try to add a replace() function to RainerScript, that could solve that
need. A message modification module would of course probably be the fastest
way. I think something that today can already be done is extract all fields and
rewrite them via a template. Have a look at this sample config here (it was for
a different use case, but...):
http://git.adiscon.com/?p=rsyslog.git;a=blob;f=doc/confsamples/normalization.conf;h=7cfd92ef86964281cd3cea8ff89ac9111c5668be;hb=HEAD
HTH
Rainer
>
> David Lang
>
> On Mon, 25 Feb 2013, Woodruff, Dan wrote:
>
> > Date: Mon, 25 Feb 2013 21:48:02 +0000
> > From: "Woodruff, Dan" <[email protected]>
> > Reply-To: rsyslog-users <[email protected]>
> > To: rsyslog-users <[email protected]>
> > Subject: [rsyslog] Replace a character in a message with a different
> > one
> >
> > We use Snare to capture logs from Windows machines to rsyslog for archival
> > and forwarding to other tools. One tool required Snare use a comma as a
> > delimiter so all the servers are configured to use a comma. The problem is
> > we have added another tool that we would like to forward logs to that wants
> > the original Snare delimiter, which is a tab character.
> >
> > Is there a way to replace all commas in rawmsg with a tab character? I
> > could not figure out a way to do this with a template because the message
> > format is strange and multiple fields from Snare are combined in the syslog
> > fields. Here is (sanitized) debug output:
> >
> > Debug line with all properties:
> > FROMHOST: '1.2.3.4', fromhost-ip: '1.2.3.4', HOSTNAME:
> > 'Hostname.domain.com', PRI: 134, syslogtag
> > 'MSWinEventLog,1,Security,695810,Mon', programname:
> > 'MSWinEventLog,1,Security,695810,Mon', APP-NAME:
> > 'MSWinEventLog,1,Security,695810,Mon', PROCID: '-', MSGID: '-',
> > TIMESTAMP: 'Feb 25 16:38:03', STRUCTURED-DATA: '-',
> > msg: ' Feb 25 16:38:00
> > 2013,4634,Microsoft-Windows-Security-Auditing,ITS-W2KS02ADR$,N/A,Success
> > Audit,Hostname.domain.com,None,,An account was logged off. Subject:
> > Security ID: S-1-5-21-299502267-36228127-1238915-1106 Account Name:
> > DCNAME$ Account Domain: DOMAIN Logon ID: 0x8178814 Logon Type: 3
> > This event is generated when a logon session is destroyed. It may be
> > positively correlated with a logon event using the Logon ID value. Logon
> > IDs are only unique between reboots on the same computer.,695323 '
> > escaped msg: ' Feb 25 16:38:00
> > 2013,4634,Microsoft-Windows-Security-Auditing,DCNAME$,N/A,Success
> > Audit,Hostname.domain.com,None,,An account was logged off. Subject:
> > Security ID: S-1-5-21-299502267-36228127-1238915-1106 Account Name:
> > DCNAME$ Account Domain: DOMAIN Logon ID: 0x8178814 Logon Type: 3
> > This event is generated when a logon session is destroyed. It may be
> > positively correlated with a logon event using the Logon ID value. Logon
> > IDs are only unique between reboots on the same computer.,695323 '
> > inputname: imudp rawmsg: '<134>Feb 25 16:38:03 Hostname.domain.com
> > MSWinEventLog,1,Security,695810,Mon Feb 25 16:38:00
> > 2013,4634,Microsoft-Windows-Security-Auditing,DCNAME$,N/A,Success
> > Audit,Hostname.domain.com,None,,An account was logged off. Subject:
> > Security ID: S-1-5-21-299502267-36228127-1238915-1106 Account Name:
> > DCNAME$ Account Domain: DOMAIN Logon ID: 0x8178814 Logon Type: 3
> > This event is generated when a logon session is destroyed. It may be
> > positively correlated with a logon event using the Logon ID value. Logon
> > IDs are only unique between reboots on the same computer.,695323 '
> >
> >
> > Any way to do that substitution? Or any other creative solutions?
> >
> > Thank you in advance,
> > Dan Woodruff
> >
> >
> > _______________________________________________
> > rsyslog mailing list
> > http://lists.adiscon.net/mailman/listinfo/rsyslog
> > http://www.rsyslog.com/professional-services/
> > What's up with rsyslog? Follow https://twitter.com/rgerhards NOTE
> > WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of
> > sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T
> > LIKE THAT.
> >
> _______________________________________________
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog
> http://www.rsyslog.com/professional-services/
> What's up with rsyslog? Follow https://twitter.com/rgerhards NOTE
> WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of sites
> beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE THAT.
_______________________________________________
rsyslog mailing list
http://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards NOTE WELL: This is
a PUBLIC mailing list, posts are ARCHIVED by a myriad of sites beyond our
control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE THAT.
_______________________________________________
rsyslog mailing list
http://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards
NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of
sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE
THAT.
