here comes the debug log :( http://pastebin.com/raw/4rZ6bGed
Dne 01/08/2017 v 05:21 PM Radoslav Bodó via rsyslog napsal(a): > the same problem is replicable on my side with 8.23.0-2 > > bodik > > > > > > Dne 01/08/2017 v 05:17 PM Radoslav Bodó via rsyslog napsal(a): >>> the system in question is debian stretch with rsyslog 8.16.0-1+b3 >> >> I'll rerun the tests with more recent rsyslog and report back >> >> bodik >> >> Dne 01/08/2017 v 05:04 PM David Lang napsal(a): >>> what version of rsyslog are you using? versions prior to ~8.20 had a >>> known problem due to json-c not being thread-safe. >>> >>> David Lang >>> >>> On Sun, 8 Jan 2017, Radoslav Bodó via rsyslog wrote: >>> >>>> Date: Sun, 8 Jan 2017 16:57:37 +0100 >>>> From: Radoslav Bodó via rsyslog <[email protected]> >>>> To: [email protected] >>>> Cc: Radoslav Bodó <[email protected]> >>>> Subject: [rsyslog] replacing message parts with regex leads to message >>>> corruption >>>> >>>> Hi, >>>> >>>> recently I was trying to create a masking template for software which >>>> logs messages including potentialy sensitive information (remctld >>>> logging whole command line executed). >>>> >>>> >>>> So I created a rule for masking that part out before storing messages: >>>> >>>> -------------- rsyslog.d/neweb2.conf >>>> $template Neweb2Format,"%timegenerated% %HOSTNAME% >>>> %syslogtag%%!msg:::drop-last-lf%\n" >>>> if ( ($programname == 'remctld') and ($msg contains 'neweb2') and ($msg >>>> contains 'pwd') ) then { >>>> set $!ext = re_extract($msg,'(pwd [^ ]+)',0,1,""); >>>> set $!msg = replace($msg, $!ext, "pwd MASKEDOUT"); >>>> action(type="omfile" template="Neweb2Format" >>>> File="/var/log/syslog") >>>> stop >>>> } >>>> ------------------------- >>>> >>>> >>>> >>>> acording to test a good behavior I've created a test case simulating >>>> remctld logging and check desired output >>>> >>>> ------------------- neweb2/tests/remctl_syslog_masks.sh >>>> #!/bin/sh >>>> >>>> . /puppet/metalib/bin/lib.sh >>>> >>>> RANDOM=$(/bin/dd if=/dev/urandom bs=100 count=1 2>/dev/null | >>>> /usr/bin/sha256sum | /usr/bin/awk '{print $1}' | sed >>>> 's/^\(......\).*/\1/') >>>> >>>> logger -t remctld "neweb2 db ${RANDOM}a --set --pwd 1234567890 --noop" >>>> logger -t remctld "neweb2 db ${RANDOM}b --set --noop --pwd 1234567890" >>>> logger -t remctld "neweb2 db --pwd 1234567890 --noop --set ${RANDOM}c" >>>> >>>> grep "neweb2 db ${RANDOM}a --set --pwd MASKEDOUT --noop" /var/log/syslog >>>> if [ $? -ne 0 ]; then >>>> rreturn 1 "$0 remctl neweb2 sensitive data not masked A" >>>> fi >>>> grep "neweb2 db ${RANDOM}b --set --noop --pwd MASKEDOUT" /var/log/syslog >>>> if [ $? -ne 0 ]; then >>>> rreturn 1 "$0 remctl neweb2 sensitive data not masked B" >>>> fi >>>> grep "neweb2 db --pwd MASKEDOUT --noop --set ${RANDOM}c" /var/log/syslog >>>> if [ $? -ne 0 ]; then >>>> rreturn 1 "$0 remctl neweb2 sensitive data not masked C" >>>> fi >>>> >>>> rreturn 0 "$0" >>>> ------------------------- >>>> >>>> >>>> >>>> but according to the test some of the messages gets garbled >>>> >>>> ---------- tail /var/log/syslog -n10 >>>> Jan 8 16:28:21 tester remctld: neweb2 db ba539ba --set --pwd MASKEDOUT >>>> --nooc >>>> Jan 8 16:28:21 tester remctld: neweb2 db ba539bb --set --noop --pwd >>>> MASKEDOUT >>>> Jan 8 16:28:21 tester remctld: neweb2 db --pwd MASKEDOUT --noop --set >>>> ba539bc >>>> ------------------- >>>> >>>> >>>> see the "--nooc" instead of "--noop" in the first case >>>> >>>> >>>> >>>> >>>> >>>> >>>> I'd suspect: >>>> >>>> a) my usage of replace() is wrong >>>> b) some memory management inside "property replacer" is not correct >>>> >>>> the system in question is debian stretch with rsyslog 8.16.0-1+b3 >>>> >>>> >>>> >>>> I'd be glad for any suggestions or cross-tests of this case. I could dig >>>> into code, make some additional testing, or propose a patch, but I'm not >>>> really sure where to start ... >>>> >>>> >>>> Thank you for any help >>>> Best regards >>>> bodik >>>> >>>> >>>> >>>> >>>> _______________________________________________ >>>> rsyslog mailing list >>>> http://lists.adiscon.net/mailman/listinfo/rsyslog >>>> http://www.rsyslog.com/professional-services/ >>>> What's up with rsyslog? Follow https://twitter.com/rgerhards >>>> NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a >>>> myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST >>>> if you DON'T LIKE THAT. >>>> >> _______________________________________________ >> rsyslog mailing list >> http://lists.adiscon.net/mailman/listinfo/rsyslog >> http://www.rsyslog.com/professional-services/ >> What's up with rsyslog? Follow https://twitter.com/rgerhards >> NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of >> sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T >> LIKE THAT. >> > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > http://www.rsyslog.com/professional-services/ > What's up with rsyslog? Follow https://twitter.com/rgerhards > NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of > sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T > LIKE THAT. > _______________________________________________ rsyslog mailing list http://lists.adiscon.net/mailman/listinfo/rsyslog http://www.rsyslog.com/professional-services/ What's up with rsyslog? Follow https://twitter.com/rgerhards NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE THAT.
