the same problem is replicable on my side with 8.23.0-2 bodik
Dne 01/08/2017 v 05:17 PM Radoslav Bodó via rsyslog napsal(a): >> the system in question is debian stretch with rsyslog 8.16.0-1+b3 > > I'll rerun the tests with more recent rsyslog and report back > > bodik > > Dne 01/08/2017 v 05:04 PM David Lang napsal(a): >> what version of rsyslog are you using? versions prior to ~8.20 had a >> known problem due to json-c not being thread-safe. >> >> David Lang >> >> On Sun, 8 Jan 2017, Radoslav Bodó via rsyslog wrote: >> >>> Date: Sun, 8 Jan 2017 16:57:37 +0100 >>> From: Radoslav Bodó via rsyslog <[email protected]> >>> To: [email protected] >>> Cc: Radoslav Bodó <[email protected]> >>> Subject: [rsyslog] replacing message parts with regex leads to message >>> corruption >>> >>> Hi, >>> >>> recently I was trying to create a masking template for software which >>> logs messages including potentialy sensitive information (remctld >>> logging whole command line executed). >>> >>> >>> So I created a rule for masking that part out before storing messages: >>> >>> -------------- rsyslog.d/neweb2.conf >>> $template Neweb2Format,"%timegenerated% %HOSTNAME% >>> %syslogtag%%!msg:::drop-last-lf%\n" >>> if ( ($programname == 'remctld') and ($msg contains 'neweb2') and ($msg >>> contains 'pwd') ) then { >>> set $!ext = re_extract($msg,'(pwd [^ ]+)',0,1,""); >>> set $!msg = replace($msg, $!ext, "pwd MASKEDOUT"); >>> action(type="omfile" template="Neweb2Format" >>> File="/var/log/syslog") >>> stop >>> } >>> ------------------------- >>> >>> >>> >>> acording to test a good behavior I've created a test case simulating >>> remctld logging and check desired output >>> >>> ------------------- neweb2/tests/remctl_syslog_masks.sh >>> #!/bin/sh >>> >>> . /puppet/metalib/bin/lib.sh >>> >>> RANDOM=$(/bin/dd if=/dev/urandom bs=100 count=1 2>/dev/null | >>> /usr/bin/sha256sum | /usr/bin/awk '{print $1}' | sed >>> 's/^\(......\).*/\1/') >>> >>> logger -t remctld "neweb2 db ${RANDOM}a --set --pwd 1234567890 --noop" >>> logger -t remctld "neweb2 db ${RANDOM}b --set --noop --pwd 1234567890" >>> logger -t remctld "neweb2 db --pwd 1234567890 --noop --set ${RANDOM}c" >>> >>> grep "neweb2 db ${RANDOM}a --set --pwd MASKEDOUT --noop" /var/log/syslog >>> if [ $? -ne 0 ]; then >>> rreturn 1 "$0 remctl neweb2 sensitive data not masked A" >>> fi >>> grep "neweb2 db ${RANDOM}b --set --noop --pwd MASKEDOUT" /var/log/syslog >>> if [ $? -ne 0 ]; then >>> rreturn 1 "$0 remctl neweb2 sensitive data not masked B" >>> fi >>> grep "neweb2 db --pwd MASKEDOUT --noop --set ${RANDOM}c" /var/log/syslog >>> if [ $? -ne 0 ]; then >>> rreturn 1 "$0 remctl neweb2 sensitive data not masked C" >>> fi >>> >>> rreturn 0 "$0" >>> ------------------------- >>> >>> >>> >>> but according to the test some of the messages gets garbled >>> >>> ---------- tail /var/log/syslog -n10 >>> Jan 8 16:28:21 tester remctld: neweb2 db ba539ba --set --pwd MASKEDOUT >>> --nooc >>> Jan 8 16:28:21 tester remctld: neweb2 db ba539bb --set --noop --pwd >>> MASKEDOUT >>> Jan 8 16:28:21 tester remctld: neweb2 db --pwd MASKEDOUT --noop --set >>> ba539bc >>> ------------------- >>> >>> >>> see the "--nooc" instead of "--noop" in the first case >>> >>> >>> >>> >>> >>> >>> I'd suspect: >>> >>> a) my usage of replace() is wrong >>> b) some memory management inside "property replacer" is not correct >>> >>> the system in question is debian stretch with rsyslog 8.16.0-1+b3 >>> >>> >>> >>> I'd be glad for any suggestions or cross-tests of this case. I could dig >>> into code, make some additional testing, or propose a patch, but I'm not >>> really sure where to start ... >>> >>> >>> Thank you for any help >>> Best regards >>> bodik >>> >>> >>> >>> >>> _______________________________________________ >>> rsyslog mailing list >>> http://lists.adiscon.net/mailman/listinfo/rsyslog >>> http://www.rsyslog.com/professional-services/ >>> What's up with rsyslog? Follow https://twitter.com/rgerhards >>> NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a >>> myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST >>> if you DON'T LIKE THAT. >>> > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > http://www.rsyslog.com/professional-services/ > What's up with rsyslog? Follow https://twitter.com/rgerhards > NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of > sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T > LIKE THAT. > _______________________________________________ rsyslog mailing list http://lists.adiscon.net/mailman/listinfo/rsyslog http://www.rsyslog.com/professional-services/ What's up with rsyslog? Follow https://twitter.com/rgerhards NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE THAT.
