OK, here is the fix: http://git.adiscon.com/?p=rsyslog.git;a=commit;h=5cd9b52ea5a75c4dff3d92683ff51d4e02cb1fd5
The bug was the missing ToPos. If you don't want to apply the patch now, you can simply set ToPos to a reasonable large number. Thanks for the report! Rainer > -----Original Message----- > From: [email protected] [mailto:rsyslog- > [email protected]] On Behalf Of Rainer Gerhards > Sent: Monday, January 07, 2013 3:17 PM > To: rsyslog-users > Subject: Re: [rsyslog] Some list templates with mmjsonparse try to get > 2GB of RAM or give OOM > > Quick info: already got it, need to check how to handle it > intelligently ;) > > > -----Original Message----- > > From: [email protected] [mailto:rsyslog- > > [email protected]] On Behalf Of Radu Gheorghe > > Sent: Monday, January 07, 2013 2:46 PM > > To: rsyslog-users > > Subject: [rsyslog] Some list templates with mmjsonparse try to get > 2GB > > of RAM or give OOM > > > > Hello, > > > > With the configuration below, rsyslog seems to try and allocate ~2GB > of > > RAM > > (and quickly free that memory afterwards, I suppose) for each line of > > log > > it gets. So if my machine has 2GB of RAM or less, I get "**OUT OF > > MEMORY**" > > for my "$!all-json" property. > > > > #####CONFIG START######## > > module(load="imuxsock") # provides support for local system logging > > module(load="mmjsonparse") #for parsing CEE-enhanced messages > > > > template(name="testTemplate" > > type="list") { > > property(name="$!all-json" position.from="2") > > } > > > > *.* :mmjsonparse: > > > > *.* action(type="omfile" > > file="/tmp/logs" > > template="testTemplate" > > ) > > #####CONFIG END######## > > > > If I run it with valgrind, I get this for every log line: > > ==2436== Warning: set address range perms: large range [0x39431040, > > 0xb078a440) (undefined) > > ==2436== Warning: set address range perms: large range [0x39431030, > > 0xb078a450) (noaccess) > > > > And after 3 logs, in the end I get this: > > ==2436== total heap usage: 1,815 allocs, 1,773 frees, 6,000,203,586 > > bytes > > allocated > > > > However, if I remove the position.from part of the template, I get a > > much > > more reasonable: > > ==2441== total heap usage: 1,805 allocs, 1,763 frees, 203,474 bytes > > allocated > > > > I'm attaching both both valgrind outputs. > > > > Another interesting thing is that the equivalent string template > works > > fine: > > template(name="testTemplate" type="string" > > string="%$!all-json:2:$::%" > > ) > > > > The problem is reproducible with rsyslog 7.2.4 and 7.3.5. I've tried > on > > CentOS 6.3 and Ubuntu 12.04, both 64-bit. I've tried with the Ubuntu > > packages and also compiled sources manually - same thing. > > > > Should I open a report in bugzilla about this? > > > > Also, I have a related question: is it OK to use $!all-json in > > production? > > I don't see it in the property replacer documentation, and I was > > wondering > > if there's a reason for that. > > > > And another question related to the related question: is there a way > > for > > contributing to the rsyslog official documentation (not wiki)? I'm > > particularly thinking about github pull requests - or maybe some > > similar > > system. I think it would be a nice way to keep the documentation > > updated. > > > > Best regards, > > Radu > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > http://www.rsyslog.com/professional-services/ > What's up with rsyslog? Follow https://twitter.com/rgerhards > NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a > myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST > if you DON'T LIKE THAT. _______________________________________________ rsyslog mailing list http://lists.adiscon.net/mailman/listinfo/rsyslog http://www.rsyslog.com/professional-services/ What's up with rsyslog? Follow https://twitter.com/rgerhards NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE THAT.
