In my case, using field extraction instead of regexp really increases performance.
> -----Original Message----- > From: Grégoire Seux > Sent: mercredi 4 décembre 2013 13:26 > To: rsyslog-users > Subject: RE: [rsyslog] regex instance shared between all workers > > I have checked at least in v7-stable branch and it seems to be the same code. > > > -----Original Message----- > > From: [email protected] [mailto:rsyslog- > > [email protected]] On Behalf Of Rainer Gerhards > > Sent: mercredi 4 décembre 2013 13:02 > > To: rsyslog-users > > Subject: Re: [rsyslog] regex instance shared between all workers > > > > ah, it's still a very interesting note. I think v8 does most probably have > > the same problem... > > > > Rainer > > > > > > On Wed, Dec 4, 2013 at 11:48 AM, David Lang <[email protected]> wrote: > > > > > 5.8 is quite old at this point, there are a lot of changes in 8.1 that are > > > designed to significantly improve multi-threaded workloads, I would > > suggest > > > looking at that because there is no further development of the 5.x > branch, > > > and things have probably changed in three major releases. > > > > > > David Lang > > > > > > On Wed, 4 Dec 2013, Grégoire Seux wrote: > > > > > > Date: Wed, 4 Dec 2013 10:41:29 +0000 > > >> From: Grégoire Seux <[email protected]> > > >> Reply-To: rsyslog-users <[email protected]> > > >> To: "rsyslog-users ([email protected])" < > > >> [email protected]> > > >> Subject: [rsyslog] regex instance shared between all workers > > >> > > >> > > >> Hello, > > >> > > >> I have noticed that all workers dequeing messages in main queue are > > using > > >> the same objects instances (at least in 5.8.10). > > >> This creates a lot of contention when using regexp since matching a > > >> string against a regexp take a lock on the regexp instance (in libc) > > >> > > >> As a result, all threads but one are waiting to acquire the lock. > > >> Here is a copy of one backtrace: > > >> > > >> Thread 4 (Thread 0x7f7ad75fe700 (LWP 25735)): > > >> #0 0x00007f7b532b306e in __lll_lock_wait_private () from > /lib64/libc.so.6 > > >> #1 0x00007f7b5328852a in _L_lock_49262 () from /lib64/libc.so.6 > > >> #2 0x00007f7b532845fc in regexec@@GLIBC_2.3.4 () from > /lib64/libc.so.6 > > >> #3 0x00007f7b541e4b80 in MsgGetProp (pMsg=<value optimized out>, > > >> pTpe=0x7f7b555fa500, > > >> propID=<value optimized out>, pPropLen=0x7f7ad75fd158, > > >> pbMustBeFreed=0x7f7ad75fd16e) > > >> at msg.c:2621 > > >> #4 0x00007f7b54204979 in tplToString (pTpl=0x7f7b555f9370, > > >> pMsg=0x7f7aa3ed1a40, > > >> ppBuf=0x7f7b519984a0, pLenBuf=0x7f7b519984c0) at > ../template.c:129 > > >> #5 0x00007f7b54202305 in prepareDoActionParams > > (pAction=0x7f7b5560b370, > > >> pBatch=0x7f7b55641e48, pbShutdownImmediate=<value optimized > > out>) at > > >> ../action.c:780 > > >> #6 prepareBatch (pAction=0x7f7b5560b370, pBatch=0x7f7b55641e48, > > >> pbShutdownImmediate=<value optimized out>) at ../action.c:1166 > > >> #7 processBatchMain (pAction=0x7f7b5560b370, > > pBatch=0x7f7b55641e48, > > >> pbShutdownImmediate=<value optimized out>) at ../action.c:1207 > > >> #8 0x00007f7b541ff93f in doQueueEnqObjDirectBatch > > >> (pAction=0x7f7b5560b370, > > >> pBatch=0x7f7b55641e48) at ../action.c:1659 > > >> #9 0x00007f7b54200f2b in doSubmitToActionQNotAllMarkBatch > > >> (pAction=0x7f7b5560b370, > > >> pBatch=0x7f7b55641e48) at ../action.c:1568 > > >> #10 0x00007f7b541fbfd3 in processBatchDoActions > > (pData=0x7f7b5560b370, > > >> pParam=<value optimized out>) at rule.c:101 > > >> #11 0x00007f7b541e653f in llExecFunc (pThis=0x7f7b556090b0, > > >> pFunc=0x7f7b541fbfa0 <processBatchDoActions>, > > pParam=0x7f7b55641e48) > > >> at linkedlist.c:389 > > >> #12 0x00007f7b541fc3b3 in processBatch (pThis=0x7f7b55609060, > > >> pBatch=0x7f7b55641e48) > > >> at rule.c:292 > > >> #13 0x00007f7b541fb3de in processBatchDoRules > > (pData=0x7f7b55609060, > > >> pParam=0x7f7b55641e48) > > >> at ruleset.c:150 > > >> #14 0x00007f7b541e653f in llExecFunc (pThis=0x7f7b555f4410, > > >> pFunc=0x7f7b541fb3b0 <processBatchDoRules>, > > pParam=0x7f7b55641e48) at > > >> linkedlist.c:389 > > >> #15 0x00007f7b541fb866 in processBatch (pBatch=0x7f7b55641e48) at > > >> ruleset.c:232 > > >> #16 0x00007f7b541d3641 in msgConsumer (notNeeded=<value > optimized > > out>, > > >> pBatch=0x7f7b55641e48, pbShutdownImmediate=0x7f7b55641028) at > > >> syslogd.c:719 > > >> #17 0x00007f7b541faedb in ConsumerReg (pThis=0x7f7b55641010, > > >> pWti=0x7f7b55641e20) > > >> at queue.c:1760 > > >> #18 0x00007f7b541f3e46 in wtiWorker (pThis=0x7f7b55641e20) at > > wti.c:313 > > >> #19 0x00007f7b541f393a in wtpWorker (arg=0x7f7b55641e20) at > wtp.c:387 > > >> #20 0x00007f7b53b77851 in start_thread () from /lib64/libpthread.so.0 > > >> #21 0x00007f7b532a390d in clone () from /lib64/libc.so.6 > > >> > > >> > > >> This is sad because each worker could (should?) create its own instance > > >> of the regex. > > >> > > >> Do you agree with that conclusion? > > >> > > >> The configuration except is: > > >> $template All,"/var/opt/logs/technicals/%msg:R,ERE,1,ZERO:^ > > >> \+([a-zA-Z/0-9\-]+)\+(.*)--end%.log" > > >> $template RemoveCR, "%TIMESTAMP:::date-rfc3339% %HOSTNAME% > > >> %msg:R,ERE,2,FIELD:^ \+([a-zA-Z/0-9\-]+)\+(.*)--end%\n" > > >> :fromhost-ip,!isequal,"127.0.0.1" -?All;RemoveCR > > >> > > >> I can include the full configuration if needed. > > >> > > >> Right now the only solution I see would be to load balance messages > > >> between several actions doing exactly the same treatment (but maybe > > this > > >> would create new issues). > > >> > > >> > > > _______________________________________________ > > > rsyslog mailing list > > > http://lists.adiscon.net/mailman/listinfo/rsyslog > > > http://www.rsyslog.com/professional-services/ > > > What's up with rsyslog? Follow https://twitter.com/rgerhards > > > NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad > > > of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if > you > > > DON'T LIKE THAT. > > > > > _______________________________________________ > > rsyslog mailing list > > http://lists.adiscon.net/mailman/listinfo/rsyslog > > http://www.rsyslog.com/professional-services/ > > What's up with rsyslog? Follow https://twitter.com/rgerhards > > NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of > > sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you > > DON'T LIKE THAT. _______________________________________________ rsyslog mailing list http://lists.adiscon.net/mailman/listinfo/rsyslog http://www.rsyslog.com/professional-services/ What's up with rsyslog? Follow https://twitter.com/rgerhards NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE THAT.
