Re: [rsyslog] Action Queue misunderstanding
Oh ok, thanks a lot! Ludovic -Message d'origine- De : rsyslog-boun...@lists.adiscon.com [mailto:rsyslog-boun...@lists.adiscon.com] De la part de David Lang Envoyé : jeudi 27 octobre 2016 21:26 À : rsyslog-users Objet : Re: [rsyslog] Action Queue misunderstanding Yes, the file does not get re-written as logs are delivered, it's just pointers into the file that change (the overhead and race conditions related to re-writing a potentially large file when one message is delivered and additional messages are arriving are ugly) David Lang On Thu, 27 Oct 2016, phrogz wrote: > Thanks for the feedback, you right pstats show 0 for the DA queue, but is it > normal that the file (inside) is not empty? Because there are still some > messages in the queue, and If I take one random msg in this file I can see > that the msg has been transmitted to the backend database. > > Thanks, > > Ludovic > -Message d'origine- > De : rsyslog-boun...@lists.adiscon.com > [mailto:rsyslog-boun...@lists.adiscon.com] De la part de David Lang > Envoyé : jeudi 27 octobre 2016 20:42 À : rsyslog-users > Objet : Re: [rsyslog] Action Queue > misunderstanding > > disk assisted queues will leave one file in place when they are fully caught > up (in case there are more files that need to be queued), when you shutdown > rsyslog (not a HUP, but a full shutdown), this 'last' queue file will go away. > > but the pstats data should show the enqueued size as 0 for the DA version of > the queue. > > when you have a DA queue, it's actually two queues, a memory queue and a disk > assist queue, they show up separately in the pstats data, and the memory > queue will fill up before the disk queue starts being used. > > David Lang > > > On Thu, 20 Oct 2016, phrogz wrote: > >> Date: 20 Oct 2016 17:42:22 +0200 >> From: phrogz >> Reply-To: rsyslog-users >> To: rsyslog@lists.adiscon.com >> Subject: Re: [rsyslog] Action Queue misunderstanding >> >> yes, but normally the queues are cleaned? isn't it? >> >> 2016-10-20 17:34 GMT+02:00 phrogz : >>> Anyway, the logs in the queue files are delivered. >> >> So you are happy? >>> >>> Thanks, >>> >>> Ludovic >>> >>> ___ >>> rsyslog mailing list >>> http://lists.adiscon.net/mailman/listinfo/rsyslog >>> http://www.rsyslog.com/professional-services/ >>> What's up with rsyslog? Follow https://twitter.com/rgerhards NOTE >>> WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of >>> sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T >>> LIKE THAT. >> ___ >> rsyslog mailing list >> http://lists.adiscon.net/mailman/listinfo/rsyslog >> http://www.rsyslog.com/professional-services/ >> What's up with rsyslog? Follow https://twitter.com/rgerhards NOTE WELL: This >> is a PUBLIC mailing list, posts are ARCHIVED by a myriad of sites beyond our >> control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE THAT. >> >> >> ___ >> rsyslog mailing list >> http://lists.adiscon.net/mailman/listinfo/rsyslog >> http://www.rsyslog.com/professional-services/ >> What's up with rsyslog? Follow https://twitter.com/rgerhards NOTE >> WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of sites >> beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE >> THAT. >> > ___ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > http://www.rsyslog.com/professional-services/ > What's up with rsyslog? Follow https://twitter.com/rgerhards NOTE WELL: This > is a PUBLIC mailing list, posts are ARCHIVED by a myriad of sites beyond our > control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE THAT. > ___ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > http://www.rsyslog.com/professional-services/ > What's up with rsyslog? Follow https://twitter.com/rgerhards NOTE > WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of sites > beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE THAT. > ___ rsyslog mailing list http://lists.adiscon.net/mailman/listinfo/rsyslog http://www.rsyslog.com/professional-services/ What's up with rsyslog? Follow https://twitter.com/rgerhards NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE THAT.
Re: [rsyslog] Action Queue misunderstanding
Yes, the file does not get re-written as logs are delivered, it's just pointers into the file that change (the overhead and race conditions related to re-writing a potentially large file when one message is delivered and additional messages are arriving are ugly) David Lang On Thu, 27 Oct 2016, phrogz wrote: Thanks for the feedback, you right pstats show 0 for the DA queue, but is it normal that the file (inside) is not empty? Because there are still some messages in the queue, and If I take one random msg in this file I can see that the msg has been transmitted to the backend database. Thanks, Ludovic -Message d'origine- De : rsyslog-boun...@lists.adiscon.com [mailto:rsyslog-boun...@lists.adiscon.com] De la part de David Lang Envoyé : jeudi 27 octobre 2016 20:42 À : rsyslog-users Objet : Re: [rsyslog] Action Queue misunderstanding disk assisted queues will leave one file in place when they are fully caught up (in case there are more files that need to be queued), when you shutdown rsyslog (not a HUP, but a full shutdown), this 'last' queue file will go away. but the pstats data should show the enqueued size as 0 for the DA version of the queue. when you have a DA queue, it's actually two queues, a memory queue and a disk assist queue, they show up separately in the pstats data, and the memory queue will fill up before the disk queue starts being used. David Lang On Thu, 20 Oct 2016, phrogz wrote: Date: 20 Oct 2016 17:42:22 +0200 From: phrogz Reply-To: rsyslog-users To: rsyslog@lists.adiscon.com Subject: Re: [rsyslog] Action Queue misunderstanding yes, but normally the queues are cleaned? isn't it? 2016-10-20 17:34 GMT+02:00 phrogz : Anyway, the logs in the queue files are delivered. So you are happy? Thanks, Ludovic ___ rsyslog mailing list http://lists.adiscon.net/mailman/listinfo/rsyslog http://www.rsyslog.com/professional-services/ What's up with rsyslog? Follow https://twitter.com/rgerhards NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE THAT. ___ rsyslog mailing list http://lists.adiscon.net/mailman/listinfo/rsyslog http://www.rsyslog.com/professional-services/ What's up with rsyslog? Follow https://twitter.com/rgerhards NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE THAT. ___ rsyslog mailing list http://lists.adiscon.net/mailman/listinfo/rsyslog http://www.rsyslog.com/professional-services/ What's up with rsyslog? Follow https://twitter.com/rgerhards NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE THAT. ___ rsyslog mailing list http://lists.adiscon.net/mailman/listinfo/rsyslog http://www.rsyslog.com/professional-services/ What's up with rsyslog? Follow https://twitter.com/rgerhards NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE THAT. ___ rsyslog mailing list http://lists.adiscon.net/mailman/listinfo/rsyslog http://www.rsyslog.com/professional-services/ What's up with rsyslog? Follow https://twitter.com/rgerhards NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE THAT. ___ rsyslog mailing list http://lists.adiscon.net/mailman/listinfo/rsyslog http://www.rsyslog.com/professional-services/ What's up with rsyslog? Follow https://twitter.com/rgerhards NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE THAT.
Re: [rsyslog] Action Queue misunderstanding
Thanks for the feedback, you right pstats show 0 for the DA queue, but is it normal that the file (inside) is not empty? Because there are still some messages in the queue, and If I take one random msg in this file I can see that the msg has been transmitted to the backend database. Thanks, Ludovic -Message d'origine- De : rsyslog-boun...@lists.adiscon.com [mailto:rsyslog-boun...@lists.adiscon.com] De la part de David Lang Envoyé : jeudi 27 octobre 2016 20:42 À : rsyslog-users Objet : Re: [rsyslog] Action Queue misunderstanding disk assisted queues will leave one file in place when they are fully caught up (in case there are more files that need to be queued), when you shutdown rsyslog (not a HUP, but a full shutdown), this 'last' queue file will go away. but the pstats data should show the enqueued size as 0 for the DA version of the queue. when you have a DA queue, it's actually two queues, a memory queue and a disk assist queue, they show up separately in the pstats data, and the memory queue will fill up before the disk queue starts being used. David Lang On Thu, 20 Oct 2016, phrogz wrote: > Date: 20 Oct 2016 17:42:22 +0200 > From: phrogz > Reply-To: rsyslog-users > To: rsyslog@lists.adiscon.com > Subject: Re: [rsyslog] Action Queue misunderstanding > > yes, but normally the queues are cleaned? isn't it? > > 2016-10-20 17:34 GMT+02:00 phrogz : >> Anyway, the logs in the queue files are delivered. > > So you are happy? >> >> Thanks, >> >> Ludovic >> >> ___ >> rsyslog mailing list >> http://lists.adiscon.net/mailman/listinfo/rsyslog >> http://www.rsyslog.com/professional-services/ >> What's up with rsyslog? Follow https://twitter.com/rgerhards NOTE >> WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of sites >> beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE >> THAT. > ___ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > http://www.rsyslog.com/professional-services/ > What's up with rsyslog? Follow https://twitter.com/rgerhards NOTE WELL: This > is a PUBLIC mailing list, posts are ARCHIVED by a myriad of sites beyond our > control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE THAT. > > > ___ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > http://www.rsyslog.com/professional-services/ > What's up with rsyslog? Follow https://twitter.com/rgerhards NOTE > WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of sites > beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE THAT. > ___ rsyslog mailing list http://lists.adiscon.net/mailman/listinfo/rsyslog http://www.rsyslog.com/professional-services/ What's up with rsyslog? Follow https://twitter.com/rgerhards NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE THAT. ___ rsyslog mailing list http://lists.adiscon.net/mailman/listinfo/rsyslog http://www.rsyslog.com/professional-services/ What's up with rsyslog? Follow https://twitter.com/rgerhards NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE THAT.
Re: [rsyslog] Action Queue misunderstanding
disk assisted queues will leave one file in place when they are fully caught up (in case there are more files that need to be queued), when you shutdown rsyslog (not a HUP, but a full shutdown), this 'last' queue file will go away. but the pstats data should show the enqueued size as 0 for the DA version of the queue. when you have a DA queue, it's actually two queues, a memory queue and a disk assist queue, they show up separately in the pstats data, and the memory queue will fill up before the disk queue starts being used. David Lang On Thu, 20 Oct 2016, phrogz wrote: Date: 20 Oct 2016 17:42:22 +0200 From: phrogz Reply-To: rsyslog-users To: rsyslog@lists.adiscon.com Subject: Re: [rsyslog] Action Queue misunderstanding yes, but normally the queues are cleaned? isn't it? 2016-10-20 17:34 GMT+02:00 phrogz : Anyway, the logs in the queue files are delivered. So you are happy? Thanks, Ludovic ___ rsyslog mailing list http://lists.adiscon.net/mailman/listinfo/rsyslog http://www.rsyslog.com/professional-services/ What's up with rsyslog? Follow https://twitter.com/rgerhards NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE THAT. ___ rsyslog mailing list http://lists.adiscon.net/mailman/listinfo/rsyslog http://www.rsyslog.com/professional-services/ What's up with rsyslog? Follow https://twitter.com/rgerhards NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE THAT. ___ rsyslog mailing list http://lists.adiscon.net/mailman/listinfo/rsyslog http://www.rsyslog.com/professional-services/ What's up with rsyslog? Follow https://twitter.com/rgerhards NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE THAT. ___ rsyslog mailing list http://lists.adiscon.net/mailman/listinfo/rsyslog http://www.rsyslog.com/professional-services/ What's up with rsyslog? Follow https://twitter.com/rgerhards NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE THAT.
Re: [rsyslog] Action Queue misunderstanding
yes, but normally the queues are cleaned? isn't it? 2016-10-20 17:34 GMT+02:00 phrogz : > Anyway, the logs in the queue files are delivered. So you are happy? > > Thanks, > > Ludovic > > ___ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > http://www.rsyslog.com/professional-services/ > What's up with rsyslog? Follow https://twitter.com/rgerhards NOTE > WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of sites > beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE THAT. ___ rsyslog mailing list http://lists.adiscon.net/mailman/listinfo/rsyslog http://www.rsyslog.com/professional-services/ What's up with rsyslog? Follow https://twitter.com/rgerhards NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE THAT. ___ rsyslog mailing list http://lists.adiscon.net/mailman/listinfo/rsyslog http://www.rsyslog.com/professional-services/ What's up with rsyslog? Follow https://twitter.com/rgerhards NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE THAT.
Re: [rsyslog] Action Queue misunderstanding
2016-10-20 17:34 GMT+02:00 phrogz : > Anyway, the logs in the queue files are delivered. So you are happy? > > Thanks, > > Ludovic > > ___ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > http://www.rsyslog.com/professional-services/ > What's up with rsyslog? Follow https://twitter.com/rgerhards > NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of > sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T > LIKE THAT. ___ rsyslog mailing list http://lists.adiscon.net/mailman/listinfo/rsyslog http://www.rsyslog.com/professional-services/ What's up with rsyslog? Follow https://twitter.com/rgerhards NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE THAT.
Re: [rsyslog] Action Queue misunderstanding
Anyway, the logs in the queue files are delivered. Thanks, Ludovic ___ rsyslog mailing list http://lists.adiscon.net/mailman/listinfo/rsyslog http://www.rsyslog.com/professional-services/ What's up with rsyslog? Follow https://twitter.com/rgerhards NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE THAT.
Re: [rsyslog] Action Queue misunderstanding
Config: global( defaultNetstreamDriver="gtls" #debug.gnutls="10" #TCP TLS defaultNetstreamDriverCAFile="/apps/rsyslog/certCA.crt" defaultNetstreamDriverCertFile="/apps/rsyslog/certHost.crt" defaultNetstreamDriverKeyFile="/apps/rsyslog/certHost.key" ) module(load="omrelp") module(load="imptcp") module( load="imtcp" streamDriver.name="gtls" streamDriver.mode="1" streamDriver.authMode="x509/name" permittedPeer=["sourcesrv1.test.lan","sourcesrv2.test.lan"] ) module( load="impstats" interval="10" resetCounters="on" format="cee" ruleset="remote_tcp_secure_6514" # send stats to the remote_tcp_secure_6514 ruleset ) template(name="SyslogSource" type="string" string="<%pri%>%timestamp:::date-rfc3339% %hostname% %fromhost-ip% %syslogtag% %msg%\n" ) template(name="SyslogSourceRelay" type="string" string="<%pri%>%timestamp:::date-rfc3339% %hostname% %syslogtag% %msg%\n" ) /* Input entries */ input(type="imptcp" port="514" ruleset="remote_tcp_secure_6514") #input(type="imptcp" port="514" ruleset="remote_relp_secure_6514") input( type="imtcp" port="6514" name="imtcp_secure" ruleset="remote_relay_secure_6514" ) main_queue( queue.size="100" # capacity of the main queue queue.dequeuebatchsize="1" # process messages in batches of 1000 and move them to the action queues queue.workerthreads="2" # 2 threads for the main queue ) /* Remote Logging */ RuleSet (name="remote_tcp_secure_6514") { action( type="omfwd" name="SendToLogStashFrom514" target="destserv.test.lan" port="6514" protocol="tcp" StreamDriverMode="1" StreamDriver="gtls" StreamDriverAuthMode="x509/name" StreamDriverPermittedPeers="destserv.test.lan" action.resumeRetryCount="-1" queue.type="linkedList" queue.filename="ToLogStash" queue.spoolDirectory="/apps/rsyslog/spool" queue.maxDiskSpace="298g" queue.saveOnShutdown="on" queue.dequeuebatchsize="2000" RebindInterval="1000" queue.workerthreads="4" queue.highwatermark="50" queue.lowwatermark="20" template="SyslogSource" queue.size="10" action.reportSuspension="on" action.reportSuspensionContinuation="on" ) } RuleSet (name="remote_relay_secure_6514") { action( type="omfwd" name="RelayToLogStashFromTLS" target="destserv.test.lan" port="6514" protocol="tcp" StreamDriverMode="1" StreamDriver="gtls" StreamDriverAuthMode="x509/name" StreamDriverPermittedPeers="destserv.test.lan" action.resumeRetryCount="-1" queue.type="linkedList" queue.filename="RelayToLogStash" queue.spoolDirectory="/apps/rsyslog/spool" queue.maxDiskSpace="200g" queue.saveOnShutdown="on" queue.dequeuebatchsize="2000" RebindInterval="1000" queue.workerthreads="4" queue.highwatermark="50" queue.lowwatermark="20" template="SyslogSourceRelay" queue.size="10" action.reportSuspension="on" action.reportSuspensionContinuation="on" ) } # ## Debug: 0253.095392417:RelayToLogStashFromTLS queue[DA]:Reg/w0: Action 2 transitioned to state: rdy 0253.095397748:RelayToLogStashFromTLS queue[DA]:Reg/w0: actionCommit, action 2, in retry loop, iRet 0 0253.095403383:RelayToLogStashFromTLS queue[DA]:Reg/w0: regular consumer finished, iret=0, szlog 24 sz phys 32 0253.095427110:RelayToLogStashFromTLS queue[DA]:Reg/w0: DeleteProcessedBatch: we deleted 8 objects and enqueued 0 objects 0253.095433364:RelayToLogStashFromTLS queue[DA]:Reg/w0: doDeleteBatch: delete batch from store, new sizes: log 24, phys 24 0253.095616161:RelayToLogStashFromTLS queue[DA]:Reg/w0: strm 0x7f17367b5e70: file 28 read 4096 bytes 0253.095663836:RelayToLogStashFromTLS queue[DA]:Reg/w0: action 'RelayToLogStashFromTLS': is transactional - executing in commit phase 0253.095669813:RelayToLogStashFromTLS queue[DA]:Reg/w0: omfwd: beginTransaction 0253.095675302:RelayToLogStashFromTLS queue[DA]:Reg/w0: omfwd: doTryResume destserv.test.lan iRet 0 0253.095680569:RelayToLogStashFromTLS queue[DA]:Reg/w0: Action 2 transitioned to state: itx 0253.095687176:RelayToLogStashFromTLS queue[DA]:Reg/w0: action 'RelayToLogStashFromTLS': is transactional - executing in commit phase 0253.095693630:RelayToLogStashFromTLS queue[DA]:Reg/w0: action 'RelayToLogStashFromTLS': is transactional - executing in commit phase 0253.095700113:RelayToLogStashFromTLS queue[DA]:Reg/w0: action 'RelayToLogStashFromTLS': is transactional - executing in commit phase 0253.095706525:RelayToLogStashFromTLS queue[DA]:Reg/w0: action 'RelayToLogStashF
Re: [rsyslog] Action Queue misunderstanding
ok, 8.21 should do (but 8.22 is better as this is what I am working on). Can you name your actions so that we know where "action2" etc refers to. It is just action(name="myname" ...) Thx, Rainer 2016-10-18 15:54 GMT+02:00 phrogz : >>2016-10-18 15:19 GMT+02:00 phrogz : >>> >>> >>> >>>>De : rsyslog-boun...@lists.adiscon.com >>>>de la part de Rainer Gerhards >>>>Envoyé : mardi 18 octobre 2016 12:48 >>>>À : rsyslog-users >>>>Objet : Re: [rsyslog] Action Queue misunderstanding >>>> >>>>2016-10-18 12:26 GMT+02:00 phrogz : >>>>> Hello, >>>>> >>>>> >>>>> 1/ I'm trying to understand why in impstats my action queue doesn't reach >>>>> an higher value than 1000 (and sometime there is a value for 'full', I >>>>> don't know why). >>>> >>>>you have commented out "queue.size", so it uses the default of 1,000. >>>>If you specify a larger size, that larger value will be used. >>>> >>>>This is the same for DA as for other queues. What did you get the idea >>>>it does not apply to DA? >>>> >>>>Rainer >>> >>> >>> Thanks, that solved the issue in the point 1! >>> I think I misundertood the following: >>> Here: http://www.rsyslog.com/doc/v8-stable/concepts/queues.html ; Chapter >>> "Limiting the Queue Size": "[...]Disk assisted queues are special in that >>> they do not have any size limit.[...]" >>> >>> I still have the issue described in the second point. >> >>I overlooked this. If you do not have at least 8.21.0, update to >>current 8.22.0. It is far more robust in regard to queue failures. >> >>Rainer >>> >>> Thanks > > I've the version 8.21, I will try an update to 8.22 > > Ludovic > >>> >>> >>>>> >>>>> >>>>> e.g. : >>>>> >>>>> "name": "main Q", >>>>> "origin": "core.queue", >>>>> "size": 10916, >>>>> "enqueued": 23215, >>>>> "full": 0, >>>>> "discarded_full": 0, >>>>> "discarded_nf": 0, >>>>> "maxqsize": 144059 >>>>> >>>>> "name": "action 1 queue", >>>>> "origin": "core.queue", >>>>> "size": 1000, >>>>> "enqueued": 17808, >>>>> "full": 63, >>>>> "discarded_full": 0, >>>>> "discarded_nf": 0, >>>>> "maxqsize": 1000 >>>>> >>>>> "name": "action 2 queue", >>>>> "origin": "core.queue", >>>>> "size": 0, >>>>> "enqueued": 0, >>>>> "full": 0, >>>>> "discarded_full": 0, >>>>> "discarded_nf": 0, >>>>> "maxqsize": 1000 >>>>> >>>>> "name": "action 1 queue[DA]", >>>>> "origin": "core.queue", >>>>> "size": 0, >>>>> "enqueued": 0, >>>>> "full": 0, >>>>> "discarded_full": 0, >>>>> "discarded_nf": 0, >>>>> "maxqsize": 0 >>>>> >>>>> "name": "action 2 queue[DA]", >>>>> "origin": "core.queue", >>>>> "size": 0, >>>>> "enqueued": 0, >>>>> "full": 0, >>>>> "discarded_full": 0, >>>>> "discarded_nf": 0, >>>>> "maxqsize": 0 >>>>> >>>>> >>>>> 2/ There is also something strange, there is two old queues in the >>>>> rsyslog spool that are not deleted. >>>>> # >>>>> # >>>>> # >>>>> >>>
Re: [rsyslog] Action Queue misunderstanding
>2016-10-18 15:19 GMT+02:00 phrogz : >> >> >> >>>De : rsyslog-boun...@lists.adiscon.com >>>de la part de Rainer Gerhards >>>Envoyé : mardi 18 octobre 2016 12:48 >>>À : rsyslog-users >>>Objet : Re: [rsyslog] Action Queue misunderstanding >>> >>>2016-10-18 12:26 GMT+02:00 phrogz : >>>> Hello, >>>> >>>> >>>> 1/ I'm trying to understand why in impstats my action queue doesn't reach >>>> an higher value than 1000 (and sometime there is a value for 'full', I >>>> don't know why). >>> >>>you have commented out "queue.size", so it uses the default of 1,000. >>>If you specify a larger size, that larger value will be used. >>> >>>This is the same for DA as for other queues. What did you get the idea >>>it does not apply to DA? >>> >>>Rainer >> >> >> Thanks, that solved the issue in the point 1! >> I think I misundertood the following: >> Here: http://www.rsyslog.com/doc/v8-stable/concepts/queues.html ; Chapter >> "Limiting the Queue Size": "[...]Disk assisted queues are special in that >> they do not have any size limit.[...]" >> >> I still have the issue described in the second point. > >I overlooked this. If you do not have at least 8.21.0, update to >current 8.22.0. It is far more robust in regard to queue failures. > >Rainer >> >> Thanks I've the version 8.21, I will try an update to 8.22 Ludovic >> >> >>>> >>>> >>>> e.g. : >>>> >>>> "name": "main Q", >>>> "origin": "core.queue", >>>> "size": 10916, >>>> "enqueued": 23215, >>>> "full": 0, >>>> "discarded_full": 0, >>>> "discarded_nf": 0, >>>> "maxqsize": 144059 >>>> >>>> "name": "action 1 queue", >>>> "origin": "core.queue", >>>> "size": 1000, >>>> "enqueued": 17808, >>>> "full": 63, >>>> "discarded_full": 0, >>>> "discarded_nf": 0, >>>> "maxqsize": 1000 >>>> >>>> "name": "action 2 queue", >>>> "origin": "core.queue", >>>> "size": 0, >>>> "enqueued": 0, >>>> "full": 0, >>>> "discarded_full": 0, >>>> "discarded_nf": 0, >>>> "maxqsize": 1000 >>>> >>>> "name": "action 1 queue[DA]", >>>> "origin": "core.queue", >>>> "size": 0, >>>> "enqueued": 0, >>>> "full": 0, >>>> "discarded_full": 0, >>>> "discarded_nf": 0, >>>> "maxqsize": 0 >>>> >>>> "name": "action 2 queue[DA]", >>>> "origin": "core.queue", >>>> "size": 0, >>>> "enqueued": 0, >>>> "full": 0, >>>> "discarded_full": 0, >>>> "discarded_nf": 0, >>>> "maxqsize": 0 >>>> >>>> >>>> 2/ There is also something strange, there is two old queues in the rsyslog >>>> spool that are not deleted. >>>> # >>>> # >>>> # >>>> >>>> here is the debug: >>>> 1768.304204149:main thread: action 1 queue: starting queue >>>> 1768.304209377:main thread: action 1 queue: is disk-assisted, disk >>>> will be used on demand >>>> 1768.304216886:main thread: action 1 queue: params: type 1, enq-only >>>> 0, disk assisted 1, spoolDir '/apps/rsyslog/spool', maxFileSz 1048576, >>>> maxQSize 1000, lqsize 0, pqsize 0, child 0, full delay 450, light delay >>>> 700, deq batch size 1000, high wtrmrk 900, low wtrmrk 700, discardmrk 980, >>>> max wrkr 4, min msgs f. wrkr 250 >>>>
Re: [rsyslog] Action Queue misunderstanding
2016-10-18 15:19 GMT+02:00 phrogz : > > > >>De : rsyslog-boun...@lists.adiscon.com de >>la part de Rainer Gerhards >>Envoyé : mardi 18 octobre 2016 12:48 >>À : rsyslog-users >>Objet : Re: [rsyslog] Action Queue misunderstanding >> >>2016-10-18 12:26 GMT+02:00 phrogz : >>> Hello, >>> >>> >>> 1/ I'm trying to understand why in impstats my action queue doesn't reach >>> an higher value than 1000 (and sometime there is a value for 'full', I >>> don't know why). >> >>you have commented out "queue.size", so it uses the default of 1,000. >>If you specify a larger size, that larger value will be used. >> >>This is the same for DA as for other queues. What did you get the idea >>it does not apply to DA? >> >>Rainer > > > Thanks, that solved the issue in the point 1! > I think I misundertood the following: > Here: http://www.rsyslog.com/doc/v8-stable/concepts/queues.html ; Chapter > "Limiting the Queue Size": "[...]Disk assisted queues are special in that > they do not have any size limit.[...]" > > I still have the issue described in the second point. I overlooked this. If you do not have at least 8.21.0, update to current 8.22.0. It is far more robust in regard to queue failures. Rainer > > Thanks > > >>> >>> >>> e.g. : >>> >>> "name": "main Q", >>> "origin": "core.queue", >>> "size": 10916, >>> "enqueued": 23215, >>> "full": 0, >>> "discarded_full": 0, >>> "discarded_nf": 0, >>> "maxqsize": 144059 >>> >>> "name": "action 1 queue", >>> "origin": "core.queue", >>> "size": 1000, >>> "enqueued": 17808, >>> "full": 63, >>> "discarded_full": 0, >>> "discarded_nf": 0, >>> "maxqsize": 1000 >>> >>> "name": "action 2 queue", >>> "origin": "core.queue", >>> "size": 0, >>> "enqueued": 0, >>> "full": 0, >>> "discarded_full": 0, >>> "discarded_nf": 0, >>> "maxqsize": 1000 >>> >>> "name": "action 1 queue[DA]", >>> "origin": "core.queue", >>> "size": 0, >>> "enqueued": 0, >>> "full": 0, >>> "discarded_full": 0, >>> "discarded_nf": 0, >>> "maxqsize": 0 >>> >>> "name": "action 2 queue[DA]", >>> "origin": "core.queue", >>> "size": 0, >>> "enqueued": 0, >>> "full": 0, >>> "discarded_full": 0, >>> "discarded_nf": 0, >>> "maxqsize": 0 >>> >>> >>> 2/ There is also something strange, there is two old queues in the rsyslog >>> spool that are not deleted. >>> # >>> # >>> # >>> >>> here is the debug: >>> 1768.304204149:main thread: action 1 queue: starting queue >>> 1768.304209377:main thread: action 1 queue: is disk-assisted, disk will >>> be used on demand >>> 1768.304216886:main thread: action 1 queue: params: type 1, enq-only 0, >>> disk assisted 1, spoolDir '/apps/rsyslog/spool', maxFileSz 1048576, >>> maxQSize 1000, lqsize 0, pqsize 0, child 0, full delay 450, light delay >>> 700, deq batch size 1000, high wtrmrk 900, low wtrmrk 700, discardmrk 980, >>> max wrkr 4, min msgs f. wrkr 250 >>> 1768.304222849:main thread: action 1 queue:Reg: finalizing construction >>> of worker thread pool (numworkerThreads 4) >>> 1768.304228002:main thread: action 1 queue:Reg/w0: finalizing >>> construction of worker instance data (for 13 actions) >>> 1768.304236533:main thread: action 1 queue:Reg/w1: finalizing >>> construction of worker instance data (for 13 actions) >>> 1768.304243582:main thread: action 1 queue:Reg/w2: finalizing
Re: [rsyslog] Action Queue misunderstanding
>De : rsyslog-boun...@lists.adiscon.com de >la part de Rainer Gerhards >Envoyé : mardi 18 octobre 2016 12:48 >À : rsyslog-users >Objet : Re: [rsyslog] Action Queue misunderstanding > >2016-10-18 12:26 GMT+02:00 phrogz : >> Hello, >> >> >> 1/ I'm trying to understand why in impstats my action queue doesn't reach an >> higher value than 1000 (and sometime there is a value for 'full', I don't >> know why). > >you have commented out "queue.size", so it uses the default of 1,000. >If you specify a larger size, that larger value will be used. > >This is the same for DA as for other queues. What did you get the idea >it does not apply to DA? > >Rainer Thanks, that solved the issue in the point 1! I think I misundertood the following: Here: http://www.rsyslog.com/doc/v8-stable/concepts/queues.html ; Chapter "Limiting the Queue Size": "[...]Disk assisted queues are special in that they do not have any size limit.[...]" I still have the issue described in the second point. Thanks >> >> >> e.g. : >> >> "name": "main Q", >> "origin": "core.queue", >> "size": 10916, >> "enqueued": 23215, >> "full": 0, >> "discarded_full": 0, >> "discarded_nf": 0, >> "maxqsize": 144059 >> >> "name": "action 1 queue", >> "origin": "core.queue", >> "size": 1000, >> "enqueued": 17808, >> "full": 63, >> "discarded_full": 0, >> "discarded_nf": 0, >> "maxqsize": 1000 >> >> "name": "action 2 queue", >> "origin": "core.queue", >> "size": 0, >> "enqueued": 0, >> "full": 0, >> "discarded_full": 0, >> "discarded_nf": 0, >> "maxqsize": 1000 >> >> "name": "action 1 queue[DA]", >> "origin": "core.queue", >> "size": 0, >> "enqueued": 0, >> "full": 0, >> "discarded_full": 0, >> "discarded_nf": 0, >> "maxqsize": 0 >> >> "name": "action 2 queue[DA]", >> "origin": "core.queue", >> "size": 0, >> "enqueued": 0, >> "full": 0, >> "discarded_full": 0, >> "discarded_nf": 0, >> "maxqsize": 0 >> >> >> 2/ There is also something strange, there is two old queues in the rsyslog >> spool that are not deleted. >> # >> # >> # >> >> here is the debug: >> 1768.304204149:main thread: action 1 queue: starting queue >> 1768.304209377:main thread: action 1 queue: is disk-assisted, disk will >> be used on demand >> 1768.304216886:main thread: action 1 queue: params: type 1, enq-only 0, >> disk assisted 1, spoolDir '/apps/rsyslog/spool', maxFileSz 1048576, maxQSize >> 1000, lqsize 0, pqsize 0, child 0, full delay 450, light delay 700, deq >> batch size 1000, high wtrmrk 900, low wtrmrk 700, discardmrk 980, max wrkr >> 4, min msgs f. wrkr 250 >> 1768.304222849:main thread: action 1 queue:Reg: finalizing construction >> of worker thread pool (numworkerThreads 4) >> 1768.304228002:main thread: action 1 queue:Reg/w0: finalizing >> construction of worker instance data (for 13 actions) >> 1768.304236533:main thread: action 1 queue:Reg/w1: finalizing >> construction of worker instance data (for 13 actions) >> 1768.304243582:main thread: action 1 queue:Reg/w2: finalizing >> construction of worker instance data (for 13 actions) >> 1768.304260052:main thread: action 1 queue:Reg/w3: finalizing >> construction of worker instance data (for 13 actions) >> 1768.304275153:main thread: action 1 queue:DAwpool: finalizing >> construction of worker thread pool (numworkerThreads 1) >> 1768.304288963:main thread: action 1 queue:DAwpool/w0: finalizing >> construction of worker instance data (for 13 actions) >> 1768.304334755:main thread: action 1 queue[DA]: starting queue >> 1768.304343275:main thread: action 1 que
Re: [rsyslog] Action Queue misunderstanding
2016-10-18 12:26 GMT+02:00 phrogz : > Hello, > > > 1/ I'm trying to understand why in impstats my action queue doesn't reach an > higher value than 1000 (and sometime there is a value for 'full', I don't > know why). you have commented out "queue.size", so it uses the default of 1,000. If you specify a larger size, that larger value will be used. This is the same for DA as for other queues. What did you get the idea it does not apply to DA? Rainer > > > e.g. : > > "name": "main Q", > "origin": "core.queue", > "size": 10916, > "enqueued": 23215, > "full": 0, > "discarded_full": 0, > "discarded_nf": 0, > "maxqsize": 144059 > > "name": "action 1 queue", > "origin": "core.queue", > "size": 1000, > "enqueued": 17808, > "full": 63, > "discarded_full": 0, > "discarded_nf": 0, > "maxqsize": 1000 > > "name": "action 2 queue", > "origin": "core.queue", > "size": 0, > "enqueued": 0, > "full": 0, > "discarded_full": 0, > "discarded_nf": 0, > "maxqsize": 1000 > > "name": "action 1 queue[DA]", > "origin": "core.queue", > "size": 0, > "enqueued": 0, > "full": 0, > "discarded_full": 0, > "discarded_nf": 0, > "maxqsize": 0 > > "name": "action 2 queue[DA]", > "origin": "core.queue", > "size": 0, > "enqueued": 0, > "full": 0, > "discarded_full": 0, > "discarded_nf": 0, > "maxqsize": 0 > > > 2/ There is also something strange, there is two old queues in the rsyslog > spool that are not deleted. > # > # > # > > here is the debug: > 1768.304204149:main thread: action 1 queue: starting queue > 1768.304209377:main thread: action 1 queue: is disk-assisted, disk will > be used on demand > 1768.304216886:main thread: action 1 queue: params: type 1, enq-only 0, > disk assisted 1, spoolDir '/apps/rsyslog/spool', maxFileSz 1048576, maxQSize > 1000, lqsize 0, pqsize 0, child 0, full delay 450, light delay 700, deq batch > size 1000, high wtrmrk 900, low wtrmrk 700, discardmrk 980, max wrkr 4, min > msgs f. wrkr 250 > 1768.304222849:main thread: action 1 queue:Reg: finalizing construction > of worker thread pool (numworkerThreads 4) > 1768.304228002:main thread: action 1 queue:Reg/w0: finalizing > construction of worker instance data (for 13 actions) > 1768.304236533:main thread: action 1 queue:Reg/w1: finalizing > construction of worker instance data (for 13 actions) > 1768.304243582:main thread: action 1 queue:Reg/w2: finalizing > construction of worker instance data (for 13 actions) > 1768.304260052:main thread: action 1 queue:Reg/w3: finalizing > construction of worker instance data (for 13 actions) > 1768.304275153:main thread: action 1 queue:DAwpool: finalizing > construction of worker thread pool (numworkerThreads 1) > 1768.304288963:main thread: action 1 queue:DAwpool/w0: finalizing > construction of worker instance data (for 13 actions) > 1768.304334755:main thread: action 1 queue[DA]: starting queue > 1768.304343275:main thread: action 1 queue[DA]: .qi file name is > '/apps/rsyslog/spool/ToLogStash.qi', len 33 > 1768.304347249:main thread: action 1 queue[DA]: I am a child > 1768.304519153:main thread: file stream /apps/rsyslog/spool/ToLogStash.qi > params: flush interval 0, async write 0 > 1768.304545844:main thread: file '/apps/rsyslog/spool/ToLogStash.qi' > opened as #10 with mode 384 > 1768.304565987:main thread: strm 0x7f43404b9890: opened file > '/apps/rsyslog/spool/ToLogStash.qi' for READ as 10 > 1768.304574600:main thread: strm 0x7f43404b9890: file 10 read 543 bytes > 1768.304603730:main thread: file stream ToLogStash params: flush interval > 0, async write 0 > 1768.304626194:main thread: file stream ToLogStash params: flush interval > 0, async write 0 > 1768.304636987:main thread: file stream ToLogStash params: flush interval > 0, async write 0 > 1768.304658150:main thread: file > '/apps/rsyslog/spool/ToLogStash.00038759' opened as #11 with mode 384 > 1768.304669771:main thread: queue 'strm 0x7f43404bac20', file > '/apps/rsyslog/spool/ToLogStash.00038759' opened for non-append write, but > already contains 358469 bytes > 1768.304674986:main thread: strm 0x7f43404bac20: opened file > '/apps/rsyslog/spool/ToLogStash.00038759' for WRITE as 11 > 1768.304679648:main thread: strm 0x7f43404bac20: file 11 seek, pos 639 > 1768.304691562:main thread: file > '/apps/rsyslog/spool/ToLogStash.00038759' opened as #22 with mode 384 > 1768.304701188:main thread: strm 0x7f43404bbeb0: opened file > '/apps/rsyslog/spool/ToLogStash.00038759' for READ as 22 > 1768.304705945:main thread: strm 0x7f43404bbeb0: file 22 seek, pos 639 > 1768.304716357:main thread: file > '/apps/rsyslog/spool/ToLogStash.00038759'