2016-10-18 15:19 GMT+02:00 phrogz <phr...@phrogz.me>:
>
>
>
>>De : rsyslog-boun...@lists.adiscon.com <rsyslog-boun...@lists.adiscon.com> de 
>>la part de Rainer Gerhards <rgerha...@hq.adiscon.com>
>>Envoyé : mardi 18 octobre 2016 12:48
>>À : rsyslog-users
>>Objet : Re: [rsyslog] Action Queue misunderstanding
>>
>>2016-10-18 12:26 GMT+02:00 phrogz <phr...@phrogz.me>:
>>> Hello,
>>>
>>>
>>> 1/ I'm trying to understand why in impstats my action queue doesn't reach 
>>> an higher value than 1000 (and sometime there is a value for 'full', I 
>>> don't know why).
>>
>>you have commented out "queue.size", so it uses the default of 1,000.
>>If you specify a larger size, that larger value will be used.
>>
>>This is the same for DA as for other queues. What did you get the idea
>>it does not apply to DA?
>>
>>Rainer
>
>
> Thanks, that solved the issue in the point 1!
> I think I misundertood the following:
> Here: http://www.rsyslog.com/doc/v8-stable/concepts/queues.html ; Chapter 
> "Limiting the Queue Size": "[...]Disk assisted queues are special in that 
> they do not have any size limit.[...]"
>
> I still have the issue described in the second point.

I overlooked this. If you do not have at least 8.21.0, update to
current 8.22.0. It is far more robust in regard to queue failures.

Rainer
>
> Thanks
>
>
>>>
>>>
>>> e.g. :
>>>
>>>       "name": "main Q",
>>>       "origin": "core.queue",
>>>       "size": 10916,
>>>       "enqueued": 23215,
>>>       "full": 0,
>>>       "discarded_full": 0,
>>>       "discarded_nf": 0,
>>>       "maxqsize": 144059
>>>
>>>       "name": "action 1 queue",
>>>       "origin": "core.queue",
>>>       "size": 1000,
>>>       "enqueued": 17808,
>>>       "full": 63,
>>>       "discarded_full": 0,
>>>       "discarded_nf": 0,
>>>       "maxqsize": 1000
>>>
>>>       "name": "action 2 queue",
>>>       "origin": "core.queue",
>>>       "size": 0,
>>>       "enqueued": 0,
>>>       "full": 0,
>>>       "discarded_full": 0,
>>>       "discarded_nf": 0,
>>>       "maxqsize": 1000
>>>
>>>       "name": "action 1 queue[DA]",
>>>       "origin": "core.queue",
>>>       "size": 0,
>>>       "enqueued": 0,
>>>       "full": 0,
>>>       "discarded_full": 0,
>>>       "discarded_nf": 0,
>>>       "maxqsize": 0
>>>
>>>       "name": "action 2 queue[DA]",
>>>       "origin": "core.queue",
>>>       "size": 0,
>>>       "enqueued": 0,
>>>       "full": 0,
>>>       "discarded_full": 0,
>>>       "discarded_nf": 0,
>>>       "maxqsize": 0
>>>
>>>
>>> 2/ There is also something strange, there is two old queues in the rsyslog 
>>> spool that are not deleted.
>>> #####################
>>> #####################
>>> #####################
>>>
>>> here is the debug:
>>> 1768.304204149:main thread    : action 1 queue: starting queue
>>> 1768.304209377:main thread    : action 1 queue: is disk-assisted, disk will 
>>> be used on demand
>>> 1768.304216886:main thread    : action 1 queue: params: type 1, enq-only 0, 
>>> disk assisted 1, spoolDir '/apps/rsyslog/spool', maxFileSz 1048576, 
>>> maxQSize 1000, lqsize 0, pqsize 0, child 0, full delay 450, light delay 
>>> 700, deq batch size 1000, high wtrmrk 900, low wtrmrk 700, discardmrk 980, 
>>> max wrkr 4, min msgs f. wrkr 250
>>> 1768.304222849:main thread    : action 1 queue:Reg: finalizing construction 
>>> of worker thread pool (numworkerThreads 4)
>>> 1768.304228002:main thread    : action 1 queue:Reg/w0: finalizing 
>>> construction of worker instance data (for 13 actions)
>>> 1768.304236533:main thread    : action 1 queue:Reg/w1: finalizing 
>>> construction of worker instance data (for 13 actions)
>>> 1768.304243582:main thread    : action 1 queue:Reg/w2: finalizing 
>>> construction of worker instance data (for 13 actions)
>>> 1768.304260052:main thread    : action 1 queue:Reg/w3: finalizing 
>>> construction of worker instance data (for 13 actions)
>>> 1768.304275153:main thread    : action 1 queue:DAwpool: finalizing 
>>> construction of worker thread pool (numworkerThreads 1)
>>> 1768.304288963:main thread    : action 1 queue:DAwpool/w0: finalizing 
>>> construction of worker instance data (for 13 actions)
>>> 1768.304334755:main thread    : action 1 queue[DA]: starting queue
>>> 1768.304343275:main thread    : action 1 queue[DA]: .qi file name is 
>>> '/apps/rsyslog/spool/ToLogStash.qi', len 33
>>> 1768.304347249:main thread    : action 1 queue[DA]: I am a child
>>> 1768.304519153:main thread    : file stream 
>>> /apps/rsyslog/spool/ToLogStash.qi params: flush interval 0, async write 0
>>> 1768.304545844:main thread    : file '/apps/rsyslog/spool/ToLogStash.qi' 
>>> opened as #10 with mode 384
>>> 1768.304565987:main thread    : strm 0x7f43404b9890: opened file 
>>> '/apps/rsyslog/spool/ToLogStash.qi' for READ as 10
>>> 1768.304574600:main thread    : strm 0x7f43404b9890: file 10 read 543 bytes
>>> 1768.304603730:main thread    : file stream ToLogStash params: flush 
>>> interval 0, async write 0
>>> 1768.304626194:main thread    : file stream ToLogStash params: flush 
>>> interval 0, async write 0
>>> 1768.304636987:main thread    : file stream ToLogStash params: flush 
>>> interval 0, async write 0
>>> 1768.304658150:main thread    : file 
>>> '/apps/rsyslog/spool/ToLogStash.00038759' opened as #11 with mode 384
>>> 1768.304669771:main thread    : queue 'strm 0x7f43404bac20', file 
>>> '/apps/rsyslog/spool/ToLogStash.00038759' opened for non-append write, but 
>>> already contains 358469 bytes
>>> 1768.304674986:main thread    : strm 0x7f43404bac20: opened file 
>>> '/apps/rsyslog/spool/ToLogStash.00038759' for WRITE as 11
>>> 1768.304679648:main thread    : strm 0x7f43404bac20: file 11 seek, pos 639
>>> 1768.304691562:main thread    : file 
>>> '/apps/rsyslog/spool/ToLogStash.00038759' opened as #22 with mode 384
>>> 1768.304701188:main thread    : strm 0x7f43404bbeb0: opened file 
>>> '/apps/rsyslog/spool/ToLogStash.00038759' for READ as 22
>>> 1768.304705945:main thread    : strm 0x7f43404bbeb0: file 22 seek, pos 639
>>> 1768.304716357:main thread    : file 
>>> '/apps/rsyslog/spool/ToLogStash.00038759' opened as #23 with mode 384
>>> 1768.304726030:main thread    : strm 0x7f43404bd2a0: opened file 
>>> '/apps/rsyslog/spool/ToLogStash.00038759' for READ as 23
>>> 1768.304731297:main thread    : strm 0x7f43404bd2a0: file 23 seek, pos 639
>>> 1768.304736880:main thread    : strm 0x7f43404b9890: file 
>>> 10(/apps/rsyslog/spool/ToLogStash.qi) closing, bDeleteOnClose 0
>>> 1768.304750072:main thread    : action 1 queue[DA]: params: type 2, 
>>> enq-only 0, disk assisted 0, spoolDir '/apps/rsyslog/spool', maxFileSz 
>>> 1048576, maxQSize 0, lqsize 0, pqsize 0, child 1, full delay -1, light 
>>> delay -1, deq batch size 8, high wtrmrk 0, low wtrmrk 1, discardmrk 0, max 
>>> wrkr 1, min msgs f. wrkr 0
>>> 1768.304758727:main thread    : action 1 queue[DA]:Reg: finalizing 
>>> construction of worker thread pool (numworkerThreads 1)
>>> 1768.304764348:main thread    : action 1 queue[DA]:Reg/w0: finalizing 
>>> construction of worker instance data (for 13 actions)
>>> 1768.304769386:main thread    : action 1 queue[DA]: queue finished 
>>> initialization
>>> 1768.304776843:main thread    : action 1 queue: DA queue initialized, disk 
>>> queue 0x7f43404b9550
>>> 1768.304781135:main thread    : action 1 queue: queue finished 
>>> initialization
>>> 1768.304789476:main thread    : Action builtin:omfwd[0x7f4340483dc0]: queue 
>>> 0x7f4340484b60 started
>>> 1768.304794344:main thread    : iterateAllActions calling into action 
>>> 0x7f4340484ea0
>>> 1768.304798231:main thread    : action 2 queue: starting queue
>>> 1768.304803919:main thread    : action 2 queue: is disk-assisted, disk will 
>>> be used on demand
>>> 1768.304811645:main thread    : action 2 queue: params: type 1, enq-only 0, 
>>> disk assisted 1, spoolDir '/apps/rsyslog/spool', maxFileSz 1048576, 
>>> maxQSize 1000, lqsize 0, pqsize 0, child 0, full delay 450, light delay 
>>> 700, deq batch size 1000, high wtrmrk 900, low wtrmrk 700, discardmrk 980, 
>>> max wrkr 4, min msgs f. wrkr 250
>>> 1768.304817986:main thread    : action 2 queue:Reg: finalizing construction 
>>> of worker thread pool (numworkerThreads 4)
>>> 1768.304823043:main thread    : action 2 queue:Reg/w0: finalizing 
>>> construction of worker instance data (for 13 actions)
>>> 1768.304840673:main thread    : action 2 queue:Reg/w1: finalizing 
>>> construction of worker instance data (for 13 actions)
>>> 1768.304862659:main thread    : action 2 queue:Reg/w2: finalizing 
>>> construction of worker instance data (for 13 actions)
>>> 1768.304879227:main thread    : action 2 queue:Reg/w3: finalizing 
>>> construction of worker instance data (for 13 actions)
>>> 1768.304894375:main thread    : action 2 queue:DAwpool: finalizing 
>>> construction of worker thread pool (numworkerThreads 1)
>>> 1768.304899909:main thread    : action 2 queue:DAwpool/w0: finalizing 
>>> construction of worker instance data (for 13 actions)
>>> 1768.304916529:main thread    : action 2 queue[DA]: starting queue
>>> 1768.304922466:main thread    : action 2 queue[DA]: .qi file name is 
>>> '/apps/rsyslog/spool/RelayToLogStash.qi', len 38
>>> 1768.304926530:main thread    : action 2 queue[DA]: I am a child
>>> 1768.304938902:main thread    : file stream 
>>> /apps/rsyslog/spool/RelayToLogStash.qi params: flush interval 0, async 
>>> write 0
>>> 1768.304955158:main thread    : file 
>>> '/apps/rsyslog/spool/RelayToLogStash.qi' opened as #10 with mode 384
>>> 1768.304965285:main thread    : strm 0x7f43404cb7c0: opened file 
>>> '/apps/rsyslog/spool/RelayToLogStash.qi' for READ as 10
>>> 1768.304985578:main thread    : strm 0x7f43404cb7c0: file 10 read 560 bytes
>>> 1768.305009054:main thread    : file stream RelayToLogStash params: flush 
>>> interval 0, async write 0
>>> 1768.305064267:main thread    : file stream RelayToLogStash params: flush 
>>> interval 0, async write 0
>>> 1768.305074762:main thread    : file stream RelayToLogStash params: flush 
>>> interval 0, async write 0
>>> 1768.305151652:main thread    : file 
>>> '/apps/rsyslog/spool/RelayToLogStash.00002857' opened as #24 with mode 384
>>> 1768.305165535:main thread    : queue 'strm 0x7f43404ccbe0', file 
>>> '/apps/rsyslog/spool/RelayToLogStash.00002857' opened for non-append write, 
>>> but already contains 1012893 bytes
>>> 1768.305170609:main thread    : strm 0x7f43404ccbe0: opened file 
>>> '/apps/rsyslog/spool/RelayToLogStash.00002857' for WRITE as 24
>>> 1768.305175510:main thread    : strm 0x7f43404ccbe0: file 24 seek, pos 
>>> 438297
>>> 1768.305187379:main thread    : file 
>>> '/apps/rsyslog/spool/RelayToLogStash.00002857' opened as #25 with mode 384
>>> 1768.305197024:main thread    : strm 0x7f43404cdfd0: opened file 
>>> '/apps/rsyslog/spool/RelayToLogStash.00002857' for READ as 25
>>> 1768.305202122:main thread    : strm 0x7f43404cdfd0: file 25 seek, pos 1817
>>> 1768.305212704:main thread    : file 
>>> '/apps/rsyslog/spool/RelayToLogStash.00002857' opened as #26 with mode 384
>>> 1768.305221830:main thread    : strm 0x7f43404cf260: opened file 
>>> '/apps/rsyslog/spool/RelayToLogStash.00002857' for READ as 26
>>> 1768.305226641:main thread    : strm 0x7f43404cf260: file 26 seek, pos 1817
>>> 1768.305232314:main thread    : strm 0x7f43404cb7c0: file 
>>> 10(/apps/rsyslog/spool/RelayToLogStash.qi) closing, bDeleteOnClose 0
>>> 1768.305244857:main thread    : action 2 queue[DA]: params: type 2, 
>>> enq-only 0, disk assisted 0, spoolDir '/apps/rsyslog/spool', maxFileSz 
>>> 1048576, maxQSize 0, lqsize 731, pqsize 731, child 1, full delay -1, light 
>>> delay -1, deq batch size 8, high wtrmrk 0, low wtrmrk 1, discardmrk 0, max 
>>> wrkr 1, min msgs f. wrkr 0
>>> 1768.305253009:main thread    : action 2 queue[DA]:Reg: finalizing 
>>> construction of worker thread pool (numworkerThreads 1)
>>> 1768.305258858:main thread    : action 2 queue[DA]:Reg/w0: finalizing 
>>> construction of worker instance data (for 13 actions)
>>> 1768.305263853:main thread    : action 2 queue[DA]: queue finished 
>>> initialization
>>> 1768.305268754:main thread    : action 2 queue[DA]:Reg: high activity - 
>>> starting 1 additional worker thread(s).
>>> 1768.305374179:main thread    : action 2 queue[DA]:Reg: started with state 
>>> 0, num workers now 1
>>> 1768.305385824:main thread    : action 2 queue: DA queue initialized, disk 
>>> queue 0x7f43404cb3f0
>>> 1768.305390224:main thread    : action 2 queue: queue finished 
>>> initialization
>>> 1768.305397556:main thread    : Action builtin:omfwd[0x7f4340484ea0]: queue 
>>> 0x7f4340485460 started
>>> 1768.305402568:main thread    : Activating Ruleset Queue[(nil)] for Ruleset 
>>> RSYSLOG_DefaultRuleset
>>> 1768.305406401:main thread    : Activating Ruleset Queue[(nil)] for Ruleset 
>>> local
>>> 1768.305417331:7f433317b700: thread created, tid 4945, name 'rs:action 2 
>>> queue[DA]:R'
>>> 1768.305498188:action 2 queue[DA]:Reg/w0: wti 0x7f43404cb8f0: worker 
>>> starting
>>> 1768.305512959:main thread    : Activating Ruleset Queue[(nil)] for Ruleset 
>>> remote_tcp_secure_6514
>>> 1768.305536551:main thread    : Activating Ruleset Queue[(nil)] for Ruleset 
>>> remote_relay_secure_6514
>>> 1768.305542715:main thread    : activateMainQueue: mainq cnf obj ptr is 
>>> 0x7f434047ebb0
>>> 1768.305554811:main thread    : nvlstGetParam: name 'queue.size', type 9, 
>>> valnode->bUsed 0
>>> 1768.305560230:main thread    : nvlstGetParam: name 
>>> 'queue.dequeuebatchsize', type 6, valnode->bUsed 0
>>> 1768.305567499:main thread    : nvlstGetParam: name 'queue.workerthreads', 
>>> type 6, valnode->bUsed 0
>>> 1768.305573965:main thread    : queue param blk:
>>> 1768.305577887:main thread    : queue.filename: (unset)
>>> 1768.305594020:action 2 queue[DA]:Reg/w0: DeleteProcessedBatch: we deleted 
>>> 0 objects and enqueued 0 objects
>>> 1768.305616369:action 2 queue[DA]:Reg/w0: doDeleteBatch: delete batch from 
>>> store, new sizes: log 731, phys 731
>>> 1768.305630344:main thread    : queue.spooldirectory: (unset)
>>> 1768.305640736:action 2 queue[DA]:Reg/w0: strm 0x7f43404cf260: file 26 read 
>>> 4096 bytes
>>>
>>> 3/Here is my config:
>>> ############################
>>> ############################
>>> ############################
>>> global(
>>>   defaultNetstreamDriver="gtls"
>>>   #debug.gnutls="10"
>>>   #TCP TLS
>>>   defaultNetstreamDriverCAFile="/apps/rsyslog/certCA.crt"
>>>   defaultNetstreamDriverCertFile="/apps/rsyslog/certHost.crt"
>>>   defaultNetstreamDriverKeyFile="/apps/rsyslog/certHost.key"
>>> )
>>> module(load="omrelp")
>>> module(load="imptcp")
>>> module(
>>>   load="imtcp"
>>>   streamDriver.name="gtls"
>>>   streamDriver.mode="1"
>>>   streamDriver.authMode="x509/name"
>>>   permittedPeer=["sourcesrv1.test.lan","sourcesrv2.test.lan"]
>>> )
>>> module(
>>>  load="impstats"
>>>  interval="10"
>>>  resetCounters="on"
>>>  format="cee"
>>>  ruleset="remote_tcp_secure_6514"      # send stats to the 
>>> remote_tcp_secure_6514 ruleset
>>>  )
>>>
>>>
>>> template(name="SyslogSource" type="string"
>>>          string="<%pri%>%timestamp:::date-rfc3339% %hostname% %fromhost-ip% 
>>> %syslogtag% %msg%\n"
>>>         )
>>> template(name="SyslogSourceRelay" type="string"
>>>          string="<%pri%>%timestamp:::date-rfc3339% %hostname% %syslogtag% 
>>> %msg%\n"
>>>         )
>>>
>>> /* Input entries */
>>>
>>> input(type="imptcp" port="514" ruleset="remote_tcp_secure_6514")
>>> #input(type="imptcp" port="514" ruleset="remote_relp_secure_6514")
>>>
>>> input(
>>>   type="imtcp"
>>>   port="6514"
>>>   name="imtcp_secure"
>>>   ruleset="remote_relay_secure_6514"
>>> )
>>>
>>> main_queue(
>>>   queue.size="1000000"   # capacity of the main queue
>>>   queue.dequeuebatchsize="10000"  # process messages in batches of 1000 and 
>>> move them to the action queues
>>>   queue.workerthreads="2"  # 2 threads for the main queue
>>> )
>>>
>>> /* Remote Logging */
>>>
>>> RuleSet (name="remote_tcp_secure_6514")
>>> {
>>>     action(
>>>       type="omfwd"
>>>       target="destserv.test.lan"
>>>       port="6514"
>>>       protocol="tcp"
>>>       StreamDriverMode="1"
>>>       StreamDriver="gtls"
>>>       StreamDriverAuthMode="x509/name"
>>>       StreamDriverPermittedPeers="destserv.test.lan"
>>>       action.resumeRetryCount="-1"
>>>       queue.type="linkedList"
>>>       queue.filename="ToLogStash"
>>>       queue.spoolDirectory="/apps/rsyslog/spool"
>>>       queue.maxDiskSpace="298g"
>>>       queue.saveOnShutdown="on"
>>>       queue.dequeuebatchsize="2000"
>>>       RebindInterval="1000"
>>>       queue.workerthreads="4"
>>>       queue.highwatermark="500000"
>>>       queue.lowwatermark="200000"
>>>       template="SyslogSource"
>>>       #queue.size="1000000000" #Not needed for DA Queue
>>>       action.reportSuspension="on"
>>>       action.reportSuspensionContinuation="on"
>>>     )
>>> }
>>>
>>>
>>> RuleSet (name="remote_relay_secure_6514")
>>> {
>>>     action(
>>>       type="omfwd"
>>>       target="destserv.test.lan"
>>>       port="6514"
>>>       protocol="tcp"
>>>       StreamDriverMode="1"
>>>       StreamDriver="gtls"
>>>       StreamDriverAuthMode="x509/name"
>>>       StreamDriverPermittedPeers="destserv.test.lan"
>>>       action.resumeRetryCount="-1"
>>>       queue.type="linkedList"
>>>       queue.filename="RelayToLogStash"
>>>       queue.spoolDirectory="/apps/rsyslog/spool"
>>>       queue.maxDiskSpace="200g"
>>>       queue.saveOnShutdown="on"
>>>       queue.dequeuebatchsize="2000"
>>>       RebindInterval="1000"
>>>       queue.workerthreads="4"
>>>       queue.highwatermark="500000"
>>>       queue.lowwatermark="200000"
>>>       template="SyslogSourceRelay"
>>>       #queue.size="1000000000" #Not needed for DA Queue
>>>       action.reportSuspension="on"
>>>       action.reportSuspensionContinuation="on"
>>>     )
>>> }
>>>
>>> Thanks,
>>>
>>> Ludovic
>>>
>>>
>>> _______________________________________________
>>> rsyslog mailing list
>>> http://lists.adiscon.net/mailman/listinfo/rsyslog
>>> http://www.rsyslog.com/professional-services/
>>> What's up with rsyslog? Follow https://twitter.com/rgerhards
>>> NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of 
>>> sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T 
>>> LIKE THAT.
>>_______________________________________________
>>rsyslog mailing list
>>http://lists.adiscon.net/mailman/listinfo/rsyslog
>>http://www.rsyslog.com/professional-services/
>>What's up with rsyslog? Follow https://twitter.com/rgerhards
>>NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of 
>>sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T 
>>LIKE THAT.
>>
> _______________________________________________
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog
> http://www.rsyslog.com/professional-services/
> What's up with rsyslog? Follow https://twitter.com/rgerhards
> NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of 
> sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T 
> LIKE THAT.
_______________________________________________
rsyslog mailing list
http://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards
NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of 
sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE 
THAT.

Reply via email to