On Thu, Jul 18, 2013 at 9:34 AM, Erik Steffl <[email protected]> wrote:
> On 07/17/2013 10:01 PM, Rainer Gerhards wrote: > >> On Thu, Jul 18, 2013 at 2:43 AM, Erik Steffl <[email protected]> wrote: >> >> On 07/16/2013 09:28 PM, David Lang wrote: >>> >>> take a look at >>> http://www.rsyslog.com/doc/****node32.html<http://www.rsyslog.com/doc/**node32.html> >>>> <http://www.**rsyslog.com/doc/node32.html<http://www.rsyslog.com/doc/node32.html> >>>> >**and see if it >>>> answers your questions. >>>> >>>> >>> since the queue is specified inside action does that mean that each >>> action needs its own queue? >>> >> >> >> Yes, and thanks to the new config format this is now clear ;) >> >> >> E.g. I have two actions that forward messages to remote log aggregator >>> and >>> would like them to be assisted by the same disk queue (so that I can >>> specify one queue.maxdiskspace). Is that possible? >>> >>> >>> No ... more below >> >> action( >>> type="omfwd" >>> template="some template" >>> queue.filename="/var/log/****myFile" >>> queue.maxdiskspace="****4294967296" >>> >>> ... >>> ) >>> ... >>> action( >>> type="omfwd" >>> template="a different template" >>> queue.filename="/var/log/****myFile" >>> queue.maxdiskspace="****4294967296" >>> >>> ... >>> ) >>> >>> Would this create one disk based queue with filename /var/log/myFile >>> that both actions would use or would this just mess things up? >>> >>> >> Old version messed things ab, current version rename the second queue file >> and emit a warning message. So you'll end up with two seperate queues in >> any case. >> >> Side-note: I guess you used omfile just as example, but one never news who >> will dig this up via google: it is absolutely counter-productive to use >> disk queus with files, as writing the actual output files is much faster >> (10 to 20 times) than writing a disk queue. So don't do that. Disk queues >> are for potentially slow outputs (databases, remote systems...). >> > > I am a bit confused, I use omfwd in the examples above (left out the > rest of omfwd args for brevity), just added the queue.filename and > queue.maxdiskspace to make sure I understand it corerectly. Do my examples > look correct? > My fault. I just saw /var/log and my mind switched to omfile, ignoring the facts. Sorry for the confusion. Yes, this sounds right (except that most folks usually do not put spool files in /var/log, what probably is a good idea). Rainer > > and yes, we plan to use the disk based queues so that the messages are > not lost if the remote log aggregator goes down or there is a network > problem or something like that... > > > Alternatively is there a way to specify total limit for all disk based >>> queues? (didn't find anything so I guess not but hope never dies :) >>> >>> >>> No, that would required an immense amount of synchronization with the >> current implementation, which makes this infeasable. >> >> Background info: the current queue system is stricly sequential. To do >> what >> you ask for would require a totally different subsystem, something along >> the lines of the OS page file system. Actually, I think such a thing to >> have would be a big plus. However, it is a very major task to do, I'd say >> 3+ month of work. I would need external funding /sponsorship for such a >> big >> task (at least a larger part of it). >> >> A work-around may be to put things into their own ruleset and create a >> ruleset queue. However, this means if one of the actions block, the other >> are not executed until that block is solved. May be acceptable based on >> the >> use case. >> >> Sorry I have no better answer. >> > > ok, I'll just make sure there is enough space for all the queues (only > have two so not a huge deal). > > thanks! > > erik > > >> Rainer >> >> thanks! >> >>> >>> erik >>> >>> >>> >>> David Lang >>>> >>>> On Tue, 16 Jul 2013, Erik Steffl wrote: >>>> >>>> Action definition: >>>> >>>>> >>>>> action( >>>>> type="omfwd" >>>>> target="somehost.com" >>>>> port="5140" >>>>> protocol="tcp" >>>>> template="json" >>>>> ) >>>>> >>>>> How to make this action use disk assisted queue? I did read >>>>> http://www.rsyslog.com/doc/****omfwd.html<http://www.rsyslog.com/doc/**omfwd.html> >>>>> <http://www.rsyslog.**com/doc/omfwd.html<http://www.rsyslog.com/doc/omfwd.html> >>>>> >and >>>>> http://www.rsyslog.com/doc/****queues.html<http://www.rsyslog.com/doc/**queues.html> >>>>> <http://www.**rsyslog.com/doc/queues.html<http://www.rsyslog.com/doc/queues.html> >>>>> > >>>>> >>>>> >>>>> Section 'Disk-Assisted Memory Queues' says 'If a disk queue name is >>>>> defined for in-memory queues (via $<object>QueueFileName), they >>>>> automatically become "disk-assisted" (DA)'. However I also see at >>>>> http://www.rsyslog.com/doc/****rsyslog_conf_actions.html<http://www.rsyslog.com/doc/**rsyslog_conf_actions.html> >>>>> <http**://www.rsyslog.com/doc/**rsyslog_conf_actions.html<http://www.rsyslog.com/doc/rsyslog_conf_actions.html>>that >>>>> legacy >>>>> >>>>> format does not affect RainerScript actions (which is what I use). >>>>> >>>>> Or is it going to automatically store it in a directory specified by >>>>> this: $WorkDirectory /var/spool/rsyslog and I don't have to worry >>>>> about it? >>>>> >>>>> thanks! >>>>> >>>>> erik >>>>> ______________________________****_________________ >>>>> rsyslog mailing list >>>>> http://lists.adiscon.net/****mailman/listinfo/rsyslog<http://lists.adiscon.net/**mailman/listinfo/rsyslog> >>>>> <http:**//lists.adiscon.net/mailman/**listinfo/rsyslog<http://lists.adiscon.net/mailman/listinfo/rsyslog> >>>>> > >>>>> http://www.rsyslog.com/****professional-services/<http://www.rsyslog.com/**professional-services/> >>>>> <http://**www.rsyslog.com/professional-**services/<http://www.rsyslog.com/professional-services/> >>>>> > >>>>> >>>>> What's up with rsyslog? Follow https://twitter.com/rgerhards >>>>> NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a >>>>> myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST >>>>> if you DON'T LIKE THAT. >>>>> >>>>> ______________________________****_________________ >>>>> >>>> rsyslog mailing list >>>> http://lists.adiscon.net/****mailman/listinfo/rsyslog<http://lists.adiscon.net/**mailman/listinfo/rsyslog> >>>> <http:**//lists.adiscon.net/mailman/**listinfo/rsyslog<http://lists.adiscon.net/mailman/listinfo/rsyslog> >>>> > >>>> http://www.rsyslog.com/****professional-services/<http://www.rsyslog.com/**professional-services/> >>>> <http://**www.rsyslog.com/professional-**services/<http://www.rsyslog.com/professional-services/> >>>> > >>>> >>>> What's up with rsyslog? Follow https://twitter.com/rgerhards >>>> NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad >>>> of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you >>>> DON'T LIKE THAT. >>>> >>>> >>> ______________________________****_________________ >>> rsyslog mailing list >>> http://lists.adiscon.net/****mailman/listinfo/rsyslog<http://lists.adiscon.net/**mailman/listinfo/rsyslog> >>> <http:**//lists.adiscon.net/mailman/**listinfo/rsyslog<http://lists.adiscon.net/mailman/listinfo/rsyslog> >>> > >>> http://www.rsyslog.com/****professional-services/<http://www.rsyslog.com/**professional-services/> >>> <http://**www.rsyslog.com/professional-**services/<http://www.rsyslog.com/professional-services/> >>> > >>> >>> What's up with rsyslog? Follow https://twitter.com/rgerhards >>> NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad >>> of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you >>> DON'T LIKE THAT. >>> >>> ______________________________**_________________ >> rsyslog mailing list >> http://lists.adiscon.net/**mailman/listinfo/rsyslog<http://lists.adiscon.net/mailman/listinfo/rsyslog> >> http://www.rsyslog.com/**professional-services/<http://www.rsyslog.com/professional-services/> >> What's up with rsyslog? Follow https://twitter.com/rgerhards >> NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad >> of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you >> DON'T LIKE THAT. >> >> > ______________________________**_________________ > rsyslog mailing list > http://lists.adiscon.net/**mailman/listinfo/rsyslog<http://lists.adiscon.net/mailman/listinfo/rsyslog> > http://www.rsyslog.com/**professional-services/<http://www.rsyslog.com/professional-services/> > What's up with rsyslog? Follow https://twitter.com/rgerhards > NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad > of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you > DON'T LIKE THAT. > _______________________________________________ rsyslog mailing list http://lists.adiscon.net/mailman/listinfo/rsyslog http://www.rsyslog.com/professional-services/ What's up with rsyslog? Follow https://twitter.com/rgerhards NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE THAT.
