On 07/17/2013 10:01 PM, Rainer Gerhards wrote:
On Thu, Jul 18, 2013 at 2:43 AM, Erik Steffl <[email protected]> wrote:
On 07/16/2013 09:28 PM, David Lang wrote:
take a look at
http://www.rsyslog.com/doc/**node32.html<http://www.rsyslog.com/doc/node32.html>and
see if it
answers your questions.
since the queue is specified inside action does that mean that each
action needs its own queue?
Yes, and thanks to the new config format this is now clear ;)
E.g. I have two actions that forward messages to remote log aggregator and
would like them to be assisted by the same disk queue (so that I can
specify one queue.maxdiskspace). Is that possible?
No ... more below
action(
type="omfwd"
template="some template"
queue.filename="/var/log/**myFile"
queue.maxdiskspace="**4294967296"
...
)
...
action(
type="omfwd"
template="a different template"
queue.filename="/var/log/**myFile"
queue.maxdiskspace="**4294967296"
...
)
Would this create one disk based queue with filename /var/log/myFile
that both actions would use or would this just mess things up?
Old version messed things ab, current version rename the second queue file
and emit a warning message. So you'll end up with two seperate queues in
any case.
Side-note: I guess you used omfile just as example, but one never news who
will dig this up via google: it is absolutely counter-productive to use
disk queus with files, as writing the actual output files is much faster
(10 to 20 times) than writing a disk queue. So don't do that. Disk queues
are for potentially slow outputs (databases, remote systems...).
I am a bit confused, I use omfwd in the examples above (left out the
rest of omfwd args for brevity), just added the queue.filename and
queue.maxdiskspace to make sure I understand it corerectly. Do my
examples look correct?
and yes, we plan to use the disk based queues so that the messages
are not lost if the remote log aggregator goes down or there is a
network problem or something like that...
Alternatively is there a way to specify total limit for all disk based
queues? (didn't find anything so I guess not but hope never dies :)
No, that would required an immense amount of synchronization with the
current implementation, which makes this infeasable.
Background info: the current queue system is stricly sequential. To do what
you ask for would require a totally different subsystem, something along
the lines of the OS page file system. Actually, I think such a thing to
have would be a big plus. However, it is a very major task to do, I'd say
3+ month of work. I would need external funding /sponsorship for such a big
task (at least a larger part of it).
A work-around may be to put things into their own ruleset and create a
ruleset queue. However, this means if one of the actions block, the other
are not executed until that block is solved. May be acceptable based on the
use case.
Sorry I have no better answer.
ok, I'll just make sure there is enough space for all the queues
(only have two so not a huge deal).
thanks!
erik
Rainer
thanks!
erik
David Lang
On Tue, 16 Jul 2013, Erik Steffl wrote:
Action definition:
action(
type="omfwd"
target="somehost.com"
port="5140"
protocol="tcp"
template="json"
)
How to make this action use disk assisted queue? I did read
http://www.rsyslog.com/doc/**omfwd.html<http://www.rsyslog.com/doc/omfwd.html>and
http://www.rsyslog.com/doc/**queues.html<http://www.rsyslog.com/doc/queues.html>
Section 'Disk-Assisted Memory Queues' says 'If a disk queue name is
defined for in-memory queues (via $<object>QueueFileName), they
automatically become "disk-assisted" (DA)'. However I also see at
http://www.rsyslog.com/doc/**rsyslog_conf_actions.html<http://www.rsyslog.com/doc/rsyslog_conf_actions.html>that
legacy
format does not affect RainerScript actions (which is what I use).
Or is it going to automatically store it in a directory specified by
this: $WorkDirectory /var/spool/rsyslog and I don't have to worry
about it?
thanks!
erik
______________________________**_________________
rsyslog mailing list
http://lists.adiscon.net/**mailman/listinfo/rsyslog<http://lists.adiscon.net/mailman/listinfo/rsyslog>
http://www.rsyslog.com/**professional-services/<http://www.rsyslog.com/professional-services/>
What's up with rsyslog? Follow https://twitter.com/rgerhards
NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a
myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST
if you DON'T LIKE THAT.
______________________________**_________________
rsyslog mailing list
http://lists.adiscon.net/**mailman/listinfo/rsyslog<http://lists.adiscon.net/mailman/listinfo/rsyslog>
http://www.rsyslog.com/**professional-services/<http://www.rsyslog.com/professional-services/>
What's up with rsyslog? Follow https://twitter.com/rgerhards
NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad
of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you
DON'T LIKE THAT.
______________________________**_________________
rsyslog mailing list
http://lists.adiscon.net/**mailman/listinfo/rsyslog<http://lists.adiscon.net/mailman/listinfo/rsyslog>
http://www.rsyslog.com/**professional-services/<http://www.rsyslog.com/professional-services/>
What's up with rsyslog? Follow https://twitter.com/rgerhards
NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad
of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you
DON'T LIKE THAT.
_______________________________________________
rsyslog mailing list
http://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards
NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of
sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE
THAT.
_______________________________________________
rsyslog mailing list
http://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards
NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of
sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE
THAT.
