Actually, I've managed to find a workaround to improve this approach.
Rather than using uptime, which has 1-second resolution, I'm using
timegenerated, which has microsecond resolution. I was concerned that running
at high rates and large numbers (eg. 100keps across 100 devices) would result
in each device being blasted by 100keps for 1 second.
This way, I flip every microsecond to a new destination. I chose not to use the event
time, since it may not have subsecond resolution. I also chose not to use the event
index number that I get inside every event (I have one), since it would require regex of
the form ".*?EventId=(\d+).*?", which could be expensive in large (6-8KB) log
lines, and slow down processing.
Next step is to go from UDP to Encrypted TCP, which should hopefully be easy.
Code snippet for the load balancing (across 3 in test);
# Load balance output based on system time
# Define a template that contains just the subsecond value of the event receipt
time (not event timestamp!).
# For timegenerated, this is in microseconds
template(name="subseconds" type="string"
string="%timegenerated:::date-subseconds%")
# Set a variable to that value
set $!subsecs = exec_template("subseconds");
# Perform a modulo of the subsecond value of the receipt time to decide which
way to send it
if ($!subsecs % 3 == 0) then call output_0
if ($!subsecs % 3 == 1) then call output_1
if ($!subsecs % 3 == 2) then call output_2
On Friday, 7 November 2014, 1:04, Damian <[email protected]> wrote:
Thanks David - I got it working:
In the end, the $$uptime % 3 == 0 property worked, and it reliably directed
traffic to each address for one-second intervals.
Also, having read field() from the link you mentioned, it's basically a substring
operator, so field(timegenerated,":",3) would return the seconds - which should
have a similar effect to the above, but based on the event timestamp. Hence, I assume
(not yet tested) it would take the numeric value and decide it's not an integer rather
than the parent string, so I could use it in a similar way. eg:
if (field($timegenerated,":",3) %3 == 0) then call destination_0
Thanks!
Damian
On Thursday, 6 November 2014, 12:56, David Lang <[email protected]> wrote:
On Thu, 6 Nov 2014, Damian Skeeles wrote:
Hi David,
Thanks, that's really good info. I'll have another go at uptime as my primary
focus.
I noticed there are some properties for the replacement properties to show the
time as epoch/Unix time, so that would be an integer (or could be converted to
one from string, if rsyslog has such an operation). Any ideas if these are also
available as properties, or only replacement properties?
all variables should be available in condition tests.
remember that you can also set a variable to
the output of a format operation.
I think that variable contents that look like numbers can be treated as numbers.
Rsyslog doesn't have types in it's variables. But if you try to do a math
operation on something that doesn't look like a number, it's not going to get
evaluated the way you want it to
Btw, any ideas on what the field() operator does? I couldn't find it anywhere
in the docs, and it's quite hard to google for by its nature.
rainerscript functions are defined at
http://www.rsyslog.com/doc/master/rainerscript/functions.html
I can't do clustering at the receiving end as there are existing products to
receive the events, and I want rsyslog to take the entire config/maintenance
load of the balancing. I need one machine, one install, one config, all free
and reliable, as the entire load balancing glue.
Ok, I'd still suggest that you take a look at the presentations. It requires
that you have access to the OS level to make changes, but it doesn't require
that the software receiving the logs know anything about it. I've used this
approach to deliver logs to proprietary software running on linux boxes in the
past with great success.
David Lang
Damian
On 6 Nov 2014, at
12:21, David Lang <[email protected]> wrote:
On Wed, 5 Nov 2014, Damian wrote:
Hi,
I'm currently working on trying to use rsyslog as a basic load balancer, by
selecting the output on a time basis. I'm using the discussion posted here as
my starting point:
http://lists.adiscon.net/pipermail/rsyslog/2013-October/034442.html
In this discussion, the authors looked at using:
if ($uptime % 3 == 0) then
action1
if ($uptime % 3 == 1) then action2
if ($uptime % 3 == 2) then action2
To use the system uptime to decide which way to send the events (so it would
average over the three destinations). However, this didn't work in 7.4, as
uptime is not available outside templates. I also found 8.4.2 to not like this
parameter.
try accessing $$uptime (yes it's ugly, but it's a combination of $ to refer to
the property name and the property name being named $uptime for legacy
reasons). In some versions I think this is magically combined so you can just
use $uptime, but I dont't remember what versions (if any) this worked in
For the original discussion, what eventually seemed to work was:
field($timegenerated,':',3);
However - it's not clear how this was used, and I can't see how it would refer
to three different destinations. It seems more of a string operation than a
modulus. When I try using this, rsyslog debug mode generates no errors, so it
seems to work. If I try something like:
if ($timegenerated % 3 == 0) then call output_0
if ($timegenerated % 3 == 1) then call output_1
if ($timegenerated % 3 == 2) then call output_2
Then it gives errors for these lines; it doesn't seem to work as an operation.
$timegenerated is a string, so it's not surprising that this fails.
Can anyone clarify what the field($...) operation does, and how I can use it.
Alternatively, any
suggestions as to how I can basically call a different ruleset if the
system/event seconds value is modulus 0, 1, or 2.
I would actually approach this on the receiving end instead.
on the sender, set the rebindinterval to something like 1000 and then on the
receiving end setup your multiple receivers to share an IP address and split
traffic between them using the iptables CLUSTERIP feature. I talk about this in
the presentation I gave at LISA 2012, video and paper are available at:
https://www.usenix.org/conference/lisa12/technical-sessions/presentation/lang_david
This will spread the traffic across the machines roughly every 1000 messages,
and while it
uses a different mechanism, I think it ends up being cleaner. It's definantly
easier to add new machines to the cluster as needed, and you can have something
like corosync (http://http://clusterlabs.org/) to detect failures to the
recieving servers and adjust the traffic load appropriately.
David Lang
_______________________________________________
rsyslog mailing list
http://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards
NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of
sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE
THAT.
