Hello all,
I'm currently trying to load balance the log traffic accross several servers.
I thought my configuration with "ActionSendTCPRebindInterval" option was
working properly, unfortunately my recent benchs show that the log flow is not
well load balanced.
Please find below a part of the architecture :
My problem is located on the log aggregators : the rsyslog send its traffic to
haproxy on localhost using relp protocol.
I monitored the tcp sessions and i can see that haproxy doesn't change the
destination servers.
watch 'ss -lap -o state established \( dport = :20514 \)'
Recv-Q Send-Q Local Address:Port Peer Address:Port
1716 0 127.0.0.1:43652 127.0.0.1:20514 users:(("rsyslogd",8409,88))
0 1138 10.17.252.4:58436 10.19.12.5:20514 timer:(on,196ms,0)
users:(("haproxy",3922
,2))
1760 0 127.0.0.1:43650 127.0.0.1:20514 users:(("rsyslogd",8409,22))
0 0 10.17.252.4:55583 10.19.12.6:20514 users:(("haproxy",3922,10))
Please find enclosed my configuration.
NB :
- the source pid (rsyslog) never change as it is expected with
"ActionSendTCPRebindInterval"
- i mixed legacy and new syntaxe because of the following bug
https://github.com/rsyslog/rsyslog/issues/96
This bug is annoying and i didn't receive any update since about 4 months
Could you please help me ?
OS : debian7
rsyslog version : 8.10
Regards ,
Smana
module(
load="impstats"
interval="30"
resetCounters="off"
format="cee"
log.file="/var/log/rsyslog-stats.log"
ruleset="monitoring"
)
module(load="mmjsonparse")
module(load="omelasticsearch")
module(load="omrelp")
module(load="imtcp")
input(type="imtcp" port="514" ruleset="forward")
template(name="stats"
type="list") {
constant(value="{")
property(name="timereported" dateFormat="rfc3339" format="jsonf"
outname="@timestamp") # the timestamp
constant(value=",")
property(name="hostname" format="jsonf" outname="host") # the host
generating stats
constant(value=",\"source\":\"impstats\",")
property(name="$!all-json" position.from="2")
}
ruleset(
name="monitoring"
queue.type="FixedArray"
queue.highwatermark="180000"
queue.filename="stats_ruleset"
queue.lowwatermark="20000"
queue.maxdiskspace="100m"
queue.size="3000000"
queue.dequeuebatchsize="1000"
queue.saveonshutdown="on"
){
action(
name="parse_impstats"
type="mmjsonparse"
)
action(
name="impstats_to_es"
type="omelasticsearch"
server="es-1"
serverport="9200"
template="stats"
searchIndex="monitoring"
searchType="rsyslog"
bulkmode="on"
action.resumeretrycount="-1"
)
}
# Ruleset: Send logs to insight plateform
ruleset(name="forward"){
$ActionName analytics
$ActionQueueType LinkedList
$ActionQueueFileName analytics-spool
$ActionResumeRetryCount -1
$ActionQueueSaveOnShutdown on
$ActionQueueHighWaterMark 10000
$ActionSendTCPRebindInterval 10000
if $programname startswith 'CDN.' then :omrelp:localhost:20514
}
_______________________________________________
rsyslog mailing list
http://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards
NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of
sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE
THAT.
