from a quick glance, the elasticsearch call is part of the local ruleset while
anything that arrives remotely only has the remote ruleset applied to it.
Am I missing something?
David Lang
On Fri, 29 Jan 2016, Brad Cox wrote:
Date: Fri, 29 Jan 2016 13:55:56 -0500
From: Brad Cox <[email protected]>
Reply-To: rsyslog-users <[email protected]>
To: rsyslog-users <[email protected]>
Subject: Re: [rsyslog] Logs from remote server not making it to Elastic Search
OK. Do you need the rsyslog.d/* files too?
# https://linux-help.org/wiki/logging/rsyslog/advanced-rsyslog
# rsyslog v7 configuration file
# For more information see /usr/share/doc/rsyslog-*/rsyslog_conf.html
# If you experience problems, see http://www.rsyslog.com/doc/troubleshoot.html
# $DebugFile /var/log/rsyslog.debug.log
# $DebugLevel 1
#### MODULES ####
module(load="imuxsock") # local system logging (e.g. via logger command)
module(load="imklog") # provides kernel logging (previously done by rklogd)
#module(load"immark") # --MARK-- message capability
module(load="mmnormalize")
module(load="omelasticsearch")
# Provides UDP syslog reception
# for parameters see http://www.rsyslog.com/doc/imudp.html
#module(load="imudp") # needs to be done just once
#input(type="imudp" port="514")
module(load="imudp")
# Provides TCP syslog reception
# for parameters see http://www.rsyslog.com/doc/imtcp.html
# module(load="imtcp") # needs to be done just once
# input(type="imtcp" port="10514")
# module(load="imtcp" MaxSessions="500")
# module(load="imrelp" RuleSet="remote")
module(load="imtcp" MaxSessions="500")
#input(type="imtcp" port="10514")
input(type="imudp" port="10514" ruleset="remote")
input(type="imtcp" port="10514" ruleset="remote")
$AllowedSender TCP, 127.0.0.1, 192.168.48.137
#### GLOBAL DIRECTIVES ####
# Templates
template(name="RemoteHost" type="string"
string="/srv/log/%HOSTNAME%/%$YEAR%/%$MONTH%/syslog-%$DAY%.log")
$IncludeConfig /etc/rsyslog.d/*.template
# Use default timestamp format
$ActionFileDefaultTemplate RSYSLOG_TraditionalFileFormat
# Include all config files in /etc/rsyslog.d/
$IncludeConfig /etc/rsyslog.d/*.conf
#### RULES ####
ruleset(name="local") {
# Log all kernel messages to the console.
# Logging much else clutters up the screen.
kern.* /var/log/kern.log
# Log anything (except mail) of level info or higher.
# Don't log private authentication messages!
*.info;mail.none;authpriv.none;cron.none /var/log/messages
# The authpriv file has restricted access.
authpriv.* /var/log/secure
# Log all the mail messages in one place.
mail.* -/var/log/maillog
# Log cron stuff
cron.* /var/log/cron
# Everybody gets emergency messages
*.emerg :omusrmsg:*
# Save news errors of level crit and higher in a special file.
uucp,news.crit /var/log/spooler
# Save boot messages also to boot.log
local7.* /var/log/boot.log
*.* action(type="omelasticsearch"
server="localhost"
serverport="9200"
template="logstash"
searchIndex="logstash-index"
dynSearchIndex="on"
searchType="syslog"
bulkmode="on"
queue.type="linkedlist"
queue.size="5000"
queue.dequeuebatchsize="300"
action.resumeretrycount="-1")
}
$DefaultRuleset local
ruleset(name="remote") {
$IncludeConfig /etc/rsyslog.d/*.remote
action(type="omfile" DynaFile="RemoteHost")
}
#input(type="imudp" port="10514" ruleset="remote")
#input(type="imtcp" port="10514" ruleset="remote")
# $InputRELPServerBindRuleset remote
# input(type="imrelp" port="20514")
Dr. Brad J. Cox Cell: 703-594-1883 Skype: dr.brad.cox
On Jan 29, 2016, at 1:20 PM, David Lang <[email protected]> wrote:
please post your config (rsyslog.conf)
David Lang
On Fri, 29 Jan 2016, Brad Cox wrote:
Date: Fri, 29 Jan 2016 12:58:34 -0500
From: Brad Cox <[email protected]>
Reply-To: rsyslog-users <[email protected]>
To: [email protected]
Subject: [rsyslog] Logs from remote server not making it to Elastic Search
I'm trying to centralize logs that originate from syslog-ng on a Security
Onion server (Ubuntu 14.04.3-11) at an central server running rsyslog
latest. I've configured SO to copy all logs to the central server on TCP
port 10514 and confirmed that Tcpdump shows log traffic (on TCP port 10514)
is leaving SO and arriving at the central server.
My goal is to ingest the remote logs into Elastic Search so I've been
following the instructions at
https://linux-help.org/wiki/logging/rsyslog/advanced-rsyslog. I had to
tweak some settings to get past parsing errors from rsyslog but that seems
OK now (according to rsyslogd -f rsyslog.conf -N1 with and without
$DebugLevel=1). At least rsyslogd now launches and runs correctly.
Problem is, only local logs are appearing in the Elastic Search index
(according to kibana). There's no sign of remote logs from Security Onion
in the ES indices. The only trace of remote logs is in
/srv/log/SecurityOnionVM/...) which seems to contain the right contents,
although I can find no signs of how they wound up there in the debug logs.
I'm new at rsyslog and have ridden the documentation about as far as it
will take me. Can someone help me get this bus on its wheels?
Dr. Brad J. Cox Cell: 703-594-1883 Blog: http://bradjcox.blogspot.com
_______________________________________________
rsyslog mailing list
http://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards
NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of
sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE
THAT.
_______________________________________________
rsyslog mailing list
http://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards
NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of
sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE
THAT.
_______________________________________________
rsyslog mailing list
http://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards
NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of
sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE
THAT.
_______________________________________________
rsyslog mailing list
http://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards
NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of
sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE
THAT.
