Strange. I copied that directly from
https://linux-help.org/wiki/logging/rsyslog/advanced-rsyslog.
I've moved that action block into the remote section as shown below and
restarted rsyslogd. Also changed the DefaultRuleSet to remote.
ruleset(name="remote") {
$IncludeConfig /etc/rsyslog.d/*.remote
action(type="omfile" DynaFile="RemoteHost")
*.* action(type="omelasticsearch"
server="localhost"
serverport="9200"
template="logstash"
searchIndex="logstash-index"
dynSearchIndex="on"
searchType="syslog"
bulkmode="on"
queue.type="linkedlist"
queue.size="5000"
queue.dequeuebatchsize="300"
action.resumeretrycount="-1")
}
$DefaultRuleset remote
But still no sign of security onion messages in elastic search. I'm doing all
this with very little understanding at this point. Any insight into how to
diagnose such problems would be most helpful.
Dr. Brad J. Cox Cell: 703-594-1883 Skype: dr.brad.cox
> On Jan 29, 2016, at 6:53 PM, David Lang <[email protected]> wrote:
>
> from a quick glance, the elasticsearch call is part of the local ruleset
> while anything that arrives remotely only has the remote ruleset applied to
> it.
>
> Am I missing something?
>
> David Lang
>
>
> On Fri, 29 Jan 2016, Brad Cox wrote:
>
>> Date: Fri, 29 Jan 2016 13:55:56 -0500
>> From: Brad Cox <[email protected]>
>> Reply-To: rsyslog-users <[email protected]>
>> To: rsyslog-users <[email protected]>
>> Subject: Re: [rsyslog] Logs from remote server not making it to Elastic
>> Search
>> OK. Do you need the rsyslog.d/* files too?
>>
>> # https://linux-help.org/wiki/logging/rsyslog/advanced-rsyslog
>> # rsyslog v7 configuration file
>>
>> # For more information see /usr/share/doc/rsyslog-*/rsyslog_conf.html
>> # If you experience problems, see
>> http://www.rsyslog.com/doc/troubleshoot.html
>> # $DebugFile /var/log/rsyslog.debug.log
>> # $DebugLevel 1
>>
>> #### MODULES ####
>>
>> module(load="imuxsock") # local system logging (e.g. via logger command)
>> module(load="imklog") # provides kernel logging (previously done by rklogd)
>> #module(load"immark") # --MARK-- message capability
>> module(load="mmnormalize")
>> module(load="omelasticsearch")
>>
>> # Provides UDP syslog reception
>> # for parameters see http://www.rsyslog.com/doc/imudp.html
>> #module(load="imudp") # needs to be done just once
>> #input(type="imudp" port="514")
>> module(load="imudp")
>>
>> # Provides TCP syslog reception
>> # for parameters see http://www.rsyslog.com/doc/imtcp.html
>> # module(load="imtcp") # needs to be done just once
>> # input(type="imtcp" port="10514")
>> # module(load="imtcp" MaxSessions="500")
>> # module(load="imrelp" RuleSet="remote")
>> module(load="imtcp" MaxSessions="500")
>> #input(type="imtcp" port="10514")
>> input(type="imudp" port="10514" ruleset="remote")
>> input(type="imtcp" port="10514" ruleset="remote")
>>
>> $AllowedSender TCP, 127.0.0.1, 192.168.48.137
>>
>> #### GLOBAL DIRECTIVES ####
>> # Templates
>> template(name="RemoteHost" type="string"
>> string="/srv/log/%HOSTNAME%/%$YEAR%/%$MONTH%/syslog-%$DAY%.log")
>> $IncludeConfig /etc/rsyslog.d/*.template
>>
>> # Use default timestamp format
>> $ActionFileDefaultTemplate RSYSLOG_TraditionalFileFormat
>>
>> # Include all config files in /etc/rsyslog.d/
>> $IncludeConfig /etc/rsyslog.d/*.conf
>>
>>
>> #### RULES ####
>> ruleset(name="local") {
>> # Log all kernel messages to the console.
>> # Logging much else clutters up the screen.
>> kern.* /var/log/kern.log
>>
>> # Log anything (except mail) of level info or higher.
>> # Don't log private authentication messages!
>> *.info;mail.none;authpriv.none;cron.none /var/log/messages
>>
>> # The authpriv file has restricted access.
>> authpriv.* /var/log/secure
>>
>> # Log all the mail messages in one place.
>> mail.* -/var/log/maillog
>>
>>
>> # Log cron stuff
>> cron.* /var/log/cron
>>
>> # Everybody gets emergency messages
>> *.emerg :omusrmsg:*
>>
>> # Save news errors of level crit and higher in a special file.
>> uucp,news.crit /var/log/spooler
>>
>> # Save boot messages also to boot.log
>> local7.* /var/log/boot.log
>>
>> *.* action(type="omelasticsearch"
>> server="localhost"
>> serverport="9200"
>> template="logstash"
>> searchIndex="logstash-index"
>> dynSearchIndex="on"
>> searchType="syslog"
>> bulkmode="on"
>> queue.type="linkedlist"
>> queue.size="5000"
>> queue.dequeuebatchsize="300"
>> action.resumeretrycount="-1")
>> }
>> $DefaultRuleset local
>>
>> ruleset(name="remote") {
>> $IncludeConfig /etc/rsyslog.d/*.remote
>> action(type="omfile" DynaFile="RemoteHost")
>> }
>> #input(type="imudp" port="10514" ruleset="remote")
>> #input(type="imtcp" port="10514" ruleset="remote")
>>
>> # $InputRELPServerBindRuleset remote
>> # input(type="imrelp" port="20514")
>>
>> Dr. Brad J. Cox Cell: 703-594-1883 Skype: dr.brad.cox
>>
>>
>>
>>
>>> On Jan 29, 2016, at 1:20 PM, David Lang <[email protected]> wrote:
>>>
>>> please post your config (rsyslog.conf)
>>>
>>> David Lang
>>>
>>> On Fri, 29 Jan 2016, Brad Cox wrote:
>>>
>>>> Date: Fri, 29 Jan 2016 12:58:34 -0500
>>>> From: Brad Cox <[email protected]>
>>>> Reply-To: rsyslog-users <[email protected]>
>>>> To: [email protected]
>>>> Subject: [rsyslog] Logs from remote server not making it to Elastic Search
>>>> I'm trying to centralize logs that originate from syslog-ng on a Security
>>>> Onion server (Ubuntu 14.04.3-11) at an central server running rsyslog
>>>> latest. I've configured SO to copy all logs to the central server on TCP
>>>> port 10514 and confirmed that Tcpdump shows log traffic (on TCP port 10514)
>>>> is leaving SO and arriving at the central server.
>>>>
>>>> My goal is to ingest the remote logs into Elastic Search so I've been
>>>> following the instructions at
>>>> https://linux-help.org/wiki/logging/rsyslog/advanced-rsyslog. I had to
>>>> tweak some settings to get past parsing errors from rsyslog but that seems
>>>> OK now (according to rsyslogd -f rsyslog.conf -N1 with and without
>>>> $DebugLevel=1). At least rsyslogd now launches and runs correctly.
>>>>
>>>> Problem is, only local logs are appearing in the Elastic Search index
>>>> (according to kibana). There's no sign of remote logs from Security Onion
>>>> in the ES indices. The only trace of remote logs is in
>>>> /srv/log/SecurityOnionVM/...) which seems to contain the right contents,
>>>> although I can find no signs of how they wound up there in the debug logs.
>>>>
>>>> I'm new at rsyslog and have ridden the documentation about as far as it
>>>> will take me. Can someone help me get this bus on its wheels?
>>>>
>>>> Dr. Brad J. Cox Cell: 703-594-1883 Blog: http://bradjcox.blogspot.com
>>>> _______________________________________________
>>>> rsyslog mailing list
>>>> http://lists.adiscon.net/mailman/listinfo/rsyslog
>>>> http://www.rsyslog.com/professional-services/
>>>> What's up with rsyslog? Follow https://twitter.com/rgerhards
>>>> NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad
>>>> of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you
>>>> DON'T LIKE THAT.
>>>>
>>> _______________________________________________
>>> rsyslog mailing list
>>> http://lists.adiscon.net/mailman/listinfo/rsyslog
>>> http://www.rsyslog.com/professional-services/
>>> What's up with rsyslog? Follow https://twitter.com/rgerhards
>>> NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of
>>> sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T
>>> LIKE THAT.
>>
>> _______________________________________________
>> rsyslog mailing list
>> http://lists.adiscon.net/mailman/listinfo/rsyslog
>> http://www.rsyslog.com/professional-services/
>> What's up with rsyslog? Follow https://twitter.com/rgerhards
>> NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of
>> sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T
>> LIKE THAT.
>>
> _______________________________________________
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog
> http://www.rsyslog.com/professional-services/
> What's up with rsyslog? Follow https://twitter.com/rgerhards
> NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of
> sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T
> LIKE THAT.
_______________________________________________
rsyslog mailing list
http://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards
NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of
sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE
THAT.
