On Jan 30, 2016, at 9:22 PM, David Lang <[email protected]> wrote:
On Sat, 30 Jan 2016, Brad Cox wrote:
Thanks!
Wasn't sure whether you meant to use your config instead of my default
rsyslog-config or to insert yours at the beginning of my module statements. I
did the former and patched a few errors to wind up with just this:
module(
load="impstats"
interval="10"
resetCounters="off"
format="legacy"
file="/var/log/pstats"
)
# load the modules to listen to the network and configure them to send their
# logs to the 'inbound' ruleset
module(load="imtcp")
module(load="imudp" timerequery="4" )
input(type="imtcp" port="10514" ruleset="inbound" )
input(type="imudp" port="10514" ruleset="inbound")
# define the inbound ruleset, all logs will go to elasticsearch
ruleset(
name="inbound"
queue.type="fixedarray"
){
action(
name = "send_es"
type="omelasticsearch"
action.resumeretrycount="-1"
)
What I'm seeing in /var/log/pstat is lots and lots of example at the end and
still still no sign of traffic from security onion in kibana. Ideas?
BTW: the goal is to keep SO clients and server completely intact and unmodified
(except for the two lines I showed earlier that copy all sensor -> server
events to a separate VM. This will grow into a spark/hadoop cluster that we'll
evolve to do advanced analytics that SO doesn't provide. The mixing of syslog-ng
and rsyslog is just because SO uses Ubuntu and bcoxVM uses Xubuntu. According to
the docs they're interchangable.
I've seen the recipe link you provided as well as a similar one that routes
traffic via kafka. I used a different one (link provided earlier) because it
didn't involve logstash/ruby to avoid unnecessary complexity. Once I get this
recipe working I'll probably try via kafka.
Sat Jan 30 12:13:09 2016: imtcp(10514): origin=imtcp submitted=0
Sat Jan 30 12:13:09 2016: imudp(*:10514): origin=imudp submitted=0
Sat Jan 30 12:13:09 2016: imudp(*:10514): origin=imudp submitted=0
these tell you how many messages have been submitted via these inputs.
Sat Jan 30 12:13:09 2016: resource-usage: origin=impstats utime=4000 stime=4000
maxrss=9724 minflt=184 majflt=0 inblock=8 oublock=176 nvcsw=124 nivcsw=4
ignore this for now, it is how many resources rsyslog has consumed in the last
10 sec
Sat Jan 30 12:13:09 2016: inbound: origin=core.queue size=0 enqueued=0 full=0
discarded.full=0 discarded.nf=0 maxqsize=0
how many messages have gone to the inbound ruleset (enqueued=0 means none)
Sat Jan 30 12:13:09 2016: main Q: origin=core.queue size=7 enqueued=422 full=0
discarded.full=0 discarded.nf=0 maxqsize=9
ignore this for now, it shows how many messages went to the main queue.
Sat Jan 30 12:13:09 2016: imudp(w0): origin=imudp called.recvmmsg=0
called.recvmsg=0 msgs.received=0
more stats on inbound udp, but you whould be getting things via tcp, so ignore
it
Sat Jan 30 12:13:19 2016: omelasticsearch: origin=omelasticsearch submitted=0
failed.http=0 failed.httprequests=0 failed.es=0
stats from the elasticsearch module
Sat Jan 30 12:13:19 2016: send_es: origin=core.action processed=0 failed=0
suspended=0 suspended.duration=0 resumed=0
stats from the action delivering logs to ES
Sat Jan 30 12:14:08 2016: inbound: origin=core.queue size=0 enqueued=0 full=0
discarded.full=0 discarded.nf=0 maxqsize=0
so this shows that no logs have been deliverd to rsylog yet. you may need to
kick syslog-ng on the senders to get it to try again.
do a grep -e imtcp -e inbount -e omelasticsearch -e send_es on the pstats file,
we need the end of it when it shows some data arriving.
David Lang
_______________________________________________
rsyslog mailing list
http://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards
NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of
sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE
THAT.
