On Sat, 30 Jan 2016, Brad Cox wrote:
Thanks!Wasn't sure whether you meant to use your config instead of my default rsyslog-config or to insert yours at the beginning of my module statements. I did the former and patched a few errors to wind up with just this: module( load="impstats" interval="10" resetCounters="off" format="legacy" file="/var/log/pstats" ) # load the modules to listen to the network and configure them to send their # logs to the 'inbound' ruleset module(load="imtcp") module(load="imudp" timerequery="4" ) input(type="imtcp" port="10514" ruleset="inbound" ) input(type="imudp" port="10514" ruleset="inbound") # define the inbound ruleset, all logs will go to elasticsearch ruleset( name="inbound" queue.type="fixedarray" ){ action( name = "send_es" type="omelasticsearch" action.resumeretrycount="-1" ) What I'm seeing in /var/log/pstat is lots and lots of example at the end and still still no sign of traffic from security onion in kibana. Ideas? BTW: the goal is to keep SO clients and server completely intact and unmodified (except for the two lines I showed earlier that copy all sensor -> server events to a separate VM. This will grow into a spark/hadoop cluster that we'll evolve to do advanced analytics that SO doesn't provide. The mixing of syslog-ng and rsyslog is just because SO uses Ubuntu and bcoxVM uses Xubuntu. According to the docs they're interchangable. I've seen the recipe link you provided as well as a similar one that routes traffic via kafka. I used a different one (link provided earlier) because it didn't involve logstash/ruby to avoid unnecessary complexity. Once I get this recipe working I'll probably try via kafka. Sat Jan 30 12:13:09 2016: imtcp(10514): origin=imtcp submitted=0 Sat Jan 30 12:13:09 2016: imudp(*:10514): origin=imudp submitted=0 Sat Jan 30 12:13:09 2016: imudp(*:10514): origin=imudp submitted=0
these tell you how many messages have been submitted via these inputs.
Sat Jan 30 12:13:09 2016: resource-usage: origin=impstats utime=4000 stime=4000 maxrss=9724 minflt=184 majflt=0 inblock=8 oublock=176 nvcsw=124 nivcsw=4
ignore this for now, it is how many resources rsyslog has consumed in the last 10 sec
Sat Jan 30 12:13:09 2016: inbound: origin=core.queue size=0 enqueued=0 full=0 discarded.full=0 discarded.nf=0 maxqsize=0
how many messages have gone to the inbound ruleset (enqueued=0 means none)
Sat Jan 30 12:13:09 2016: main Q: origin=core.queue size=7 enqueued=422 full=0 discarded.full=0 discarded.nf=0 maxqsize=9
ignore this for now, it shows how many messages went to the main queue.
Sat Jan 30 12:13:09 2016: imudp(w0): origin=imudp called.recvmmsg=0 called.recvmsg=0 msgs.received=0
more stats on inbound udp, but you whould be getting things via tcp, so ignore it
Sat Jan 30 12:13:19 2016: omelasticsearch: origin=omelasticsearch submitted=0 failed.http=0 failed.httprequests=0 failed.es=0
stats from the elasticsearch module
Sat Jan 30 12:13:19 2016: send_es: origin=core.action processed=0 failed=0 suspended=0 suspended.duration=0 resumed=0
stats from the action delivering logs to ES
Sat Jan 30 12:14:08 2016: inbound: origin=core.queue size=0 enqueued=0 full=0 discarded.full=0 discarded.nf=0 maxqsize=0
so this shows that no logs have been deliverd to rsylog yet. you may need to kick syslog-ng on the senders to get it to try again.
do a grep -e imtcp -e inbount -e omelasticsearch -e send_es on the pstats file, we need the end of it when it shows some data arriving.
David Lang _______________________________________________ rsyslog mailing list http://lists.adiscon.net/mailman/listinfo/rsyslog http://www.rsyslog.com/professional-services/ What's up with rsyslog? Follow https://twitter.com/rgerhards NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE THAT.
