Ok David, I got it working. It turns out I had issues on two sides. One was in the file paths for the rsyslog.conf and then on the other side in the kibana webroot. I had the wrong url for the elasticsearch config.js. It is working now! :) Thanks for the help.
Josh -----Original Message----- From: [email protected] [mailto:[email protected]] On Behalf Of David Lang Sent: Wednesday, April 30, 2014 4:27 PM To: rsyslog-users Subject: Re: [rsyslog] Rsyslog w/ logstash-elasticsearch-kibana server no, the file is probably huge, look for 'error' in it grep -i error debug.log or search for the contents of a message that you know should be going to elasticsearch. The error message should be pretty obvious, and when you see it you will probably be able to figure out what's wrong. David Lang On Wed, 30 Apr 2014, Josh Bitto wrote: > Date: Wed, 30 Apr 2014 10:44:14 -0700 > From: Josh Bitto <[email protected]> > Reply-To: rsyslog-users <[email protected]> > To: rsyslog-users <[email protected]> > Subject: Re: [rsyslog] Rsyslog w/ logstash-elasticsearch-kibana server > > Ok I think I got it to debug... > > I stopped the rsyslog service and then entered this at the command line. > /sbin/rsyslogd -c3 -dn > /var/log/debug.log > > Then it gave me a file. Should I post the entire contents? > > > > Josh > > > > > > -----Original Message----- > From: [email protected] > [mailto:[email protected]] On Behalf Of David Lang > Sent: Tuesday, April 29, 2014 4:44 PM > To: rsyslog-users > Subject: Re: [rsyslog] Rsyslog w/ logstash-elasticsearch-kibana server > > Ok, I woould guess that your elasticsarch actions are 10 and 11 (you > would have to count the outputs in the file or add the name parameter > to the action() > calls) > > So this is saying that it's attempting to deliver the messages to > elasticsearch but is failing. > > try starting in debug mode and see exactly what's happening when it tries to > deliver a message. > > David Lang > > On Tue, 29 Apr 2014, Josh Bitto wrote: > >> Date: Tue, 29 Apr 2014 16:24:42 -0700 >> From: Josh Bitto <[email protected]> >> Reply-To: rsyslog-users <[email protected]> >> To: rsyslog-users <[email protected]> >> Subject: Re: [rsyslog] Rsyslog w/ logstash-elasticsearch-kibana >> server >> >> This is the output >> >> Apr 29 16:23:15 syslogtest rsyslogd-pstats: imuxsock: submitted=1 >> ratelimit.discarded=0 ratelimit.numratelimiters=1 Apr 29 16:23:15 >> syslogtest rsyslogd-pstats: omelasticsearch: submitted=2 >> failed.http=2 >> failed.httprequests=2 failed.es=0 Apr 29 16:23:15 syslogtest >> rsyslogd-pstats: action 1: processed=10 failed=10 suspended=1 >> suspended.duration=60 resumed=0 Apr 29 16:23:15 syslogtest >> rsyslogd-pstats: action 2: processed=0 failed=0 suspended=0 >> suspended.duration=0 resumed=0 Apr 29 16:23:15 syslogtest >> rsyslogd-pstats: action 3: processed=9 failed=0 suspended=0 >> suspended.duration=0 resumed=0 Apr 29 16:23:15 syslogtest >> rsyslogd-pstats: action 4: processed=0 failed=0 suspended=0 >> suspended.duration=0 resumed=0 Apr 29 16:23:15 syslogtest >> rsyslogd-pstats: action 5: processed=0 failed=0 suspended=0 >> suspended.duration=0 resumed=0 Apr 29 16:23:15 syslogtest >> rsyslogd-pstats: action 6: processed=1 failed=0 suspended=0 >> suspended.duration=0 resumed=0 Apr 29 16:23:15 syslogtest >> rsyslogd-pstats: action 7: processed=0 failed=0 suspended=0 >> suspended.duration=0 resumed=0 Apr 29 16:23:15 syslogtest >> rsyslogd-pstats: action 8: processed=0 failed=0 suspended=0 >> suspended.duration=0 resumed=0 Apr 29 16:23:15 syslogtest >> rsyslogd-pstats: action 9: processed=0 failed=0 suspended=0 >> suspended.duration=0 resumed=0 Apr 29 16:23:15 syslogtest >> rsyslogd-pstats: action 10: processed=10 failed=10 suspended=1 >> suspended.duration=60 resumed=0 Apr 29 16:23:15 syslogtest >> rsyslogd-pstats: action 11: processed=10 failed=10 suspended=1 >> suspended.duration=60 resumed=0 Apr 29 16:23:15 syslogtest >> rsyslogd-pstats: resource-usage: apache-error:=2999 stime=15997 >> maxrss=2988 minflt=561 majflt=0 inblock=0 oublock=32 nvcsw=68 >> nivcsw=31 Apr 29 16:23:15 syslogtest rsyslogd-pstats: main Q: size=14 >> enqueued=24 full=0 discarded.full=0 discarded.nf=0 maxqsize=14 Apr 29 >> 16:23:15 syslogtest rsyslogd-pstats: imudp(w0): called.recvmmsg=0 >> called.recvmsg=0 msgs.received=0 Apr 29 16:23:46 syslogtest >> rsyslogd-pstats: imuxsock: submitted=0 ratelimit.discarded=0 >> ratelimit.numratelimiters=0 Apr 29 16:23:46 syslogtest >> rsyslogd-pstats: omelasticsearch: submitted=2 failed.http=2 >> failed.httprequests=2 failed.es=0 Apr 29 16:23:46 syslogtest >> rsyslogd-pstats: action 1: processed=9 failed=9 suspended=1 >> suspended.duration=30 resumed=0 Apr 29 16:23:46 syslogtest >> rsyslogd-pstats: action 2: processed=0 failed=0 suspended=0 >> suspended.duration=0 resumed=0 Apr 29 16:23:46 syslogtest >> rsyslogd-pstats: action 3: processed=9 failed=0 suspended=0 >> suspended.duration=0 resumed=0 Apr 29 16:23:46 syslogtest >> rsyslogd-pstats: action 4: processed=0 failed=0 suspended=0 >> suspended.duration=0 resumed=0 Apr 29 16:23:46 syslogtest >> rsyslogd-pstats: action 5: processed=0 failed=0 suspended=0 >> suspended.duration=0 resumed=0 Apr 29 16:23:46 syslogtest >> rsyslogd-pstats: action 6: processed=0 failed=0 suspended=0 >> suspended.duration=0 resumed=0 Apr 29 16:23:46 syslogtest >> rsyslogd-pstats: action 7: processed=0 failed=0 suspended=0 >> suspended.duration=0 resumed=0 Apr 29 16:23:46 syslogtest >> rsyslogd-pstats: action 8: processed=0 failed=0 suspended=0 >> suspended.duration=0 resumed=0 Apr 29 16:23:46 syslogtest >> rsyslogd-pstats: action 9: processed=0 failed=0 suspended=0 >> suspended.duration=0 resumed=0 Apr 29 16:23:46 syslogtest >> rsyslogd-pstats: action 10: processed=9 failed=9 suspended=1 >> suspended.duration=30 resumed=0 Apr 29 16:23:46 syslogtest >> rsyslogd-pstats: action 11: processed=9 failed=9 suspended=1 >> suspended.duration=30 resumed=0 Apr 29 16:23:46 syslogtest >> rsyslogd-pstats: imudp(*:514): submitted=0 Apr 29 16:23:46 syslogtest >> rsyslogd-pstats: imudp(*:514): submitted=0 Apr 29 16:23:46 syslogtest >> rsyslogd-pstats: resource-usage: apache-error:=1999 stime=999 >> maxrss=2916 minflt=523 majflt=0 inblock=0 oublock=16 nvcsw=10 >> nivcsw=31 Apr 29 16:23:46 syslogtest rsyslogd-pstats: main Q: size=16 >> enqueued=25 full=0 discarded.full=0 discarded.nf=0 maxqsize=16 Apr 29 >> 16:23:46 syslogtest rsyslogd-pstats: imudp(w0): called.recvmmsg=0 >> called.recvmsg=0 msgs.received=0 >> >> >> >> >> >> -----Original Message----- >> From: [email protected] >> [mailto:[email protected]] On Behalf Of David Lang >> Sent: Tuesday, April 29, 2014 4:20 PM >> To: rsyslog-users >> Subject: Re: [rsyslog] Rsyslog w/ logstash-elasticsearch-kibana >> server >> >> On Tue, 29 Apr 2014, Josh Bitto wrote: >> >>> I didn't have it running, but I added it and waiting on the 10 >>> minute interval. If I set it to 300 would be go down to 5 minutes? >> >> Yes, for a test like this where it doesn't look like anything is >> getting through, I'd suggest setting it to something really short, >> say 10s so that you can debug quickly >> >> David Lang >> >>> -----Original Message----- >>> From: [email protected] >>> [mailto:[email protected]] On Behalf Of David Lang >>> Sent: Tuesday, April 29, 2014 4:10 PM >>> To: rsyslog-users >>> Subject: Re: [rsyslog] Rsyslog w/ logstash-elasticsearch-kibana >>> server >>> >>> On Tue, 29 Apr 2014, Josh Bitto wrote: >>> >>>> Ok so after everyone's input I decided to go with >>>> Rsyslog->Elasticsearch->Kibana setup. >>>> >>>> So I'm running CentOS 6.5 with apache. On a virtualbox machine. >>>> Rsyslog version rsyslog-7.6.3-1.el6.x86_64 Kibana and elasticsearch >>>> are the latest editions. I added a repo to just do a yum install of ES, >>>> and kibana is in my webroot directory. >>>> >>>> Basically I've just created a test server to see how well the setup will >>>> be compared to a live server and I'm running into some issues. I've >>>> followed the instructions from here. >>>> http://blog.sematext.com/2013/07/01/recipe-rsyslog-elasticsearch-kibana/ >>>> and I added the config part in that tutorial to the rsyslog.conf. >>>> >>>> I've tested to make sure that rsyslog is running "logger blah blah blah" >>>> at the command line and It returns in the messages logs. So I think where >>>> I'm missing is from rsyslog to elasticsearch.. >>>> >>>> When I go to my kibana webpage and try to search for logs that I know are >>>> there it doesn't return anything. >>>> >>>> Here is my rsyslog.conf >>> >>> do you have impstats running? what does it have to say about the action to >>> put logs into elasticsearch? >>> >>> David Lang >>> _______________________________________________ >>> rsyslog mailing list >>> http://lists.adiscon.net/mailman/listinfo/rsyslog >>> http://www.rsyslog.com/professional-services/ >>> What's up with rsyslog? Follow https://twitter.com/rgerhards NOTE >>> WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of >>> sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T >>> LIKE THAT. >>> >> _______________________________________________ >> rsyslog mailing list >> http://lists.adiscon.net/mailman/listinfo/rsyslog >> http://www.rsyslog.com/professional-services/ >> What's up with rsyslog? Follow https://twitter.com/rgerhards NOTE WELL: This >> is a PUBLIC mailing list, posts are ARCHIVED by a myriad of sites beyond our >> control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE THAT. >> _______________________________________________ >> rsyslog mailing list >> http://lists.adiscon.net/mailman/listinfo/rsyslog >> http://www.rsyslog.com/professional-services/ >> What's up with rsyslog? Follow https://twitter.com/rgerhards NOTE >> WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of sites >> beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE >> THAT. >> > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > http://www.rsyslog.com/professional-services/ > What's up with rsyslog? Follow https://twitter.com/rgerhards NOTE WELL: This > is a PUBLIC mailing list, posts are ARCHIVED by a myriad of sites beyond our > control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE THAT. > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > http://www.rsyslog.com/professional-services/ > What's up with rsyslog? Follow https://twitter.com/rgerhards NOTE > WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of sites > beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE THAT. > _______________________________________________ rsyslog mailing list http://lists.adiscon.net/mailman/listinfo/rsyslog http://www.rsyslog.com/professional-services/ What's up with rsyslog? Follow https://twitter.com/rgerhards NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE THAT. _______________________________________________ rsyslog mailing list http://lists.adiscon.net/mailman/listinfo/rsyslog http://www.rsyslog.com/professional-services/ What's up with rsyslog? Follow https://twitter.com/rgerhards NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE THAT.
