I wasn't able to find any reason's for my error until I did the debug. The rsyslog-stats log wasn't much help. I know that it was user-error on my part for wrong file paths. I don't think anything needs to be added. Well...maybe an idiot message that says...."Make sure your file paths are correct noob" :P
I had read documentation for doing a "yum install rsyslog-elasticsearch" which didn't explain much other than that. So I figured it was a built in that auto configured rsyslog to talk with elasticsearch. I think you guys do an amazing job on rsyslog and covered the documentation well enough. Josh -----Original Message----- From: [email protected] [mailto:[email protected]] On Behalf Of Rainer Gerhards Sent: Thursday, May 01, 2014 11:15 AM To: rsyslog-users Subject: Re: [rsyslog] Rsyslog w/ logstash-elasticsearch-kibana server Is there anything that could be added to the doc to help the next person? Any error messages rsyslog did not emit but in the debug log? Sent from phone, thus brief. Am 01.05.2014 20:09 schrieb "Josh Bitto" <[email protected]>: > Ok David, > > I got it working. It turns out I had issues on two sides. One was in > the file paths for the rsyslog.conf and then on the other side in the > kibana webroot. I had the wrong url for the elasticsearch config.js. > It is working now! :) Thanks for the help. > > > Josh > > > -----Original Message----- > From: [email protected] [mailto: > [email protected]] On Behalf Of David Lang > Sent: Wednesday, April 30, 2014 4:27 PM > To: rsyslog-users > Subject: Re: [rsyslog] Rsyslog w/ logstash-elasticsearch-kibana server > > no, the file is probably huge, look for 'error' in it > > grep -i error debug.log > > or search for the contents of a message that you know should be going > to elasticsearch. > > The error message should be pretty obvious, and when you see it you > will probably be able to figure out what's wrong. > > David Lang > > On Wed, 30 Apr 2014, Josh Bitto wrote: > > > Date: Wed, 30 Apr 2014 10:44:14 -0700 > > From: Josh Bitto <[email protected]> > > Reply-To: rsyslog-users <[email protected]> > > To: rsyslog-users <[email protected]> > > Subject: Re: [rsyslog] Rsyslog w/ logstash-elasticsearch-kibana > > server > > > > Ok I think I got it to debug... > > > > I stopped the rsyslog service and then entered this at the command line. > > /sbin/rsyslogd -c3 -dn > /var/log/debug.log > > > > Then it gave me a file. Should I post the entire contents? > > > > > > > > Josh > > > > > > > > > > > > -----Original Message----- > > From: [email protected] > > [mailto:[email protected]] On Behalf Of David Lang > > Sent: Tuesday, April 29, 2014 4:44 PM > > To: rsyslog-users > > Subject: Re: [rsyslog] Rsyslog w/ logstash-elasticsearch-kibana > > server > > > > Ok, I woould guess that your elasticsarch actions are 10 and 11 (you > > would have to count the outputs in the file or add the name > > parameter to the action() > > calls) > > > > So this is saying that it's attempting to deliver the messages to > elasticsearch but is failing. > > > > try starting in debug mode and see exactly what's happening when it > tries to deliver a message. > > > > David Lang > > > > On Tue, 29 Apr 2014, Josh Bitto wrote: > > > >> Date: Tue, 29 Apr 2014 16:24:42 -0700 > >> From: Josh Bitto <[email protected]> > >> Reply-To: rsyslog-users <[email protected]> > >> To: rsyslog-users <[email protected]> > >> Subject: Re: [rsyslog] Rsyslog w/ logstash-elasticsearch-kibana > >> server > >> > >> This is the output > >> > >> Apr 29 16:23:15 syslogtest rsyslogd-pstats: imuxsock: submitted=1 > >> ratelimit.discarded=0 ratelimit.numratelimiters=1 Apr 29 16:23:15 > >> syslogtest rsyslogd-pstats: omelasticsearch: submitted=2 > >> failed.http=2 > >> failed.httprequests=2 failed.es=0 Apr 29 16:23:15 syslogtest > >> rsyslogd-pstats: action 1: processed=10 failed=10 suspended=1 > >> suspended.duration=60 resumed=0 Apr 29 16:23:15 syslogtest > >> rsyslogd-pstats: action 2: processed=0 failed=0 suspended=0 > >> suspended.duration=0 resumed=0 Apr 29 16:23:15 syslogtest > >> rsyslogd-pstats: action 3: processed=9 failed=0 suspended=0 > >> suspended.duration=0 resumed=0 Apr 29 16:23:15 syslogtest > >> rsyslogd-pstats: action 4: processed=0 failed=0 suspended=0 > >> suspended.duration=0 resumed=0 Apr 29 16:23:15 syslogtest > >> rsyslogd-pstats: action 5: processed=0 failed=0 suspended=0 > >> suspended.duration=0 resumed=0 Apr 29 16:23:15 syslogtest > >> rsyslogd-pstats: action 6: processed=1 failed=0 suspended=0 > >> suspended.duration=0 resumed=0 Apr 29 16:23:15 syslogtest > >> rsyslogd-pstats: action 7: processed=0 failed=0 suspended=0 > >> suspended.duration=0 resumed=0 Apr 29 16:23:15 syslogtest > >> rsyslogd-pstats: action 8: processed=0 failed=0 suspended=0 > >> suspended.duration=0 resumed=0 Apr 29 16:23:15 syslogtest > >> rsyslogd-pstats: action 9: processed=0 failed=0 suspended=0 > >> suspended.duration=0 resumed=0 Apr 29 16:23:15 syslogtest > >> rsyslogd-pstats: action 10: processed=10 failed=10 suspended=1 > >> suspended.duration=60 resumed=0 Apr 29 16:23:15 syslogtest > >> rsyslogd-pstats: action 11: processed=10 failed=10 suspended=1 > >> suspended.duration=60 resumed=0 Apr 29 16:23:15 syslogtest > >> rsyslogd-pstats: resource-usage: apache-error:=2999 stime=15997 > >> maxrss=2988 minflt=561 majflt=0 inblock=0 oublock=32 nvcsw=68 > >> nivcsw=31 Apr 29 16:23:15 syslogtest rsyslogd-pstats: main Q: > >> size=14 > >> enqueued=24 full=0 discarded.full=0 discarded.nf=0 maxqsize=14 Apr > >> 29 > >> 16:23:15 syslogtest rsyslogd-pstats: imudp(w0): called.recvmmsg=0 > >> called.recvmsg=0 msgs.received=0 Apr 29 16:23:46 syslogtest > >> rsyslogd-pstats: imuxsock: submitted=0 ratelimit.discarded=0 > >> ratelimit.numratelimiters=0 Apr 29 16:23:46 syslogtest > >> rsyslogd-pstats: omelasticsearch: submitted=2 failed.http=2 > >> failed.httprequests=2 failed.es=0 Apr 29 16:23:46 syslogtest > >> rsyslogd-pstats: action 1: processed=9 failed=9 suspended=1 > >> suspended.duration=30 resumed=0 Apr 29 16:23:46 syslogtest > >> rsyslogd-pstats: action 2: processed=0 failed=0 suspended=0 > >> suspended.duration=0 resumed=0 Apr 29 16:23:46 syslogtest > >> rsyslogd-pstats: action 3: processed=9 failed=0 suspended=0 > >> suspended.duration=0 resumed=0 Apr 29 16:23:46 syslogtest > >> rsyslogd-pstats: action 4: processed=0 failed=0 suspended=0 > >> suspended.duration=0 resumed=0 Apr 29 16:23:46 syslogtest > >> rsyslogd-pstats: action 5: processed=0 failed=0 suspended=0 > >> suspended.duration=0 resumed=0 Apr 29 16:23:46 syslogtest > >> rsyslogd-pstats: action 6: processed=0 failed=0 suspended=0 > >> suspended.duration=0 resumed=0 Apr 29 16:23:46 syslogtest > >> rsyslogd-pstats: action 7: processed=0 failed=0 suspended=0 > >> suspended.duration=0 resumed=0 Apr 29 16:23:46 syslogtest > >> rsyslogd-pstats: action 8: processed=0 failed=0 suspended=0 > >> suspended.duration=0 resumed=0 Apr 29 16:23:46 syslogtest > >> rsyslogd-pstats: action 9: processed=0 failed=0 suspended=0 > >> suspended.duration=0 resumed=0 Apr 29 16:23:46 syslogtest > >> rsyslogd-pstats: action 10: processed=9 failed=9 suspended=1 > >> suspended.duration=30 resumed=0 Apr 29 16:23:46 syslogtest > >> rsyslogd-pstats: action 11: processed=9 failed=9 suspended=1 > >> suspended.duration=30 resumed=0 Apr 29 16:23:46 syslogtest > >> rsyslogd-pstats: imudp(*:514): submitted=0 Apr 29 16:23:46 > >> syslogtest > >> rsyslogd-pstats: imudp(*:514): submitted=0 Apr 29 16:23:46 > >> syslogtest > >> rsyslogd-pstats: resource-usage: apache-error:=1999 stime=999 > >> maxrss=2916 minflt=523 majflt=0 inblock=0 oublock=16 nvcsw=10 > >> nivcsw=31 Apr 29 16:23:46 syslogtest rsyslogd-pstats: main Q: > >> size=16 > >> enqueued=25 full=0 discarded.full=0 discarded.nf=0 maxqsize=16 Apr > >> 29 > >> 16:23:46 syslogtest rsyslogd-pstats: imudp(w0): called.recvmmsg=0 > >> called.recvmsg=0 msgs.received=0 > >> > >> > >> > >> > >> > >> -----Original Message----- > >> From: [email protected] > >> [mailto:[email protected]] On Behalf Of David Lang > >> Sent: Tuesday, April 29, 2014 4:20 PM > >> To: rsyslog-users > >> Subject: Re: [rsyslog] Rsyslog w/ logstash-elasticsearch-kibana > >> server > >> > >> On Tue, 29 Apr 2014, Josh Bitto wrote: > >> > >>> I didn't have it running, but I added it and waiting on the 10 > >>> minute interval. If I set it to 300 would be go down to 5 minutes? > >> > >> Yes, for a test like this where it doesn't look like anything is > >> getting through, I'd suggest setting it to something really short, > >> say 10s so that you can debug quickly > >> > >> David Lang > >> > >>> -----Original Message----- > >>> From: [email protected] > >>> [mailto:[email protected]] On Behalf Of David Lang > >>> Sent: Tuesday, April 29, 2014 4:10 PM > >>> To: rsyslog-users > >>> Subject: Re: [rsyslog] Rsyslog w/ logstash-elasticsearch-kibana > >>> server > >>> > >>> On Tue, 29 Apr 2014, Josh Bitto wrote: > >>> > >>>> Ok so after everyone's input I decided to go with > Rsyslog->Elasticsearch->Kibana setup. > >>>> > >>>> So I'm running CentOS 6.5 with apache. On a virtualbox machine. > >>>> Rsyslog version rsyslog-7.6.3-1.el6.x86_64 Kibana and > >>>> elasticsearch are the latest editions. I added a repo to just do > >>>> a yum install of > ES, and kibana is in my webroot directory. > >>>> > >>>> Basically I've just created a test server to see how well the > >>>> setup > will be compared to a live server and I'm running into some issues. > I've followed the instructions from here. > >>>> > http://blog.sematext.com/2013/07/01/recipe-rsyslog-elasticsearch-kibana/and I > added the config part in that tutorial to the rsyslog.conf. > >>>> > >>>> I've tested to make sure that rsyslog is running "logger blah > >>>> blah > blah" at the command line and It returns in the messages logs. So I > think where I'm missing is from rsyslog to elasticsearch.. > >>>> > >>>> When I go to my kibana webpage and try to search for logs that I > >>>> know > are there it doesn't return anything. > >>>> > >>>> Here is my rsyslog.conf > >>> > >>> do you have impstats running? what does it have to say about the > action to put logs into elasticsearch? > >>> > >>> David Lang > >>> _______________________________________________ > >>> rsyslog mailing list > >>> http://lists.adiscon.net/mailman/listinfo/rsyslog > >>> http://www.rsyslog.com/professional-services/ > >>> What's up with rsyslog? Follow https://twitter.com/rgerhards NOTE > >>> WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a > >>> myriad of > sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you > DON'T LIKE THAT. > >>> > >> _______________________________________________ > >> rsyslog mailing list > >> http://lists.adiscon.net/mailman/listinfo/rsyslog > >> http://www.rsyslog.com/professional-services/ > >> What's up with rsyslog? Follow https://twitter.com/rgerhards NOTE > WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of > sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you > DON'T LIKE THAT. > >> _______________________________________________ > >> rsyslog mailing list > >> http://lists.adiscon.net/mailman/listinfo/rsyslog > >> http://www.rsyslog.com/professional-services/ > >> What's up with rsyslog? Follow https://twitter.com/rgerhards NOTE > >> WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad > >> of > sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you > DON'T LIKE THAT. > >> > > _______________________________________________ > > rsyslog mailing list > > http://lists.adiscon.net/mailman/listinfo/rsyslog > > http://www.rsyslog.com/professional-services/ > > What's up with rsyslog? Follow https://twitter.com/rgerhards NOTE WELL: > This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of sites > beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T > LIKE THAT. > > _______________________________________________ > > rsyslog mailing list > > http://lists.adiscon.net/mailman/listinfo/rsyslog > > http://www.rsyslog.com/professional-services/ > > What's up with rsyslog? Follow https://twitter.com/rgerhards NOTE > > WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad > > of > sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you > DON'T LIKE THAT. > > > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > http://www.rsyslog.com/professional-services/ > What's up with rsyslog? Follow https://twitter.com/rgerhards NOTE WELL: > This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of sites > beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T > LIKE THAT. > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > http://www.rsyslog.com/professional-services/ > What's up with rsyslog? Follow https://twitter.com/rgerhards NOTE > WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of > sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you > DON'T LIKE THAT. > _______________________________________________ rsyslog mailing list http://lists.adiscon.net/mailman/listinfo/rsyslog http://www.rsyslog.com/professional-services/ What's up with rsyslog? Follow https://twitter.com/rgerhards NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE THAT. _______________________________________________ rsyslog mailing list http://lists.adiscon.net/mailman/listinfo/rsyslog http://www.rsyslog.com/professional-services/ What's up with rsyslog? Follow https://twitter.com/rgerhards NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE THAT.
