Heka's own output is very simple, it just emits console output to stdout and stderr, as appropriate. I'm no upstart expert, but it seems like you have to figure out the magic incantation to get it to correctly route the output. I did a quick web search, the following page seems to provide the most alternatives:
http://serverfault.com/questions/114052/logging-a-daemons-output-with-upstart One of the suggestions on that page matches what you're using, which is clearly not working well. It also recommends some other options, though, such as `console log`, and a brute force redirect (i.e. `exec /opt/heka/bin/hekad -config=/usr/local/etc/hekad.toml >> /var/log/hekad.log 2>&1`). I'd experiment with those to see what works best. -r On 04/07/2015 10:42 AM, Ali wrote:
Rob, Unless I'm missing something, I'm not getting any related output from hekad. I am running it from an upstart script on CentOS 6.6 and thought I was getting console output at /var/log/messages, but it looks like I was mistaken. I think I was only getting upstart messages (heka started, heka stopped) there. I installed heka by downloading the compiled binaries. Here's my upstart script: https://gist.github.com/hourback/6af38b17793887796e54 I didn't see any options for configuring logging for hekad. What do you recommend? -Ali On Tue, Apr 7, 2015 at 11:57 AM Rob Miller <[email protected] <mailto:[email protected]>> wrote: Any time there is an error that causes a buffered record to not be sent to ElasticSearch Heka should be emitting that error to its own console output. Do you not see any errors in your Heka output? -r On 04/07/2015 09:47 AM, Ali wrote: > I fixed my original problem, but am wondering how to better diagnose > what's going on with Heka in the future. > > I checked the ElasticSearch log and saw an error I mentioned on here a > few days ago. An earlier incarnation of my decoder had a type > containing capital letters. This was causing ElasticSearch to throw an > error because it wouldn't create an index that wasn't lower case. > Restarting the server didn't help because the output_queue still > existed. I stopped hekad, deleted the output_queue, restarted hekad, > and the records are now appearing in ElasticSearch. > > Is there a way I could have diagnosed this only using Heka, without > looking at the ElasticSearch logs? > > -Ali > > On Tue, Apr 7, 2015 at 11:20 AM Ali <[email protected] <mailto:[email protected]> > <mailto:[email protected] <mailto:[email protected]>>> wrote: > > Morning, all. > > Here's my hekad.toml: > > https://gist.github.com/__hourback/36f29939a804becd9723 <https://gist.github.com/hourback/36f29939a804becd9723> > > I am using a TcpInput for IBM HTTPServer access log data. I'm using > a similar pipeline for Rsyslog messages coming from the same remote > host (via nxlog) and things are working okay. However, for this > TcpInput, nothing seems to be coming through, i.e., the input and > decoder show up in the Heka dashboard, but the number of messages > processed is always zero. > > I was originally using the unfortunately named > IbmHttpserverAccessCommonLogDe__coder as my TcpInput. When that > wasn't showing results, I decided to test things by creating another > decoder, the ApacheRegexDecoder of type PayloadRegexDecoder. My > intention was to just grab everything sent to the decoder and throw > it back out to ElasticSearch with the rest of the log data so I > could verify that the data is actually getting to Heka from the > remote host. However, nothing seems to be coming through here either. > > I'm sure I'm getting dozens of things wrong so whatever tips you > have would be appreciated. :-) > > TIA, > Ali > > > > _________________________________________________ > Heka mailing list > [email protected] <mailto:[email protected]> > https://mail.mozilla.org/__listinfo/heka <https://mail.mozilla.org/listinfo/heka> >
_______________________________________________ Heka mailing list [email protected] https://mail.mozilla.org/listinfo/heka
