BTW, as I am still new to golang and writing plugins, any feedback will be very useful!
On Mon, May 18, 2015 at 9:15 PM, Alex Jiao <[email protected]> wrote: > Hey Rob, thanks a lot for your input. It clears up a lot of doubts > regarding my code. I have rewritten it according to your suggestions and it > works. The ticker now fires at regular intervals as expected, with the > added bonus that the plugin is able to shut down cleanly upon killing the > process. I have add other features like gzip compression as well. My > company will be using this plugin in production soon. Thanks all for your > help. > > On Thu, May 14, 2015 at 6:06 AM, Rob Miller <[email protected]> wrote: > >> Your code is put together very strangely. The ticker channel is firing >> correctly, but once it fires the first time you immediately get trapped in >> the for loop that is over the range of the inChan. So you aren't getting >> any of the input messages until the first tick fires, but after it fires >> you never escape. >> >> Also, your Stop method will never be called. Stop methods are only for >> input plugins, output plugins are signaled to exit by closing the inChan. >> >> You want to use a select on the inChan and the tickerChan, both of them >> enclosed in a loop that will exit when the inChan is closed. The simplest >> plugin I can think of that demonstrates this pattern is the CounterFilter: >> >> >> https://github.com/mozilla-services/heka/blob/dev/pipeline/counter_filter.go >> >> Hope this helps, >> >> -r >> >> >> >> On 05/13/2015 03:42 AM, Alex Jiao wrote: >> >>> Hi, I'm writing an S3 output plugin for heka. As I am new to golang and >>> heka, I'm not too sure what's the best way to go about writing the >>> plugin. Hence I wish to seek you guys' help to resolve some issues that >>> are plaguing my code. >>> >>> heka-s3 plugin code: >>> https://github.com/uohzxela/heka-s3/blob/master/s3.go >>> >>> My plugin needs to send message packs from the pipeline at regular >>> intervals as specified in the .toml file. It writes to a buffer during >>> each interval and upload to a S3 bucket at the next tick. The interval >>> is specified by the 'ticker_interval' option. However, I'm not too sure >>> whether you need to take care of the ticker logic in the for loop or >>> that the output Run function will be invoked at the specified ticker >>> interval by Heka service itself. >>> >>> I have implemented the first case as seen in my code and it led to a >>> weird issue. The plugin can be loaded upon starting the Heka service, >>> however the ticker can only run once after the elapsed time interval. >>> It's only after I sent a TERM signal to the Heka process using kill >>> (kill <heka-pid>), that the ticker starts to work. >>> >>> Could you guys shed light on resolving this issue? Why does the ticker >>> not work upon starting the plugin? Does the programmer have to take care >>> of the ticker logic or the output runner will automatically be invoked >>> at regular intervals by the Heka service? I'd appreciate it as well if >>> you can point me to some output plugins that have the ticker >>> functionality. >>> >>> Thank you. >>> >>> -- >>> Alex >>> >>> >>> _______________________________________________ >>> Heka mailing list >>> [email protected] >>> https://mail.mozilla.org/listinfo/heka >>> >>> > > > -- > Alex > -- Alex
_______________________________________________ Heka mailing list [email protected] https://mail.mozilla.org/listinfo/heka
