Hey sorry, could you guys delete/edit my previous message? I have realised I accidentally exposed my access key and secret key.
On Thu, May 14, 2015 at 2:17 AM, Alex Jiao <[email protected]> wrote: > [input_log] > type = "LogstreamerInput" > log_directory = "/tmp" > file_match = "hello.log" > > [output_s3] > type = "S3Output" > message_matcher = "TRUE" > secret_key = "4fvHMt5lekgPWzxLwUgwb+kcFu7lVi695NxUJ0a7" > access_key = "0BRRE6DFWV2WSVJD3882" > bucket_name = "wego-logs-staging" > path_name = "/hello" > region = "ap-southeast-1" > ticker_interval = 10 > max_buffer_size = 10 > encoder = "PayloadEncoder" > > [PayloadEncoder] > append_newlines = false > > It is just a simple pipeline (from local filesystem to S3 bucket) for > testing purposes. However in production, we will configure the plugin to > have kafka as the input and S3 bucket as the output, as required by my > company. > > On Thu, May 14, 2015 at 12:20 AM, Jarek Śmiejczak <[email protected]> > wrote: > >> Could you post your config file for heka .toml? What is the value for >> ticker_interval? >> >> On Wed, May 13, 2015 at 12:42 PM, Alex Jiao <[email protected]> wrote: >> >>> Hi, I'm writing an S3 output plugin for heka. As I am new to golang and >>> heka, I'm not too sure what's the best way to go about writing the plugin. >>> Hence I wish to seek you guys' help to resolve some issues that are >>> plaguing my code. >>> >>> heka-s3 plugin code: >>> https://github.com/uohzxela/heka-s3/blob/master/s3.go >>> >>> My plugin needs to send message packs from the pipeline at regular >>> intervals as specified in the .toml file. It writes to a buffer during each >>> interval and upload to a S3 bucket at the next tick. The interval is >>> specified by the 'ticker_interval' option. However, I'm not too sure >>> whether you need to take care of the ticker logic in the for loop or that >>> the output Run function will be invoked at the specified ticker interval by >>> Heka service itself. >>> >>> I have implemented the first case as seen in my code and it led to a >>> weird issue. The plugin can be loaded upon starting the Heka service, >>> however the ticker can only run once after the elapsed time interval. It's >>> only after I sent a TERM signal to the Heka process using kill (kill >>> <heka-pid>), that the ticker starts to work. >>> >>> Could you guys shed light on resolving this issue? Why does the ticker >>> not work upon starting the plugin? Does the programmer have to take care of >>> the ticker logic or the output runner will automatically be invoked at >>> regular intervals by the Heka service? I'd appreciate it as well if you can >>> point me to some output plugins that have the ticker functionality. >>> >>> Thank you. >>> >>> -- >>> Alex >>> >>> _______________________________________________ >>> Heka mailing list >>> [email protected] >>> https://mail.mozilla.org/listinfo/heka >>> >>> >> >> >> -- >> jarek@reijutsu:~$ fortune >> You have Egyptian flu: you're going to be a mummy. >> jarek@reijutsu:~$ fortune >> You now have Asian Flu. >> > > > > -- > Alex > -- Alex
_______________________________________________ Heka mailing list [email protected] https://mail.mozilla.org/listinfo/heka
