Hi Christian, Tony Thanks for the suggestion and the link. I can see these event in ElasticSearch now. Appreciate your prompt response.
Cheers, Ming -----Original Message----- From: Christian Kniep [mailto:[email protected]] Sent: Thursday, August 27, 2015 11:26 PM To: Ho, Ming Cc: [email protected] Subject: Re: [heka] ElasticSearchOutput can’t initialize buffer Hey Ming, you have to set the max_file_size like this: #### [ElasticSearchOutput] server = "http://10.211.62.132:9200” message_matcher = "Type == 'heka.memstat’" encoder = “ESJsonEncoder" flush_interval = 10 [ElasticSearchOutput.buffering] max_file_size = 268435456 #### To what size it’s up to debate… It seems at least greater then 65794 as the error message suggest… Cheers Christian ### > On 28 Aug 2015, at 08:09, Ho, Ming <[email protected]> wrote: > > Heka community, > > I try the following config file to send memstats data to Elasticsearch using > [ElasticSearchOutput]. > > [MemStats] > type = "FilePollingInput" > ticker_interval = 5 > file_path = "/proc/meminfo" > decoder = "MemStatsDecoder" > > [MemStatsDecoder] > type = "SandboxDecoder" > filename = "lua_decoders/linux_memstats.lua" > > [ESJsonEncoder] > index = "test" > es_index_from_timestamp = true > type_name = "%{Type}" > > [ElasticSearchOutput] > server = "http://10.211.62.132:9200"; > message_matcher = "Type == 'heka.memstat'" > encoder = "ESJsonEncoder" > flush_interval = 10 > > > But got “ ElasticSearchOutput can’t initialize buffer” error as shown below > when I run heka. I don’t see max_file_size is a configurable parameter in the > heka doc. Appreciate any help on this > > 2015/08/27 21:49:14 Pre-loading: [MemStatsDecoder] > 2015/08/27 21:49:14 Pre-loading: [ESJsonEncoder] > 2015/08/27 21:49:14 Pre-loading: [ElasticSearchOutput] > 2015/08/27 21:49:14 Pre-loading: [MemStats] > 2015/08/27 21:49:14 Pre-loading: [ProtobufDecoder] > 2015/08/27 21:49:14 Loading: [ProtobufDecoder] > 2015/08/27 21:49:14 Pre-loading: [ProtobufEncoder] > 2015/08/27 21:49:14 Loading: [ProtobufEncoder] > 2015/08/27 21:49:14 Pre-loading: [TokenSplitter] > 2015/08/27 21:49:14 Loading: [TokenSplitter] > 2015/08/27 21:49:14 Pre-loading: [HekaFramingSplitter] > 2015/08/27 21:49:14 Loading: [HekaFramingSplitter] > 2015/08/27 21:49:14 Pre-loading: [NullSplitter] > 2015/08/27 21:49:14 Loading: [NullSplitter] > 2015/08/27 21:49:14 Loading: [MemStatsDecoder] > 2015/08/27 21:49:14 Loading: [ESJsonEncoder] > 2015/08/27 21:49:14 Loading: [MemStats] > 2015/08/27 21:49:14 Loading: [ElasticSearchOutput] > 2015/08/27 21:49:14 Starting hekad... > 2015/08/27 21:49:14 Output 'ElasticSearchOutput' failed to start: can't > initialize buffer: `max_file_size` must be greater than maximum record size > of 65794 > 2015/08/27 21:49:14 MessageRouter started. > 2015/08/27 21:49:14 Input started: MemStats > 2015/08/27 21:49:14 Shutdown initiated. > 2015/08/27 21:49:14 Stop message sent to input 'MemStats' > 2015/08/27 21:49:14 Waiting for decoders shutdown > 2015/08/27 21:49:14 Decoders shutdown complete > 2015/08/27 21:49:14 Stop message sent to output 'ElasticSearchOutput' > 2015/08/27 21:49:14 Shutdown complete. > > Ming > > _______________________________________________ > Heka mailing list > [email protected] > https://mail.mozilla.org/listinfo/heka _______________________________________________ Heka mailing list [email protected] https://mail.mozilla.org/listinfo/heka
