Thanks all. It works fine now. -----Original Message----- From: Rob Miller [mailto:[email protected]] Sent: Friday, August 28, 2015 12:56 PM To: Giordano, J C. Cc: [email protected]; Ho, Ming Subject: Re: [heka] ElasticSearchOutput can’t initialize buffer
Also, the fact that you're seeing this error is a bug in the default buffer config settings that has recently been fixed (https://github.com/mozilla-services/heka/pull/1685). The fix will be in the next release. -r On 08/28/2015 12:52 PM, Giordano, J C. wrote: > Ming, > > Buffering has it’s own documentation page: > > http://hekad.readthedocs.org/en/v0.10.0b1/buffering.html > > Here’s an example for the TCPOutput: > > [TcpOutput] message_matcher = "Type !~ /^heka/" address = > "upstream.example.com <http://upstream.example.com>:5565" keep_alive = > true use_buffering = true [TcpOutput.buffering] max_file_size = > 268435456 # 256MiB max_buffer_size = 1073741824 # 1GiB full_action = > "block" cursor_update_count = 100 > > Chris > > > > On Aug 28, 2015, at 1:09 AM, Ho, Ming <[email protected] > > <mailto:[email protected]>> wrote: > > > > Heka community, > > I try the following config file to send memstats data to > > Elasticsearch using [ElasticSearchOutput]. > > [MemStats] > > type = "FilePollingInput" > > ticker_interval = 5 > > file_path = "/proc/meminfo" > > decoder = "MemStatsDecoder" > > [MemStatsDecoder] > > type = "SandboxDecoder" > > filename = "lua_decoders/linux_memstats.lua" > > [ESJsonEncoder] > > index = "test" > > es_index_from_timestamp = true > > type_name = "%{Type}" > > [ElasticSearchOutput] > > server = "http://10.211.62.132:9200 <http://10.211.62.132:9200/>" > > message_matcher = "Type == 'heka.memstat'" > > encoder = "ESJsonEncoder" > > flush_interval = 10 > > But got “ ElasticSearchOutput can’t initialize buffer” error as > > shown below when I run heka. I don’t see max_file_size is a > > configurable parameter in the heka doc. Appreciate any help on this > > 2015/08/27 21:49:14 Pre-loading: [MemStatsDecoder] > > 2015/08/27 21:49:14 Pre-loading: [ESJsonEncoder] > > 2015/08/27 21:49:14 Pre-loading: [ElasticSearchOutput] > > 2015/08/27 21:49:14 Pre-loading: [MemStats] > > 2015/08/27 21:49:14 Pre-loading: [ProtobufDecoder] > > 2015/08/27 21:49:14 Loading: [ProtobufDecoder] > > 2015/08/27 21:49:14 Pre-loading: [ProtobufEncoder] > > 2015/08/27 21:49:14 Loading: [ProtobufEncoder] > > 2015/08/27 21:49:14 Pre-loading: [TokenSplitter] > > 2015/08/27 21:49:14 Loading: [TokenSplitter] > > 2015/08/27 21:49:14 Pre-loading: [HekaFramingSplitter] > > 2015/08/27 21:49:14 Loading: [HekaFramingSplitter] > > 2015/08/27 21:49:14 Pre-loading: [NullSplitter] > > 2015/08/27 21:49:14 Loading: [NullSplitter] > > 2015/08/27 21:49:14 Loading: [MemStatsDecoder] > > 2015/08/27 21:49:14 Loading: [ESJsonEncoder] > > 2015/08/27 21:49:14 Loading: [MemStats] > > 2015/08/27 21:49:14 Loading: [ElasticSearchOutput] > > 2015/08/27 21:49:14 Starting hekad... > > 2015/08/27 21:49:14 Output 'ElasticSearchOutput' failed to start: > > can't initialize buffer: `max_file_size` must be greater than > > maximum record size of 65794 > > 2015/08/27 21:49:14 MessageRouter started. > > 2015/08/27 21:49:14 Input started: MemStats > > 2015/08/27 21:49:14 Shutdown initiated. > > 2015/08/27 21:49:14 Stop message sent to input 'MemStats' > > 2015/08/27 21:49:14 Waiting for decoders shutdown > > 2015/08/27 21:49:14 Decoders shutdown complete > > 2015/08/27 21:49:14 Stop message sent to output 'ElasticSearchOutput' > > 2015/08/27 21:49:14 Shutdown complete. > > Ming > > _______________________________________________ > > Heka mailing list > > [email protected] <mailto:[email protected]> > > https://mail.mozilla.org/listinfo/heka > > > > _______________________________________________ > Heka mailing list > [email protected] > https://mail.mozilla.org/listinfo/heka > _______________________________________________ Heka mailing list [email protected] https://mail.mozilla.org/listinfo/heka
