Ming, Buffering has it’s own documentation page:
http://hekad.readthedocs.org/en/v0.10.0b1/buffering.html Here’s an example for the TCPOutput: [TcpOutput] message_matcher = "Type !~ /^heka/" address = "upstream.example.com<http://upstream.example.com>:5565" keep_alive = true use_buffering = true [TcpOutput.buffering] max_file_size = 268435456 # 256MiB max_buffer_size = 1073741824 # 1GiB full_action = "block" cursor_update_count = 100 Chris On Aug 28, 2015, at 1:09 AM, Ho, Ming <[email protected]<mailto:[email protected]>> wrote: Heka community, I try the following config file to send memstats data to Elasticsearch using [ElasticSearchOutput]. [MemStats] type = "FilePollingInput" ticker_interval = 5 file_path = "/proc/meminfo" decoder = "MemStatsDecoder" [MemStatsDecoder] type = "SandboxDecoder" filename = "lua_decoders/linux_memstats.lua" [ESJsonEncoder] index = "test" es_index_from_timestamp = true type_name = "%{Type}" [ElasticSearchOutput] server = "http://10.211.62.132:9200<http://10.211.62.132:9200/>" message_matcher = "Type == 'heka.memstat'" encoder = "ESJsonEncoder" flush_interval = 10 But got “ ElasticSearchOutput can’t initialize buffer” error as shown below when I run heka. I don’t see max_file_size is a configurable parameter in the heka doc. Appreciate any help on this 2015/08/27 21:49:14 Pre-loading: [MemStatsDecoder] 2015/08/27 21:49:14 Pre-loading: [ESJsonEncoder] 2015/08/27 21:49:14 Pre-loading: [ElasticSearchOutput] 2015/08/27 21:49:14 Pre-loading: [MemStats] 2015/08/27 21:49:14 Pre-loading: [ProtobufDecoder] 2015/08/27 21:49:14 Loading: [ProtobufDecoder] 2015/08/27 21:49:14 Pre-loading: [ProtobufEncoder] 2015/08/27 21:49:14 Loading: [ProtobufEncoder] 2015/08/27 21:49:14 Pre-loading: [TokenSplitter] 2015/08/27 21:49:14 Loading: [TokenSplitter] 2015/08/27 21:49:14 Pre-loading: [HekaFramingSplitter] 2015/08/27 21:49:14 Loading: [HekaFramingSplitter] 2015/08/27 21:49:14 Pre-loading: [NullSplitter] 2015/08/27 21:49:14 Loading: [NullSplitter] 2015/08/27 21:49:14 Loading: [MemStatsDecoder] 2015/08/27 21:49:14 Loading: [ESJsonEncoder] 2015/08/27 21:49:14 Loading: [MemStats] 2015/08/27 21:49:14 Loading: [ElasticSearchOutput] 2015/08/27 21:49:14 Starting hekad... 2015/08/27 21:49:14 Output 'ElasticSearchOutput' failed to start: can't initialize buffer: `max_file_size` must be greater than maximum record size of 65794 2015/08/27 21:49:14 MessageRouter started. 2015/08/27 21:49:14 Input started: MemStats 2015/08/27 21:49:14 Shutdown initiated. 2015/08/27 21:49:14 Stop message sent to input 'MemStats' 2015/08/27 21:49:14 Waiting for decoders shutdown 2015/08/27 21:49:14 Decoders shutdown complete 2015/08/27 21:49:14 Stop message sent to output 'ElasticSearchOutput' 2015/08/27 21:49:14 Shutdown complete. Ming _______________________________________________ Heka mailing list [email protected]<mailto:[email protected]> https://mail.mozilla.org/listinfo/heka
_______________________________________________ Heka mailing list [email protected] https://mail.mozilla.org/listinfo/heka
