Bonus question, but I don’t want to stress it to much… What is different if I send the string via ‘logger’? #### logger -n 127.0.0.1 -P 5514 -T Test6 #### This only shows the closing TCP connection but now decoding/splitting… #### 2015/08/19 23:32:20 Decoder 'TcpInput-newline_decoder-127.0.0.1': stopped ####
Good n8 from Berlin Christian > On 19 Aug 2015, at 23:13, Nathan Williams <[email protected]> wrote: > > i also noticed that that echo isn't actually inserting newlines, but literal > '\n' instead. fwiw, it works a bit better using printf instead. > > On Wed, Aug 19, 2015 at 2:10 PM, Christian Kniep <[email protected] > <mailto:[email protected]>> wrote: > Cool, > > thanks guys… that seems to solve my issue! > > #### > # cat /etc/heka/hekad.toml > [hekad] > maxprocs = 2 > > [TcpInput] > address = ":5514" > splitter = "newline_splitter" > decoder = "better_decoder" > > [newline_splitter] > type = "TokenSplitter" > delimiter = '\n' > > [better_decoder] > type = "ScribbleDecoder" > [better_decoder.message_fields] > Type = "MyType" > > [gross_decoder] > type = "PayloadRegexDecoder" > match_regex = "^(?P<Message>.+)" > > [PayloadEncoder] > append_newlines = true > > [LogOutput] > message_matcher = "TRUE" > encoder = “PayloadEncoder" > ##### > > Cheers > Christian > > > On 19 Aug 2015, at 23:06, Rob Miller <[email protected] > > <mailto:[email protected]>> wrote: > > > > What you're seeing is normal. When your client shuts down, the TCP > > connection closes, so Heka spins down the goroutines for handling that > > connection. Each connection gets its own decoder instance, so you're seeing > > the ProtobufDecoder for that connection exiting. > > > > Since TcpInput defaults to using ProtobufDecoder, you'll need to specify > > something to override the default. Ideally you'd do some actual parsing > > here, but if you don't need to do any parsing the most minimal performance > > impact would probably be to use a ScribbleDecoder to scribble some value > > onto each message. > > > > -r > > > > > > On 08/19/2015 01:44 PM, Christian Kniep wrote: > >> Hey y’all, > >> > >> after opening an issue on github I am still not sure how to create the > >> TCP input to allow me sending arbitrary strings. > >> https://github.com/mozilla-services/heka/issues/1683 > >> <https://github.com/mozilla-services/heka/issues/1683> > >> > >> I am creating a docker image out of it: > >> https://github.com/qnib/docker-heka <https://github.com/qnib/docker-heka> > >> > >> My heka.toml looks as follows: > >> ############## > >> [hekad] > >> maxprocs = 2 > >> > >> [TcpInput] > >> address = ":5514" > >> splitter = "newline_splitter" > >> > >> [newline_splitter] > >> type = "RegexSplitter" > >> delimiter = '\n' > >> delimiter_eol = true > >> > >> [split_on_space] > >> type = "TokenSplitter" > >> delimiter = " " > >> > >> [PayloadEncoder] > >> append_newlines = true > >> > >> [LogOutput] > >> message_matcher = "TRUE" > >> encoder = “PayloadEncoder" > >> ############## > >> > >> If I send a string the TCPInput stops: > >> ############## > >> [root@heka tmp]# /opt/heka/bin/hekad -config=/etc/heka/hekad.toml > >> 2015/08/19 22:38:14 Pre-loading: [LogOutput] > >> 2015/08/19 22:38:14 Pre-loading: [TcpInput] > >> 2015/08/19 22:38:14 Pre-loading: [newline_splitter] > >> 2015/08/19 22:38:14 Pre-loading: [split_on_space] > >> 2015/08/19 22:38:14 Pre-loading: [PayloadEncoder] > >> 2015/08/19 22:38:14 Pre-loading: [HekaFramingSplitter] > >> 2015/08/19 22:38:14 Loading: [HekaFramingSplitter] > >> 2015/08/19 22:38:14 Pre-loading: [NullSplitter] > >> 2015/08/19 22:38:14 Loading: [NullSplitter] > >> 2015/08/19 22:38:14 Pre-loading: [ProtobufDecoder] > >> 2015/08/19 22:38:14 Loading: [ProtobufDecoder] > >> 2015/08/19 22:38:14 Pre-loading: [ProtobufEncoder] > >> 2015/08/19 22:38:14 Loading: [ProtobufEncoder] > >> 2015/08/19 22:38:14 Pre-loading: [TokenSplitter] > >> 2015/08/19 22:38:14 Loading: [TokenSplitter] > >> 2015/08/19 22:38:14 Loading: [PayloadEncoder] > >> 2015/08/19 22:38:14 Loading: [newline_splitter] > >> 2015/08/19 22:38:14 Loading: [split_on_space] > >> 2015/08/19 22:38:14 Loading: [TcpInput] > >> 2015/08/19 22:38:14 Loading: [LogOutput] > >> 2015/08/19 22:38:14 Starting hekad... > >> 2015/08/19 22:38:14 Output started: LogOutput > >> 2015/08/19 22:38:14 MessageRouter started. > >> 2015/08/19 22:38:14 Input started: TcpInput > >> 2015/08/19 22:38:18 Decoder 'TcpInput-ProtobufDecoder-::1': stopped > >> 2015/08/19 22:38:18 > >> ############## > >> While sending something like this: > >> # echo "huhu\n"|nc -w1 localhost 5514 > >> > >> Cheers > >> Christian > >> > >> > >> _______________________________________________ > >> Heka mailing list > >> [email protected] <mailto:[email protected]> > >> https://mail.mozilla.org/listinfo/heka > >> <https://mail.mozilla.org/listinfo/heka> > >> > > > _______________________________________________ > Heka mailing list > [email protected] <mailto:[email protected]> > https://mail.mozilla.org/listinfo/heka > <https://mail.mozilla.org/listinfo/heka> > > > _______________________________________________ > Heka mailing list > [email protected] > https://mail.mozilla.org/listinfo/heka
signature.asc
Description: Message signed with OpenPGP using GPGMail
_______________________________________________ Heka mailing list [email protected] https://mail.mozilla.org/listinfo/heka
