great, that's the info we needed. so you can drop the TcpOutput for now since it appears that you are trying to write in the native heka format to a heka process from your own app. having heka write to another endpoint might be useful later, but it doesn't need to write to itself or the LogOutput plugin to display your messages in stdout.
i'm not sure what it's in message_bytes, but here's a snippet that you can use as a reference. https://github.com/imgix/hekametrics/blob/master/hekalogger.go On Thu, Jan 7, 2016 at 3:48 PM, Emily Gu <[email protected]> wrote: > Thanks you both Rob and David very much! > > Not sure where I need to define "base_dir"? > > I'm going to write a Heka plugin to pass our metrics data into Heka. > > For now, I have a hard time to see the data I send in through > TCP programmatically through TcpInput in the output.log file. > I don't see any output. The configs are: > > tcp_input.toml > ============ > > [hekad] > > maxprocs = 1 > > share_dir = "/Users/egu/heka/share/heka" > > > [tcp_in:3242] > > type = "TcpInput" > > splitter = "HekaFramingSplitter" > > decoder = "ProtobufDecoder" > > address = ":3242" > > > tcp_output.toml > > ============== > > [hekad] > > maxprocs = 1 > > share_dir = "/Users/egu/heka/share/heka" > > > [tcp_out:3242] > > type = "TcpOutput" > > message_matcher = "TRUE" > > address = "127.0.0.1:3242" > > > [tcp_heka_output_log] > > type = "FileOutput" > > message_matcher = "TRUE" > > path = "/tmp/output.log" > > perm = "664" > > encoder = "tcp_heka_output_encoder" > > > [tcp_heka_output_encoder] > > type = "PayloadEncoder" > > append_newlines = false > > > The client: > > package main > > > import ( > > "fmt" > > "github.com/mozilla-services/heka/client" > > ) > > > > func main() { > > message_bytes := []byte {100} > > > sender, err := client.NewNetworkSender("tcp", "127.0.0.1:3242") > > if err != nil { > > fmt.Println("Could not connect to", "127.0.0.1:3242") > > return > > } > > fmt.Println("Connected") > > > > var i int > > for i = 0; i < 10; i++ { > > fmt.Println("message byte:", string(message_bytes)) > > err = sender.SendMessage(message_bytes) > > if err != nil { > > break > > } > > } > > fmt.Println("sent", i, "messages") > > } > > > Please let me know what else I need to change. > > Thanks, > > Emily > > > > > > > On Thu, Jan 7, 2016 at 3:28 PM, David Birdsong <[email protected]> > wrote: > >> >> >> On Thu, Jan 7, 2016 at 3:22 PM, Rob Miller <[email protected]> wrote: >> >>> On 01/07/2016 03:09 PM, Emily Gu wrote: >>> >>>> Thanks David for all the help! I'll give it a try. >>>> >>>> Please bear with me as some parts I still not understand. >>>> >>>> 1. Why do I have to run two Heka instances where one for input and >>>> another for output? >>>> >>> >>> Because if you send the output from a Heka instance back into itself, >>> then you're likely setting up an infinite loop of traffic that will spin >>> out of control. >>> >>> 2. Did you mean I need to specify different share_dirs in input and >>>> output Toml configs? >>>> >>> >>> If you're running multiple Heka instances on a single machine, it >>> *should* be fine for them to use the same share_dir, which is read-only. >>> It's very important that each specifies a unique base_dir, however, since >>> that's used by Heka for internal bookkeeping data. Two Heka's using the >>> same base_dir is asking for trouble. >>> >>> 3. Do I need both TcpOutput and FileOutput in order for me to see >>>> messages inside an output file? What if I didn't specify TcpOutput? >>>> >>> >>> Um, TcpOutput sends output data over a TCP connection. It expects that >>> there is a listener on the other side which will accept that TCP >>> connection, and will know how to correctly handle the data that Heka is >>> sending over the TCP connection. >>> >>> FileOutput sends data to a file on the local file system. >>> >>> It's of course fine to specify a FileOutput without specifying a >>> TcpOutput. >>> >>> -r >>> >> >> whoops, yes I meant base_dir for where heka writes various internal state >> information to. >> >> Emily, >> >> Maybe you could share what data you're trying to read into heka and what >> you would like to do with it and we could help get you going. >> >> Heka intended to a uni-directional pipeline. It can read data in from >> many places into various formats, aggregate into interesting new formats, >> and finally emit data somewhere else. >> >> >
_______________________________________________ Heka mailing list [email protected] https://mail.mozilla.org/listinfo/heka
