My point was not accumulating strings in output buffer but injecting several messages instead to make them smaller. Or do you mean that inject_message() run several times in a single run should work? I’ll try that — haven’t done that yet.
Timur On 19 Jan 2016 at 22:50:28, Rob Miller ([email protected]) wrote: Yes, that's right. A SandboxEncoder can only make a single inject call. If you call `inject_message`, then you will be emitting a protobuf encoded Heka message. If you call `inject_payload`, then you will be emitting a UTF8 encoded set of bytes. If you want to accumulate multiple string values in the output buffer, you can use `add_to_payload` before a final `inject_payload` call. -r On 01/16/2016 12:08 PM, Timur Batyrshin wrote: > An encoder I was using for reference: > > function process_message() > inject_payload("msg1", "", "msg1") > inject_payload("msg2", "", "msg2") > return 0 > end > > This produces only “msg2”. > > > Timur > > > On 16 Jan 2016 at 22:09:35, Timur Batyrshin ([email protected] > <mailto:[email protected]>) wrote: > > > Hi, > > > > > > I’ve been trying to do some buffering and splitting of messages in an > > encoder and I’ve run in a following issue. > > > > Looks like calling inject_payload() several times in a single run of > > encoder’s process_message() doesn’t work and only the last message is > > injected. > > Is it so? > > > > If yes, any ideas how can I solve the following task? > > * I have incoming messages consisting of up to several hundreds of > > custom fields > > * I’m encoding the fields into JSON array like [{“metric": > > “foobar.count", "value": 1000}, {“metric": “bazqux.rate”, “value”: > > 10}] and sending them out via HTTP to a different service > > * I’d like to keep outgoing HTTP messages relatively small (no more > > than a few dozens of fields) and for that reason I’m going to split > > messages with too many fields into several smaller ones > > > > For that I was going to do the splitting inside encoder attached to > > HTTPOutput and calling inject_payload() several times but it seems > > that only the last message is sent out. > > I’ve tried setting max_process_inject = 1000 in [hekad] config section > > but this didn’t help either. > > > > I could try using a filter to split messages and reinject them back to > > Heka (haven’t tested yet if it is going to work in this scenario) but > > this looks like a bit overweight to me. > > > > Any other ideas how to handle the task? > > > > > > Thanks, > > Timur > > > _______________________________________________ > Heka mailing list > [email protected] > https://mail.mozilla.org/listinfo/heka >
_______________________________________________ Heka mailing list [email protected] https://mail.mozilla.org/listinfo/heka
