I have a heka instance that's upstream of another heka instance that's queued ~60GB of output. So the moment the upstream heka starts, it gets pounded with a heavy workload.
The upstream heka runs for exactly 20 mins before halting all progress and prints out the idle packs error message. As always, the error includes every plugin attributing the same count to every one of them: https://gist.github.com/davidbirdsong/f5d1bf140c865f9b2d72 Prior to the error, the http dashboard indicates that the decoder plugin channels are usually full or close to full, but are still moving messages through them--roughly ~8-11k/sec decode rates depending on whether there are 1 or 2 downstream heka's connected. After the error, message matchers and plugin channels all show 0 messages. What's a good course of action to rooting out and fixing the cause? Since this heka is upstream and is well positioned to apply back pressure, I'm trying out configuring the plugin channel to 0. It seems that's the best way to cut down on messages lost on restart, but I don't get why it would impact back pressure since a full channel is the same as a blocking channel (right?). All hail Heka son of Atum.
_______________________________________________ Heka mailing list [email protected] https://mail.mozilla.org/listinfo/heka
