Couple of things to notice just from the screenshot... As Andy points out the store in kite dataset processor appears to be doing very little. The preceding two processors are
ConvertJSONtoAvro at 270 messages per second. The resulting data is far smaller than the input which is interesting. ReplaceText is processing at 70 messages per second StoreInKite is doing very very little. And I doubt back pressure is a factor in this config as-is. Neither of those processors appear to be actively executing threads when this snapshot was taken. That makes sense since neither has much of a queue. ListenSyslog has 4 active threads. You likely won't need that many. I would avoid adding 100 parallel threads to any processor. But as for speed of the convert and replace they seem to be going as fast as needed to receive the data... On Tue, Jun 21, 2016 at 1:18 PM, Andy LoPresto <[email protected]> wrote: > It looks like the StoreInKiteDataset processor is the issue. It has read > 1.85 MB in, but written 0 B out. What are the back pressure settings on that > processor? > > > Andy LoPresto > [email protected] > [email protected] > PGP Fingerprint: 70EC B3E5 98A6 5A3F D3C4 BACE 3C6E F65B 2F7D EF69 > > On Jun 21, 2016, at 9:07 AM, pradeepbill <[email protected]> wrote: > > Hi there, I am seeing huge delays in ConvertJSONToAvro processor where the > converted data trickles down to the next step.Please see attachment.I have > increased the concurrent tasks =100. Is there anything else I can > configure/tune up. > > nifi_Capture.PNG > <http://apache-nifi-developer-list.39713.n7.nabble.com/file/n11957/nifi_Capture.PNG> > > Thanks > Pradeep > > > > -- > View this message in context: > http://apache-nifi-developer-list.39713.n7.nabble.com/ConvertJSONToAvro-processor-is-slow-tp11957.html > Sent from the Apache NiFi Developer List mailing list archive at Nabble.com. > >
