Re: Sink metric numRecordsIn drops temporarily

2017-09-17 Thread Michael Fong
Hi, Thank you for your description. What I tried to understand is what the counter value is at that moment of spikes. Grafana would take the average out of a continuous data values before rendering result to UI. That is, if the metrics value is not transmitted continuously, where at some data

Re: Sink metric numRecordsIn drops temporarily

2017-09-17 Thread Philipp Bussche
Hi, thank you for your answer. So for September 11th which is shown on the screenshot I had the counter sitting at 26.91k where when the drop happened it was going down to 26.01k. This happened 3 times during that day and it was always going back to the same value. Philipp -- Sent from:

Re: Sink metric numRecordsIn drops temporarily

2017-09-17 Thread Michael Fong
Missed to cc to user@flink.apache.org Hi, > > Just wondering what is the value of that counter (wo/ applying > NonNegativeDerivative > function) when you observe the spikes? If I remember correctly, Grafana > is known to aggregate those values by averaging them across the time > duration

Sink metric numRecordsIn drops temporarily

2017-09-17 Thread Philipp Bussche
Hi there, I witnessed an interesting behaviour on my Grafana dashboard where sink related metrics would show activity where there should not be any. I am saying this because for this particular sink activity is triggered by data being ingested through a cronjob at a particular time, however the