Prior to this point in time, the Profiler had received a message indicating that the current time is Mon Jan 27 2020 17:46:44 GMT. It then received a message with a timestamp of Tue Jan 28 2020 09:02:52 GMT, about 15 hours in the future. Since this time gap is significantly larger than your profile period, a log message is written to warn you of a data quality issue. I can think of a few possible causes of this that you will want to investigate further.
(1) There is a large time gap in the data that you are processing. Is this expected? Was there an outage that caused an interruption in your data feed? If the outage is expected, then you can safely ignore this warning. (2) Your data is significantly out-of-order for some reason. You can accommodate some out-of-order data by adjusting the Profiler's time lag <https://metron.apache.org/current-book/metron-analytics/metron-profiler-storm/index.html#profiler.window.lag>, but it does not seem reasonable to account for 15 hours. (3) If your profile is processing telemetry from different sources, perhaps there is a timestamp in one of these sources that is significantly different than all the others. Hope this helps On Tue, Jan 28, 2020 at 5:00 AM Gonçalo Pedras <goncalo.ped...@ctd.pt> wrote: > Hi again, > > I found something in the profiler storm logs that proves the delay: > > “2020-01-28 09:46:37.061 o.a.m.p.s.FixedFrequencyFlushSignal > watermark-event-generator-0 [WARN] Timestamp out-of-order by -54968000 ms. > This may indicate a problem in the data. timestamp=1580202172000, > maxKnown=1580147204000, flushFreq=900000 ms” > > > > The profiler is delayed 15 hours and a half. >
