[
https://issues.apache.org/jira/browse/METRON-590?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Nick Allen updated METRON-590:
------------------------------
Description:
There are at least two different times that are important to consider when
handling the telemetry messages received by Metron.
(1) Processing time is the time at which Metron processed the message.
(2) Event time is the time at which the event actually occurred.
If Metron is consuming live data and all is well, the processing and event
times may remain close and consistent. When processing time differs from event
time the data produced by the Profiler may be inaccurate. There are a few
scenarios under which these times might differ greatly which would negatively
impact the feature set produced by the Profiler.
(1) When the system has experienced an outage, for example, a scheduled
maintenance window. When restarted a high volume of messages will need to be
processed by the Profiler. The output of the Profiler will indicate an
increase in activity, although no change in activity actually occurred on the
target network. This could happen whether the outage was Metron itself or an
upstream system that feeds data to Metron.
(2) If the user attempts to replay historical telemetry through the Profiler,
the Profiler will attribute the activity to the time period in which it was
processed. Obviously the activity should be attributed to the time period in
which the raw telemetry events originated in.
There are some scenarios when processing time might be preferred and other use
cases where event time is preferred. The Profiler should be enhanced to allow
it to produce profiles based on either processing time or event time.
was:
There are at least two different times that are important to consider when
handling the telemetry messages received by Metron.
(1) Processing time is the time at which Metron processed the message.
(2) Event time is the time at which the event actually occurred.
If Metron is consuming live data and all is well, these times may remain close
and consistent. There are many scenarios under which these times might differ
greatly. For example, if there are delays in processing messages due to heavy
load or if older, archived telemetry is being replayed through the system.
When processing time differs from event time the data produced by the Profiler
may be inaccurate. There are some use cases when processing time might be
preferred and other use cases where event time is preferred. The Profiler
should be enhanced to allow it to produce profiles based on either processing
time or event time.
> Enable Use of Event Time in Profiler
> ------------------------------------
>
> Key: METRON-590
> URL: https://issues.apache.org/jira/browse/METRON-590
> Project: Metron
> Issue Type: Improvement
> Reporter: Nick Allen
> Assignee: Nick Allen
>
> There are at least two different times that are important to consider when
> handling the telemetry messages received by Metron.
> (1) Processing time is the time at which Metron processed the message.
> (2) Event time is the time at which the event actually occurred.
> If Metron is consuming live data and all is well, the processing and event
> times may remain close and consistent. When processing time differs from
> event time the data produced by the Profiler may be inaccurate. There are a
> few scenarios under which these times might differ greatly which would
> negatively impact the feature set produced by the Profiler.
> (1) When the system has experienced an outage, for example, a scheduled
> maintenance window. When restarted a high volume of messages will need to be
> processed by the Profiler. The output of the Profiler will indicate an
> increase in activity, although no change in activity actually occurred on the
> target network. This could happen whether the outage was Metron itself or an
> upstream system that feeds data to Metron.
> (2) If the user attempts to replay historical telemetry through the Profiler,
> the Profiler will attribute the activity to the time period in which it was
> processed. Obviously the activity should be attributed to the time period in
> which the raw telemetry events originated in.
> There are some scenarios when processing time might be preferred and other
> use cases where event time is preferred. The Profiler should be enhanced to
> allow it to produce profiles based on either processing time or event time.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
