[
https://issues.apache.org/jira/browse/METRON-590?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15751641#comment-15751641
]
ASF GitHub Bot commented on METRON-590:
---------------------------------------
Github user nickwallen commented on the issue:
https://github.com/apache/incubator-metron/pull/395
So now I'm thinking about changes to the Profiler Client API based on those
usage scenarios.
For the **Live Data** use case, the existing `PROFILE_GET(profile, entity,
look-back)` function should work just fine. I think this is the most simple
function signature to use in this case.
For the **Replayed Data** use case, the existing `PROFILE_GET` signature is
not going to work. To query the Profiler for replayed data, we need one or
both of the following functions. Or maybe there is a better function signature
that I'm not thinking of.
#### PROFILE_GET_RANGE
Return the measurements taken for a profile and entity between a range of
time defined by a start and end.
The only problem here is how do I make it easy for the user to specify a
timestamp? Do I have to accept a number of different formats and try to coerce
to a timestamp? What formats should I accept?
```
PROFILE_GET_RANGE(profile, entity, start timestamp, end timestamp)
```
#### PROFILE_GET_FROM
Return the measurements taken for a profile and entity starting from a
point in time, the offset, and looking back from that offset. This might seem
initially hard to grok, but also might seem easier to use once you have your
mind wrapped around it. This follows closely the existing `PROFILE_GET`
function signature.
```
PROFILE_GET_FROM(profile, entity, look-back, offset)
```
For example, I replayed data that I know is 2 months old and now I need to
look at that data. I am interested in a 15 minute window from a profile that I
know is offset by 2 months.
```
PROFILE_GET_FROM('replayed-profile', '10.0.0.1', 15, "MINUTES", 2, "MONTHS")
```
> Enable Use of Event Time in Profiler
> ------------------------------------
>
> Key: METRON-590
> URL: https://issues.apache.org/jira/browse/METRON-590
> Project: Metron
> Issue Type: Improvement
> Reporter: Nick Allen
> Assignee: Nick Allen
>
> There are at least two different times that are important to consider when
> handling the telemetry messages received by Metron.
> (1) Processing time is the time at which Metron processed the message.
> (2) Event time is the time at which the event actually occurred.
> If Metron is consuming live data and all is well, the processing and event
> times may remain close and consistent. When processing time differs from
> event time the data produced by the Profiler may be inaccurate. There are a
> few scenarios under which these times might differ greatly which would
> negatively impact the feature set produced by the Profiler.
> (1) When the system has experienced an outage, for example, a scheduled
> maintenance window. When restarted a high volume of messages will need to be
> processed by the Profiler. The output of the Profiler will indicate an
> increase in activity, although no change in activity actually occurred on the
> target network. This could happen whether the outage was Metron itself or an
> upstream system that feeds data to Metron.
> (2) If the user attempts to replay historical telemetry through the Profiler,
> the Profiler will attribute the activity to the time period in which it was
> processed. Obviously the activity should be attributed to the time period in
> which the raw telemetry events originated in.
> There are some scenarios when processing time might be preferred and other
> use cases where event time is preferred. The Profiler should be enhanced to
> allow it to produce profiles based on either processing time or event time.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
