[
https://issues.apache.org/jira/browse/METRON-668?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15834725#comment-15834725
]
ASF GitHub Bot commented on METRON-668:
---------------------------------------
Github user cestella commented on the issue:
https://github.com/apache/incubator-metron/pull/420
@nickwallen running as we speak. The test plan will be to generate some
data like we do for the MAD outliers and construct a couple simple profiles.
I'll make it a bit more explicit after I do it. :)
> Remove the "tickUpdate" profile config and make the "init" phase not reset
> variables
> ------------------------------------------------------------------------------------
>
> Key: METRON-668
> URL: https://issues.apache.org/jira/browse/METRON-668
> Project: Metron
> Issue Type: Improvement
> Reporter: Casey Stella
> Assignee: Casey Stella
>
> Originally during work on the MAD outlier work, I conceived of a need for a
> new callback in the profile configuration, "tickUpdate" that ran at the tick
> and had the variables from the tick that just completed available to it.
> This was done so that I could merge the state accumulated in the tick with a
> lookback window of state for MAD. The problem is that the "init" phase
> happens after this and blows away the changes done in "tickUpdate", so it
> never worked like intended.
> It occurs to me that what we really want is not to have two separate config
> phases, but only one, "init" and to not reset the variables on the tick for
> the profile. You can, of course, choose to update them by overwriting them
> in the "init" phase *or* you can choose to use them as part of your init.
> For context, this would make the example for MAD:
> {code:javascript}
> {
> "profiles": [
> {
> "profile": "sketchy_mad",
> "foreach": "'global'",
> "onlyif": "true",
> "init" : {
> "s": "OUTLIER_MAD_STATE_MERGE(PROFILE_GET('sketchy_mad',
> 'global', 5, 'MINUTES'))"
> },
> "tickUpdate": {
> "s": "OUTLIER_MAD_STATE_MERGE(PROFILE_GET('sketchy_mad',
> 'global', 5, 'MINUTES'), s)"
> },
> "update": {
> "s": "OUTLIER_MAD_ADD(s, value)"
> },
> "result": "s"
> }
> ]
> }
> {code}
> is functionally equivalent to
> {code:javascript}
> {
> "profiles": [
> {
> "profile": "sketchy_mad",
> "foreach": "'global'",
> "onlyif": "true",
> "init" : {
> "s": "OUTLIER_MAD_STATE_MERGE(PROFILE_GET('sketchy_mad',
> 'global', 5, 'MINUTES'))"
> },
> "update": {
> "s": "OUTLIER_MAD_ADD(s, value)"
> },
> "result": "s"
> }
> ]
> }
> {code}
> This resets the MAD state to the last 5 minute window. If we did NOT reset
> the state and keep accumulating state (provided we did not clear the
> variables on init, we could do the following:
> {code:javascript}
> {
> "profiles": [
> {
> "profile": "sketchy_mad",
> "foreach": "'global'",
> "onlyif": "true",
> "init" : {
> "s": "if exists(s) then s else
> OUTLIER_MAD_STATE_MERGE(PROFILE_GET('sketchy_mad',
> 'global', 5, 'MINUTES'))"
> },
> "update": {
> "s": "OUTLIER_MAD_ADD(s, value)"
> },
> "result": "s"
> }
> ]
> }
> {code}
> s would get initialized sensibly and then always accumulate as long as the
> topology continued (rather than having a fixed lookback).
> In short, making init to not reset the variables shouldn't cause any harm and
> should provide another set of use-cases for the profiler. Also, tickUpdate
> has no function whatsoever and should be removed because it gets overwritten
> by init directly after being called.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
