Casey Stella created METRON-668:
-----------------------------------
Summary: Remove the "tickUpdate" profile config and make the
"init" phase not reset variables
Key: METRON-668
URL: https://issues.apache.org/jira/browse/METRON-668
Project: Metron
Issue Type: Improvement
Reporter: Casey Stella
Assignee: Casey Stella
Originally during work on the MAD outlier work, I conceived of a need for a new
callback in the profile configuration, "tickUpdate" that ran at the tick and
had the variables from the tick that just completed available to it. This was
done so that I could merge the state accumulated in the tick with a lookback
window of state for MAD. The problem is that the "init" phase happens after
this and blows away the changes done in "tickUpdate", so it never worked like
intended.
It occurs to me that what we really want is not to have two separate config
phases, but only one, "init" and to not reset the variables on the tick for the
profile. You can, of course, choose to update them by overwriting them in the
"init" phase *or* you can choose to use them as part of your init.
For context, this would make the example for MAD:
{code:javascript}
{
"profiles": [
{
"profile": "sketchy_mad",
"foreach": "'global'",
"onlyif": "true",
"init" : {
"s": "OUTLIER_MAD_STATE_MERGE(PROFILE_GET('sketchy_mad',
'global', 5, 'MINUTES'))"
},
"tickUpdate": {
"s": "OUTLIER_MAD_STATE_MERGE(PROFILE_GET('sketchy_mad',
'global', 5, 'MINUTES'), s)"
},
"update": {
"s": "OUTLIER_MAD_ADD(s, value)"
},
"result": "s"
}
]
}
{code}
is functionally equivalent to
{code:javascript}
{
"profiles": [
{
"profile": "sketchy_mad",
"foreach": "'global'",
"onlyif": "true",
"init" : {
"s": "OUTLIER_MAD_STATE_MERGE(PROFILE_GET('sketchy_mad',
'global', 5, 'MINUTES'))"
},
"update": {
"s": "OUTLIER_MAD_ADD(s, value)"
},
"result": "s"
}
]
}
{code}
This resets the MAD state to the last 5 minute window. If we did NOT reset the
state and keep accumulating state (provided we did not clear the variables on
init, we could do the following:
{code:javascript}
{
"profiles": [
{
"profile": "sketchy_mad",
"foreach": "'global'",
"onlyif": "true",
"init" : {
"s": "if exists(s) then s else
OUTLIER_MAD_STATE_MERGE(PROFILE_GET('sketchy_mad',
'global', 5, 'MINUTES'))"
},
"update": {
"s": "OUTLIER_MAD_ADD(s, value)"
},
"result": "s"
}
]
}
{code}
s would get initialized sensibly and then always accumulate as long as the
topology continued (rather than having a fixed lookback).
In short, making init to not reset the variables shouldn't cause any harm and
should provide another set of use-cases for the profiler. Also, tickUpdate has
no function whatsoever and should be removed because it gets overwritten by
init directly after being called.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
