GitHub user nickwallen opened a pull request:
https://github.com/apache/metron/pull/708
Metron 1120
[METRON-1120](https://issues.apache.org/jira/browse/METRON-1120)
- [ ] This is built on top of METRON-1120 so this should not be committed
before METRON-1120.
The `groupBy` expression can now reference any of these variables.
* `profile` The name of the profile.
* `entity` The name of the entity being profiled.
* `start` The start time of the profile period in epoch milliseconds.
* `end` The end time of the profile period in epoch milliseconds.
* `duration` The duration of the profile period in milliseconds.
* `result` The result of executing the `result` expression.
Unit tests have been added to validate this functionality. The README has
also been updated to describe the fields available to the `groupBy` expression.
This can also be tested manually in either a live Profiler or using the
Profiler debugging functions.
The following shows how this change would be used to implement the
problematic profile described in.
Create a profile that references the start of the profile period in the
`groupBy` expression.
```
[Stellar]>>> conf := SHELL_EDIT()
[Stellar]>>> conf
{
"profiles": [
{
"profile": "calender-effects",
"onlyif": "exists(ip_src_addr) and exists(timestamp)",
"foreach": "ip_src_addr",
"init":{ "count": 0 },
"update": { "count": "count + 1" },
"result": "count",
"groupBy": ["DAY_OF_WEEK(start)"]
}
]
}
```
Create a message to exercise the profiler.
```
[Stellar]>>> msg := SHELL_EDIT()
[Stellar]>>> msg
{
"ip_src_addr":"10.0.0.1",
"timestamp":"2017-08-18 09:00:00"
}
```
Create a Profiler and apply the messages to it.
```
[Stellar]>>> p := PROFILER_INIT(conf)
[Stellar]>>> PROFILER_APPLY(msg, p)
org.apache.metron.profiler.StandAloneProfiler@4572b5b4
[Stellar]>>> PROFILER_APPLY(msg, p)
org.apache.metron.profiler.StandAloneProfiler@4572b5b4
[Stellar]>>> PROFILER_APPLY(msg, p)
org.apache.metron.profiler.StandAloneProfiler@4572b5b4
```
Flush the profile and validate the result of executing the `groupBy`. The
value is 6, which indicates Friday, which is correct in this case.
```
[Stellar]>>> PROFILER_FLUSH(p)
[{period={duration=90, period=1670094, start=150308460,
end=150308550}, profile=calender-effects, groups=[6], value=3,
entity=10.0.0.1}]
```
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/nickwallen/metron METRON-1120
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/metron/pull/708.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #708
commit 5579748ad4336a7c1a15c319d59fd6cbdeb6531e
Author: Nick Allen
Date: 2017-08-18T17:37:01Z
METRON-1121 Ignore Profile with Bad 'init', 'update' or 'groupBy'
commit 893b7db84f155ea6af975ee51338f39b763eaedb
Author: Nick Allen
Date: 2017-08-18T17:45:50Z
Rm errant comment
commit da365c8b546678bbe07011e10ab3cd222faa8297
Author: Nick Allen
Date: 2017-08-18T19:01:26Z
METRON-1120 Profile's 'groupBy' Expression Has No Reference to Time
commit 5d8a7a06096d5aa725a0ce3b47fef36a8e14ac72
Author: Nick Allen
Date: 2017-08-18T19:04:35Z
Rm artifacts that should not be in Git
commit c52dce2be6146127eed9af0d2b311ff65f0de551
Author: Nick Allen
Date: 2017-08-18T19:32:07Z
Updated README
commit 54f1c5969268032e0841f0dd4b5e76449b8b3b6f
Author: Nick Allen
Date: 2017-08-18T19:35:31Z
Fix README
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
