Probably, though there are things (unlikely things) you can do to templates that would prevent that.
> On 26 Sep 2017, at 17:25, Laurens Vets <[email protected]> wrote: > > Why would I need to update my ES template? I should see the field (possibly > with the wrong type) anyways in the event after I refreshed the fields in > Kibana right? > > On 2017-09-26 09:16, Simon Elliston Ball wrote: > >> There should be, though you may need to update your templates in ES if >> you've got any custom templates there, and make sure you refresh the fields >> in kibana's index config. >> >> Simon >> >> >>> On 26 Sep 2017, at 17:13, Laurens Vets <[email protected] >>> <mailto:[email protected]>> wrote: >>> >>> After setting is_alert to true, this field is now shown in my event in >>> Kibana. I would expect there also to be a field "threat:triage:level" in >>> those same events (if my rules work?) >>> >>> On 2017-09-25 16:46, [email protected] <mailto:[email protected]> wrote: >>> >>> I was quickly reading through this on my mobile device so sorry if I'm off >>> base here, but it may be because threat.triage.level is changed to >>> threat:triage:level just before indexing due to the inability to use a >>> period in keys on older versions of ES. Not sure exactly what you mean by >>> you don't get a threat.triage.level field. >>> >>> Jon >>> >>> >>> On Mon, Sep 25, 2017, 19:34 Laurens Vets <[email protected] >>> <mailto:[email protected]>> wrote: >>> Next problem: >>> >>> I'm setting the "is_alert" field to true. It shows up in Kibana, but I >>> don't get a threat.triage.level field which means that either my >>> riskLevelRules rules don't trigger or something else goes wrong. >>> >>> How and where can I look for additional information on why my rules >>> might not be working? (Metron UI accepts my JSON without issues) >>> >>> On 2017-09-25 13:39, Laurens Vets wrote: >>> > Thanks! >>> > >>> > On 2017-09-25 13:16, Simon Elliston Ball wrote: >>> >> The second statement overwrites the first, but also uses the previous >>> >> value. >>> >> >>> >> Technically that is an or. Note this construct is designed to allow >>> >> multiple different trigger conditions to make is_alert true, hence the >>> >> second one being is_alert := is_alert || something_else. >>> >> >>> >> && is bitwise and >>> >> || is bitwise or >>> >> >>> >> Simon >>> >> >>> >>> On 25 Sep 2017, at 21:12, Laurens Vets <[email protected] >>> >>> <mailto:[email protected]>> wrote: >>> >>> >>> >>> Thanks! Followup question, the below is_alert 'rules' in the snippet >>> >>> from >>> >>> http://apache.website-solution.net/metron/0.4.1/site-book/use-cases/geographic_login_outliers/index.html >>> >>> >>> >>> <http://apache.website-solution.net/metron/0.4.1/site-book/use-cases/geographic_login_outliers/index.html>, >>> >>> are those an AND or OR? >>> >>> >>> >>> "threatIntel": { >>> >>> "fieldMap": { >>> >>> "stellar" : { >>> >>> "config" : [ >>> >>> "geo_distance_distr:= STATS_MERGE( PROFILE_GET( >>> >>> 'geo_distribution_from_centroid', 'global', PROFILE_FIXED( 2, >>> >>> 'MINUTES')))", >>> >>> "dist_median := STATS_PERCENTILE(geo_distance_distr, 50.0)", >>> >>> "dist_sd := STATS_SD(geo_distance_distr)", >>> >>> "geo_outlier := ABS(dist_median - geo_distance) >= >>> >>> 5*dist_sd", >>> >>> "is_alert := exists(is_alert) && is_alert", >>> >>> "is_alert := is_alert || (geo_outlier != null && geo_outlier >>> >>> == true)", >>> >>> "geo_distance_distr := null" >>> >>> ] >>> >>> } >>> >>> }, >>> >>> >>> >>> For instance, can the 2nd is_alert line overwrite the value assigned >>> >>> in the first is_alert rule? >>> >>> >>> >>> On 2017-09-25 11:13, Simon Elliston Ball wrote: >>> >>>> Usually you would have the is_alert set based on more complex rules, >>> >>>> and then potentially have different rules to determine the >>> >>>> importance >>> >>>> of the alert, so they do tend to serve different purposes. >>> >>>> For example a triage rule might be set on levels of an indicator >>> >>>> after >>> >>>> is_alert has been triggered by a simple presence of a non-zero >>> >>>> result >>> >>>> for that indicator, e.g. is it 2x std_devs, or 4x std_devs as >>> >>>> different rule levels. We're adding the ability to make score a >>> >>>> stellar statement which simplifies this further by allowing score to >>> >>>> be a function, but thresholds are still useful to determine the text >>> >>>> content of the alert for example. >>> >>>> Simon >>> >>>>> On 25 Sep 2017, at 19:09, Laurens Vets <[email protected] >>> >>>>> <mailto:[email protected]>> wrote: >>> >>>>> Oh, I didn't know I had to set is_alert to True. >>> >>>>> Doesn't that mean that we have to add all rules twice? First to >>> >>>>> check whether is_alert needs to be set to True. Next to apply the >>> >>>>> actual scores? >>> >>>>> On 2017-09-25 11:00, Simon Elliston Ball wrote: >>> >>>>>> the _score field is actually an elastic search matching score >>> >>>>>> field, >>> >>>>>> and is not relevant to metron. You should see the scores in the >>> >>>>>> threat:triage:score field. However, your rules will only be run if >>> >>>>>> the >>> >>>>>> telemetry has is_alert set true, so you should ensure that the >>> >>>>>> enrichment phase sets is_alert: true somewhere for alerts you want >>> >>>>>> to >>> >>>>>> go to triage? >>> >>>>>> Simon >>> >>>>>>> On 25 Sep 2017, at 18:46, Laurens Vets <[email protected] >>> >>>>>>> <mailto:[email protected]>> wrote: >>> >>>>>>> I have the following configuration: >>> >>>>>>> "threatIntel": { >>> >>>>>>> "fieldMap": {}, >>> >>>>>>> "fieldToTypeMap": {}, >>> >>>>>>> "config": {}, >>> >>>>>>> "triageConfig": { >>> >>>>>>> "riskLevelRules": [ >>> >>>>>>> { >>> >>>>>>> "name": "Rule1", >>> >>>>>>> "comment": "Checks whatever 1.", >>> >>>>>>> "rule": "test == \"false\"", >>> >>>>>>> "score": 20, >>> >>>>>>> "reason": null >>> >>>>>>> }, >>> >>>>>>> { >>> >>>>>>> "name": "Rule1", >>> >>>>>>> "comment": "Checks whatever 2.", >>> >>>>>>> "rule": "test2 == \"False\"", >>> >>>>>>> "score": 20, >>> >>>>>>> "reason": null >>> >>>>>>> }, >>> >>>>>>> { >>> >>>>>>> "name": "Rule3", >>> >>>>>>> "comment": "Checks whatever 2.", >>> >>>>>>> "rule": "test3 == \"No\"", >>> >>>>>>> "score": 20, >>> >>>>>>> "reason": null >>> >>>>>>> } >>> >>>>>>> ], >>> >>>>>>> "aggregator": "SUM", >>> >>>>>>> "aggregationConfig": {} >>> >>>>>>> } >>> >>>>>>> }, >>> >>>>>>> I have no additional configuration in enrichment besides filling >>> >>>>>>> a specific with true or false based on a Stellar expression. >>> >>>>>>> I expected that when events would match my above rules, the >>> >>>>>>> _score field would be filled in. That does not seem to be the >>> >>>>>>> case. >>> >>>>>>> Does anyone know what I might be missing? >>> -- >>> Jon >>> >>> >>> >
