I was quickly reading through this on my mobile device so sorry if I'm off base here, but it may be because threat.triage.level is changed to threat:triage:level just before indexing due to the inability to use a period in keys on older versions of ES. Not sure exactly what you mean by you don't get a threat.triage.level field.
Jon On Mon, Sep 25, 2017, 19:34 Laurens Vets <[email protected]> wrote: > Next problem: > > I'm setting the "is_alert" field to true. It shows up in Kibana, but I > don't get a threat.triage.level field which means that either my > riskLevelRules rules don't trigger or something else goes wrong. > > How and where can I look for additional information on why my rules > might not be working? (Metron UI accepts my JSON without issues) > > On 2017-09-25 13:39, Laurens Vets wrote: > > Thanks! > > > > On 2017-09-25 13:16, Simon Elliston Ball wrote: > >> The second statement overwrites the first, but also uses the previous > >> value. > >> > >> Technically that is an or. Note this construct is designed to allow > >> multiple different trigger conditions to make is_alert true, hence the > >> second one being is_alert := is_alert || something_else. > >> > >> && is bitwise and > >> || is bitwise or > >> > >> Simon > >> > >>> On 25 Sep 2017, at 21:12, Laurens Vets <[email protected]> wrote: > >>> > >>> Thanks! Followup question, the below is_alert 'rules' in the snippet > >>> from > >>> > http://apache.website-solution.net/metron/0.4.1/site-book/use-cases/geographic_login_outliers/index.html > , > >>> are those an AND or OR? > >>> > >>> "threatIntel": { > >>> "fieldMap": { > >>> "stellar" : { > >>> "config" : [ > >>> "geo_distance_distr:= STATS_MERGE( PROFILE_GET( > >>> 'geo_distribution_from_centroid', 'global', PROFILE_FIXED( 2, > >>> 'MINUTES')))", > >>> "dist_median := STATS_PERCENTILE(geo_distance_distr, 50.0)", > >>> "dist_sd := STATS_SD(geo_distance_distr)", > >>> "geo_outlier := ABS(dist_median - geo_distance) >= > >>> 5*dist_sd", > >>> "is_alert := exists(is_alert) && is_alert", > >>> "is_alert := is_alert || (geo_outlier != null && geo_outlier > >>> == true)", > >>> "geo_distance_distr := null" > >>> ] > >>> } > >>> }, > >>> > >>> For instance, can the 2nd is_alert line overwrite the value assigned > >>> in the first is_alert rule? > >>> > >>> On 2017-09-25 11:13, Simon Elliston Ball wrote: > >>>> Usually you would have the is_alert set based on more complex rules, > >>>> and then potentially have different rules to determine the > >>>> importance > >>>> of the alert, so they do tend to serve different purposes. > >>>> For example a triage rule might be set on levels of an indicator > >>>> after > >>>> is_alert has been triggered by a simple presence of a non-zero > >>>> result > >>>> for that indicator, e.g. is it 2x std_devs, or 4x std_devs as > >>>> different rule levels. We’re adding the ability to make score a > >>>> stellar statement which simplifies this further by allowing score to > >>>> be a function, but thresholds are still useful to determine the text > >>>> content of the alert for example. > >>>> Simon > >>>>> On 25 Sep 2017, at 19:09, Laurens Vets <[email protected]> wrote: > >>>>> Oh, I didn't know I had to set is_alert to True. > >>>>> Doesn't that mean that we have to add all rules twice? First to > >>>>> check whether is_alert needs to be set to True. Next to apply the > >>>>> actual scores? > >>>>> On 2017-09-25 11:00, Simon Elliston Ball wrote: > >>>>>> the _score field is actually an elastic search matching score > >>>>>> field, > >>>>>> and is not relevant to metron. You should see the scores in the > >>>>>> threat:triage:score field. However, your rules will only be run if > >>>>>> the > >>>>>> telemetry has is_alert set true, so you should ensure that the > >>>>>> enrichment phase sets is_alert: true somewhere for alerts you want > >>>>>> to > >>>>>> go to triage? > >>>>>> Simon > >>>>>>> On 25 Sep 2017, at 18:46, Laurens Vets <[email protected]> wrote: > >>>>>>> I have the following configuration: > >>>>>>> "threatIntel": { > >>>>>>> "fieldMap": {}, > >>>>>>> "fieldToTypeMap": {}, > >>>>>>> "config": {}, > >>>>>>> "triageConfig": { > >>>>>>> "riskLevelRules": [ > >>>>>>> { > >>>>>>> "name": "Rule1", > >>>>>>> "comment": "Checks whatever 1.", > >>>>>>> "rule": "test == \"false\"", > >>>>>>> "score": 20, > >>>>>>> "reason": null > >>>>>>> }, > >>>>>>> { > >>>>>>> "name": "Rule1", > >>>>>>> "comment": "Checks whatever 2.", > >>>>>>> "rule": "test2 == \"False\"", > >>>>>>> "score": 20, > >>>>>>> "reason": null > >>>>>>> }, > >>>>>>> { > >>>>>>> "name": "Rule3", > >>>>>>> "comment": "Checks whatever 2.", > >>>>>>> "rule": "test3 == \"No\"", > >>>>>>> "score": 20, > >>>>>>> "reason": null > >>>>>>> } > >>>>>>> ], > >>>>>>> "aggregator": "SUM", > >>>>>>> "aggregationConfig": {} > >>>>>>> } > >>>>>>> }, > >>>>>>> I have no additional configuration in enrichment besides filling > >>>>>>> a specific with true or false based on a Stellar expression. > >>>>>>> I expected that when events would match my above rules, the > >>>>>>> _score field would be filled in. That does not seem to be the > >>>>>>> case. > >>>>>>> Does anyone know what I might be missing? > -- Jon
