After setting is_alert to true, this field is now shown in my event in Kibana. I would expect there also to be a field "threat:triage:level" in those same events (if my rules work?)
On 2017-09-25 16:46, zeo...@gmail.com wrote: > I was quickly reading through this on my mobile device so sorry if I'm off > base here, but it may be because threat.triage.level is changed to > threat:triage:level just before indexing due to the inability to use a period > in keys on older versions of ES. Not sure exactly what you mean by you don't > get a threat.triage.level field. > > Jon > > On Mon, Sep 25, 2017, 19:34 Laurens Vets <laur...@daemon.be> wrote: > >> Next problem: >> >> I'm setting the "is_alert" field to true. It shows up in Kibana, but I >> don't get a threat.triage.level field which means that either my >> riskLevelRules rules don't trigger or something else goes wrong. >> >> How and where can I look for additional information on why my rules >> might not be working? (Metron UI accepts my JSON without issues) >> >> On 2017-09-25 13:39, Laurens Vets wrote: >>> Thanks! >>> >>> On 2017-09-25 13:16, Simon Elliston Ball wrote: >>>> The second statement overwrites the first, but also uses the previous >>>> value. >>>> >>>> Technically that is an or. Note this construct is designed to allow >>>> multiple different trigger conditions to make is_alert true, hence the >>>> second one being is_alert := is_alert || something_else. >>>> >>>> && is bitwise and >>>> || is bitwise or >>>> >>>> Simon >>>> >>>>> On 25 Sep 2017, at 21:12, Laurens Vets <laur...@daemon.be> wrote: >>>>> >>>>> Thanks! Followup question, the below is_alert 'rules' in the snippet >>>>> from >>>>> http://apache.website-solution.net/metron/0.4.1/site-book/use-cases/geographic_login_outliers/index.html, >>>>> are those an AND or OR? >>>>> >>>>> "threatIntel": { >>>>> "fieldMap": { >>>>> "stellar" : { >>>>> "config" : [ >>>>> "geo_distance_distr:= STATS_MERGE( PROFILE_GET( >>>>> 'geo_distribution_from_centroid', 'global', PROFILE_FIXED( 2, >>>>> 'MINUTES')))", >>>>> "dist_median := STATS_PERCENTILE(geo_distance_distr, 50.0)", >>>>> "dist_sd := STATS_SD(geo_distance_distr)", >>>>> "geo_outlier := ABS(dist_median - geo_distance) >= >>>>> 5*dist_sd", >>>>> "is_alert := exists(is_alert) && is_alert", >>>>> "is_alert := is_alert || (geo_outlier != null && geo_outlier >>>>> == true)", >>>>> "geo_distance_distr := null" >>>>> ] >>>>> } >>>>> }, >>>>> >>>>> For instance, can the 2nd is_alert line overwrite the value assigned >>>>> in the first is_alert rule? >>>>> >>>>> On 2017-09-25 11:13, Simon Elliston Ball wrote: >>>>>> Usually you would have the is_alert set based on more complex rules, >>>>>> and then potentially have different rules to determine the >>>>>> importance >>>>>> of the alert, so they do tend to serve different purposes. >>>>>> For example a triage rule might be set on levels of an indicator >>>>>> after >>>>>> is_alert has been triggered by a simple presence of a non-zero >>>>>> result >>>>>> for that indicator, e.g. is it 2x std_devs, or 4x std_devs as >>>>>> different rule levels. We're adding the ability to make score a >>>>>> stellar statement which simplifies this further by allowing score to >>>>>> be a function, but thresholds are still useful to determine the text >>>>>> content of the alert for example. >>>>>> Simon >>>>>>> On 25 Sep 2017, at 19:09, Laurens Vets <laur...@daemon.be> wrote: >>>>>>> Oh, I didn't know I had to set is_alert to True. >>>>>>> Doesn't that mean that we have to add all rules twice? First to >>>>>>> check whether is_alert needs to be set to True. Next to apply the >>>>>>> actual scores? >>>>>>> On 2017-09-25 11:00, Simon Elliston Ball wrote: >>>>>>>> the _score field is actually an elastic search matching score >>>>>>>> field, >>>>>>>> and is not relevant to metron. You should see the scores in the >>>>>>>> threat:triage:score field. However, your rules will only be run if >>>>>>>> the >>>>>>>> telemetry has is_alert set true, so you should ensure that the >>>>>>>> enrichment phase sets is_alert: true somewhere for alerts you want >>>>>>>> to >>>>>>>> go to triage? >>>>>>>> Simon >>>>>>>>> On 25 Sep 2017, at 18:46, Laurens Vets <laur...@daemon.be> wrote: >>>>>>>>> I have the following configuration: >>>>>>>>> "threatIntel": { >>>>>>>>> "fieldMap": {}, >>>>>>>>> "fieldToTypeMap": {}, >>>>>>>>> "config": {}, >>>>>>>>> "triageConfig": { >>>>>>>>> "riskLevelRules": [ >>>>>>>>> { >>>>>>>>> "name": "Rule1", >>>>>>>>> "comment": "Checks whatever 1.", >>>>>>>>> "rule": "test == \"false\"", >>>>>>>>> "score": 20, >>>>>>>>> "reason": null >>>>>>>>> }, >>>>>>>>> { >>>>>>>>> "name": "Rule1", >>>>>>>>> "comment": "Checks whatever 2.", >>>>>>>>> "rule": "test2 == \"False\"", >>>>>>>>> "score": 20, >>>>>>>>> "reason": null >>>>>>>>> }, >>>>>>>>> { >>>>>>>>> "name": "Rule3", >>>>>>>>> "comment": "Checks whatever 2.", >>>>>>>>> "rule": "test3 == \"No\"", >>>>>>>>> "score": 20, >>>>>>>>> "reason": null >>>>>>>>> } >>>>>>>>> ], >>>>>>>>> "aggregator": "SUM", >>>>>>>>> "aggregationConfig": {} >>>>>>>>> } >>>>>>>>> }, >>>>>>>>> I have no additional configuration in enrichment besides filling >>>>>>>>> a specific with true or false based on a Stellar expression. >>>>>>>>> I expected that when events would match my above rules, the >>>>>>>>> _score field would be filled in. That does not seem to be the >>>>>>>>> case. >>>>>>>>> Does anyone know what I might be missing? > > -- > > Jon
