Why would I need to update my ES template? I should see the field (possibly with the wrong type) anyways in the event after I refreshed the fields in Kibana right?
On 2017-09-26 09:16, Simon Elliston Ball wrote: > There should be, though you may need to update your templates in ES if you've > got any custom templates there, and make sure you refresh the fields in > kibana's index config. > > Simon > > On 26 Sep 2017, at 17:13, Laurens Vets <laur...@daemon.be> wrote: > > After setting is_alert to true, this field is now shown in my event in > Kibana. I would expect there also to be a field "threat:triage:level" in > those same events (if my rules work?) > > On 2017-09-25 16:46, zeo...@gmail.com wrote: > > I was quickly reading through this on my mobile device so sorry if I'm off > base here, but it may be because threat.triage.level is changed to > threat:triage:level just before indexing due to the inability to use a period > in keys on older versions of ES. Not sure exactly what you mean by you don't > get a threat.triage.level field. > > Jon > > On Mon, Sep 25, 2017, 19:34 Laurens Vets <laur...@daemon.be> wrote: Next > problem: > > I'm setting the "is_alert" field to true. It shows up in Kibana, but I > don't get a threat.triage.level field which means that either my > riskLevelRules rules don't trigger or something else goes wrong. > > How and where can I look for additional information on why my rules > might not be working? (Metron UI accepts my JSON without issues) > > On 2017-09-25 13:39, Laurens Vets wrote: >> Thanks! >> >> On 2017-09-25 13:16, Simon Elliston Ball wrote: >>> The second statement overwrites the first, but also uses the previous >>> value. >>> >>> Technically that is an or. Note this construct is designed to allow >>> multiple different trigger conditions to make is_alert true, hence the >>> second one being is_alert := is_alert || something_else. >>> >>> && is bitwise and >>> || is bitwise or >>> >>> Simon >>> >>>> On 25 Sep 2017, at 21:12, Laurens Vets <laur...@daemon.be> wrote: >>>> >>>> Thanks! Followup question, the below is_alert 'rules' in the snippet >>>> from >>>> http://apache.website-solution.net/metron/0.4.1/site-book/use-cases/geographic_login_outliers/index.html, >>>> are those an AND or OR? >>>> >>>> "threatIntel": { >>>> "fieldMap": { >>>> "stellar" : { >>>> "config" : [ >>>> "geo_distance_distr:= STATS_MERGE( PROFILE_GET( >>>> 'geo_distribution_from_centroid', 'global', PROFILE_FIXED( 2, >>>> 'MINUTES')))", >>>> "dist_median := STATS_PERCENTILE(geo_distance_distr, 50.0)", >>>> "dist_sd := STATS_SD(geo_distance_distr)", >>>> "geo_outlier := ABS(dist_median - geo_distance) >= >>>> 5*dist_sd", >>>> "is_alert := exists(is_alert) && is_alert", >>>> "is_alert := is_alert || (geo_outlier != null && geo_outlier >>>> == true)", >>>> "geo_distance_distr := null" >>>> ] >>>> } >>>> }, >>>> >>>> For instance, can the 2nd is_alert line overwrite the value assigned >>>> in the first is_alert rule? >>>> >>>> On 2017-09-25 11:13, Simon Elliston Ball wrote: >>>>> Usually you would have the is_alert set based on more complex rules, >>>>> and then potentially have different rules to determine the >>>>> importance >>>>> of the alert, so they do tend to serve different purposes. >>>>> For example a triage rule might be set on levels of an indicator >>>>> after >>>>> is_alert has been triggered by a simple presence of a non-zero >>>>> result >>>>> for that indicator, e.g. is it 2x std_devs, or 4x std_devs as >>>>> different rule levels. We're adding the ability to make score a >>>>> stellar statement which simplifies this further by allowing score to >>>>> be a function, but thresholds are still useful to determine the text >>>>> content of the alert for example. >>>>> Simon >>>>>> On 25 Sep 2017, at 19:09, Laurens Vets <laur...@daemon.be> wrote: >>>>>> Oh, I didn't know I had to set is_alert to True. >>>>>> Doesn't that mean that we have to add all rules twice? First to >>>>>> check whether is_alert needs to be set to True. Next to apply the >>>>>> actual scores? >>>>>> On 2017-09-25 11:00, Simon Elliston Ball wrote: >>>>>>> the _score field is actually an elastic search matching score >>>>>>> field, >>>>>>> and is not relevant to metron. You should see the scores in the >>>>>>> threat:triage:score field. However, your rules will only be run if >>>>>>> the >>>>>>> telemetry has is_alert set true, so you should ensure that the >>>>>>> enrichment phase sets is_alert: true somewhere for alerts you want >>>>>>> to >>>>>>> go to triage? >>>>>>> Simon >>>>>>>> On 25 Sep 2017, at 18:46, Laurens Vets <laur...@daemon.be> wrote: >>>>>>>> I have the following configuration: >>>>>>>> "threatIntel": { >>>>>>>> "fieldMap": {}, >>>>>>>> "fieldToTypeMap": {}, >>>>>>>> "config": {}, >>>>>>>> "triageConfig": { >>>>>>>> "riskLevelRules": [ >>>>>>>> { >>>>>>>> "name": "Rule1", >>>>>>>> "comment": "Checks whatever 1.", >>>>>>>> "rule": "test == \"false\"", >>>>>>>> "score": 20, >>>>>>>> "reason": null >>>>>>>> }, >>>>>>>> { >>>>>>>> "name": "Rule1", >>>>>>>> "comment": "Checks whatever 2.", >>>>>>>> "rule": "test2 == \"False\"", >>>>>>>> "score": 20, >>>>>>>> "reason": null >>>>>>>> }, >>>>>>>> { >>>>>>>> "name": "Rule3", >>>>>>>> "comment": "Checks whatever 2.", >>>>>>>> "rule": "test3 == \"No\"", >>>>>>>> "score": 20, >>>>>>>> "reason": null >>>>>>>> } >>>>>>>> ], >>>>>>>> "aggregator": "SUM", >>>>>>>> "aggregationConfig": {} >>>>>>>> } >>>>>>>> }, >>>>>>>> I have no additional configuration in enrichment besides filling >>>>>>>> a specific with true or false based on a Stellar expression. >>>>>>>> I expected that when events would match my above rules, the >>>>>>>> _score field would be filled in. That does not seem to be the >>>>>>>> case. >>>>>>>> Does anyone know what I might be missing? > -- > > Jon
