[
https://issues.apache.org/jira/browse/METRON-141?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15274884#comment-15274884
]
ASF GitHub Bot commented on METRON-141:
---------------------------------------
GitHub user cestella opened a pull request:
https://github.com/apache/incubator-metron/pull/108
METRON-141: The ability to do threat triage
We have the ability to mark messages as part of the enrichment topology as
threat alerts, but we have no ability to prioritize those alerts.
We should allow for the prioritization of messages that have some threat
intelligence alert via a scoring mechanism.
The general idea is for each message from each sensor:
1. Map threat conditions as defined by a lightweight query language to
threat levels (a number).
2. More than one condition may match a given message, so allow for the
choice of aggregation functions.
3. If the aggregated score is positive, then add a field
(`threat.triage.level`) with the level to the JSON message.
The above configuration is done at the per-sensor level.
As part of this PR, the following were added out of necessity:
* A query language DSL which is used in defining the conditions which map
to threat levels. Docs at
[here](https://github.com/cestella/incubator-metron/tree/METRON-141/metron-platform/metron-common#query-language)
* The addition of a `triageConfig` section to the sensor enrichment config.
Description at
[here](https://github.com/cestella/incubator-metron/tree/METRON-141/metron-platform/metron-common#the-threatintel-configuration)
* Actual documentation of our enrichment configuration. Documentation at
[here](https://github.com/cestella/incubator-metron/tree/METRON-141/metron-platform/metron-common#enrichment-configuration)
* A rewrite of the `zk_load_configs.sh` utility to support both pushing and
pulling configs (the threat triage rules are part of the configs and until we
have a UI, we need to add them). Docs at
[here](https://github.com/cestella/incubator-metron/tree/METRON-141/metron-platform/metron-common#management-utility)
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/cestella/incubator-metron METRON-141
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/incubator-metron/pull/108.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #108
----
commit cfdffd48d6a5ea7c228c4b66f23c16cc55691f79
Author: cstella <[email protected]>
Date: 2016-05-06T22:46:02Z
METRON-141: The ability to do threat triage
----
> The ability to do threat triage
> -------------------------------
>
> Key: METRON-141
> URL: https://issues.apache.org/jira/browse/METRON-141
> Project: Metron
> Issue Type: New Feature
> Reporter: Casey Stella
> Assignee: Casey Stella
>
> We have the ability to mark messages as part of the enrichment topology as
> threat alerts, but we have no ability to prioritize those alerts.
> We should allow for the prioritization of messages that have some threat
> intelligence alert via a scoring mechanism. The one implemented here allows
> the user to map conditions expressed via a light-weight DSL to a score and
> allow a configurable aggregation strategy.
> The DSL to express conditions should allow for the following:
> * Referencing fields in the enriched JSON
> * Simple boolean operations: and, not, or
> * The ability to have parenthesis to make order of operations explicit
> * A fixed set of functions which take strings and return boolean (currently
> IN_SUBNET(ip, cidr1, cidr2, ...), IS_EMPTY(str), STARTS_WITH(str, prefix),
> ENDS_WITH(str, suffix), REGEXP_MATCH(str, pattern) )
> * A fixed set of string to string transformation functions: TO_LOWER,
> TO_UPPER, TRIM
> For each message, if the rule as expressed by the DSL matches on the message,
> then we are given a list of numbers to aggregate into a single score.
> Aggregation functions supported are as follows:
> * MAX
> * MEAN
> * POSITIVE_MEAN - the mean of the positive scores
> If an aggregated score that is positive is yielded, then a field
> 'threat.triage.level' with the score is added to the indexed JSON.
> This configuration will be done on a per-sensor basis and added to the
> SensorEnrichmentConfig.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
