[
https://issues.apache.org/jira/browse/METRON-686?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15860422#comment-15860422
]
ASF GitHub Bot commented on METRON-686:
---------------------------------------
Github user nickwallen commented on a diff in the pull request:
https://github.com/apache/incubator-metron/pull/438#discussion_r100442785
--- Diff:
metron-platform/metron-enrichment/src/main/java/org/apache/metron/threatintel/triage/ThreatTriageProcessor.java
---
@@ -52,15 +74,36 @@ public ThreatTriageProcessor( SensorEnrichmentConfig
config
@Nullable
@Override
- public Double apply(@Nullable Map input) {
- List<Number> scores = new ArrayList<>();
+ public ThreatScore apply(@Nullable Map input) {
+
+ ThreatScore threatScore = new ThreatScore();
StellarPredicateProcessor predicateProcessor = new
StellarPredicateProcessor();
+ StellarProcessor processor = new StellarProcessor();
VariableResolver resolver = new MapVariableResolver(input,
sensorConfig.getConfiguration(), threatIntelConfig.getConfig());
+
+ // attempt to apply each rule to the threat
for(RiskLevelRule rule : threatTriageConfig.getRiskLevelRules()) {
if(predicateProcessor.parse(rule.getRule(), resolver,
functionResolver, context)) {
- scores.add(rule.getScore());
+
+ // add the rule's score to the overall threat score
+ String reason = execute(rule.getReason(), processor, resolver,
String.class);
+ RuleScore score = new RuleScore(rule, reason);
+ threatScore.addRuleScore(score);
}
}
- return threatTriageConfig.getAggregator().aggregate(scores,
threatTriageConfig.getAggregationConfig());
+
+ // calculate the aggregate threat score
+ Aggregators aggregators = threatTriageConfig.getAggregator();
+ List<Number> allScores =
threatScore.getRuleScores().stream().map(score ->
score.getRule().getScore()).collect(Collectors.toList());
+ Double aggregateScore = aggregators.aggregate(allScores,
threatTriageConfig.getAggregationConfig());
+
+ // return the overall threat score
+ threatScore.setScore(aggregateScore);
+ return threatScore;
+ }
+
+ private <T> T execute(String expression, StellarProcessor processor,
VariableResolver resolver, Class<T> clazz) {
+ Object result = processor.parse(expression, resolver,
functionResolver, context);
--- End diff --
It is a cool idea @justinleet.
If we could build the abstract syntax tree and then cache that, we could
certainly improve performance. Of course, we'd have to confirm how much time
we spend building the AST versus execution, but I'd be willing to bet you are
right.
I don't know that we have the ability to do this today though. At least, I
have not seen us do something like this elsewhere. But I could be wrong.
Might be a good general strategy to explore for improving performance in a
lot of places considering how much we rely on Stellar executions nowadays.
> Record Rule Set that Fired During Threat Triage
> -----------------------------------------------
>
> Key: METRON-686
> URL: https://issues.apache.org/jira/browse/METRON-686
> Project: Metron
> Issue Type: Improvement
> Reporter: Nick Allen
> Assignee: Nick Allen
>
> h3. Problem
> There is little transparency into the Threat Triage process itself. When
> Threat Triage runs, all I get is a score. I don't know how that score was
> arrived at, which rules were triggered, and the specific values that caused a
> rule to trigger.
> More specifically, there is no way to generate a message that looks like "The
> host 'powned.svr.bank.com' has '230' inbound flows, exceeding the threshold
> of '202'". This makes it difficult for an analyst to action the alert.
> h3. Proposed Solution
> To improve the transparency of the Threat Triage process, I am proposing
> these enhancements.
> (1) Threat Triage should attach to each message all of the rules that fired
> in addition to the total calculated threat triage score.
> (2) Threat Triage should allow a custom message to be generated for each
> rule. The custom message would allow for some form of string interpolation
> so that I can add specific values from each message to the generated alert.
> We could allow this in one or both of the new fields that Casey just added,
> name and comment.
> (3) The specific method of string interpolation will be implemented under a
> separate issue.
> h3. Example
> (1) In this example, we have a telemetry message with a field called 'value'
> that we need to monitor. In Enrichment, I calculate some sort of value
> threshold, over which an alert should be generated.
> (2) In Threat Triage, I use the calculated value threshold to alert on any
> message that has a value exceeding this threshold.
> (3) By leveraging a new field called 'reason', I can embed values from the
> message, like the hostname, value, and value threshold, into the alert
> produced by Threat Triage.
> {code}
> "triageConfig" : {
> "riskLevelRules" : [ {
> "name" : "Abnormal DNS Port",
> "rule" : "source.type == 'bro' and protocol == 'dns' and ip_dst_port
> != 53",
> "score" : 10.0,
> "reason" : "FORMAT('Abnormal DNS Port: expected: 53, found: %s:%d',
> ip_dst_addr, ip_dst_port)"
> } ],
> "aggregator" : "MAX",
> "aggregationConfig" : { }
> }
> {code}
> (4) The Threat Triage process today would add only the total calculated score.
> {code}
> "threat.triage.level": 10.0
> {code}
> With this proposal, Threat Triage would add the following to the message.
> {code}
> "threat.triage.level":{
> "score":10.0,
> "rules":[
> {
> "name":"Abnormal DNS Port",
> "comment":null
> "score":10.0,
> "reason":"Abnormal DNS Port: expected: 53, found: 224.0.0.251:5353",
> }
> ]
> }
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
