[jira] [Updated] (METRON-2326) Unable to Call ENRICHMENT_GET from Threat Triage Rule Reason Field
[
https://issues.apache.org/jira/browse/METRON-2326?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Michael Miklavcic updated METRON-2326:
--
Fix Version/s: Next + 1
> Unable to Call ENRICHMENT_GET from Threat Triage Rule Reason Field
> --
>
> Key: METRON-2326
> URL: https://issues.apache.org/jira/browse/METRON-2326
> Project: Metron
> Issue Type: Bug
>Reporter: Nick Allen
>Assignee: Nick Allen
>Priority: Major
> Fix For: Next + 1
>
> Time Spent: 1h 10m
> Remaining Estimate: 0h
>
> A Threat Triage Rule's "reason" field can contain executable Stellar to
> provide an operator context as to why a rule fired during Threat Triage. I
> am unable to call any function that requires a StellarContext during
> initialization, from the 'Reason' field of a Threat Triage Rule. For
> example, I cannot call `ENRICHMENT_GET`.
> h3. Steps to Replicate
> 1. Create a simple file called `user.csv`.
> {code:java}
> [root@node1 ~]# cat user.csv
> jdoe,192.168.138.2
> jane,192.168.66.1
> ciana,192.168.138.158
> danixa,95.163.121.204
> jim,192.168.66.121
> {code}
> 2 . Create a file called `user-extractor.json`.
> {code:java}
> {
> "config": {
> "columns": {
> "user": 0,
> "ip": 1
> },
> "indicator_column": "ip",
> "separator": ",",
> "type": "user"
> },
> "extractor": "CSV"
> }
> {code}
> 3. Import the enrichment data.
> {code:java}
> source /etc/default/metron
> $METRON_HOME/bin/flatfile_loader.sh -i ./user.csv -t enrichment -c t -e
> ./user-extractor.json
> {code}
> 4. Validate that the enrichment loaded successfully.
> {code:java}
> [root@node1 0.7.2]# source /etc/default/metron
> [root@node1 0.7.2]# $METRON_HOME/bin/stellar -z $ZOOKEEPER
>
> [Stellar]>>> ip_dst_addr := "192.168.138.2"
> 192.168.138.2
>
> [Stellar]>>> ENRICHMENT_GET('user', ip_dst_addr, 'enrichment', 't')
> \{ip=192.168.138.2, user=jdoe}
> {code}
> 5. Create a threat triage rule that attempts an ENRICHMENT_GET.
> {code}
> [Stellar]>>> conf := SHELL_EDIT()
> {
> "enrichment": {
> "fieldMap": {
> "stellar": {
> "config": {
> "is_alert": "true"
> }
> }
> },
> "fieldToTypeMap": {},
> "config": {}
> },
> "threatIntel": {
> "fieldMap": {},
> "fieldToTypeMap": {},
> "config": {},
> "triageConfig": {
> "riskLevelRules": [
> {
> "name": "Rule",
> "comment": "This rule does not work when executing the 'reason' field.",
> "rule": "true",
> "reason": "FORMAT('Call to ENRICHMENT_GET=%s', ENRICHMENT_GET('user',
> ip_dst_addr, 'enrichment', 't'))",
> "score": "100"
> }
> ],
> "aggregator": "MAX",
> "aggregationConfig": {}
> }
> },
> "configuration": {}
> }
>
> [Stellar]>>> CONFIG_PUT("ENRICHMENT", conf, "snort")
> {code}
>
> 6. The Storm worker logs for Enrichment show the following error.
> {code:java}
> 2019-11-21 03:54:34.370 o.a.c.f.r.c.TreeCache Curator-TreeCache-4 [ERROR]
> org.apache.metron.jackson.databind.JsonMappingException: Unable to find
> capability GLOBAL_CONFIG; it may not be available in your context.
> at [Source: java.io.ByteArrayInputStream@1f55bdda; line: 24, column: 11]
> (through reference chain:
> org.apache.metron.common.configuration.enrichment.SensorEnrichmentConfig["threatIntel"]->org.apache.metron.common.configuration.enrichment.threatintel.ThreatIntelConfig["triageConfig"]->org.apache.metron.common.configuration.enrichment.threatintel.ThreatTriageConfig["riskLevelRules"])
> at
> org.apache.metron.jackson.databind.JsonMappingException.from(JsonMappingException.java:262)
> ~[stormjar.jar:?]
> at
> org.apache.metron.jackson.databind.deser.SettableBeanProperty._throwAsIOE(SettableBeanProperty.java:537)
> ~[stormjar.jar:?]
> at
> org.apache.metron.jackson.databind.deser.SettableBeanProperty._throwAsIOE(SettableBeanProperty.java:518)
> ~[stormjar.jar:?]
> at
> org.apache.metron.jackson.databind.deser.impl.MethodProperty.deserializeAndSet(MethodProperty.java:99)
> ~[stormjar.jar:?]
> at
> org.apache.metron.jackson.databind.deser.BeanDeserializer.vanillaDeserialize(BeanDeserializer.java:260)
> ~[stormjar.jar:?]
> at
> org.apache.metron.jackson.databind.deser.BeanDeserializer.deserialize(BeanDeserializer.java:125)
> ~[stormjar.jar:?]
> at
> org.apache.metron.jackson.databind.deser.SettableBeanProperty.deserialize(SettableBeanProperty.java:490)
> ~[stormjar.jar:?]
> at
> org.apache.metron.jackson.databind.deser.impl.MethodProperty.deserializeAndSet(MethodProperty.java:95)
> ~[stormjar.jar:?]
> at
> org.apache.metron.jackson.databind.deser.BeanDeserializer.vanillaDeserialize(BeanDeserializer.java:260)
> ~[stormjar.jar:?]
> at
> org.apache.metron.jackson.databind.deser.BeanDeserializer.deserialize(BeanDeserializer.java:125)
> ~[stormjar.jar:?]
> at
>
[jira] [Updated] (METRON-2326) Unable to Call ENRICHMENT_GET from Threat Triage Rule Reason Field
[
https://issues.apache.org/jira/browse/METRON-2326?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Nick Allen updated METRON-2326:
---
Summary: Unable to Call ENRICHMENT_GET from Threat Triage Rule Reason Field
(was: Unable to Call ENRICHMENT_GET from Threat Triage Rule 'Reason' Field)
> Unable to Call ENRICHMENT_GET from Threat Triage Rule Reason Field
> --
>
> Key: METRON-2326
> URL: https://issues.apache.org/jira/browse/METRON-2326
> Project: Metron
> Issue Type: Bug
>Reporter: Nick Allen
>Assignee: Nick Allen
>Priority: Major
>
> A Threat Triage Rule's "reason" field can contain executable Stellar to
> provide an operator context as to why a rule fired during Threat Triage. I
> am unable to call any function that requires a StellarContext during
> initialization, from the 'Reason' field of a Threat Triage Rule. For
> example, I cannot call `ENRICHMENT_GET`.
> h3. Steps to Replicate
> 1. Create a simple file called `user.csv`.
> {code:java}
> [root@node1 ~]# cat user.csv
> jdoe,192.168.138.2
> jane,192.168.66.1
> ciana,192.168.138.158
> danixa,95.163.121.204
> jim,192.168.66.121
> {code}
> 2 . Create a file called `user-extractor.json`.
> {code:java}
> {
> "config": {
> "columns": {
> "user": 0,
> "ip": 1
> },
> "indicator_column": "ip",
> "separator": ",",
> "type": "user"
> },
> "extractor": "CSV"
> }
> {code}
> 3. Import the enrichment data.
> {code:java}
> source /etc/default/metron
> $METRON_HOME/bin/flatfile_loader.sh -i ./user.csv -t enrichment -c t -e
> ./user-extractor.json
> {code}
> 4. Validate that the enrichment loaded successfully.
> {code:java}
> [root@node1 0.7.2]# source /etc/default/metron
> [root@node1 0.7.2]# $METRON_HOME/bin/stellar -z $ZOOKEEPER
>
> [Stellar]>>> ip_dst_addr := "192.168.138.2"
> 192.168.138.2
>
> [Stellar]>>> ENRICHMENT_GET('user', ip_dst_addr, 'enrichment', 't')
> \{ip=192.168.138.2, user=jdoe}
> {code}
> 5. Create a threat triage rule that attempts an ENRICHMENT_GET.
> {code}
> [Stellar]>>> conf := SHELL_EDIT()
> {
> "enrichment": {
> "fieldMap": {
> "stellar": {
> "config": {
> "is_alert": "true"
> }
> }
> },
> "fieldToTypeMap": {},
> "config": {}
> },
> "threatIntel": {
> "fieldMap": {},
> "fieldToTypeMap": {},
> "config": {},
> "triageConfig": {
> "riskLevelRules": [
> {
> "name": "Rule",
> "comment": "This rule does not work when executing the 'reason' field.",
> "rule": "true",
> "reason": "FORMAT('Call to ENRICHMENT_GET=%s', ENRICHMENT_GET('user',
> ip_dst_addr, 'enrichment', 't'))",
> "score": "100"
> }
> ],
> "aggregator": "MAX",
> "aggregationConfig": {}
> }
> },
> "configuration": {}
> }
>
> [Stellar]>>> CONFIG_PUT("ENRICHMENT", conf, "snort")
> {code}
>
> 6. The Storm worker logs for Enrichment show the following error.
> {code:java}
> 2019-11-21 03:54:34.370 o.a.c.f.r.c.TreeCache Curator-TreeCache-4 [ERROR]
> org.apache.metron.jackson.databind.JsonMappingException: Unable to find
> capability GLOBAL_CONFIG; it may not be available in your context.
> at [Source: java.io.ByteArrayInputStream@1f55bdda; line: 24, column: 11]
> (through reference chain:
> org.apache.metron.common.configuration.enrichment.SensorEnrichmentConfig["threatIntel"]->org.apache.metron.common.configuration.enrichment.threatintel.ThreatIntelConfig["triageConfig"]->org.apache.metron.common.configuration.enrichment.threatintel.ThreatTriageConfig["riskLevelRules"])
> at
> org.apache.metron.jackson.databind.JsonMappingException.from(JsonMappingException.java:262)
> ~[stormjar.jar:?]
> at
> org.apache.metron.jackson.databind.deser.SettableBeanProperty._throwAsIOE(SettableBeanProperty.java:537)
> ~[stormjar.jar:?]
> at
> org.apache.metron.jackson.databind.deser.SettableBeanProperty._throwAsIOE(SettableBeanProperty.java:518)
> ~[stormjar.jar:?]
> at
> org.apache.metron.jackson.databind.deser.impl.MethodProperty.deserializeAndSet(MethodProperty.java:99)
> ~[stormjar.jar:?]
> at
> org.apache.metron.jackson.databind.deser.BeanDeserializer.vanillaDeserialize(BeanDeserializer.java:260)
> ~[stormjar.jar:?]
> at
> org.apache.metron.jackson.databind.deser.BeanDeserializer.deserialize(BeanDeserializer.java:125)
> ~[stormjar.jar:?]
> at
> org.apache.metron.jackson.databind.deser.SettableBeanProperty.deserialize(SettableBeanProperty.java:490)
> ~[stormjar.jar:?]
> at
> org.apache.metron.jackson.databind.deser.impl.MethodProperty.deserializeAndSet(MethodProperty.java:95)
> ~[stormjar.jar:?]
> at
> org.apache.metron.jackson.databind.deser.BeanDeserializer.vanillaDeserialize(BeanDeserializer.java:260)
> ~[stormjar.jar:?]
> at
> org.apache.metron.jackson.databind.deser.BeanDeserializer.deserialize(BeanDeserializer.java:125)
> ~[stormjar.jar:?]
> at
>
