[
https://issues.apache.org/jira/browse/METRON-1341?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16284150#comment-16284150
]
ASF GitHub Bot commented on METRON-1341:
----------------------------------------
Github user ottobackwards commented on the issue:
https://github.com/apache/metron/pull/861
I ran the following test:
Modified the default snort parser configuration such that it was :
```json
{
"parserClassName":"org.apache.metron.parsers.snort.BasicSnortParser",
"sensorTopic":"snort",
"parserConfig": {},
"fieldTransformations" : [
{
"output" : ["msg" ],
"transformation" : "SELECT"
}
]
}
```
And the default snort enrichment configuration such that it was :
```json
{
"enrichment" : {
},
"threatIntel" : {
}
}
}
```
I got the following:
```
2.168.138.158,49189,62.75.195.236,80,00:00:00:00:00:00,00:00:00:00:00:00,0x3C,***A****,0x9DFB1927,0xF1BD72CC,,0xFAF0,128,0,2360,40,40960,,,,","enrichmentsplitterbolt.splitter.end.ts":"1512763453749","enrichmentsplitterbolt.splitter.begin.ts":"1512763453749","guid":"08a84757-bf05-431b-9d81-5fa95fb99938","timestamp":1512763452000}
at
org.apache.metron.enrichment.bolt.EnrichmentJoinBolt.getStreamIds(EnrichmentJoinBolt.java:53)
~[stormjar.jar:?]
at
org.apache.metron.enrichment.bolt.EnrichmentJoinBolt.getStreamIds(EnrichmentJoinBolt.java:33)
~[stormjar.jar:?]
at
org.apache.metron.enrichment.bolt.JoinBolt.execute(JoinBolt.java:138)
[stormjar.jar:?]
at
org.apache.storm.daemon.executor$fn__6573$tuple_action_fn__6575.invoke(executor.clj:734)
[storm-core-1.0.1.2.5.3.0-37.jar:1.0.1.2.5.3.0-37]
at
org.apache.storm.daemon.executor$mk_task_receiver$fn__6494.invoke(executor.clj:466)
[storm-core-1.0.1.2.5.3.0-37.jar:1.0.1.2.5.3.0-37]
at
org.apache.storm.disruptor$clojure_handler$reify__6007.onEvent(disruptor.clj:40)
[storm-core-1.0.1.2.5.3.0-37.jar:1.0.1.2.5.3.0-37]
at
org.apache.storm.utils.DisruptorQueue.consumeBatchToCursor(DisruptorQueue.java:451)
[storm-core-1.0.1.2.5.3.0-37.jar:1.0.1.2.5.3.0-37]
at
org.apache.storm.utils.DisruptorQueue.consumeBatchWhenAvailable(DisruptorQueue.java:430)
[storm-core-1.0.1.2.5.3.0-37.jar:1.0.1.2.5.3.0-37]
at
org.apache.storm.disruptor$consume_batch_when_available.invoke(disruptor.clj:73)
[storm-core-1.0.1.2.5.3.0-37.jar:1.0.1.2.5.3.0-37]
at
org.apache.storm.daemon.executor$fn__6573$fn__6586$fn__6639.invoke(executor.clj:853)
[storm-core-1.0.1.2.5.3.0-37.jar:1.0.1.2.5.3.0-37]
at org.apache.storm.util$async_loop$fn__554.invoke(util.clj:484)
[storm-core-1.0.1.2.5.3.0-37.jar:1.0.1.2.5.3.0-37]
at clojure.lang.AFn.run(AFn.java:22) [clojure-1.7.0.jar:?]
at java.lang.Thread.run(Thread.java:745) [?:1.8.0_77]
2017-12-08 20:04:17.171 o.a.m.e.b.EnrichmentSplitterBolt [ERROR] Trying to
retrieve a field map with sensor type of null
2017-12-08 20:04:17.171 o.a.m.e.b.EnrichmentSplitterBolt [ERROR] Trying to
retrieve a field map with sensor type of null
2017-12-08 20:04:17.171 o.a.m.e.b.EnrichmentSplitterBolt [ERROR] Trying to
retrieve a field map with sensor type of null
2017-12-08 20:04:17.171 o.a.m.e.b.EnrichmentSplitterBolt [ERROR] Trying to
retrieve a field map with sensor type of null
2017-12-08 20:04:17.171 o.a.m.e.b.EnrichmentSplitterBolt [ERROR] Trying to
retrieve a field map with sensor type of null
2017-12-08 20:04:17.171 o.a.m.e.b.EnrichmentSplitterBolt [ERROR] Trying to
retrieve a field map with sensor type of null
2017-12-08 20:04:17.171 o.a.m.e.b.EnrichmentSplitterBolt [ERROR] Trying to
retrieve a field map with sensor type of null
2017-12-08 20:04:17.171 o.a.m.e.b.EnrichmentSplitterBolt [ERROR] Trying to
retrieve a field map with sensor type of null
2017-12-08 20:04:17.171 o.a.m.e.b.EnrichmentSplitterBolt [ERROR] Trying to
retrieve a field map with sensor type of null
```
So it looks like there are more fields to protect.
> Projection FieldTransformation
> ------------------------------
>
> Key: METRON-1341
> URL: https://issues.apache.org/jira/browse/METRON-1341
> Project: Metron
> Issue Type: Improvement
> Affects Versions: 0.4.2
> Reporter: Simon Elliston Ball
> Assignee: Simon Elliston Ball
> Labels: newbie
>
> It would be useful to have a projection transformation for Parsers which
> could use configured to limit the fields output from the parser.
> The configuration would look like this:
> {code:java}
> {
> "fieldTransformations": [
> {
> "transformation": "STELLAR",
> "config": [
> "ipSrc = TRIM(raw_ip_src)"
> "ip_src_addr := ipSrc"
> ]
> },
> {
> "transformation": "SELECT",
> "output" : [ "ip_src_addr", "ip_dst_addr", "message"]
> }
> ]
> }
> {code}
> This would lead to only the fields in the output definition of the SELECT
> transformation being put into the outbound message.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
