http://git-wip-us.apache.org/repos/asf/metron/blob/cbdaee17/use-cases/parser_chaining/README.md ---------------------------------------------------------------------- diff --git a/use-cases/parser_chaining/README.md b/use-cases/parser_chaining/README.md new file mode 100644 index 0000000..26fd333 --- /dev/null +++ b/use-cases/parser_chaining/README.md @@ -0,0 +1,235 @@ +<!-- +Licensed to the Apache Software Foundation (ASF) under one +or more contributor license agreements. See the NOTICE file +distributed with this work for additional information +regarding copyright ownership. The ASF licenses this file +to you under the Apache License, Version 2.0 (the +"License"); you may not use this file except in compliance +with the License. You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +--> +# Problem Statement + +Aggregating many different types sensors into a single data source (e.g. +syslog) and ingesting that aggregate sensor into Metron is a common pattern. It +is not obvious precisely how to manage these types of aggregate sensors +as they require two-pass parsing. This document will walk through an +example of supporting this kind of multi-pass ingest. + +Multi-pass parser involves the following requirements: +* The enveloping parser (e.g. the aggregation format such as syslog or + plain CSV) may contain metadata which should be ingested along with the data. +* The enveloping sensor contains many different sensor types + +# High Level Solution + +![High Level Approach](message_routing_high_level.svg) + +At a high level, we continue to maintain the architectural invariant of +a 1-1 relationship between logical sensors and storm topologies. +Eventually this relationship may become more complex, but at the moment +the approach is to construct a routing parser which will have two +responsibilities: +* Parse the envelope (e.g. syslog data) and extract any metadata fields + from the envelope to pass along +* Route the unfolded data to the appropriate kafka topic associated with + the enveloped sensor data + +Because the data emitted from the routing parser is just like any data +emitted from any other parser, in that it is a JSON blob like any +data emitted from any parser, we will need to adjust the downstream +parsers to extract the enveloped data from the JSON blob and treat it as +the data to parse. + +# Example + +## Preliminaries + +We assume that the following environment variables are set: +* `METRON_HOME` - the home directory for metron +* `ZOOKEEPER` - The zookeeper quorum (comma separated with port specified: e.g. `node1:2181` for full-dev) +* `BROKERLIST` - The Kafka broker list (comma separated with port specified: e.g. `node1:6667` for full-dev) +* `ES_HOST` - The elasticsearch master (and port) e.g. `node1:9200` for full-dev. + +Before editing configurations, be sure to pull the configs from zookeeper locally via +``` +$METRON_HOME/bin/zk_load_configs.sh --mode PULL -z $ZOOKEEPER -o $METRON_HOME/config/zookeeper/ -f +``` + + +## The Scenario + +Consider the following situation, we have some logs from a Cisco PIX +device that we would like to ingest. The format is syslog, but multiple +scenarios exist in the same log file. Specificaly, let's consider the +sample logs +[here](http://www.monitorware.com/en/logsamples/cisco-pix-61(2).php). + +The log lines in general have the following components: +* A timestamp +* A message type tag +* The message payload that is dependent upon the tag + +Let's consider two types of messages that we'd like to parse: +* Tag `6-302*` which are connection creation and teardown messages e.g. `Built UDP connection for faddr 198.207.223.240/53337 gaddr 10.0.0.187/53 laddr 192.168.0.2/53` +* Tag `5-304*` which are URL access events e.g. `192.168.0.2 Accessed URL 66.102.9.99:/` + +A couple things are apparent from this: +* The formats we care about are easy to represent in grok, but are very + different and logically represent very different sensors. +* The syslog loglines output by this device has many types of events that I do not care + about (yet). + +We will proceed to create 3 separate parsers: +* A `pix_syslog_router` parser which will: + * Parse the timestamp field + * Parse the payload into a field called `data` + * Parse the tag into a field called `pix_type` + * Route the enveloped messages to the appropriate kafka topic + based on the tag +* A `cisco-6-302` and `cisco-5-304` parser which will append to the existing fields from + the `pix_syslog_router` the sensor specific fields based on the tag type. + +## Cisco PIX Grok Patterns +In order to assist in these parsers, we're going to accumulate some grok +expressions which will help us deal with these various parsers. + +* Open a file `~/cisco_patterns` and place the following in there +``` +CISCO_ACTION Built|Teardown|Deny|Denied|denied|requested|permitted|denied by ACL|discarded|est-allowed|Dropping|created|deleted +CISCO_REASON Duplicate TCP SYN|Failed to locate egress interface|Invalid transport field|No matching connection|DNS Response|DNS Query|(?:%{WORD}\s*)* +CISCO_DIRECTION Inbound|inbound|Outbound|outbound +CISCOFW302020_302021 %{CISCO_ACTION:action}(?:%{CISCO_DIRECTION:direction})? %{WORD:protocol} connection %{GREEDYDATA:ignore} faddr %{IP:ip_dst_addr}/%{INT:icmp_seq_num}(?:\(%{DATA:fwuser}\))? gaddr %{IP:ip_src_xlated}/%{INT:icmp_code_xlated} laddr %{IP:ip_src_addr}/%{INT:icmp_code}( \(%{DATA:user}\))? +ACCESSED %{URIHOST:ip_src_addr} Accessed URL %{IP:ip_dst_addr}:%{URIPATHPARAM:uri_path} +CISCO_PIX %{GREEDYDATA:timestamp}: %PIX-%{NOTSPACE:pix_type}: %{GREEDYDATA:data} +``` +* Place this pattern in HDFS at `/tmp/cisco_patterns` via `hadoop fs -put ~/cisco_patterns /tmp` + * NOTE: In production, we'd have more battle hardened patterns as well as place them in a more sensible location. + +## The `pix_syslog_router` Parser + +* Create the `pix_syslog_router` kafka topic via: +``` +/usr/hdp/current/kafka-broker/bin/kafka-topics.sh --zookeeper $ZOOKEEPER --create --topic pix_syslog_router --partitions 1 --replication-factor 1 +``` +* Create the `pix_syslog_router` parser by opening `$METRON_HOME/config/zookeeper/parsers/pix_syslog_router.json` and placing the following: +``` +{ + "parserClassName" : "org.apache.metron.parsers.GrokParser" + ,"sensorTopic" : "pix_syslog_router" + , "parserConfig": { + "grokPath": "/tmp/cisco_patterns", + "batchSize" : 1, + "patternLabel": "CISCO_PIX", + "timestampField": "timestamp", + "timeFields" : [ "timestamp" ], + "dateFormat" : "MMM dd yyyy HH:mm:ss", + "kafka.topicField" : "logical_source_type" + } + ,"fieldTransformations" : [ + { + "transformation" : "REGEX_SELECT" + ,"input" : "pix_type" + ,"output" : "logical_source_type" + ,"config" : { + "cisco-6-302" : "^6-302.*", + "cisco-5-304" : "^5-304.*" + } + } + ] +} +``` +A couple of things to note about this config: +* In the `parserConfig` section, note that we are specifying `kafka.topicField` is `logical_source_field`. This specifies that the parser will send messages to the topic specified in the `logical_source_type` field. If the field does not exist, then the message is not sent. +* The `REGEX_SELECT` field transformation sets the `logical_source_type` field based on the value in the `pix_type` field, which recall is our tag. This will enable us to route the broad category of cisco firewall messages along to the specific parser. + + +## The `cisco-6-302` Parser + +* Create the `cisco-6-302` kafka topic via: +``` +/usr/hdp/current/kafka-broker/bin/kafka-topics.sh --zookeeper $ZOOKEEPER --create --topic cisco-6-302 --partitions 1 --replication-factor 1 +``` +* Create the `cisco-6-302` parser by opening `$METRON_HOME/config/zookeeper/parsers/cisco-6-302.json` and placing the following: +``` +{ + "parserClassName" : "org.apache.metron.parsers.GrokParser" + ,"sensorTopic" : "cisco-6-302" + ,"rawMessageStrategy" : "ENVELOPE" + ,"rawMessageStrategyConfig" : { + "messageField" : "data", + "metadataPrefix" : "" + } + , "parserConfig": { + "grokPath": "/tmp/cisco_patterns", + "batchSize" : 1, + "patternLabel": "CISCOFW302020_302021" + } +} +``` + +Note a couple of things: +* We are specifying the `rawMessageStrategy` to be `ENVELOPE` to indicate that it is not a straight data feed, but rather it's enveloped in a JSON map (i.e. the output of the `pix_syslog_router) +* Because this is enveloped, we must specify the field which contains the actual raw data by setting `messageField` in `rawMessageStrategyConfig` +* You may be wondering why we specify `metadataPrefix` to be empty string. We want some of the fields in the enveloped message to be merged in without prefix. Most specifically, we want the `timestamp` field. By default, the prefix is `metron.metadata`. + +## The `cisco-5-304` Parser + +* Create the `cisco-5-304` kafka topic via: +``` +/usr/hdp/current/kafka-broker/bin/kafka-topics.sh --zookeeper $ZOOKEEPER --create --topic cisco-5-304 --partitions 1 --replication-factor 1 +``` +* Create the `cisco-5-304` parser by opening `$METRON_HOME/config/zookeeper/parsers/cisco-5-304.json` and placing the following: +``` +{ + "parserClassName" : "org.apache.metron.parsers.GrokParser" + ,"sensorTopic" : "cisco-5-304" + ,"rawMessageStrategy" : "ENVELOPE" + ,"rawMessageStrategyConfig" : { + "messageField" : "data", + "metadataPrefix" : "" + } + , "parserConfig": { + "grokPath": "/tmp/cisco_patterns", + "batchSize" : 1, + "patternLabel": "ACCESSED" + } +} +``` + +Mostly the same comments from the previous parser apply here; we are just using a different pattern label. + +# Start the Parsers +Now we should start the parsers +* Push the configs that we've created for the 3 parsers: +``` +$METRON_HOME/bin/zk_load_configs.sh --mode PUSH -z $ZOOKEEPER -i $METRON_HOME/config/zookeeper/ +``` +* Start the `cisco-6-302` parser via +``` +$METRON_HOME/bin/start_parser_topology.sh -k $BROKERLIST -z $ZOOKEEPER -s cisco-6-302 +``` +* Start the `cisco-5-304` parser via +``` +$METRON_HOME/bin/start_parser_topology.sh -k $BROKERLIST -z $ZOOKEEPER -s cisco-5-304 +``` +* Start the `pix_syslog_router` parser via +``` +$METRON_HOME/bin/start_parser_topology.sh -k $BROKERLIST -z $ZOOKEEPER -s pix_syslog_router +``` + +# Send Data +* Create a file called `~/data.log` with the sample syslog loglines [here](http://www.monitorware.com/en/logsamples/cisco-pix-61(2).php). +* Send the data in via kafka console producer +``` +cat ~/data.log | /usr/hdp/current/kafka-broker/bin/kafka-console-producer.sh --broker-list $BROKERLIST --topic pix_syslog_router +``` + +You should see indices created for the `cisco-5-304` and `cisco-6-302` data with appropriate fields created for each type.
http://git-wip-us.apache.org/repos/asf/metron/blob/cbdaee17/use-cases/parser_chaining/message_routing_high_level.svg ---------------------------------------------------------------------- diff --git a/use-cases/parser_chaining/message_routing_high_level.svg b/use-cases/parser_chaining/message_routing_high_level.svg new file mode 100644 index 0000000..1d80766 --- /dev/null +++ b/use-cases/parser_chaining/message_routing_high_level.svg @@ -0,0 +1,14 @@ +<!-- + Licensed to the Apache Software + Foundation (ASF) under one or more contributor license agreements. See the + NOTICE file distributed with this work for additional information regarding + copyright ownership. The ASF licenses this file to You under the Apache License, + Version 2.0 (the "License"); you may not use this file except in compliance + with the License. You may obtain a copy of the License at http://www.apache.org/licenses/LICENSE-2.0 + Unless required by applicable law or agreed to in writing, software distributed + under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES + OR CONDITIONS OF ANY KIND, either express or implied. See the License for + the specific language governing permissions and limitations under the License. + --> +<!DOCTYPE svg PUBLIC "-//W3C//DTD SVG 1.1//EN" "http://www.w3.org/Graphics/SVG/1.1/DTD/svg11.dtd"> +<svg xmlns="http://www.w3.org/2000/svg" xmlns:xlink="http://www.w3.org/1999/xlink" width="749px" height="474px" version="1.1" content="<mxfile userAgent="Mozilla/5.0 (Macintosh; Intel Mac OS X 10_11_5) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/67.0.3396.62 Safari/537.36" version="8.7.6" editor="www.draw.io" type="device"><diagram id="35f3a7a3-d1c2-8c18-e1fc-d490fc6a3b68" name="Page-1">7ZrNcpswEICfxsdmABmDj7GTtjNNp5n60PYogwBNZOQRcmz36bsY8Sfs2HUJdhNyyEgrWKTdj12tzABNF5tPAi+jr9wnbGAZ/maA7gaWNR4i+J8Ktplg6NqZIBTUz0RmKZjR30QJDSVdUZ8ktQsl50zSZV3o8TgmnqzJsBB8Xb8s4Kz+1CUOSUMw8zBrSn9QX0aZ1LVGpfwzoWGUP9kcjbOROfaeQsFXsXrewELB7i8bXuBcl1poEmGfrysidD9AU8G5zFqLzZSw1LS52bL7Ph4YLeYtSCxPuUFN+xmzlVr6d76SNA5B+IhFQoSap9zmtllHVJLZEntpfw3uH6BJJBcMeiY0cbLMPBLQDYGnTHbWSFu7YYbnhE0KK0054wKGYh6DukkiBX8iuRCMZ5Ox56NiJHcG2GkSUMa02wMeSwWTOUrnwmgYQ8cDa8BK1NMfeUIl5bWBZyIkBf8/aBfMuZR8UbngVqmUPF23sh6Mkc1BD5iFX+F1IXxBpNjCJfkNjkJBvSqmrfrrCnhKFFWYy 2VYoR4Wmkt3Q0N5/AAuRsP9t2EoSIglAfGMxAmY1zK+4OAJN0AAdpdp09syCh5OzXiEjXmGwsO8EBQgfFtJ0EKUvGfmRWbs48iMXgsZs4HMQTACRja3aTSGVZPYV807j+EkoV4djapnyqDZdA9YSGx/Qs/IO7/Szo2ddjdU/lT60nY5kk2R+I2or1kclsFXwtOykcQiJLISMJuOqVje3mP5XCYIw5I+12exzx3qCY+cwvzKWGHUHY8czaPZ7NVd1bCvK9IIQpamKFtyQ9GOjmLZpwFjNYCZQYagARBQCTF9sukkcKC637vMNaOeg6vlALkdguC8nwxy0WyBDFRz8nh8Y5+XLpBzTFOL+cLt8bgIHpbdGh97VLUISLNm7RaQtwzB0PwHCLScMn61PaXVLEL27SX6svW6dh16mOiwbM1jxHvIKsWJaaVszaqwS0WZ4fDAgcXfxhh7pBH0innGQicFmb5guUTBUhygd1CwWMM+2/yHyDQ2pUaH6cbukXkDyBRb2i6QedN178V2HlreGBpn1ze25RxT1eLm49JF7rugwTp7H6qx0FDUHgn5lCsk3MeCetGC7GbU55DryiGOpTFmdpdD0JX+OOu0VOVWK1pk7HdDN5FkZGp55fxzM8dwjqlqMZpc+hzkIAUFOx+MG8N0tLTjurngkQgKC07f8h1mp9NTOyNB104UQs6ZPJlHklyLNDWPSK4i3pgvb1Na5Cj/uuGKONK/J3PR2ZHJ1ZBsCyTolp84ZpeXn5Gi+z8=</diagram></mxfile>"><defs/><g transform="translate(0.5,0.5)"><rect x="128" y="101" width="80" height="80" rx="12" ry="12" fill="none" stroke="#5e9cd3" stroke-width="2" pointer-events="none"/><g transform="translate(12 9.5,188.5)"><switch><foreignObject style="overflow:visible;" pointer-events="all" width="75" height="36" requiredFeatures="http://www.w3.org/TR/SVG11/feature#Extensibility"><div xmlns="http://www.w3.org/1999/xhtml" style="display: inline-block; font-size: 16px; font-family: Helvetica; color: rgb(0, 0, 0); line-height: 1.2; vertical-align: top; width: 75px; white-space: normal; word-wrap: normal; text-align: center;"><div xmlns="http://www.w3.org/1999/xhtml" style="display:inline-block;text-align:inherit;text-decoration:inherit;">Routing Parser</div></div></foreignObject><text x="38" y="26" fill="#000000" text-anchor="middle" font-size="16px" font-family="Helvetica">Routing Parser</text></switch></g><path d="M 8 117 C 8 133 68 133 68 117" fill="none" stroke="#5e9cd3" stroke-width="2" stroke-miterlimit="10" pointer-events="none"/><path d="M 8 117 C 8 95.67 68 95.67 68 117 L 68 165 C 68 186.33 8 186.33 8 165 Z" fill="none" stroke="#5e9cd3" stroke-width="2" stroke-miterlimit="10" pointe r-events="none"/><g transform="translate(-0.5,188.5)"><switch><foreignObject style="overflow:visible;" pointer-events="all" width="75" height="55" requiredFeatures="http://www.w3.org/TR/SVG11/feature#Extensibility"><div xmlns="http://www.w3.org/1999/xhtml" style="display: inline-block; font-size: 16px; font-family: Helvetica; color: rgb(0, 0, 0); line-height: 1.2; vertical-align: top; width: 75px; white-space: normal; word-wrap: normal; text-align: center;"><div xmlns="http://www.w3.org/1999/xhtml" style="display:inline-block;text-align:inherit;text-decoration:inherit;">Aggregate Sensor Kafka</div></div></foreignObject><text x="38" y="36" fill="#000000" text-anchor="middle" font-size="16px" font-family="Helvetica">Aggregate Sensor Kafka</text></switch></g><path d="M 68.5 146 L 68.5 136 L 108.5 136 L 108.5 125.5 L 127.5 141 L 108.5 156.5 L 108.5 146 Z" fill="#ffffff" stroke="#000000" stroke-linejoin="round" stroke-miterlimit="10" pointer-events="none"/><rect x="488" y="1" width="80" height="80" rx="12" ry="12" fill="none" stroke="#5e9cd3" stroke-width="2" pointer-events="none"/><g transform="translate(489.5,88.5)"><switch><foreignObject style="overflow:visible;" pointer-events="all" width="75" height="55" requiredFeatures="http://www.w3.org/TR/SVG11/feature#Extensibility"><div xmlns="http://www.w3.org/1999/xhtml" style="display: inline-block; font-size: 16px; font-family: Helvetica; color: rgb(0, 0, 0); line-height: 1.2; vertical-align: top; width: 75px; white-space: normal; word-wrap: normal; text-align: center;"><div xmlns="http://www.w3.org/1999/xhtml" style="display:inline-block;text-align:inherit;text-decoration:inherit;">Specific Sensor Parser</div></div></foreignObject><text x="38" y="36" fill="#000000" text-anchor="middle" font-size="16px" font-family="Helvetica">Specific Sensor Parser</text></switch></g><rect x="488" y="331" width="80" height="80" rx="12" ry="12" fill="none" stroke="#5e9cd3" stroke-width="2" pointer-events="none"/><g transform="transla te(489.5,418.5)"><switch><foreignObject style="overflow:visible;" pointer-events="all" width="75" height="55" requiredFeatures="http://www.w3.org/TR/SVG11/feature#Extensibility"><div xmlns="http://www.w3.org/1999/xhtml" style="display: inline-block; font-size: 16px; font-family: Helvetica; color: rgb(0, 0, 0); line-height: 1.2; vertical-align: top; width: 75px; white-space: normal; word-wrap: normal; text-align: center;"><div xmlns="http://www.w3.org/1999/xhtml" style="display:inline-block;text-align:inherit;text-decoration:inherit;">Specific Sensor Parser</div></div></foreignObject><text x="38" y="36" fill="#000000" text-anchor="middle" font-size="16px" font-family="Helvetica">Specific Sensor Parser</text></switch></g><path d="M 211.37 144.73 L 205.44 136.68 L 312.33 58.03 L 306.11 49.57 L 330.6 50.8 L 324.48 74.54 L 318.26 66.08 Z" fill="#ffffff" stroke="#000000" stroke-linejoin="round" stroke-miterlimit="10" pointer-events="none"/><path d="M 205.98 145.6 L 210.9 136.89 L 316.48 1 96.55 L 321.65 187.41 L 330.56 210.25 L 306.4 214.4 L 311.56 205.26 Z" fill="#ffffff" stroke="#000000" stroke-linejoin="round" stroke-miterlimit="10" pointer-events="none"/><path d="M 203.97 144 L 212.54 138.86 L 322.25 321.71 L 331.26 316.3 L 327.74 340.57 L 304.68 332.25 L 313.68 326.85 Z" fill="#ffffff" stroke="#000000" stroke-linejoin="round" stroke-miterlimit="10" pointer-events="none"/><path d="M 331 17 C 331 33 391 33 391 17" fill="none" stroke="#5e9cd3" stroke-width="2" stroke-miterlimit="10" pointer-events="none"/><path d="M 331 17 C 331 -4.33 391 -4.33 391 17 L 391 65 C 391 86.33 331 86.33 331 65 Z" fill="none" stroke="#5e9cd3" stroke-width="2" stroke-miterlimit="10" pointer-events="none"/><g transform="translate(331.5,88.5)"><switch><foreignObject style="overflow:visible;" pointer-events="all" width="57" height="55" requiredFeatures="http://www.w3.org/TR/SVG11/feature#Extensibility"><div xmlns="http://www.w3.org/1999/xhtml" style="display: inline-block; font-size: 16px; f ont-family: Helvetica; color: rgb(0, 0, 0); line-height: 1.2; vertical-align: top; width: 57px; white-space: normal; word-wrap: normal; text-align: center;"><div xmlns="http://www.w3.org/1999/xhtml" style="display:inline-block;text-align:inherit;text-decoration:inherit;">Specific Sensor Kafka</div></div></foreignObject><text x="29" y="36" fill="#000000" text-anchor="middle" font-size="16px" font-family="Helvetica">[Not supported by viewer]</text></switch></g><path d="M 391.5 46 L 391.5 36 L 468.5 36 L 468.5 25.5 L 487.5 41 L 468.5 56.5 L 468.5 46 Z" fill="#ffffff" stroke="#000000" stroke-linejoin="round" stroke-miterlimit="10" pointer-events="none"/><rect x="488" y="171" width="80" height="80" rx="12" ry="12" fill="none" stroke="#5e9cd3" stroke-width="2" pointer-events="none"/><g transform="translate(489.5,258.5)"><switch><foreignObject style="overflow:visible;" pointer-events="all" width="75" height="55" requiredFeatures="http://www.w3.org/TR/SVG11/feature#Extensibility"><div xmlns ="http://www.w3.org/1999/xhtml" style="display: inline-block; font-size: 16px; font-family: Helvetica; color: rgb(0, 0, 0); line-height: 1.2; vertical-align: top; width: 75px; white-space: normal; word-wrap: normal; text-align: center;"><div xmlns="http://www.w3.org/1999/xhtml" style="display:inline-block;text-align:inherit;text-decoration:inherit;">Specific Sensor Parser</div></div></foreignObject><text x="38" y="36" fill="#000000" text-anchor="middle" font-size="16px" font-family="Helvetica">Specific Sensor Parser</text></switch></g><path d="M 331 167 C 331 183 391 183 391 167" fill="none" stroke="#5e9cd3" stroke-width="2" stroke-miterlimit="10" pointer-events="none"/><path d="M 331 167 C 331 145.67 391 145.67 391 167 L 391 215 C 391 236.33 331 236.33 331 215 Z" fill="none" stroke="#5e9cd3" stroke-width="2" stroke-miterlimit="10" pointer-events="none"/><g transform="translate(331.5,238.5)"><switch><foreignObject style="overflow:visible;" pointer-events="all" width="57" height="55" requiredFeatures="http://www.w3.org/TR/SVG11/feature#Extensibility"><div xmlns="http://www.w3.org/1999/xhtml" style="display: inline-block; font-size: 16px; font-family: Helvetica; color: rgb(0, 0, 0); line-height: 1.2; vertical-align: top; width: 57px; white-space: normal; word-wrap: normal; text-align: center;"><div xmlns="http://www.w3.org/1999/xhtml" style="display:inline-block;text-align:inherit;text-decoration:inherit;">Specific Sensor Kafka</div></div></foreignObject><text x="29" y="36" fill="#000000" text-anchor="middle" font-size="16px" font-family="Helvetica">[Not supported by viewer]</text></switch></g><path d="M 331 337 C 331 353 391 353 391 337" fill="none" stroke="#5e9cd3" stroke-width="2" stroke-miterlimit="10" pointer-events="none"/><path d="M 331 337 C 331 315.67 391 315.67 391 337 L 391 385 C 391 406.33 331 406.33 331 385 Z" fill="none" stroke="#5e9cd3" stroke-width="2" stroke-miterlimit="10" pointer-events="none"/><g transform="translate(331.5,408.5)"><switch><fo reignObject style="overflow:visible;" pointer-events="all" width="57" height="55" requiredFeatures="http://www.w3.org/TR/SVG11/feature#Extensibility"><div xmlns="http://www.w3.org/1999/xhtml" style="display: inline-block; font-size: 16px; font-family: Helvetica; color: rgb(0, 0, 0); line-height: 1.2; vertical-align: top; width: 57px; white-space: normal; word-wrap: normal; text-align: center;"><div xmlns="http://www.w3.org/1999/xhtml" style="display:inline-block;text-align:inherit;text-decoration:inherit;">Specific Sensor Kafka</div></div></foreignObject><text x="29" y="36" fill="#000000" text-anchor="middle" font-size="16px" font-family="Helvetica">[Not supported by viewer]</text></switch></g><path d="M 388.5 365.5 L 388.5 355.5 L 465.5 355.5 L 465.5 345 L 484.5 360.5 L 465.5 376 L 465.5 365.5 Z" fill="#ffffff" stroke="#000000" stroke-linejoin="round" stroke-miterlimit="10" pointer-events="none"/><path d="M 388.5 206 L 388.5 196 L 465.5 196 L 465.5 185.5 L 484.5 201 L 465.5 216.5 L 465.5 206 Z" fill="#ffffff" stroke="#000000" stroke-linejoin="round" stroke-miterlimit="10" pointer-events="none"/><path d="M 678 177 C 678 193 738 193 738 177" fill="none" stroke="#5e9cd3" stroke-width="2" stroke-miterlimit="10" pointer-events="none"/><path d="M 678 177 C 678 155.67 738 155.67 738 177 L 738 225 C 738 246.33 678 246.33 678 225 Z" fill="none" stroke="#5e9cd3" stroke-width="2" stroke-miterlimit="10" pointer-events="none"/><g transform="translate(666.5,248.5)"><switch><foreignObject style="overflow:visible;" pointer-events="all" width="82" height="36" requiredFeatures="http://www.w3.org/TR/SVG11/feature#Extensibility"><div xmlns="http://www.w3.org/1999/xhtml" style="display: inline-block; font-size: 16px; font-family: Helvetica; color: rgb(0, 0, 0); line-height: 1.2; vertical-align: top; width: 82px; white-space: normal; word-wrap: normal; text-align: center;"><div xmlns="http://www.w3.org/1999/xhtml" style="display:inline-block;text-align:inherit;text-decoration:inhe rit;">Enrichment Kafka</div></div></foreignObject><text x="41" y="26" fill="#000000" text-anchor="middle" font-size="16px" font-family="Helvetica">Enrichment Kafka</text></switch></g><path d="M 572.36 373.01 L 564.23 367.18 L 662.58 229.94 L 654.04 223.82 L 677.71 217.41 L 679.24 241.88 L 670.71 235.76 Z" fill="#ffffff" stroke="#000000" stroke-linejoin="round" stroke-miterlimit="10" pointer-events="none"/><path d="M 568.64 215.98 L 568.36 205.99 L 657.37 203.54 L 657.08 193.04 L 676.5 208.01 L 657.93 224.03 L 657.64 213.53 Z" fill="#ffffff" stroke="#000000" stroke-linejoin="round" stroke-miterlimit="10" pointer-events="none"/><path d="M 564.48 44.59 L 572.16 38.18 L 669.36 154.82 L 677.42 148.1 L 677.68 172.62 L 653.61 167.94 L 661.68 161.22 Z" fill="#ffffff" stroke="#000000" stroke-linejoin="round" stroke-miterlimit="10" pointer-events="none"/></g></svg> http://git-wip-us.apache.org/repos/asf/metron/blob/cbdaee17/use-cases/parser_chaining/message_routing_high_level.xml ---------------------------------------------------------------------- diff --git a/use-cases/parser_chaining/message_routing_high_level.xml b/use-cases/parser_chaining/message_routing_high_level.xml new file mode 100644 index 0000000..127fb3c --- /dev/null +++ b/use-cases/parser_chaining/message_routing_high_level.xml @@ -0,0 +1,14 @@ +<!-- + Licensed to the Apache Software + Foundation (ASF) under one or more contributor license agreements. See the + NOTICE file distributed with this work for additional information regarding + copyright ownership. The ASF licenses this file to You under the Apache License, + Version 2.0 (the "License"); you may not use this file except in compliance + with the License. You may obtain a copy of the License at http://www.apache.org/licenses/LICENSE-2.0 + Unless required by applicable law or agreed to in writing, software distributed + under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES + OR CONDITIONS OF ANY KIND, either express or implied. See the License for + the specific language governing permissions and limitations under the License. + --> +<!-- This is a draw.io diagram. You can load it from http://www.draw.io --> +<mxfile userAgent="Mozilla/5.0 (Macintosh; Intel Mac OS X 10_11_5) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/67.0.3396.62 Safari/537.36" version="8.7.6" editor="www.draw.io" type="device"><diagram id="35f3a7a3-d1c2-8c18-e1fc-d490fc6a3b68" name="Page-1">7ZrNctowEICfhmMztoQxHANJ25mm00w5tD0KW9iaCIuRRYA+fddY/pMhUOoYmjiHjLSy19Lu512tcA9PFptPkizDr8KnvIcsf9PDdz2ERn0M/xPBNhU4aJQKAsn8VGQXgin7TbXQ0tIV82lcuVAJwRVbVoWeiCLqqYqMSCnW1cvmglefuiQBrQmmHuF16Q/mqzCVDtGgkH+mLAizJ9sDvb4Z8Z4CKVaRfl4P4fnuLx1ekEyXXmgcEl+sSyJ838MTKYRKW4vNhPLEtJnZ0vs+HhjN5y1ppE65QU/7mfCVXvp3sVIsCkD4SGRMpZ6n2ma2WYdM0emSeEl/De7v4XGoFhx6NjRJvEw9MmcbCk8Z76yRtHbDnMwoH+dWmgguJAxFIgJ141hJ8UQzIRjPoSPPx/lI5gyw03jOODdun4tIaZjsQTIXzoIIOh5YA1ain/4oYqaYqAw8U6kY+P/BuGAmlBKL0gW3WqUSybq19WCMbg56wM79Cq8LFQuq5BYuyW5wNQr6VbEd3V+XwNOisMRcJiMa9SDXXLgbGtrjB3Cxau6/DQJJA6IoiKc0isG8yPpC5k+kBgKwu0ya3pYz8HBixiNszFIUHma5IAfh20qBFqrlHTMvMuMcR2bwWsjYNWQOgjHndHObRGNYNY183bzzOIlj5lXRKHumCJp194CF5PYn9Kys8yvp3DhJd8PUT60vaRcj6RSpX4v6hsVhGWIlPSMbKSIDqkoBs+6Y kuWdPZbPZJJyothzdRb73KGf8CgYzK+IFVbV8dg1PJrOXt9VDvumIoMgjAxF6ZJrinZ05Ms+DRhUA2YKGYLNgYBSiOmSTSuBA1f93mauGXQcXC0HeNgiCO77ySAXzRbYwhUnj0awkrPSBXaPaWowXww7PC6CB3Ia42OPqgYBqdes7QLyliHo2/8AgZFTRq+2p0T1ImTfXqIrW69r12GGiRbL1ixGvIeskp+YlsrWtAq7VJTp9w8cWPxtjHEGBkGvmGcQPinIdAXLJQqW/AC9hYIF9bts8x8iU9uUWi2mG6dD5g0gk29p20DmTde9F9t5GHmjb51d3zjIPaaqwc3HpYvcd0EDOnsfarBQU9QcCdmUSyTcR5J54YLuZtTlkOvKIS4yGLPbyyH4Sn+cdRuqcssVLbb2u6GdSDKwjbxy/rmZa7nHVDUYTS59DnKQgpydD9aNZbtG2hkOM8EjlQwWnLzlO8xOp6dyRoKvnSiM3TN5so8kuQZpqh+RXEW8sV/epjTIUfZ1wxVxZH5PNsRnR6ahgWRTIEG3+MQxvbz4jBTf/wE=</diagram></mxfile>