Github user cestella commented on a diff in the pull request:
https://github.com/apache/metron/pull/1099#discussion_r202813546
--- Diff: metron-platform/metron-parsers/README.md ---
@@ -82,6 +82,12 @@ topology in kafka. Errors are collected with the
context of the error
(e.g. stacktrace) and original message causing the error and sent to an
`error` queue. Invalid messages as determined by global validation
functions are also treated as errors and sent to an `error` queue.
+
+Multiple sensors can be aggregated into a single Storm topology. When this
is done, there will be
+multiple Kafka spouts, but only a single parser bolt which will handle
delegating to the correct
--- End diff --
Well, parser chaining allows for DAGs of parsers, not just one level.
Also, you might not want to group parsers based on chained units, but rather
based on velocity or some other metric (e.g. I don't want to group a high
velocity sensor with a bunch of low velcoity sensors in the syslog case). In
that case, you would need the intermediate kafka topics.
---
