Github user cestella commented on a diff in the pull request:
https://github.com/apache/metron/pull/1099#discussion_r202809295
--- Diff: metron-platform/metron-parsers/README.md ---
@@ -82,6 +82,12 @@ topology in kafka. Errors are collected with the
context of the error
(e.g. stacktrace) and original message causing the error and sent to an
`error` queue. Invalid messages as determined by global validation
functions are also treated as errors and sent to an `error` queue.
+
+Multiple sensors can be aggregated into a single Storm topology. When this
is done, there will be
+multiple Kafka spouts, but only a single parser bolt which will handle
delegating to the correct
--- End diff --
Yeah, in order to do that, we'd need to execute the DAG without the
intermediate kafka step. That'd be a follow-on. This is the first step in
that. We have all the information here (the input -> output mapping of kafka
topics), so it shouldn't be so hard to intervene and cut the intermediate kafka
writing step out.
---
