Github user ottobackwards commented on a diff in the pull request:
https://github.com/apache/metron/pull/1099#discussion_r202797418
--- Diff: metron-platform/metron-parsers/README.md ---
@@ -82,6 +82,12 @@ topology in kafka. Errors are collected with the
context of the error
(e.g. stacktrace) and original message causing the error and sent to an
`error` queue. Invalid messages as determined by global validation
functions are also treated as errors and sent to an `error` queue.
+
+Multiple sensors can be aggregated into a single Storm topology. When this
is done, there will be
+multiple Kafka spouts, but only a single parser bolt which will handle
delegating to the correct
--- End diff --
There is another, more likely use case where we have a transport wrapper on
another message, and 1 topic split into many parsers as well. How can we
handle that?
Specifically -> Syslog (Many Msg types) -> kafka -> bolt -> Split per
message
I expect to add the ability for syslog parsing later, so set that aside.
The issue is we *will* have more than one use case wrt topics.
I am not going to say this PR needs to address it, but I would want us to
understand our path forward and minimize the churn.
It would be best if we did not have to redo this work when accounting for
that.
---
