Github user nickwallen commented on a diff in the pull request:
https://github.com/apache/metron/pull/1082#discussion_r199287320
--- Diff: metron-platform/metron-parsers/README.md ---
@@ -166,6 +166,15 @@ then it is assumed to be a regex and will match any
topic matching the pattern (
* `batchTimeout` : The timeout after which a batch will be flushed even
if batchSize has not been met. Optional.
If unspecified, or set to `0`, it defaults to a system-determined
duration which is a fraction of the Storm
parameter `topology.message.timeout.secs`. Ignored if batchSize is
`1`, since this disables batching.
+ * The kafka writer can be configured within the parser config as well.
(This is all configured a priori, but this is convenient for overriding the
settings) :
--- End diff --
The ability to route messages based on a message field applies for all
topologies that use a `KafkaWriter`, not just the Parsers, right? That would
include Enrichment and Profiler?
This is documented under the Parsers. Would it be worth mentioning that
the same settings would work in any topology that uses a `KafkaWriter` although
it may not be advised.
It might make sense to put these docs in `metron-writer` (with the
`KafkaWriter` class) and then link to those docs from the Parser docs here.
Then in `matron-writer` it would make sense to mention that this potentially
could be used by any topology that uses a `KafkaWriter`.
---
