Github user mmiklavc commented on the issue:
https://github.com/apache/metron/pull/1184
@ottobackwards I see what you're saying. It looks like that could
definitely work. Thinking out loud here, but might that conflate the semantics
of our validation a bit? Validate currently does things like ensure that a
timestamp exists on the message, though I don't see why we couldn't expand it
to validations outside of our global Metron context.
One class that might be worth checking out is the unified enrichment
topology. This was changed to include a parallel enricher that handles errors
and message results in an EnrichmentResult class.
1.
https://github.com/apache/metron/blob/master/metron-platform/metron-enrichment/src/main/java/org/apache/metron/enrichment/bolt/UnifiedEnrichmentBolt.java#L270
2.
https://github.com/apache/metron/blob/master/metron-platform/metron-enrichment/src/main/java/org/apache/metron/enrichment/parallel/ParallelEnricher.java#L63
It looks to me like there might be some possible collaboration opportunity
and overlap with the work you're doing here and the work @merrimanr is doing on
this PR - https://github.com/apache/metron/pull/1213#pullrequestreview-161248142
I'm just wondering if we might be able to kill 2 birds with one stone. We
probably don't want to change the MessageParser interface, but maybe we can
manage the bulk processing through a more generalized bridge between the
ParserBolt and parser implementations. I haven't dug too deep into
implementation feasibility, but it seems worth considering.
---
