Just one thing to bear in mind, publishing an error may cause some
operational challenges as it fills up the error topic as well as storm logs
which may not be necessary. To wear a Metron user hat, dropping a message
with a debug/trace level log to specify the event is filter out makes
sense. I
Completely agreed on the acking. The reason I posed the question to begin
with was because, while I believe dropping+acking is the correct
functionality, I could see a few alternative patterns for handling this:
1. Require filtering to be handled by the message filter infrastructure
and
We absolutely should be acking the dropped messages otherwise they'll be in
a replay loop. Not acking is a flat-out bug IMO.
On Wed, Dec 19, 2018 at 2:37 PM Michael Miklavcic <
michael.miklav...@gmail.com> wrote:
> When a message is filtered by the message filtering mechanism, we
> explicitly
When a message is filtered by the message filtering mechanism, we
explicitly drop the message (and presumably ack it in Storm), as explained
here -
https://github.com/apache/metron/tree/master/metron-platform/metron-parsing#filtered.
When using the REGEX_SELECT field transformation (see here -
