Github user cestella commented on the issue:
https://github.com/apache/incubator-metron/pull/359
Hey @DomenicPuzio thanks man :)
The problem with placing the ack there is that in the situation where the
enrichment adapter worker gets killed, we would *like* the data to replay from
the anchor point (the SplitBolt) because Storm will have respawned the worker
elsewhere and we'd like that enrichment to be captured. If you keep the ack
where you have it, the message will never join if the enrichment worker gets
killed, so the message will ultimately get dropped, which isn't ideal.
Ultimately, I think we can all agree that we need to ack at LEAST the tuple
on the `message` stream, but I think that we want to ack it AFTER the join has
happened.
Regarding seeing dup data, I believe you. I am just trying to wrap my head
around how it happens, but, as they say, the proof of the pudding is in the
eating. :)
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
