[
https://issues.apache.org/jira/browse/METRON-569?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16037013#comment-16037013
]
ASF GitHub Bot commented on METRON-569:
---------------------------------------
Github user cestella commented on a diff in the pull request:
https://github.com/apache/metron/pull/603#discussion_r120116377
--- Diff:
metron-platform/metron-enrichment/src/main/java/org/apache/metron/enrichment/bolt/JoinBolt.java
---
@@ -141,11 +141,12 @@ public void execute(Tuple tuple) {
collector.emit( "message"
, tuple
, new Values( key
- , joinMessages(streamMessageMap)
+ , joinMessages(streamMessageMap,
this.messageGetStrategy)
)
);
cache.invalidate(key);
- collector.ack(tuple);
+ Tuple messageTuple = streamMessageMap.get("message:");
+ collector.ack(messageTuple);
--- End diff --
Sounds good!
> Enrichment topology duplicates messages
> ---------------------------------------
>
> Key: METRON-569
> URL: https://issues.apache.org/jira/browse/METRON-569
> Project: Metron
> Issue Type: Bug
> Reporter: Domenic Puzio
> Original Estimate: 1h
> Remaining Estimate: 1h
>
> When running the 'enrichment' topology, I get duplicate message being
> indexed. For example, I put 100 messages into the 'enrichment' Kafka queue
> and I get 175 messages onto the 'indexing' Kafka queue. This happens when I
> am running the 'enrichment' topology with one or more enrichment bolt.
> This is an acking issue within the JoinBolt class. When a message does not
> "complete" the join (like when it is the first message in a pair of message
> to get joined) it does not get acked. This means that this message will get
> replayed through Storm, causing message duplication further down the road and
> tons of additional overhead. Adding the correct acking resolves this problem.
> I will add the PR for this shortly.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
