Github user srdo commented on a diff in the pull request:
https://github.com/apache/storm/pull/1999#discussion_r106283741
--- Diff:
external/storm-kafka-client/src/main/java/org/apache/storm/kafka/spout/KafkaSpout.java
---
@@ -383,23 +403,24 @@ private void commitOffsetsForAckedTuples() {
@Override
public void ack(Object messageId) {
final KafkaSpoutMessageId msgId = (KafkaSpoutMessageId) messageId;
- if(!emitted.contains(msgId)) {
- LOG.debug("Received ack for tuple this spout is no longer
tracking. Partitions may have been reassigned. Ignoring message [{}]", msgId);
- return;
- }
-
- if (!consumerAutoCommitMode) { // Only need to keep track of
acked tuples if commits are not done automatically
- acked.get(msgId.getTopicPartition()).add(msgId);
+ if (!emitted.contains(msgId)) {
+ LOG.debug("Received ack for message [{}], which is associated
with either a null tuple that was never emitted, " +
--- End diff --
I mean that the debug log here is now getting written both when the spout
receives an ack for a tuple from a partition that has been reassigned since it
was emitted, and when the RecordTranslator emits a null tuple. If you add msgId
to emitted in L357 before calling ack(), we don't have to change the log here
to also mention null tuples.
I think logging something when null tuples are being dropped might fit
better in RecordTranslator.apply (or in L357) than here, because then the log
can be clear about what is happening.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
