Prateek Maheshwari created SAMZA-1644: -----------------------------------------
Summary: Blocking poll on KafkaConsumer can miss the last batch of messages Key: SAMZA-1644 URL: https://issues.apache.org/jira/browse/SAMZA-1644 Project: Samza Issue Type: Bug Reporter: Prateek Maheshwari BrokerProxy is calling setIsAtHighWatermark before adding the new messages to the BEM (i.e., as the first step in moveMessagesToTheirQueue). This can cause the caller of blocking poll to miss the latest batch of messages in the current poll, since the poll logic is: while (envelope == null && !isAtHead(systemStreamPartition)) { metrics.incBlockingPoll(systemStreamPartition); envelope = queue.poll(1000, TimeUnit.MILLISECONDS); } We should move the setIsAtHead call to after the messages have been added to the BEM. -- This message was sent by Atlassian JIRA (v7.6.3#76005)