[
https://issues.apache.org/jira/browse/KAFKA-7114?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16596035#comment-16596035
]
Gnanasoundari commented on KAFKA-7114:
--------------------------------------
[~wangzzu]
We are able to reproduce this issue in the below scenario as well. Though the
consumer is actively associated with the partition, few messages are not
consumed by client and shows it in LAG.
Scenario:
Topic will get messages once in a day or two. And message flow is not
continuous and predictable rate. Message will be pushed on demand by the user
when they need.
There is no error/warning messages during the period.
Need your help to identify the root cause for this issue.
|TOPIC|PARTITION|CURRENT-OFFSET|LOG-END-OFFSET|LAG|CONSUMER-ID|HOST|CLIENT-ID|
|asset-action2|0|0|0|0|consumer-1-3c56c5a4-b629-4044-9570-3fddaf3aff84|/CN4|consumer-1|
|asset-action2|1|1|1|0|consumer-1-3c56c5a4-b629-4044-9570-3fddaf3aff84|/CN4|consumer-1|
|asset-action2|2|9|9|0|consumer-1-3c56c5a4-b629-4044-9570-3fddaf3aff84|/CN4|consumer-1|
|asset-action2|3|20|20|0|consumer-1-3c56c5a4-b629-4044-9570-3fddaf3aff84|/CN4|consumer-1|
|asset-action3|0|6|6|0|consumer-2-4baad662-d0cd-4d1b-af8d-9e7f9c3f5028|/CN4|consumer-2|
|asset-action3|1|0|0|0|consumer-2-4baad662-d0cd-4d1b-af8d-9e7f9c3f5028|/CN4|consumer-2|
|asset-action-request4|0|0|0|0|consumer-5-d9dbeb5b-57d8-43b5-9999-8ee4688c51b8|/CN1|consumer-5|
|asset-action-request4|1|0|0|0|consumer-5-d9dbeb5b-57d8-43b5-9999-8ee4688c51b8|/CN1|consumer-5|
|asset-action-request0|0|9|14|{color:#FF0000}5{color}|consumer-10-5b16308f-2386-4bcc-a8e9-819e921a8100|/CN4|consumer-10|
|asset-action-request0|1|4|6|{color:#FF0000}2{color}|consumer-10-5b16308f-2386-4bcc-a8e9-819e921a8100|/CN4|consumer-10|
|asset-action-request3|0|0|0|0|consumer-5-32893946-4835-479d-9cd2-96a6b1ae5185|/CN4|consumer-5|
|asset-action-request3|1|0|0|0|consumer-5-32893946-4835-479d-9cd2-96a6b1ae5185|/CN4|consumer-5|
|asset-action-request2|0|0|0|0|consumer-6-62ac1703-de40-46f9-9a45-484eff411752|/CN2|consumer-6|
|asset-action-request2|1|0|0|0|consumer-6-62ac1703-de40-46f9-9a45-484eff411752|/CN2|consumer-6|
|asset-action4|2|14|14|0|consumer-6-39d83700-9ca4-44db-a8b4-6e294b8112a5|/CN3|consumer-6|
|asset-action4|3|0|0|0|consumer-6-39d83700-9ca4-44db-a8b4-6e294b8112a5|/CN3|consumer-6|
|asset-action-request1|0|0|0|0|consumer-10-70f886b3-e1d9-413c-bb67-10af165c45e5|/CN2|consumer-10|
|asset-action-request1|1|0|0|0|consumer-10-70f886b3-e1d9-413c-bb67-10af165c45e5|/CN2|consumer-10|
|asset-action3|2|2|2|0|consumer-3-c3421e00-45e7-45c2-ab4e-dde96f655a77|/CN1|consumer-3|
|asset-action3|3|6|6|0|consumer-3-c3421e00-45e7-45c2-ab4e-dde96f655a77|/CN1|consumer-3|
|asset-action-request3|2|0|0|0|consumer-8-ccae17d1-a48b-4242-bb91-130ef0e54eac|/CN2|consumer-8|
|asset-action-request3|3|0|0|0|consumer-8-ccae17d1-a48b-4242-bb91-130ef0e54eac|/CN2|consumer-8|
|asset-action4|0|0|0|0|consumer-6-2803e5b5-ba05-43a2-b057-b02d49a33a67|/CN4|consumer-6|
|asset-action4|1|5|5|0|consumer-6-2803e5b5-ba05-43a2-b057-b02d49a33a67|/CN4|consumer-6|
|asset-action1|2|0|0|0|consumer-7-2d807ea1-ee8a-40b2-8a02-4f25ad5354af|/CN1|consumer-7|
|asset-action-request2|2|0|0|0|consumer-7-ea0b61be-4187-4c93-abe5-0130726d6655|/CN4|consumer-7|
|asset-action-request1|3|0|0|0|consumer-8-f39ebd37-dde8-45df-b7f9-49a3821ba584|/CN1|consumer-8|
|asset-action0|0|14|14|0|consumer-3-8e90b176-af02-47f7-8e06-a03b37839fcf|/CN4|consumer-3|
|asset-action1|1|4|4|0|consumer-2-13961583-6545-4e00-9ce8-8a6f039ee699|/CN2|consumer-2|
|asset-action-request0|2|14|14|0|consumer-10-b2016d59-6698-4bf5-9888-829f808b6856|/CN3|consumer-10|
|asset-action-request1|2|0|0|0|consumer-4-623975e3-8c8b-48ae-b49c-fe5b6d18eea5|/CN4|consumer-4|
|asset-action0|3|0|0|0|consumer-9-cd0f8733-811f-42b5-96c3-b96763f1ed0b|/CN1|consumer-9|
|asset-action-request0|3|4|4|0|consumer-5-90ed68b6-4f51-4363-9f87-86a2c3d779a6|/CN2|consumer-5|
|asset-action-request4|2|0|0|0|consumer-7-92539217-2ecc-43c4-8b41-36a17cbe76fa|/CN3|consumer-7|
|asset-action0|2|19|19|0|consumer-7-f6a53366-2c53-48b4-8d42-42367f794420|/CN2|consumer-7|
|asset-action0|1|4|6|2|consumer-5-2b29b017-56bb-4c84-8da0-1b8a9af98a3c|/CN3|consumer-5|
|asset-action-request4|3|0|0|0|consumer-9-d61ca364-37dd-4299-ad9b-85c3dbbe0bd5|/CN4|consumer-9|
|asset-action1|0|0|0|0|consumer-1-6660179f-d5e6-4f38-b637-29415e255d6f|/CN3|consumer-1|
|asset-action-request2|3|0|0|0|consumer-9-143cd443-c4c3-45c8-8c19-40974bbf4fa9|/CN3|consumer-9|
|asset-action1|3|0|0|0|consumer-8-b437a0f3-2afd-4f5e-90ed-9204127387f1|/CN4|consumer-8|
> Kafka Not sending few partitions messages to consumers
> ------------------------------------------------------
>
> Key: KAFKA-7114
> URL: https://issues.apache.org/jira/browse/KAFKA-7114
> Project: Kafka
> Issue Type: Bug
> Components: consumer
> Affects Versions: 1.1.0
> Reporter: Gnanasoundari
> Priority: Blocker
>
> *Infrastructure:*
> Kafka is running in cluster mode with 3 brokers and 3 zookeeper instances.
> Kafka broker is running in t2.xlarge AWS instance.
> 3 Zookeeper instances is running in single t2.xlarge AWS instance.
> Kafka Version: 1.1.0
> *Issue:*
> Kafka is not sending messages to consumer though the consumer is still active
> and subscribed to the partition.
> *How to reproduce:*
> * Continuously send 50,000 records/sec where each record size is about 200
> bytes approx. for 24hour duration.
> * We have 25 topics with 4 partitions each. All 25 topics are consumed by 4
> consumers with auto assignment of partition by kafka.
> * Kafka clears all records in partitions except 2 partitions. For two
> partitions , only 27 messages are delivered. Remaining messages has not
> delivered to consumer.
> * During the run kafka brokers cpu utilization is 5%
> Problematic partitions details:
> |TOPIC|PARTITION|CURRENT-OFFSET|LOG-END-OFFSET|LAG|CONSUMER-ID|HOST|CLIENT-ID|
> |topic-di14|3|27|3919844|3919817|consumer-7-dfaf4c6f-5b71-455e-ab12-9984446a9bf4|/10.10.9.22|consumer-7|
> |topic-di9|0|27|12231117|12231090|consumer-22-7922babd-8f8b-49db-a111-df523962cca8|/10.10.9.22|consumer-22|
> I am not able to see any error in kafka as well as my consumer application.
> Consumer is active when i ran ./bin/kafka-consumer-groups.sh command in
> kafka.
>
> *Kafka Consumer settings:*
> enable.auto.commit = true
> session.timeout.ms = 90000
> heartbeat.interval.ms = 30000
> max.poll.records = 1000
> poll-interval = 60ms
> poll-timeout = 60ms
>
> Please help us to know why this behavior is happening in kafka.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
