[
https://issues.apache.org/jira/browse/KAFKA-3553?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Manikumar resolved KAFKA-3553.
------------------------------
Resolution: Auto Closed
{color:#000000}Closing inactive issue. Please raise issue with kafka-python
project, if the issue still exists with newer versions.{color}
> Issue with getting data of next offset after restarting the consumer and
> producer is always running
> ---------------------------------------------------------------------------------------------------
>
> Key: KAFKA-3553
> URL: https://issues.apache.org/jira/browse/KAFKA-3553
> Project: Kafka
> Issue Type: Bug
> Components: consumer, offset manager
> Affects Versions: 0.9.0.0
> Environment: operating system : ubuntu 14.04
> using kafka-python
> Reporter: pooja sadashiv deokar
> Assignee: Neha Narkhede
> Priority: Major
> Labels: performance
>
> I am pushing data to kafka topic after every 1 second in python. And written
> consumer to fetch data from topic with consumer timeout as 500 ms and
> enable_auto_commit as false.
> Following are the scripts:
> 1) prod.py
> from kafka import KafkaClient,SimpleConsumer
> from kafka.producer import SimpleProducer
> import random, time
> kafka = KafkaClient("localhost:9092")
> producer = SimpleProducer(kafka)
> i=0
> while(True):
> l1 = ['Rahul','Narendra','NaMo','ManMohan','Sonia']
> msg = str(i) + ',' + str(random.choice(l1))
> print('putting data : '+msg)
> producer.send_messages("test", msg)
> i = i + 1
> time.sleep(1)
> 2) con.py
> from kafka import KafkaConsumer
> consumer =
> KafkaConsumer('test',bootstrap_servers=['localhost:9092'],consumer_timeout_ms
> = 500, enable_auto_commit = False)
> for message in consumer:
> print ("%s:%d:%d: key=%s value=%s" % (message.topic, message.partition,
> message.offset, message.key,
> message.value))
> consumer.commit()
> My producer is continuously running. And I am running consumer again when it
> stops. For first time consumer works well and gets data for me on time. But
> after first time its taking minimum 1 minute to maximum 5 minutes (or above
> 5 min also) to get next data. It should give me data after 1 second or
> maximum after 5 seconds. But its giving problem :( .
> Also 1 thing which I observe is If I wait for starting consumer about 1 min
> then data comes as expected.
> Please correct me if I am going wrong.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
