Re: Kafka Consumer not consuming new messages randomly

2016-11-26 Thread Ewen Cheslack-Postava
The REST proxy cannot guarantee that if there are messages in Kafka it will definitely return them. There will always be some latency between the request to the REST proxy and fetching data from Kafka, and because of the way the Kafka protocol works this could be delayed by the fetch timeout. The

Kafka Consumer not consuming new messages randomly

2016-11-21 Thread marcel bichon
Hello ! I contact you because I'm have a problem with my architecture using kafka. I'm using Confluent 1.0 with kafka 0.8.2. I have the following architecture : - A job who calls every hour the kafka REST API behind a load-balancer. - The Kafka REST API servers points on a kafka cluster of three