Hi, should have found a better subject...
If I read the source correctly then offsets can be committed at any time (whenever there is a change in consumer or broker zk registry). Our application doesn't use auto-commit in order to batch some messages together, process them and if everything went fine, we call commitOffsets(). If, for any reason, the processing of messages does not succeed, we rely on Kafka's promise to re-deliver the messages. But if ZKRebalancerListener triggers a rebalance before our "batch" of messages is full, then offsets will be committed even if the messages have not been processed yet by our application. So if then processing of these messages fails, we basically lost them, right? Are there any recipes how to handle this? I really would like to stick to ZookeeperConsumerConnector, so switching to SimpleConsumer would be considered as "last resort". Regards Peter -- 404 Signature Not Found
