I set up a simple Kafka configuration, with one topic and one partition. I have a Python producer to continuously publish messages to the Kafka server and a Python consumer to receive messages from the server. Each message is about 10K bytes, far smaller than socket.request.max.bytes=104857600. What I found is that the consumer intermittently missed some messages. I checked the server log file and found that these messages are missing there as well. So it looks like these message were never stored by the server. I also made sure that the producer did not receive any error for every message that it published (using send()).
Any clues what could have caused the problem? Thanks, Bo
