Thank you all for the comments. Yes, I understand concern from community
members with extra burden of having the complexity to drop message, but if
ability to inject implementation of the Queue which will make this
completely transparent to Kafka.
I just need fine-grained control of the
Basically, requirement is to support message dropping policy in event when
queue is full. When you get storm of data (in our case logging due to
buggy application code), we would like to retain current message instead of
first one in queue. We will mitigate this by rate limiting on producer
Is it possible there is another solution to the problem? I think if you
could better describe the problem(s) you are facing and how you are
architected some then you may get responses from others that perhaps have
faced the same problem with similar architectures ... or maybe folks can
chime in
