Hi Neeraj,
Sorry, I was wrong, the error rate is only records that failed to send
after all retries were exhausted (or if retries weren't possible). So if
that's gone up, then records have been dropped by the producer's sender.
Cheers,
Liam
On Tue, 5 Apr 2022 at 10:35, Neeraj Vaidya
wrote:
Thanks Liam.
Yes, I do believe the following should really help:
A producer metric which shows records which did not make their way to the topic
because of retries being exhausted or timeout being exhausted .
If the metric is at a batch level, then we will need to work out the math to
Hi Neeraj,
However, I am unclear what the record-error-rate|total metric for a
> producer means,
> Does the metric get incremented only when the record could not make it to
> the topic or even when there was a transient/retriable error trying to send
> the message to the topic ?
The latter - so
Thank you David and Liam for your excellent responses.
Checking in the consumer will be extremely difficult.
However, I am unclear what the record-error-rate|total metric for a producer
means,
Does the metric get incremented only when the record could not make it to the
topic or even when there
Hi Liam,
I think you've said it well.
David Finnie
Infrasoft Pty Limited
On 4/04/2022 15:10, Liam Clarke-Hutchinson wrote:
Thanks Neeraj,
>From reading code, I am reasonably certain that no data loss occurred - the
producer reset the batch sequence numbers, and then tried again.
I refer
Hi Neeraj,
I don't know what might be causing the first Produce error. Is the
OUT_OF_ORDER_SEQUENCE_NUMBER the first Produce error? From the error
that you included (Invalid sequence number for new epoch) it would seem
that the broker doesn't (yet) know about the Producer's epoch - possibly
Thanks Neeraj,
>From reading code, I am reasonably certain that no data loss occurred - the
producer reset the batch sequence numbers, and then tried again.
I refer you to this comment in the code of the producer's Sender:
// tell the user the result of their request. We only
Hi Liam,
Brokers are on Apache Kafka v2.7.0
However, the Producer client is using the v2.6 libraries.
Regards,
Neeraj On Monday, 4 April, 2022, 02:17:42 pm GMT+10, Liam Clarke-Hutchinson
wrote:
Hi Neeraj,
Not sure just yet, I'm diving into the code to find out. Oh, what version
Kafka
Hi Neeraj,
Not sure just yet, I'm diving into the code to find out. Oh, what version
Kafka are you running please?
Cheers,
Liam
On Mon, 4 Apr 2022 at 14:50, Neeraj Vaidya
wrote:
> Hi Liam,
> Thanks for getting back.
>
> 1) Producer settings ( I am guessing these are the ones you are
Hi Liam,
Thanks for getting back.
1) Producer settings ( I am guessing these are the ones you are interested in)
enable.idempotence=true
max.in.flight.requests.per.connection=5
2) Sample broker logs corresponding to the timestamp in the application logs of
the Producer
[2022-04-03
Hi Neeraj,
First off, what are your producer settings?
Secondly, do you have brokers logs for the leaders of some of your affected
topics on hand at all?
Cheers,
Liam Clarke-Hutchinson
On Mon, 4 Apr 2022 at 14:04, Neeraj Vaidya
wrote:
> Hi All,
> For one of the Kafka producers that I have, I
Hi All,
For one of the Kafka producers that I have, I see that the Producer Record
Error rate is non-zero i.e. out of the expected 3000 messages per second which
I a expect to be producing to the topic, I can see that this metric shows a
rate of about 200.
Does this indicate that the records
12 matches
Mail list logo
