Hi David
01 Same as Java Client's existing behaviour of requesting a full expedited
metadata-refresh done asynchronously, for error NOT_LEADER OR FENCED_EPOCH,
the KIP proposes to continue doing so. As this would help prevent similar
errors on future requests to other partitions affected by
Hi Mayank,
Thanks again for the KIP and thanks for adding the new analysis. Overall, I
am fine with it. I have a few minor comments.
01. If I understand correctly, the client will still request a metadata
update even when it gets the new leader if the produce response or the
fetch response. Is
Adding to Crispin. The new micro-benchmark shows improvements of 88% in
p99.9 with the KIP changes Vs baseline(& rejected alternate). Its
hypothesised improvements are seen as KIP avoids a full metadata refresh(Vs
baseline/rejected alternate), and the full metadata refresh can be slow due
to
Hi,
I've updated the KIP with benchmark results focusing more directly on the
redirect case, please review and +1 in the voting thread if convinced.
Thank you,
Crispin
On Wed, Jul 26, 2023 at 11:13 PM Luke Chen wrote:
> Thanks for adding the benchmark results, Crispin!
> IMO, 2~5% performance
Thanks for adding the benchmark results, Crispin!
IMO, 2~5% performance improvement is small, but given the change is small,
cost is also small (only append endpoint info when NOT_LEADER_OR_FOLLOWER..
etc error), I think it is worth doing it.
Thank you.
Luke
On Wed, Jul 26, 2023 at 12:33 AM
Thanks Crispin!
Ismael
On Mon, Jul 24, 2023 at 8:16 PM Crispin Bernier
wrote:
> I updated the wiki to include both results along with their average.
>
> Thank you,
> Crispin
>
> On Mon, Jul 24, 2023 at 10:58 AM Ismael Juma wrote:
>
> > Hi Crispin,
> >
> > One additional question, the wiki
I updated the wiki to include both results along with their average.
Thank you,
Crispin
On Mon, Jul 24, 2023 at 10:58 AM Ismael Juma wrote:
> Hi Crispin,
>
> One additional question, the wiki says "The results are averaged over 2
> runs.". Can you please provide some measure of variance in the
David,
We never backport new features to old releases. This new feature will be
> only available from 3.6 (or 3.7) onwards for both client and server.
Good to know. I think that makes the argument for bumping the version even
stronger.
On Mon, Jul 24, 2023 at 5:01 PM David Jacot
wrote:
> Hi
Hi Mayank,
We never backport new features to old releases. This new feature will be
only available from 3.6 (or 3.7) onwards for both client and server.
Best,
David
On Mon, Jul 24, 2023 at 5:20 PM Mayank Shekhar Narula <
mayanks.nar...@gmail.com> wrote:
> Thanks Jose/David/Ismael for your
Hi Mayank,
On Mon, Jul 24, 2023 at 8:21 AM Mayank Shekhar Narula
wrote:
>
> Thanks Jose/David/Ismael for your inputs.
>
> Not bumping the version, would require both broker & client to backport
> changes. Especially for FetchResponse, as backporting would have to be done
> all the way back to
Thanks Jose/David/Ismael for your inputs.
Not bumping the version, would require both broker & client to backport
changes. Especially for FetchResponse, as backporting would have to be done
all the way back to 3.1, so this effort isn't trivial, and was originally
underestimated.
Considering
Hi Crispin,
One additional question, the wiki says "The results are averaged over 2
runs.". Can you please provide some measure of variance in the
distribution, i.e. were both results similar to each other for both cases?
Ismael
On Fri, Jul 21, 2023 at 11:31 AM Ismael Juma wrote:
> Thanks for
On Mon, Jul 24, 2023 at 3:32 PM David Jacot
wrote:
> 01. Hum... I understand your reasoning. I think that this is mainly
> beneficial for clients lagging behind in terms of supported versions.
> However, it is the opposite for the java client which is up to date.
> Personally, I would rather
Hi Mayank,
01. Hum... I understand your reasoning. I think that this is mainly
beneficial for clients lagging behind in terms of supported versions.
However, it is the opposite for the java client which is up to date.
Personally, I would rather prefer to bump both versions and to add the
tagged
Hey Mayank,
It is probably binary compatible to have the NodeEndponts fielld at
taggedVersion 12+ but I think it is misleading as a code reviewer. The
Java Kafka client at version 12 will never be able to handle those
fields. Or are you planning to backport these improvements to those
clients and
The downsides of bumping the version is that clients have to have all
the latest features implemented before being able to benefit from this
performance improvement.
One of the benefits of using a tagged field is to make the field
available to previous versions too.
Choosing a minimum value
David
03. Fixed as well, to remove ignorable. In MetadataResponse, Rack came a
version later, hence was marked ignorable.
Thanks.
On Fri, Jul 21, 2023 at 1:38 PM Mayank Shekhar Narula <
mayanks.nar...@gmail.com> wrote:
> Hi David
>
> 01. My reasoning noted in the KIP is that CurrentLeader was
Hi David
01. My reasoning noted in the KIP is that CurrentLeader was first added in
version 12, so 12 is the least version where clients could get these
optimisations. So any client can now choose to implement this with version
12 of the protocol itself. If the version is bumped to X(say 16),
Thanks for the update Crispin - very helpful to have actual performance
data. 2-5% for the default configuration is a bit on the low side for this
kind of proposal.
Ismael
On Thu, Jul 20, 2023 at 11:33 PM Crispin Bernier
wrote:
> Benchmark numbers have been posted on the KIP, please review.
>
Hi Mayank,
Thanks for the KIP. This is an interesting idea that I have been thinking
about for a long time so I am happy to see it. The gain is smaller than I
expected but still worth it in my opinion.
01. In the FetchResponse, what's the reason for using version `12+` for the
new tagged field
Benchmark numbers have been posted on the KIP, please review.
On 2023/07/20 13:03:00 Mayank Shekhar Narula wrote:
> Jun
>
> Thanks for the feedback.
>
> Numbers to follow.
>
> If we don't plan to
> > bump up the FetchResponse version, we could just remove the reference to
> > version 16.
>
>
Emanuele
I agree with this. That's why i quoted below -
> I wonder if non-Kafka clients might benefit from not bumping the
> version. If versions are bumped, say for FetchResponse to 16, I believe
> that client would have to support all versions until 16 to fully
> utilise this feature. Whereas,
The downsides of bumping the version is that clients have to have all
the latest features implemented before being able to benefit from this
performance improvement.
One of the benefits of using a tagged field is to make the field
available to previous versions too.
Choosing a minimum value
23 matches
Mail list logo
