Hi all,
Please grant permission for Create KIP to wiki ID: LinShunKang.
Thanks.
Thanks.
Chris Egerton 于2022年8月21日周日 23:00写道：
> I've added you to the Wiki and Jira; you should be good to go now.
>
> On Sun, Aug 21, 2022 at 10:02 AM ShunKang Lin
> wrote:
>
> > Hi all,
> > Please grant permission for Create KIP to wiki ID: LinShunKang.
> >
> > Thanks.
> >
>
Hi all，
I'd like to start a discussion on KIP-863 which is Reduce
Fetcher#parseRecord() memory copy. This KIP can reduce Kafka Consumer
memory allocation by nearly 50% during fetch records.
Please check
https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=225152035
and
m of the opinion that any user provided data should be
> copied to internal data structures at the interface of an opaque library
> (like client) so that the user doesn't have to guess about memory lifetime
> of the objects they provided to the opaque library.
>
> What do you think?
>
iki.apache.org/confluence/display/KAFKA/Kafka+Improvement+Proposals#KafkaImprovementProposals-Whatisconsidereda%22majorchange%22thatneedsaKIP?>
> )
>
>
> Guozhang
>
> On Sun, Sep 25, 2022 at 6:52 AM ShunKang Lin
> wrote:
>
> > Hi all,
> >
> > I'd l
> I
> > > think adding an overloaded method with ByteBuffer would help with other
> > > serde places on the client side as well.
> > >
> > >
> > > Guozhang
> > >
> > > On Thu, Sep 22, 2022 at 9:41 AM ShunKang Lin <
> linshunka
;
> On Fri, Oct 21, 2022 at 9:45 PM ShunKang Lin
> wrote:
>
> > Hi everyone,
> >
> > Thank you for the vote. I've got three +1 votes (Guozhang, Luke, Chris),
> > can this vote be concluded?
> >
> > Best,
> > ShunKang
> >
> &
Bump this thread to see if there are any comments/thoughts.
Best,
ShunKang
ShunKang Lin 于2022年9月30日周五 23:58写道：
> Hi Divij Vaidya,
>
> 3. Sounds good, but `ByteBuffer#asReadOnlyBuffer()` returns a read-only
> `ByteBuffer` which `ByteBuffer#hasArray()` returns false, then it will m
Sounds good, thanks for clarifying.
> >
> > On Tue, Sep 20, 2022 at 9:50 AM ShunKang Lin
> > wrote:
> >
> >> Hi Guozhang,
> >>
> >> Thanks for your comments!
> >>
> >> 1. We can reduce memory allocation if the key/value types
Hi all, I'd like to start a new discussion thread on KIP-872 (Kafka Client)
which proposes that add Serializer#serializeToByteBuffer() to reduce memory
copying.
KIP:
https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=228495828
Thanks, ShunKang
Hi all,
I'd like to start a new discussion thread on KIP-871 (Kafka Client) which
proposes that fix ByteBufferSerializer#serialize(String, ByteBuffer)
compatible problem.
Links:
KIP:
https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=228495816
Jira:
Thanks Guozhang!
Best,
ShunKang
Guozhang Wang 于2022年9月23日周五 00:27写道：
> Could you start a separate VOTE email thread calling for votes?
>
> On Thu, Sep 22, 2022 at 9:19 AM ShunKang Lin
> wrote:
>
> > Hi Guozhang,
> >
> > Thanks for your help! By the way, wh
Hi everyone,
I'd like to open the vote for KIP-863, which proposes to reduce memory
allocation and memory copying in Fetcher#parseRecord(TopicPartition,
RecordBatch, Record).
The proposal is here:
https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=225152035
Thanks to all who
Hi Guozhang,
Thanks for your help! By the way, what should I do next?
Best,
ShunKang
Guozhang Wang 于2022年9月22日周四 23:21写道：
> Thanks ShunKang,
>
> I made a few nit edits on the Motivation section as well. LGTM for me now.
>
> On Thu, Sep 22, 2022 at 7:33 AM ShunKang Lin
&g
r as well, otherwise we would
> > still do the `return deserialize(topic, headers, Utils.toArray(data));`
> > from default impls unless the user customized deserializers is augmented
> to
> > handle ByteBuffer directly, right?
> >
> >
> > Guozhang
> >
> >
&
pes happen to be ByteBuffer as well, otherwise we would
> still do the `return deserialize(topic, headers, Utils.toArray(data));`
> from default impls unless the user customized deserializers is augmented to
> handle ByteBuffer directly, right?
>
>
> Guozhang
>
>
>
&
e, Oct 11, 2022 at 8:36 AM Luke Chen wrote:
>
> > Hi ShunKang,
> >
> > Had a quick look, I think It's a good idea.
> > I'll check it again tomorrow, and let you know if I have any questions.
> >
> > Luke
> >
> > On Sun, Sep 25, 2022 at 3:35 PM Shun
Hi Luke,
Thanks for your comments.
Best,
ShunKang
Luke Chen 于2022年10月11日周二 20:36写道：
> Hi ShunKang,
>
> Had a quick look, I think It's a good idea.
> I'll check it again tomorrow, and let you know if I have any questions.
>
> Luke
>
> On Sun, Sep 25, 2022 at 3:35
rovided input buffer nor the pointers).
>
> --
> Divij Vaidya
>
>
>
> On Wed, Sep 28, 2022 at 5:35 PM ShunKang Lin
> wrote:
>
> > Hi Divij Vaidya,
> >
> > Thanks for your comments.
> >
> > 1. I checked the code of KafkaProdu
e old one will never be called. That way, implementations
> can just throw an exception on that method instead of implementing both.
>
> Thanks again!
> -John
>
> On Wed, Nov 2, 2022, at 20:14, ShunKang Lin wrote:
> > Bump this thread again : )
> >
> > ShunKang Lin
ng of deprecating the original method. But if you think it’s better
> to keep it the way it is, I’m fine with it.
>
> I have no other comments.
>
> Thanks again for the KIP,
> John
>
> On Sat, Nov 5, 2022, at 11:59, ShunKang Lin wrote:
> > Hi John,
> >
> > Thanks
Bump this thread again : )
ShunKang Lin 于2022年9月25日 周日23:59写道：
> Hi all, I'd like to start a new discussion thread on KIP-872 (Kafka
> Client) which proposes that add Serializer#serializeToByteBuffer() to
> reduce memory copying.
>
> KIP:
> https://cwiki.apache.or
t;
> On Mon, Apr 10, 2023 at 6:47 AM ShunKang Lin
> wrote:
>
> > Thanks for your comment.
> >
> > This KIP does not modify ByteBufferSerializer#serialize(), so do we need
> to
> > clarify this aspect on motivation?
> >
> > Best,
> > ShunKang
e avoid (eg direct byte buffers, byte buffer views, etc.).
>
> Ismael
>
> On Sun, Sep 25, 2022 at 8:59 AM ShunKang Lin
> wrote:
>
> > Hi all, I'd like to start a new discussion thread on KIP-872 (Kafka
> Client)
> > which proposes that add Serializ
antics and want to correct it later, we’ll be forced to
> > introduce yet another method later. This especially makes sense if we’re
> > thinking of deprecating the original method. But if you think it’s better
> > to keep it the way it is, I’m fine with it.
> >
> > I ha
sfy the condition
> (data.arrayOffset() == 0 && arr.length == data.remaining()).
>
> --
> Divij Vaidya
>
>
>
> On Wed, Apr 19, 2023 at 6:24 AM ShunKang Lin
> wrote:
>
> > Thank you for your comment, Divij.
> >
> > 4. Do you still have any questions
me.
>
> 4. nit -> please add javadoc in the KIP as well since we are voting on KIP
> in this discussion thread and not on the PR.
>
> --
> Divij Vaidya
>
>
>
> On Fri, Apr 28, 2023 at 1:28 PM ShunKang Lin
> wrote:
>
> > Thank you for your com
Hi all,
Bump this thread again and see if we could get a few more votes. Currently
we have +2 non-binding and +1 binding.
Hoping we can get this approved, reviewed, and merged in time for 3.6.0.
Best,
ShunKang
ShunKang Lin 于2023年5月7日周日 15:24写道：
> Hi everyone,
>
> I'd like to open
+1 (non binding)
> > >
> > > I think that this is a nice improvement as it prevents an unnecessary
> data
> > > copy for users who are using ByteBuffer serialization on the producer.
> > >
> > > --
> > > Divij Vaidya
> > >
> > >
>
for 3.6.0.
Best,
ShunKang
ShunKang Lin 于2023年5月7日周日 15:24写道：
> Hi everyone,
>
> I'd like to open the vote for KIP-872, which proposes to add
> Serializer#serializeToByteBuffer() to reduce memory copying.
>
> The proposal is here:
> https://cwiki.apache.org/confluence/pages/v
Hi Satish,
Could we add "KIP-863: Reduce CompletedFetch#parseRecord() memory copy" [1]
and "KIP-872: Add Serializer#serializeToByteBuffer() to reduce memory
copying" [2] to the release plan?
Thanks!
[1]
https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=225152035
[2]
t;
> On 2023/06/20 13:37:04 ShunKang Lin wrote:
> > Hi all,
> >
> > Bump this thread again and see if we could get a few more votes.
> > Currently we have +3 non-binding (from Divij Vaidya, Kirk True and Kamal
> > Chandraprakash) and +2 binding (from Luke Chen and zi
vij and ShunKang,
> >
> > I pulled open this thread to see if you needed my vote, but FYI, Divij is
> > a committer now, so he can re-cast his vote as binding.
> >
> > Thanks,
> > -John
> >
> > On 2023/06/20 13:37:04 ShunKang Lin wrote:
> > > Hi all,
Congratulations, Divij!
Bill Bejeck 于2023年6月16日周五 09:43写道：
> Congratulations!!
>
> On Tue, Jun 13, 2023 at 7:14 PM ziming deng
> wrote:
>
> > Congratulations, Divij!
> > Well deserved!
> >
> > --
> > Ziming
> >
> > > On Jun 14, 2023, at 09:41, Luke Chen wrote:
> > >
> > > Congratulations,
,
ShunKang
ShunKang Lin 于2023年5月7日周日 15:24写道：
> Hi everyone,
>
> I'd like to open the vote for KIP-872, which proposes to add
> Serializer#serializeToByteBuffer() to reduce memory copying.
>
> The proposal is here:
> https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=2
Hi everyone,
I'd like to open the vote for KIP-872, which proposes to add
Serializer#serializeToByteBuffer() to reduce memory copying.
The proposal is here:
https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=228495828
The pull request is here:
36 matches
Mail list logo
