Re: DISCUSS KIP-984 Add pluggable compression interface to Kafka

2024-06-17 Thread Colin McCabe
: https://github.com/apache/kafka/releases/tag/2.1.0 > > > -----Original Message- > From: Greg Harris > Sent: Monday, May 13, 2024 1:23 PM > To: dev@kafka.apache.org > Subject: Re: DISCUSS KIP-984 Add pluggable compression interface to Kafka > > Hi Assane,

RE: DISCUSS KIP-984 Add pluggable compression interface to Kafka

2024-06-17 Thread Diop, Assane
Subject: Re: DISCUSS KIP-984 Add pluggable compression interface to Kafka Hi Assane, Thank you for the further information about your motivation and intended use-cases, that adds a lot of context. > Our motivation here is to accelerate compression with the use of hardware > accelerators.

Re: DISCUSS KIP-984 Add pluggable compression interface to Kafka

2024-05-13 Thread Greg Harris
arification on this topic. > Again, thank you for sharing your concerns about balancing this proposal > against the impact to the ecosystem. We think the additional performance that > this could provide along with the improved flexibility to add or accelerate > compression codecs outweigh

RE: DISCUSS KIP-984 Add pluggable compression interface to Kafka

2024-05-13 Thread Diop, Assane
, > and we are looking at alternative approaches for brokers and clients to > coordinate the plugin. > > One way to do this coordination is each client should have a configuration > mapping of the plugin name to its implementation. > > Assane > > > > > > > --

RE: DISCUSS KIP-984 Add pluggable compression interface to Kafka

2024-05-07 Thread Diop, Assane
increased complexity for the operators. Assane -Original Message- From: Greg Harris Sent: Wednesday, May 1, 2024 12:09 PM To: dev@kafka.apache.org Subject: Re: DISCUSS KIP-984 Add pluggable compression interface to Kafka Hi Assane, Thanks for the update. Unfortunately, I don't

Re: DISCUSS KIP-984 Add pluggable compression interface to Kafka

2024-05-01 Thread Greg Harris
ve approaches for brokers and clients to > coordinate the plugin. > > One way to do this coordination is each client should have a configuration > mapping of the plugin name to its implementation. > > Assane > > > > > > > -Original Message- > Fr

RE: DISCUSS KIP-984 Add pluggable compression interface to Kafka

2024-04-29 Thread Diop, Assane
o its implementation. Assane -Original Message- From: Luke Chen Sent: Monday, February 26, 2024 7:50 PM To: dev@kafka.apache.org Subject: Re: DISCUSS KIP-984 Add pluggable compression interface to Kafka Hi Assane, I also share the same concern as Greg has, which is that the KIP i

RE: DISCUSS KIP-984 Add pluggable compression interface to Kafka

2024-03-01 Thread Diop, Assane
ssane -Original Message- From: Luke Chen Sent: Monday, February 26, 2024 7:50 PM To: dev@kafka.apache.org Subject: Re: DISCUSS KIP-984 Add pluggable compression interface to Kafka Hi Assane, I also share the same concern as Greg has, which is that the KIP is not kafka ecosystem friendly. And this

Re: DISCUSS KIP-984 Add pluggable compression interface to Kafka

2024-02-26 Thread Luke Chen
ble interface, you should use topic level compression. > > I would like to have your valuable inputs on this!! > > Thanks before end, > Assane > > -Original Message----- > From: Greg Harris > Sent: Wednesday, February 14, 2024 2:36 PM > To: dev@kafka.apache.org

RE: DISCUSS KIP-984 Add pluggable compression interface to Kafka

2024-02-26 Thread Diop, Assane
ded as far as Kafka. > I have added more information to the > https://cwiki.apache.org/confluence/display/KAFKA/KIP-984%3A+Add+plugg > able+compression+interface+to+Kafka I am ready for a PR if this KIP > gets accepted > > Assane > > -Original Message- > From:

Re: DISCUSS KIP-984 Add pluggable compression interface to Kafka

2024-02-14 Thread Greg Harris
ion to the > https://cwiki.apache.org/confluence/display/KAFKA/KIP-984%3A+Add+pluggable+compression+interface+to+Kafka > I am ready for a PR if this KIP gets accepted > > Assane > > -Original Message- > From: Diop, Assane > Sent: Wednesday, January 31, 2024 10:24 A

RE: DISCUSS KIP-984 Add pluggable compression interface to Kafka

2024-02-14 Thread Diop, Assane
KIP gets accepted Assane -Original Message- From: Diop, Assane Sent: Wednesday, January 31, 2024 10:24 AM To: dev@kafka.apache.org Subject: RE: DISCUSS KIP-984 Add pluggable compression interface to Kafka Hi Divij, Thank you for your response! Although compression is not a new problem

RE: DISCUSS KIP-984 Add pluggable compression interface to Kafka

2024-01-31 Thread Diop, Assane
ocess. Assane -Original Message- From: Divij Vaidya Sent: Friday, December 22, 2023 2:27 AM To: dev@kafka.apache.org Subject: Re: DISCUSS KIP-984 Add pluggable compression interface to Kafka Thank you for writing the KIP Assane. In general, exposing a "pluggable" interface is

Re: DISCUSS KIP-984 Add pluggable compression interface to Kafka

2023-12-22 Thread Divij Vaidya
Thank you for writing the KIP Assane. In general, exposing a "pluggable" interface is not a decision made lightly because it limits our ability to remove / change that interface in future. Any future changes to the interface will have to remain compatible with existing plugins which limits the fle

RE: DISCUSS KIP-984 Add pluggable compression interface to Kafka

2023-12-18 Thread Diop, Assane
I would like to bring some attention to this KIP. We have added an interface to the compression code that allow anyone to build their own compression plugin and integrate easily back to kafka. Assane -Original Message- From: Diop, Assane Sent: Monday, October 2, 2023 9:27 AM To: dev