Hey Mickael Since this KIP was written, we have a new proposal to make the compression completely pluggable https://cwiki.apache.org/confluence/display/KAFKA/KIP-984%3A+Add+pluggable+compression+interface+to+Kafka. If we implement that KIP, would it supersede the need for adding fine grain compression controls in Kafka?
It might be beneficial to have a joint proposal of these two KIPs which may satisfy both use cases. -- Divij Vaidya On Wed, Feb 7, 2024 at 11:14 AM Mickael Maison <mickael.mai...@gmail.com> wrote: > Hi, > > I'm resurrecting this old thread as this KIP would be a nice > improvement and almost 3 years later the PR for this KIP has still not > been merged! > > The reason is that during reviews we noticed the proposed > configuration, compression.level, was not easy to use as each codec > has its own valid range of levels [0]. > > As proposed by Jun in the PR [1], I updated the KIP to use > compression.<codec>.level configurations instead of a single > compression.level setting. This syntax would also line up with the > proposal to add per-codec configuration options from KIP-780 [2] > (still to be voted). I moved the original proposal to the rejected > section. > > I've put the original voters and KIP author on CC. Let me know if you > have any feedback. > > 0: https://github.com/apache/kafka/pull/10826 > 1: https://github.com/apache/kafka/pull/10826#issuecomment-1795952612 > 2: > https://cwiki.apache.org/confluence/display/KAFKA/KIP-780%3A+Support+fine-grained+compression+options > > Thanks, > Mickael > > > On Fri, Jun 11, 2021 at 10:00 AM Dongjin Lee <dong...@apache.org> wrote: > > > > This KIP is now passed with: > > > > - binding: +3 (Ismael, Tom, Konstantine) > > - non-binding: +1 (Ryanne) > > > > Thanks again to all the supporters. I also updated the KIP by moving the > > compression buffer option into the 'Future Works' section, as Ismael > > proposed. > > > > Best, > > Dongjin > > > > > > > > On Fri, Jun 11, 2021 at 3:03 AM Konstantine Karantasis > > <konstant...@confluent.io.invalid> wrote: > > > > > Makes sense. Looks like a good improvement. Thanks for including the > > > evaluation in the proposal Dongjin. > > > > > > +1 (binding) > > > > > > Konstantine > > > > > > On Wed, Jun 9, 2021 at 6:59 PM Dongjin Lee <dong...@apache.org> wrote: > > > > > > > Thanks Ismel, Tom and Ryanne, > > > > > > > > I am now updating the KIP about the further works. Sure, You won't be > > > > disappointed. > > > > > > > > As of Present: > > > > > > > > - binding: +2 (Ismael, Tom) > > > > - non-binding: +1 (Ryanne) > > > > > > > > Anyone else? > > > > > > > > Best, > > > > Dongjin > > > > > > > > On Thu, Jun 10, 2021 at 2:03 AM Tom Bentley <tbent...@redhat.com> > wrote: > > > > > > > > > Hi Dongjin, > > > > > > > > > > Thanks for the KIP, +1 (binding). > > > > > > > > > > Kind regards, > > > > > > > > > > Tom > > > > > > > > > > On Wed, Jun 9, 2021 at 5:16 PM Ismael Juma <ism...@juma.me.uk> > wrote: > > > > > > > > > > > I'm +1 on the proposed change. As I stated in the discuss > thread, I > > > > don't > > > > > > think we should rule out the buffer size config, but we could > list > > > that > > > > > as > > > > > > future work vs rejected alternatives. > > > > > > > > > > > > Ismael > > > > > > > > > > > > On Sat, Jun 5, 2021 at 2:37 PM Dongjin Lee <dong...@apache.org> > > > wrote: > > > > > > > > > > > > > Hi all, > > > > > > > > > > > > > > I'd like to open a voting thread for KIP-390: Support > Compression > > > > Level > > > > > > > (rebooted): > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > https://cwiki.apache.org/confluence/display/KAFKA/KIP-390%3A+Support+Compression+Level > > > > > > > > > > > > > > Best, > > > > > > > Dongjin > > > > > > > > > > > > > > -- > > > > > > > *Dongjin Lee* > > > > > > > > > > > > > > *A hitchhiker in the mathematical world.* > > > > > > > > > > > > > > > > > > > > > > > > > > > > *github: <http://goog_969573159/>github.com/dongjinleekr > > > > > > > <https://github.com/dongjinleekr>keybase: > > > > > > https://keybase.io/dongjinleekr > > > > > > > <https://keybase.io/dongjinleekr>linkedin: > > > > > > kr.linkedin.com/in/dongjinleekr > > > > > > > <https://kr.linkedin.com/in/dongjinleekr>speakerdeck: > > > > > > > speakerdeck.com/dongjin > > > > > > > <https://speakerdeck.com/dongjin>* > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > -- > > > > *Dongjin Lee* > > > > > > > > *A hitchhiker in the mathematical world.* > > > > > > > > > > > > > > > > *github: <http://goog_969573159/>github.com/dongjinleekr > > > > <https://github.com/dongjinleekr>keybase: > > > https://keybase.io/dongjinleekr > > > > <https://keybase.io/dongjinleekr>linkedin: > > > kr.linkedin.com/in/dongjinleekr > > > > <https://kr.linkedin.com/in/dongjinleekr>speakerdeck: > > > > speakerdeck.com/dongjin > > > > <https://speakerdeck.com/dongjin>* > > > > > > > > > > > > > -- > > *Dongjin Lee* > > > > *A hitchhiker in the mathematical world.* > > > > > > > > *github: <http://goog_969573159/>github.com/dongjinleekr > > <https://github.com/dongjinleekr>keybase: > https://keybase.io/dongjinleekr > > <https://keybase.io/dongjinleekr>linkedin: > kr.linkedin.com/in/dongjinleekr > > <https://kr.linkedin.com/in/dongjinleekr>speakerdeck: > speakerdeck.com/dongjin > > <https://speakerdeck.com/dongjin>* >
