Something that I neglected to add originally. - in general I’m a fan of making features / implementations pluggable. It allows us to A/B test different implementations & strategies. Cassandra does this well in some areas already but not all. We should generally define interfaces that are pluggable. This helps keep the door open on experimentation.
Whether zstd works out is not known until we have the framework in place that allows us to plugin zstd and test it out. Dinesh On Tue, Aug 18, 2026 at 9:13 AM Jon Haddad <[email protected]> wrote: > Agreed. > > On Tue, Aug 18, 2026 at 7:39 AM Josh McKenzie <[email protected]> > wrote: > >> +1 to making all locations w/compression configurable, but definitely +1 >> to this. It's a well understood, cleanly defined API boundary so supporting >> multiple algos for different optimization axes should be minimal toil. >> >> On Tue, Aug 18, 2026, at 3:55 AM, Shailaja Koppu wrote: >> >> +1 to making internode compression algo configurable. >> >> >> On Aug 18, 2026, at 8:44 AM, Dinesh Joshi <[email protected]> wrote: >> >> I vaguely recall discussing this a while ago. I'm in favor of this idea. >> >> On Mon, Aug 17, 2026 at 3:04 AM Štefan Miklošovič <[email protected]> >> wrote: >> >> There is a ticket for this (1) and discussion nobody answered to (2). >> >> Is there any reason why we are compressing with lz4 only? For inter-dc >> communication where dc's are in geographically distant areas etc. I >> think that compressing it with zstd instead of lz4 would be an option >> as well as it would, presumably, transfer less data. I know there are >> caveats attached to that like (likely) increased CPU when >> de/compressing with it, especially when zstd compression levels would >> be higher than default etc. but otherwise this is worthy of giving it >> a shot? This is something people would need to opt into, of course, >> but zstd seems like a good compression algo to add for this stuff, >> especially when we declare (3) that we get the best compression ratio >> (better than lz4) at the expense of slightly worse de/compression >> times (which are tunable based on compression levels). If a traffic >> goes from Europe to Asia and it costs dozens of milliseconds then it >> is basically irrelevant if we add a few more while we compress it 30% >> less than lz4, also when inter-continental / inter-cloud traffic is >> paid for based on volume or similar. >> >> I have a working prototype locally where I can configure what the >> internode will be compressed with, I just want to check with ML this >> is something we might eventually consider adding so I can measure the >> performance and similar more deeply. >> >> (1) https://issues.apache.org/jira/browse/CASSANDRA-20488 >> (2) https://lists.apache.org/thread/dzjd3kqtwb8bncsfs36msbfv8v4kgfyy >> (3) >> https://cassandra.apache.org/doc/latest/cassandra/managing/operating/compression.html >> >> >>
