[
https://issues.apache.org/jira/browse/CASSANDRA-21533?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Yuqi Yan updated CASSANDRA-21533:
---------------------------------
Description:
Allow an LCS table to use a different sstable compressor per level. The most
useful case: keep the high-churn L0 tier on a fast codec (LZ4) while deeper
levels retain a high-ratio codec (Zstd). L0 is repeatedly rewritten by
STCS-in-L0 and L0→L1 compactions, so paying full Zstd cost there wastes
compaction CPU for data that is short-lived at that level; the ratio only
matters once data settles into the deeper, stable levels.
*Motivation*
This generalizes the idea already accepted in CASSANDRA-15379
({{{}flush_compression{}}}), which flushes with fast LZ4 and lets compaction
re-compress to the table codec. Today there is no way to keep the churny
compaction tier on a fast codec — the table's single compressor applies to
every level.
*Proposed change*
* New LCS compaction sub-option {{{}per_level_compression{}}}: a JSON object
mapping level → a compression spec, e.g. `per_level_compression = '\{"0":
{"class":"LZ4Compressor"},"1":\{"class":"ZstdCompressor","compression_level":"9"}}.
Levels not listed inherit the table {{{}compression{}}}.
* Each level's value is a full, independent compression spec, specified and
validated exactly like the table {{compression}} option
* Applied on every level-preserving SSTable write keyed on the target level
(compaction, cleanup, scrub, upgradesstables, anticompaction, …), not flush —
flush keeps its own {{flush_compression}} policy.
I'll share a patch for 4.1 targeting LCS and will create a trunk PR targeting
both UCS / LCS. Not very sure how UCS handles the "level" concept if configured
with size tiered like "T4" but I think the idea of using faster compression in
shallow level for "L10" like setup still make sense
was:
Allow an LCS table to use a different sstable compressor per level. The most
useful case: keep the high-churn L0 tier on a fast codec (LZ4) while deeper
levels retain a high-ratio codec (Zstd). L0 is repeatedly rewritten by
STCS-in-L0 and L0→L1 compactions, so paying full Zstd cost there wastes
compaction CPU for data that is short-lived at that level; the ratio only
matters once data settles into the deeper, stable levels.
*Motivation*
This generalizes the idea already accepted in CASSANDRA-15379
({{{}flush_compression{}}}), which flushes with fast LZ4 and lets compaction
re-compress to the table codec. Today there is no way to keep the churny
compaction tier on a fast codec — the table's single compressor applies to
every level.
*Proposed change*
* New LCS compaction sub-option {{{}per_level_compression{}}}: a JSON object
mapping level → a compression spec, e.g. `per_level_compression =
'\{"0":{"class":"LZ4Compressor"},"1":\{"class":"ZstdCompressor","compression_level":"9"}}'`
. Levels not listed inherit the table {{{}compression{}}}.
* Each level's value is a full, independent compression spec, specified and
validated exactly like the table {{compression}} option
* Applied on every level-preserving SSTable write keyed on the target level
(compaction, cleanup, scrub, upgradesstables, anticompaction, …), not flush —
flush keeps its own {{flush_compression}} policy.
I'll share a patch for 4.1 targeting LCS and will create a trunk PR targeting
both UCS / LCS. Not very sure how UCS handles the "level" concept if configured
with size tiered like "T4" but I think the idea of using faster compression in
shallow level for "L10" like setup still make sense
> Configurable compression algorithm for each LCS level
> -----------------------------------------------------
>
> Key: CASSANDRA-21533
> URL: https://issues.apache.org/jira/browse/CASSANDRA-21533
> Project: Apache Cassandra
> Issue Type: Improvement
> Components: Local/Compaction
> Reporter: Yuqi Yan
> Assignee: Yuqi Yan
> Priority: Normal
> Fix For: 4.1.x, trunk
>
>
> Allow an LCS table to use a different sstable compressor per level. The most
> useful case: keep the high-churn L0 tier on a fast codec (LZ4) while deeper
> levels retain a high-ratio codec (Zstd). L0 is repeatedly rewritten by
> STCS-in-L0 and L0→L1 compactions, so paying full Zstd cost there wastes
> compaction CPU for data that is short-lived at that level; the ratio only
> matters once data settles into the deeper, stable levels.
> *Motivation*
> This generalizes the idea already accepted in CASSANDRA-15379
> ({{{}flush_compression{}}}), which flushes with fast LZ4 and lets compaction
> re-compress to the table codec. Today there is no way to keep the churny
> compaction tier on a fast codec — the table's single compressor applies to
> every level.
> *Proposed change*
> * New LCS compaction sub-option {{{}per_level_compression{}}}: a JSON object
> mapping level → a compression spec, e.g. `per_level_compression = '\{"0":
> {"class":"LZ4Compressor"},"1":\{"class":"ZstdCompressor","compression_level":"9"}}.
> Levels not listed inherit the table {{{}compression{}}}.
> * Each level's value is a full, independent compression spec, specified and
> validated exactly like the table {{compression}} option
> * Applied on every level-preserving SSTable write keyed on the target level
> (compaction, cleanup, scrub, upgradesstables, anticompaction, …), not flush —
> flush keeps its own {{flush_compression}} policy.
>
> I'll share a patch for 4.1 targeting LCS and will create a trunk PR targeting
> both UCS / LCS. Not very sure how UCS handles the "level" concept if
> configured with size tiered like "T4" but I think the idea of using faster
> compression in shallow level for "L10" like setup still make sense
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
