[
https://issues.apache.org/jira/browse/CASSANDRA-21550?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Chris Lohfink updated CASSANDRA-21550:
--------------------------------------
Description:
## Context
This is phase 1 of
[CASSANDRA-21550](https://issues.apache.org/jira/browse/CASSANDRA-21550).
Anticompaction and range streaming currently redo work whose result already
exists on disk. They deserialize, serialize, and recompress rows even when the
required data is a contiguous run of the parent SSTable’s compression chunks.
CASSANDRA-21550 introduces machinery to retain those compressed chunks
verbatim—using extent sharing where available—and rebuild the remaining
components from index-only passes.
## Phase 1 scope
This PR deliberately limits the first phase to compressed BIG-format SSTables
and exposes the machinery through the opt-in:
```text
sstablesplit --zero-copy
```
Starting with `sstablesplit` provides a bounded, offline caller for validating
the new on-disk representation and splitting machinery before integrating it
into anticompaction and streaming.
This phase:
- Supports compressed BIG `pa`-or-later SSTables and writes `pb` children.
- Copies existing compression chunks without deserializing or recompressing
rows.
- Uses byte-range reflinks on supported Linux filesystems to share extents.
- Falls back to copying compressed bytes when reflinks are unavailable.
- Rebuilds indexes, summaries, filters, statistics, digests, and other child
components.
- Persists the first indexed partition position so readers can safely skip
retained compression-chunk prefixes.
- Updates loading, scanning, streaming, scrub, verification, and upgrade paths
to understand split-prefix SSTables.
- Requires `storage_compatibility_mode: NONE`.
- Rejects uncompressed SSTables, unsupported components, and SSTable-attached
indexes.
The requested split size remains a maximum unless one partition’s
compression-chunk span exceeds it. Compression chunks crossing a split boundary
may be retained by both adjacent children.
## Follow-up phases
The intended sequence for CASSANDRA-21550 is:
1. **BIG zero-copy splitting** — this PR.
2. Add the equivalent splitting implementation for **BTI**.
3. Add support for SSTables with **SAI and legacy secondary indexes (2i)**.
4. Use the machinery for **zero-copy anticompaction**, initially disabled by
default.
5. Use it to make **range streaming zero-copy**, also initially disabled by
default.
Anticompaction and general range streaming integration are intentionally
outside first PR.
was:
## Context
This is phase 1 of
[CASSANDRA-21550](https://issues.apache.org/jira/browse/CASSANDRA-21550).
Anticompaction and range streaming currently redo work whose result already
exists on disk. They deserialize, serialize, and recompress rows even when the
required data is a contiguous run of the parent SSTable’s compression chunks.
CASSANDRA-21550 introduces machinery to retain those compressed chunks
verbatim—using extent sharing where available—and rebuild the remaining
components from index-only passes.
> Zero-copy sstable splitting for anticompaction, and partial-sstable zero-copy
> streaming
> ---------------------------------------------------------------------------------------
>
> Key: CASSANDRA-21550
> URL: https://issues.apache.org/jira/browse/CASSANDRA-21550
> Project: Apache Cassandra
> Issue Type: New Feature
> Reporter: Chris Lohfink
> Priority: Normal
> Time Spent: 0.5h
> Remaining Estimate: 0h
>
> ## Context
> This is phase 1 of
> [CASSANDRA-21550](https://issues.apache.org/jira/browse/CASSANDRA-21550).
> Anticompaction and range streaming currently redo work whose result already
> exists on disk. They deserialize, serialize, and recompress rows even when
> the required data is a contiguous run of the parent SSTable’s compression
> chunks.
> CASSANDRA-21550 introduces machinery to retain those compressed chunks
> verbatim—using extent sharing where available—and rebuild the remaining
> components from index-only passes.
> ## Phase 1 scope
> This PR deliberately limits the first phase to compressed BIG-format SSTables
> and exposes the machinery through the opt-in:
> ```text
> sstablesplit --zero-copy
> ```
> Starting with `sstablesplit` provides a bounded, offline caller for
> validating the new on-disk representation and splitting machinery before
> integrating it into anticompaction and streaming.
> This phase:
> - Supports compressed BIG `pa`-or-later SSTables and writes `pb` children.
> - Copies existing compression chunks without deserializing or recompressing
> rows.
> - Uses byte-range reflinks on supported Linux filesystems to share extents.
> - Falls back to copying compressed bytes when reflinks are unavailable.
> - Rebuilds indexes, summaries, filters, statistics, digests, and other child
> components.
> - Persists the first indexed partition position so readers can safely skip
> retained compression-chunk prefixes.
> - Updates loading, scanning, streaming, scrub, verification, and upgrade
> paths to understand split-prefix SSTables.
> - Requires `storage_compatibility_mode: NONE`.
> - Rejects uncompressed SSTables, unsupported components, and SSTable-attached
> indexes.
> The requested split size remains a maximum unless one partition’s
> compression-chunk span exceeds it. Compression chunks crossing a split
> boundary may be retained by both adjacent children.
> ## Follow-up phases
> The intended sequence for CASSANDRA-21550 is:
> 1. **BIG zero-copy splitting** — this PR.
> 2. Add the equivalent splitting implementation for **BTI**.
> 3. Add support for SSTables with **SAI and legacy secondary indexes (2i)**.
> 4. Use the machinery for **zero-copy anticompaction**, initially disabled by
> default.
> 5. Use it to make **range streaming zero-copy**, also initially disabled by
> default.
> Anticompaction and general range streaming integration are intentionally
> outside first PR.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
