[
https://issues.apache.org/jira/browse/CASSANDRA-21423?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=18095753#comment-18095753
]
Isaac Philip Liu commented on CASSANDRA-21423:
----------------------------------------------
Picking this up.
RCA: AbstractSSTableSimpleWriter.createWriter() hardcodes partition count of 0
when constructing SSTableTxnWriter, since neither SSTableSimpleWriter or
SSTableSimpleUnsortedWriter can know the true partition count upfront. The
produced filter is sized for 0 elements + BITSET_EXCESS, producing ~16 byte
Filter.db files.
Since filter is present but broken, Cassandra's existing load=time filter
rebuilding path never is triggered.
Initial patch: [https://github.com/apache/cassandra/pull/4929]
Approach: adds SSTableReaderLoadingBuilder.rebuildFilter(), reuses existing
key-reading infra (handles for Big and BTI) to rebuild a correctly-sized filter
from the finished SSTable
Sizing is done via totalRows in Statistics.db, which upper bounds partition
count. Filter is repopulated from on-disk primary index.
AbstractSSTableSimpleWriter is modified to call this before notifying
sstableProducedListener.
Since finish(openResult) could have opened a reader holding an in-mem copy of
the original filter, this patch also swaps the corrected filter into any
already-opened reader via existing cloneAndReplace methods so the fix is
available for openSSTableOnProduced callers.
Rebuild failure is handled by deleting the partial Filter.db so a node can
rebuild it safely on import.
Tests cover both SSTable formats and disabled-filter case. Happy to adjust the
apporach if theres any cases I'm missing, curious to hear what maintainers
think about this approach.
> CQLSSTableWriter produces empty bloom filter
> --------------------------------------------
>
> Key: CASSANDRA-21423
> URL: https://issues.apache.org/jira/browse/CASSANDRA-21423
> Project: Apache Cassandra
> Issue Type: Improvement
> Reporter: Lukasz Antoniak
> Assignee: Isaac Philip Liu
> Priority: Normal
> Time Spent: 10m
> Remaining Estimate: 0h
>
> CQLSSTableWriter produces empty Filter.db files when flushing SSTables. This
> causes Cassandra nodes to skip Bloom filter checks on imported SSTables,
> resulting in unnecessary disk reads for every partition key lookup.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
