liuisaac opened a new pull request, #4929: URL: https://github.com/apache/cassandra/pull/4929
Issue: [CASSANDRA-21423](https://issues.apache.org/jira/browse/CASSANDRA-21423) ### Purpose `CQLSSTableWriter` does not know the key size on `INSERT` and thus produces near-empty Filter.db files. This saturates every flushed SSTable's Bloom Filter and recalculation never gets triggered, causing unnecessary disk reads for every partition key lookup to tables written via `CQLSSTableWriter`. ### Context `CQLSSTableWriter` allows callers to generate SSTable entries outside of a running Cassandra process. `CQLSSTableWriter` initializes an `AbstractSSTableSimpleWriter` writer that gets instantiated to either `SSTableSimpleWriter` or an `SSTableSimpleUnsortedWriter`. Either writer implementation calls `AbstractSSTableSimpleWriter.createWriter()` which hardcodes a partition count of 0, as neither path knows the true key count. ### Problem Hardcoded 0 flows into `FilterFactory.getFilter(0, fpChance)`, which allocates a filter of 0 elements + `BITSET_EXCESS` (20 bits). Resulting Filter.db is [~16 bytes in size](https://issues.apache.org/jira/browse/CASSANALYTICS-167). Effectively every bloom filter check will fall into a disk read. ### Solution Add `SSTableReaderLoadingBuilder.rebuildFilter()`, which pulls `totalRows` (upper bound of partition count) from Statistics.db, then walk the on-disk primary index to repopulate the filter with every key, and overwrites Filter.db. `AbstractSSTableSimpleWriter` implementations call `rebuildFilter()` after each SSTable is produced before notifying `sstableProducedListener`. Because `finish(openResult)` may already have opened an `SSTableReader` holding a shared in-memory copy of the original (broken) filter, copied in via `getFilterCopy()` before this method runs, independent of what gets written to disk afterward, the fix also swaps the corrected filter into any already-opened readers via `cloneAndReplace`, releasing the original. Skips entirely when bloom filtering is disabled for the table. On rebuild failure, failed Filter.db is deleted so a node will safely build on load instead. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
