[
https://issues.apache.org/jira/browse/CASSANDRA-21534?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=18098832#comment-18098832
]
Stefan Miklosovic edited comment on CASSANDRA-21534 at 7/24/26 11:12 AM:
-------------------------------------------------------------------------
https://app.circleci.com/pipelines/gh/instaclustr/cassandra/6606/details?useNewPipelines=true&workflowId=027bb545-37ed-4b04-8d7e-128fe68aa93a&job=564c1fce-88e1-4f0f-a344-a71bbc9d563b&buildNumber=577197&jobType=build#step-103-
multiplex seems one, failed once, I think that is CI infra or some glitch.
all unit tests
https://app.circleci.com/pipelines/gh/instaclustr/cassandra/6606/details?useNewPipelines=true&workflowId=027bb545-37ed-4b04-8d7e-128fe68aa93a&job=39d167e2-a69e-4130-97e7-b10211135c2a&buildNumber=577227&jobType=build#step-103-
was (Author: smiklosovic):
https://app.circleci.com/pipelines/gh/instaclustr/cassandra/6606/details?useNewPipelines=true&workflowId=027bb545-37ed-4b04-8d7e-128fe68aa93a&job=564c1fce-88e1-4f0f-a344-a71bbc9d563b&buildNumber=577197&jobType=build#step-103-
multiplex seems one, failed once, I think that is CI infra or some glitch.
> Reduce disk space usage by CommitLogSegmentManagerCDCTest
> ----------------------------------------------------------
>
> Key: CASSANDRA-21534
> URL: https://issues.apache.org/jira/browse/CASSANDRA-21534
> Project: Apache Cassandra
> Issue Type: Improvement
> Components: Test/unit
> Reporter: Dmitry Konstantinov
> Assignee: Dmitry Konstantinov
> Priority: Normal
> Fix For: 6.x, 7.x
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
> Problem
> -------
> A single run of CommitLogSegmentManagerCDCTest transiently consumes ~9 GB of
> local disk under build/test/cassandra (peak measured on JDK 11) and causes
> periodical pre-ci run failures due to lack of ephemeral storage on nodes.
> Root cause
> ----------
> 1. The test extends CQLTester, whose afterTest() only clears schema-tracking
> state - it does not truncate or drop tables. Each of the 10 test methods
> bulk-writes enough data to fill the CDC space limit, flushing GBs of SSTables
> that accumulate in data/ across the whole suite (nothing reclaims them until
> the suite ends).
> 2. bulkWrite() loops a hard-coded 1000 times. In non-blocking mode the loop is
> not stopped by CDCWriteException, so it writes ~1000 x
> (commitlog_segment_size / 3)
> of mutations per method. With the 5 MiB test segment size that is ~1.7 GB of
> SSTables per non-blocking method.
> Changes (test-only, CommitLogSegmentManagerCDCTest)
> ---------------------------------------------------
> 1. Override afterTest() to reclaim data between methods: truncate the per-test
> keyspace's tables with ColumnFamilyStore.truncateBlockingWithoutSnapshot(),
> then LifecycleTransaction.waitForDeletions(), then super.afterTest(). The
> WithoutSnapshot variant is required - a plain TRUNCATE/DROP hard-links the
> SSTables into snapshots/ under the default auto_snapshot=true and frees
> nothing.
> 2. Shrink the commit log segment to the 1 MiB minimum in @BeforeClass via
> DatabaseDescriptor.setCommitLogSegmentSize(1). Mutation size is segment/3,
> so this reduces per-write data ~5x. Test-local only; production default is
> unchanged.
> 3. Replace the magic 1000 in bulkWrite() with a value derived from config:
> int writes = 2 * (int) (DatabaseDescriptor.getCDCTotalSpace() / mutationSize);
> Blocking mode still stops early on the first CDCWriteException; non-blocking
> mode writes 2x the fill count. This removes the fragile "1000 writes fills
> the CDC space" assumption and stops the small-CDC cases from over-writing.
> 4. Reduce the default CDC total space from 1024 MiB to 100 MiB in
> @BeforeClass.
> Only testSwitchingCDCWriteModes relies on the default, and with the computed
> write count it no longer needs to be large.
> Result
> ------
> Peak build/test/cassandra during the run (JDK 11, all 10 tests passing):
> {code:java}
> Step Peak disk Wall time
> baseline (as-is) 9.15 GB ~95 s
> + afterTest truncate cleanup ~6.5 GB ~92 s
> + 1 MiB commit-log segments 1.03 GB ~35 s
> + computed write count 0.82 GB ~30 s
> + CDC total space 100 MiB 0.29 GB ~26 s
> {code}
> Net: ~31x less peak disk (9.15 GB -> ~292 MB) and ~3.6x faster
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
