(removing dev) commitlog_segment_size_in_mb isn't going to help, in fact you probably don't want to modify this as it'll reduce the maximum size of your mutations. Reducing the total space on its own will help, however definitely test this as such a large drop could result in a massive increase in SSTables and thus compaction overhead. You'll in general want to look into any property that makes memtables flush more frequently (which is based on heap size and some tuning properties in cassandra.yaml).
I'm typically not a fan of using a database as a streaming/workflow service, so I have to ask have you considered managing this from your clients rather than using CDC in C*? raft.so - Cassandra consulting, support, and managed services On Tue, May 4, 2021 at 4:16 AM Bingqin Zhou <bingq...@wepay.com> wrote: > Hi, > > We're working with the CDC feature to develop an agent to stream changes > in Cassandra DB into Kafka. However, the CDC feature doesn't work well for > us so far because CommitLog files are rarely flushed into cdc_raw > directory, and the frequency can be as low as a few months. > > Is there any suggested and feasible way to increase the frequency for > Cassandra to flush CommitLog files please? > > We're thinking about decreasing *commitlog_segment_size_in_mb* from 32 to > 16, and decreasing *commitlog_total_space_in_mb* from 8192 to 160. Does > this sound like a reasonable approach? Is there any concern or anything we > need to be warned about trying this please? > > Thank you! > > Bingqin Zhou >
