[
https://issues.apache.org/jira/browse/CASSANDRA-21487?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=18094028#comment-18094028
]
Sam Lightfoot edited comment on CASSANDRA-21487 at 7/6/26 9:20 PM:
-------------------------------------------------------------------
Slightly unrelated but interesting nevertheless. Read latency p99 impact of
enabling trickle_fsync with the default 10MB flush windows, showing significant
dampening via reduction of max disk throughput during flush window. Enalbement
complements omission of memtables from direct I/O path to benefit from page
cache population.
The 'writeback' mode is not in trunk. It syncs a file range without the durable
write barrier that fsync imposes (possible as we fsync with barrier after the
SSTable has been fully written).
!image-2026-07-06-22-19-56-956.png|width=870,height=602!
was (Author: JIRAUSER302824):
Slightly unrelated but interesting nevertheless. Read latency p99 impact of
enabling trickle_fsync with the default 10MB flush windows, showing significant
dampening via reduction of max disk throughput during flush window. Enalbement
complements omission of memtables from direct I/O path to benefit from page
cache population.
The 'writeback' mode is not in trunk. It syncs a file range without the durable
write barrier that fsync imposes (possible as we fsync with barrier after the
SSTable has been fully written).
!image-2026-07-06-15-28-25-963.png|width=590,height=585!
> trickle_fsync is silently ignored for compressed SSTables
> ---------------------------------------------------------
>
> Key: CASSANDRA-21487
> URL: https://issues.apache.org/jira/browse/CASSANDRA-21487
> Project: Apache Cassandra
> Issue Type: Bug
> Components: Local/SSTable
> Reporter: Sam Lightfoot
> Assignee: Sam Lightfoot
> Priority: Low
> Fix For: 5.0.x, 6.0.x
>
> Attachments: image-2026-07-06-15-28-25-963.png,
> image-2026-07-06-22-19-56-956.png
>
>
> h2. What's wrong
> {{trickle_fsync: true}} is meant to fsync a large sequential write at
> intervals, so a flush trickles its dirty pages to disk instead of dumping
> them all at once and spiking read latency. On compressed SSTables it does
> nothing. Compression is the default, so the setting has been silently ignored
> on the data file of every flush and every buffered compaction.
> h2. Why
> The compressed writer throws the setting away.
> {{CompressedSequentialWriter.buildOption()}} rebuilds the writer's options
> and copies only three of them:
> {code:java}
> return SequentialWriterOption.newBuilder()
> .bufferSize(parameters.chunkLength())
>
> .bufferType(parameters.getSstableCompressor().preferredBufferType())
> .finishOnClose(option.finishOnClose()) //
> trickleFsync + interval lost here
> .build();
> {code}
> {{trickleFsync}} and {{trickleFsyncByteInterval}} never make it through, so
> the rebuilt option defaults to {{{}trickleFsync = false{}}}. The trickle
> branch in {{SequentialWriter.doFlush}}
> ({{{}if (option.trickleFsync()) …{}}}) never runs, and no interval fsync ever
> fires.
> {{IOOptions}} and {{BigTableWriter}} set the options correctly; only the
> compressed writer drops them. Commit {{fb221095cb}} (June 2016) moved trickle
> config into the passed-in option but forgot this path. It has been broken
> since.
> h2. What still works
> The primary index (plain {{{}SequentialWriter{}}}) and uncompressed data
> files ({{{}ChecksummedSequentialWriter{}}}) pass the option through
> untouched, so trickle works there. But the index is tiny; the data file is
> where the bytes and the writeback storm are.
> h2. Fix
> Pass the two fields through:
> {code:java}
> .trickleFsync(option.trickleFsync())
> .trickleFsyncByteInterval(option.trickleFsyncByteInterval())
> {code}
> The interval counter in {{doFlush}} already tracks uncompressed bytes
> correctly; only the enable flag was missing.
> h2. How it was found
> Found while benchmarking flush write-pacing: {{trickle_fsync}} on and off
> gave an identical dirty window.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
