[
https://issues.apache.org/jira/browse/CASSANDRA-21487?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sam Lightfoot reassigned CASSANDRA-21487:
-----------------------------------------
Assignee: Sam Lightfoot
> trickle_fsync is silently ignored for compressed SSTables
> ---------------------------------------------------------
>
> Key: CASSANDRA-21487
> URL: https://issues.apache.org/jira/browse/CASSANDRA-21487
> Project: Apache Cassandra
> Issue Type: Bug
> Components: Local/SSTable
> Reporter: Sam Lightfoot
> Assignee: Sam Lightfoot
> Priority: Normal
>
> h2. What's wrong
> {{trickle_fsync: true}} is meant to fsync a large sequential write at
> intervals, so a flush
> trickles its dirty pages to disk instead of dumping them all at once and
> spiking read latency.
> On compressed SSTables it does nothing. Compression is the default, so the
> setting has been silently ignored on the data file of every flush and every
> buffered compaction. No error, no log line - it just doesn't work.
> h2. Why
> The compressed writer throws the setting away.
> {{CompressedSequentialWriter.buildOption()}} rebuilds the writer's options
> and copies only three of them:
> {code:java}
> return SequentialWriterOption.newBuilder()
> .bufferSize(parameters.chunkLength())
>
> .bufferType(parameters.getSstableCompressor().preferredBufferType())
> .finishOnClose(option.finishOnClose()) //
> trickleFsync + interval lost here
> .build();
> {code}
> {{trickleFsync}} and {{trickleFsyncByteInterval}} never make it through, so
> the rebuilt option defaults to {{{}trickleFsync = false{}}}. The trickle
> branch in {{SequentialWriter.doFlush}}
> ({{{}if (option.trickleFsync()) …{}}}) never runs, and no interval fsync ever
> fires.
> {{IOOptions}} and {{BigTableWriter}} set the options correctly; only the
> compressed writer drops them. Commit {{fb221095cb}} (June 2016) moved trickle
> config into the passed-in option but forgot this path. It has been broken
> ever since.
> h2. What still works
> The primary index (plain {{{}SequentialWriter{}}}) and uncompressed data files
> ({{{}ChecksummedSequentialWriter{}}}) pass the option through untouched, so
> trickle works there. But the index is tiny; the data file is where the bytes
> and the writeback storm are.
> h2. Fix
> Pass the two fields through:
> {code:java}
> .trickleFsync(option.trickleFsync())
> .trickleFsyncByteInterval(option.trickleFsyncByteInterval())
> {code}
> The interval counter in {{doFlush}} already tracks uncompressed bytes
> correctly; only the enable flag was missing.
> h2. How it was found
> Found while benchmarking flush write-pacing: {{trickle_fsync}} on and off
> gave an identical dirty window, which led here. Fix this on its own. The
> proposed {{trickle_fsync_mode: writeback }}improvement is separate work that
> adds a third field to propagate through the same method.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
