Sam Lightfoot created CASSANDRA-21487:
-----------------------------------------
Summary: trickle_fsync is silently ignored for compressed SSTables
Key: CASSANDRA-21487
URL: https://issues.apache.org/jira/browse/CASSANDRA-21487
Project: Apache Cassandra
Issue Type: Bug
Components: Local/SSTable
Reporter: Sam Lightfoot
h2. What's wrong
{{trickle_fsync: true}} is meant to fsync a large sequential write at
intervals, so a flush
trickles its dirty pages to disk instead of dumping them all at once and
spiking read latency.
On compressed SSTables it does nothing. Compression is the default, so the
setting has been silently ignored on the data file of every flush and every
buffered compaction. No error, no log line - it just doesn't work.
h2. Why
The compressed writer throws the setting away.
{{CompressedSequentialWriter.buildOption()}} rebuilds the writer's options and
copies only three of them:
{code:java}
return SequentialWriterOption.newBuilder()
.bufferSize(parameters.chunkLength())
.bufferType(parameters.getSstableCompressor().preferredBufferType())
.finishOnClose(option.finishOnClose()) //
trickleFsync + interval lost here
.build();
{code}
{{trickleFsync}} and {{trickleFsyncByteInterval}} never make it through, so the
rebuilt option defaults to {{{}trickleFsync = false{}}}. The trickle branch in
{{SequentialWriter.doFlush}}
({{{}if (option.trickleFsync()) …{}}}) never runs, and no interval fsync ever
fires.
{{IOOptions}} and {{BigTableWriter}} set the options correctly; only the
compressed writer drops them. Commit {{fb221095cb}} (June 2016) moved trickle
config into the passed-in option but forgot this path. It has been broken ever
since.
h2. What still works
The primary index (plain {{{}SequentialWriter{}}}) and uncompressed data files
({{{}ChecksummedSequentialWriter{}}}) pass the option through untouched, so
trickle works there. But the index is tiny; the data file is where the bytes
and the writeback storm are.
h2. Fix
Pass the two fields through:
{code:java}
.trickleFsync(option.trickleFsync())
.trickleFsyncByteInterval(option.trickleFsyncByteInterval())
{code}
The interval counter in {{doFlush}} already tracks uncompressed bytes
correctly; only the enable flag was missing.
h2. How it was found
Found while benchmarking flush write-pacing: {{trickle_fsync}} on and off gave
an identical dirty window, which led here. Fix this on its own. The proposed
{{trickle_fsync_mode: writeback }}improvement is separate work that adds a
third field to propagate through the same method.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
