[
https://issues.apache.org/jira/browse/CASSANDRA-21487?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sam Lightfoot updated CASSANDRA-21487:
--------------------------------------
Complexity: Low Hanging Fruit
Fix Version/s: 5.0.x
6.0.x
Severity: Low
> trickle_fsync is silently ignored for compressed SSTables
> ---------------------------------------------------------
>
> Key: CASSANDRA-21487
> URL: https://issues.apache.org/jira/browse/CASSANDRA-21487
> Project: Apache Cassandra
> Issue Type: Bug
> Components: Local/SSTable
> Reporter: Sam Lightfoot
> Assignee: Sam Lightfoot
> Priority: Low
> Fix For: 5.0.x, 6.0.x
>
>
> h2. What's wrong
> {{trickle_fsync: true}} is meant to fsync a large sequential write at
> intervals, so a flush trickles its dirty pages to disk instead of dumping
> them all at once and spiking read latency. On compressed SSTables it does
> nothing. Compression is the default, so the setting has been silently ignored
> on the data file of every flush and every buffered compaction.
> h2. Why
> The compressed writer throws the setting away.
> {{CompressedSequentialWriter.buildOption()}} rebuilds the writer's options
> and copies only three of them:
> {code:java}
> return SequentialWriterOption.newBuilder()
> .bufferSize(parameters.chunkLength())
>
> .bufferType(parameters.getSstableCompressor().preferredBufferType())
> .finishOnClose(option.finishOnClose()) //
> trickleFsync + interval lost here
> .build();
> {code}
> {{trickleFsync}} and {{trickleFsyncByteInterval}} never make it through, so
> the rebuilt option defaults to {{{}trickleFsync = false{}}}. The trickle
> branch in {{SequentialWriter.doFlush}}
> ({{{}if (option.trickleFsync()) …{}}}) never runs, and no interval fsync ever
> fires.
> {{IOOptions}} and {{BigTableWriter}} set the options correctly; only the
> compressed writer drops them. Commit {{fb221095cb}} (June 2016) moved trickle
> config into the passed-in option but forgot this path. It has been broken
> since.
> h2. What still works
> The primary index (plain {{{}SequentialWriter{}}}) and uncompressed data
> files ({{{}ChecksummedSequentialWriter{}}}) pass the option through
> untouched, so trickle works there. But the index is tiny; the data file is
> where the bytes and the writeback storm are.
> h2. Fix
> Pass the two fields through:
> {code:java}
> .trickleFsync(option.trickleFsync())
> .trickleFsyncByteInterval(option.trickleFsyncByteInterval())
> {code}
> The interval counter in {{doFlush}} already tracks uncompressed bytes
> correctly; only the enable flag was missing.
> h2. How it was found
> Found while benchmarking flush write-pacing: {{trickle_fsync}} on and off
> gave an identical dirty window.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
