[
https://issues.apache.org/jira/browse/CASSANDRA-12366?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15404623#comment-15404623
]
T Jake Luciani commented on CASSANDRA-12366:
--------------------------------------------
[branch|https://github.com/tjake/cassandra/tree/fix-compaction-ratelimit]
[testall|http://cassci.datastax.com/view/Dev/view/tjake/job/tjake-fix-compaction-ratelimit-testall/]
[dtest|http://cassci.datastax.com/view/Dev/view/tjake/job/tjake-fix-compaction-ratelimit-dtest/]
To show the fix:
I set a compaction limit of 8mb/sec
Before fix:
{code}
459.971MiB to 458.545MiB (~99% of original) in 99,556ms. Read Throughput =
4.620MiB/s, Write Throughput = 4.606MiB/s, Row Throughput = ~298,272/s. 30,083
total partitions merged to 30,078. Partition merge counts were {1:30073, 2:5, }
{code}
Branch fix:
{code}
459.971MiB to 458.545MiB (~99% of original) in 57,124ms. Read Throughput =
8.052MiB/s, Write Throughput = 8.027MiB/s, Row Throughput = ~514,263/s. 30,083
total partitions merged to 30,078. Partition merge counts were {1:30073, 2:5, }
{code}
The output of compactionstats and CompactionInfo is still showing the
uncompressed data size but I see no simple way to fetching the actual on disk
pointer after CASSANDRA-5863
> Fix compaction throttle
> -----------------------
>
> Key: CASSANDRA-12366
> URL: https://issues.apache.org/jira/browse/CASSANDRA-12366
> Project: Cassandra
> Issue Type: Bug
> Components: Compaction
> Reporter: T Jake Luciani
> Assignee: T Jake Luciani
> Fix For: 3.x
>
>
> Compaction throttling is broken in the following ways:
> * It throttles bytes read after being decompressed
> * Compaction creates multiple scanners which share the rate limiter causing
> too much throttling
> * It bears no resemblance to the reported compaction time remaining
> calculation (Bytes of source sstables processed since start of compaction)
> To fix this we need to simplify the throttling to be only at the
> CompactionIterator level.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
