Re: compaction_throughput: Difference between 0 (unthrottled) and large value

2018-06-13 Thread Joshua Galbraith
Thomas,

This post from Ryan Svihla has a few notes in it that may or may not be
useful to you:

>If you read the original throttling Jira you can see that there is a hurry
up and wait component to unthrottled compaction (CASSANDRA-2156- Compaction
Throttling). Ultimately you will saturate your IO in bursts, backing up
other processes and making different bottlenecks spike up a long the way,
potentially causing something OTHER than compaction to get so far behind
that the server becomes unresponsive (such as GC).

via https://medium.com/@foundev/how-i-tune-cassandra-compaction-7c16fb0b1d99

On Mon, Jun 11, 2018 at 12:05 AM, Steinmaurer, Thomas <
thomas.steinmau...@dynatrace.com> wrote:

> Sorry, should have first looked at the source code. In case of 0, it is
> set to Double.MAX_VALUE.
>
>
>
> Thomas
>
>
>
> *From:* Steinmaurer, Thomas [mailto:thomas.steinmau...@dynatrace.com]
> *Sent:* Montag, 11. Juni 2018 08:53
> *To:* user@cassandra.apache.org
> *Subject:* compaction_throughput: Difference between 0 (unthrottled) and
> large value
>
>
>
> Hello,
>
>
>
> on a 3 node loadtest cluster with very capable machines (32 physical
> cores, 512G RAM, 20T storage (26 disk RAID)), I’m trying to max out
> compaction, thus currently testing with:
>
>
>
> concurrent_compactors: 16
>
> compaction_throughput_mb_per_sec: 0
>
>
>
> With our simulated incoming load + compaction etc., the Linux volume shows
> ~ 20 Mbyte/s Read IO + 50 Mbyte/s Write IO in AVG, constantly.
>
>
>
>
>
> Setting throughput to 0 should mean unthrottled, right? Is this really
> unthrottled from a throughput perspective and then is basically limited by
> disk capabilities only? Or should it be better set to a very high value
> instead of 0. Is there any semantical difference here?
>
>
>
>
>
> Thanks,
>
> Thomas
>
>
>
> The contents of this e-mail are intended for the named addressee only. It
> contains information that may be confidential. Unless you are the named
> addressee or an authorized designee, you may not copy or use it, or
> disclose it to anyone else. If you received it in error please notify us
> immediately and then destroy it. Dynatrace Austria GmbH (registration
> number FN 91482h) is a company registered in Linz whose registered office
> is at 4040 Linz, Austria, Freistädterstraße 313
> The contents of this e-mail are intended for the named addressee only. It
> contains information that may be confidential. Unless you are the named
> addressee or an authorized designee, you may not copy or use it, or
> disclose it to anyone else. If you received it in error please notify us
> immediately and then destroy it. Dynatrace Austria GmbH (registration
> number FN 91482h) is a company registered in Linz whose registered office
> is at 4040 Linz, Austria, Freistädterstraße 313
>



-- 
*Joshua Galbraith *| Senior Software Engineer | New Relic
C: 907-209-1208 | jgalbra...@newrelic.com


RE: compaction_throughput: Difference between 0 (unthrottled) and large value

2018-06-11 Thread Steinmaurer, Thomas
Sorry, should have first looked at the source code. In case of 0, it is set to 
Double.MAX_VALUE.

Thomas

From: Steinmaurer, Thomas [mailto:thomas.steinmau...@dynatrace.com]
Sent: Montag, 11. Juni 2018 08:53
To: user@cassandra.apache.org
Subject: compaction_throughput: Difference between 0 (unthrottled) and large 
value

Hello,

on a 3 node loadtest cluster with very capable machines (32 physical cores, 
512G RAM, 20T storage (26 disk RAID)), I'm trying to max out compaction, thus 
currently testing with:

concurrent_compactors: 16
compaction_throughput_mb_per_sec: 0

With our simulated incoming load + compaction etc., the Linux volume shows ~ 20 
Mbyte/s Read IO + 50 Mbyte/s Write IO in AVG, constantly.


Setting throughput to 0 should mean unthrottled, right? Is this really 
unthrottled from a throughput perspective and then is basically limited by disk 
capabilities only? Or should it be better set to a very high value instead of 
0. Is there any semantical difference here?


Thanks,
Thomas

The contents of this e-mail are intended for the named addressee only. It 
contains information that may be confidential. Unless you are the named 
addressee or an authorized designee, you may not copy or use it, or disclose it 
to anyone else. If you received it in error please notify us immediately and 
then destroy it. Dynatrace Austria GmbH (registration number FN 91482h) is a 
company registered in Linz whose registered office is at 4040 Linz, Austria, 
Freistädterstraße 313
The contents of this e-mail are intended for the named addressee only. It 
contains information that may be confidential. Unless you are the named 
addressee or an authorized designee, you may not copy or use it, or disclose it 
to anyone else. If you received it in error please notify us immediately and 
then destroy it. Dynatrace Austria GmbH (registration number FN 91482h) is a 
company registered in Linz whose registered office is at 4040 Linz, Austria, 
Freistädterstraße 313