Re: Change STCS to TWCS

[wxn...@zjqunshuo.com]

Thanks Jeff. Will have a try using JMX on one node.
 
From: Jeff Jirsa
Date: 2017-12-29 15:57
To: user
Subject: Re: Change STCS to TWCS
It’s going to cause a lot of compactions - this is especially true with stcs 
where many of your sstables (especially the big ones) will overlap and be joined

Monitor free space (and stop compactions as needed), free memory (bloom filters 
during compaction will take a big chunk as you build), and of course cpu and IO 
- compaction touches just about everything 

You can test the operation impact by changing it on just one instance using JMX 
- compaction strategy can be set as a json string and it won’t change the 
cluster wide schema (or persist through reboot).


-- 
Jeff Jirsa


On Dec 28, 2017, at 11:40 PM, "wxn...@zjqunshuo.com" <wxn...@zjqunshuo.com> 
wrote:

Hi All,
My production cluster is running 2.2.8. It is used to store time series data 
with only insertion with TTL, no update and deletion. From the mail lists seems 
TWCS is more suitable than STCS for my use case. I'm thinking about changing 
STCS to TWCS in production. I have read the 
guide(http://thelastpickle.com/blog/2016/12/08/TWCS-part1.html) someone have 
posted.

The cluster info:
UN XX.XX.44.149 939.23 GB 256 25.8% 9180b7c9-fa0b-4bbe-bf62-64a599c01e58 rack1 
UN XX.XX.106.218 995.4 GB 256 26.0% e24d13e2-96cb-4e8c-9d94-22498ad67c85 rack1 
UN XX.XX.42.113 905.85 GB 256 23.8% 385ad28c-0f3f-415f-9e0a-7fe8bef97e17 rack1 
UN XX.XX.41.165 859.85 GB 256 23.1% 46f37f06-9c45-492d-bd25-6fef7f926e38 rack1 
UN XX.XX.106.210 1.15 TB 256 26.8% a31b6088-0cb2-40b4-ac22-aec718dbd035 rack1 
UN XX.XX.104.41 900.21 GB 256 23.6% db08f0d7-d71f-400a-85a6-1f637fa839ee rack1 
UN XX.XX.41.95 960.89 GB 256 26.3% cf80924b-885f-42fb-b8f8-f9e1946ec30a rack1 
UN XX.XX.103.239 919.14 GB 256 24.7% c3f883a8-3643-46a1-ac7a-ea1b1046b400 rack1

I plan to use "alter table" to switch STCS to TWCS in production. My concern is:
1. Does the switch have a big impact on cluster performance?
2. To ensure a smooth switch, what could I pay attention to?

Best Regards,
-Simon

Re: Change STCS to TWCS

[Jeff Jirsa]

It’s going to cause a lot of compactions - this is especially true with stcs 
where many of your sstables (especially the big ones) will overlap and be joined

Monitor free space (and stop compactions as needed), free memory (bloom filters 
during compaction will take a big chunk as you build), and of course cpu and IO 
- compaction touches just about everything 

You can test the operation impact by changing it on just one instance using JMX 
- compaction strategy can be set as a json string and it won’t change the 
cluster wide schema (or persist through reboot).


-- 
Jeff Jirsa


> On Dec 28, 2017, at 11:40 PM, "wxn...@zjqunshuo.com"  
> wrote:
> 
> Hi All,
> My production cluster is running 2.2.8. It is used to store time series data 
> with only insertion with TTL, no update and deletion. From the mail lists 
> seems TWCS is more suitable than STCS for my use case. I'm thinking about 
> changing STCS to TWCS in production. I have read the 
> guide(http://thelastpickle.com/blog/2016/12/08/TWCS-part1.html) someone have 
> posted.
> 
> The cluster info:
> UN  XX.XX.44.149   939.23 GB  25625.8%
> 9180b7c9-fa0b-4bbe-bf62-64a599c01e58  rack1
> UN  XX.XX.106.218  995.4 GB   25626.0%
> e24d13e2-96cb-4e8c-9d94-22498ad67c85  rack1
> UN  XX.XX.42.113   905.85 GB  25623.8%
> 385ad28c-0f3f-415f-9e0a-7fe8bef97e17  rack1
> UN  XX.XX.41.165   859.85 GB  25623.1%
> 46f37f06-9c45-492d-bd25-6fef7f926e38  rack1
> UN  XX.XX.106.210  1.15 TB25626.8%
> a31b6088-0cb2-40b4-ac22-aec718dbd035  rack1
> UN  XX.XX.104.41   900.21 GB  25623.6%
> db08f0d7-d71f-400a-85a6-1f637fa839ee  rack1
> UN  XX.XX.41.95960.89 GB  25626.3%
> cf80924b-885f-42fb-b8f8-f9e1946ec30a  rack1
> UN  XX.XX.103.239  919.14 GB  25624.7%
> c3f883a8-3643-46a1-ac7a-ea1b1046b400  rack1
> 
> I plan to use "alter table" to switch STCS to TWCS in production. My concern 
> is:
> 1. Does the switch have a big impact on cluster performance?
> 2. To ensure a smooth switch, what could I pay attention to?
> 
> Best Regards,
> -Simon