RE: TWCS Compactions & Tombstones
Awesome, thanks again! From: Jeff Jirsa [mailto:jji...@gmail.com] Sent: Wednesday, March 27, 2019 1:36 PM To: cassandra Subject: Re: TWCS Compactions & Tombstones You would need to swap your class from the com.jeffjirsa variant (probably from 2.1 / 2.2) to the official TWCS class. Once that happens I suspect it'll happen quite quickly, but I'm not sure. On Wed, Mar 27, 2019 at 7:30 AM Nick Hatfield mailto:nick.hatfi...@metricly.com>> wrote: Awesome, thank you Jeff. Sorry I had not seen this yet. So we have this enabled, I guess it will just take time to finally chew through it all? From: Jeff Jirsa [mailto:jji...@gmail.com<mailto:jji...@gmail.com>] Sent: Tuesday, March 26, 2019 9:41 PM To: user@cassandra.apache.org<mailto:user@cassandra.apache.org> Subject: Re: TWCS Compactions & Tombstones Or Upgrade to a version with https://issues.apache.org/jira/browse/CASSANDRA-13418 and enable that feature -- Jeff Jirsa On Mar 26, 2019, at 6:23 PM, Rahul Singh mailto:rahul.xavier.si...@gmail.com>> wrote: What's your timewindow? Roughly how much data is in each window? If you examine the sstable data and see that is truly old data with little chance that it has any new data, you can just remove the SStables. You can do a rolling restart -- take down a node, remove mc-254400-* and then start it up. rahul.xavier.si...@gmail.com<mailto:rahul.xavier.si...@gmail.com> http://cassandra.link On Tue, Mar 26, 2019 at 8:01 AM Nick Hatfield mailto:nick.hatfi...@metricly.com>> wrote: How does one properly rid of sstables that have fallen victim to overlapping timestamps? I realized that we had TWCS set in our CF which also had a read_repair = 0.1 and after correcting this to 0.0 I can clearly see the affects over time on the new sstables. However, I still have old sstables that date back some time last year, and I need to remove them: Max: 09/05/2018 Min: 09/04/2018 Estimated droppable tombstones: 0.883205790993204613G Mar 26 11:34 mc-254400-big-Data.db What is the best way to do this? This is on a production system so any help would be greatly appreciated. Thanks,
Re: TWCS Compactions & Tombstones
You would need to swap your class from the com.jeffjirsa variant (probably from 2.1 / 2.2) to the official TWCS class. Once that happens I suspect it'll happen quite quickly, but I'm not sure. On Wed, Mar 27, 2019 at 7:30 AM Nick Hatfield wrote: > Awesome, thank you Jeff. Sorry I had not seen this yet. So we have this > enabled, I guess it will just take time to finally chew through it all? > > > > *From:* Jeff Jirsa [mailto:jji...@gmail.com] > *Sent:* Tuesday, March 26, 2019 9:41 PM > *To:* user@cassandra.apache.org > *Subject:* Re: TWCS Compactions & Tombstones > > > > Or Upgrade to a version with > https://issues.apache.org/jira/browse/CASSANDRA-13418 and enable that feature > > > > -- > > Jeff Jirsa > > > > > On Mar 26, 2019, at 6:23 PM, Rahul Singh > wrote: > > What's your timewindow? Roughly how much data is in each window? > > > > If you examine the sstable data and see that is truly old data with little > chance that it has any new data, you can just remove the SStables. You can > do a rolling restart -- take down a node, remove mc-254400-* and then start > it up. > > > > > rahul.xavier.si...@gmail.com > > > > http://cassandra.link > > > > > > > > On Tue, Mar 26, 2019 at 8:01 AM Nick Hatfield > wrote: > > How does one properly rid of sstables that have fallen victim to > overlapping timestamps? I realized that we had TWCS set in our CF which > also had a read_repair = 0.1 and after correcting this to 0.0 I can clearly > see the affects over time on the new sstables. However, I still have old > sstables that date back some time last year, and I need to remove them: > > > > Max: 09/05/2018 Min: 09/04/2018 Estimated droppable tombstones: > 0.883205790993204613G Mar 26 11:34 mc-254400-big-Data.db > > > > > > What is the best way to do this? This is on a production system so any > help would be greatly appreciated. > > > > Thanks, > >
RE: TWCS Compactions & Tombstones
Awesome, thank you Jeff. Sorry I had not seen this yet. So we have this enabled, I guess it will just take time to finally chew through it all? From: Jeff Jirsa [mailto:jji...@gmail.com] Sent: Tuesday, March 26, 2019 9:41 PM To: user@cassandra.apache.org Subject: Re: TWCS Compactions & Tombstones Or Upgrade to a version with https://issues.apache.org/jira/browse/CASSANDRA-13418 and enable that feature -- Jeff Jirsa On Mar 26, 2019, at 6:23 PM, Rahul Singh mailto:rahul.xavier.si...@gmail.com>> wrote: What's your timewindow? Roughly how much data is in each window? If you examine the sstable data and see that is truly old data with little chance that it has any new data, you can just remove the SStables. You can do a rolling restart -- take down a node, remove mc-254400-* and then start it up. rahul.xavier.si...@gmail.com<mailto:rahul.xavier.si...@gmail.com> http://cassandra.link On Tue, Mar 26, 2019 at 8:01 AM Nick Hatfield mailto:nick.hatfi...@metricly.com>> wrote: How does one properly rid of sstables that have fallen victim to overlapping timestamps? I realized that we had TWCS set in our CF which also had a read_repair = 0.1 and after correcting this to 0.0 I can clearly see the affects over time on the new sstables. However, I still have old sstables that date back some time last year, and I need to remove them: Max: 09/05/2018 Min: 09/04/2018 Estimated droppable tombstones: 0.883205790993204613G Mar 26 11:34 mc-254400-big-Data.db What is the best way to do this? This is on a production system so any help would be greatly appreciated. Thanks,
Re: TWCS Compactions & Tombstones
Have you tried enabling 'unchecked_tombstone_compaction' on the affected tables? On Tue, Mar 26, 2019 at 5:01 AM Nick Hatfield wrote: > How does one properly rid of sstables that have fallen victim to > overlapping timestamps? I realized that we had TWCS set in our CF which > also had a read_repair = 0.1 and after correcting this to 0.0 I can clearly > see the affects over time on the new sstables. However, I still have old > sstables that date back some time last year, and I need to remove them: > > Max: 09/05/2018 Min: 09/04/2018 Estimated droppable tombstones: > 0.883205790993204613G Mar 26 11:34 mc-254400-big-Data.db > > > What is the best way to do this? This is on a production system so any > help would be greatly appreciated. > > Thanks, > -- James Brown Engineer
RE: TWCS Compactions & Tombstones
Thanks for the insight, Rahul. We’re using 1 day for the time window.
compaction = {'class':
'com.jeffjirsa.cassandra.db.compaction.TimeWindowCompactionStrategy',
'compaction_window_size': '1',
'compaction_window_unit': 'DAYS',
'max_threshold': '32',
'min_threshold': '4',
'timestamp_resolution': 'MILLISECONDS',
'tombstone_compaction_interval': '86400',
'tombstone_threshold': '0.2',
'unchecked_tombstone_compaction': 'true'}'
AND
default_time_to_live = 7884009
AND
gc_grace_seconds = 86400
AND
read_repair_chance = 0
Whats the best way to examine the sstable data so that I can verify that it is
old data, other than by the min / max timestamps?
Thanks for your help
From: Rahul Singh [mailto:rahul.xavier.si...@gmail.com]
Sent: Tuesday, March 26, 2019 9:24 PM
To: user
Subject: Re: TWCS Compactions & Tombstones
What's your timewindow? Roughly how much data is in each window?
If you examine the sstable data and see that is truly old data with little
chance that it has any new data, you can just remove the SStables. You can do a
rolling restart -- take down a node, remove mc-254400-* and then start it up.
rahul.xavier.si...@gmail.com<mailto:rahul.xavier.si...@gmail.com>
http://cassandra.link
On Tue, Mar 26, 2019 at 8:01 AM Nick Hatfield
mailto:nick.hatfi...@metricly.com>> wrote:
How does one properly rid of sstables that have fallen victim to overlapping
timestamps? I realized that we had TWCS set in our CF which also had a
read_repair = 0.1 and after correcting this to 0.0 I can clearly see the
affects over time on the new sstables. However, I still have old sstables that
date back some time last year, and I need to remove them:
Max: 09/05/2018 Min: 09/04/2018 Estimated droppable tombstones:
0.883205790993204613G Mar 26 11:34 mc-254400-big-Data.db
What is the best way to do this? This is on a production system so any help
would be greatly appreciated.
Thanks,
Re: TWCS Compactions & Tombstones
Or Upgrade to a version with https://issues.apache.org/jira/browse/CASSANDRA-13418 and enable that feature -- Jeff Jirsa > On Mar 26, 2019, at 6:23 PM, Rahul Singh wrote: > > What's your timewindow? Roughly how much data is in each window? > > If you examine the sstable data and see that is truly old data with little > chance that it has any new data, you can just remove the SStables. You can do > a rolling restart -- take down a node, remove mc-254400-* and then start it > up. > > > rahul.xavier.si...@gmail.com > > http://cassandra.link > > > >> On Tue, Mar 26, 2019 at 8:01 AM Nick Hatfield >> wrote: >> How does one properly rid of sstables that have fallen victim to overlapping >> timestamps? I realized that we had TWCS set in our CF which also had a >> read_repair = 0.1 and after correcting this to 0.0 I can clearly see the >> affects over time on the new sstables. However, I still have old sstables >> that date back some time last year, and I need to remove them: >> >> Max: 09/05/2018 Min: 09/04/2018 Estimated droppable tombstones: >> 0.8832057909932046 >> 13G Mar 26 11:34 mc-254400-big-Data.db >> >> >> What is the best way to do this? This is on a production system so any help >> would be greatly appreciated. >> >> Thanks,
Re: TWCS Compactions & Tombstones
What's your timewindow? Roughly how much data is in each window? If you examine the sstable data and see that is truly old data with little chance that it has any new data, you can just remove the SStables. You can do a rolling restart -- take down a node, remove mc-254400-* and then start it up. rahul.xavier.si...@gmail.com http://cassandra.link On Tue, Mar 26, 2019 at 8:01 AM Nick Hatfield wrote: > How does one properly rid of sstables that have fallen victim to > overlapping timestamps? I realized that we had TWCS set in our CF which > also had a read_repair = 0.1 and after correcting this to 0.0 I can clearly > see the affects over time on the new sstables. However, I still have old > sstables that date back some time last year, and I need to remove them: > > Max: 09/05/2018 Min: 09/04/2018 Estimated droppable tombstones: > 0.883205790993204613G Mar 26 11:34 mc-254400-big-Data.db > > > What is the best way to do this? This is on a production system so any > help would be greatly appreciated. > > Thanks, >
TWCS Compactions & Tombstones
How does one properly rid of sstables that have fallen victim to overlapping timestamps? I realized that we had TWCS set in our CF which also had a read_repair = 0.1 and after correcting this to 0.0 I can clearly see the affects over time on the new sstables. However, I still have old sstables that date back some time last year, and I need to remove them: Max: 09/05/2018 Min: 09/04/2018 Estimated droppable tombstones: 0.883205790993204613G Mar 26 11:34 mc-254400-big-Data.db What is the best way to do this? This is on a production system so any help would be greatly appreciated. Thanks,
