Re: ALTER default_time_to_live

[Alain RODRIGUEZ]

Hello Vlad,

I don't remember the context of the sentences, but I see no
incompatibilities. I believe the misunderstanding comes from the word
'compaction' that is a bit large. In the first sentence it is about
tombstone compactions. in sentence two it is about standard, regular
compaction according to the chosen compaction strategy. Both are slightly
different things:

A compaction strategy can be done compacting data like in those buckets you
mentioned but still do the tombstones compactions. Tombstone compactions
are done specially for tombstone eviction. Historically they are involving
only one sstable (in recent versions can be multiple SSTables), they are
ran with a lower priority than normal compactions and are not disabled
unless the 'tombstone_threshold' is set to 1 (subproperty of compaction,
table level).

And expiring tables does not involve any compactions.

I hope this is more clear now.

C*heers,
---
Alain Rodriguez - @arodream - al...@thelastpickle.com
France / Spain

The Last Pickle - Apache Cassandra Consulting
http://www.thelastpickle.com


2018-01-18 19:21 GMT+00:00 Vlad :

> Hi, thanks for answer!
>
> I've read article about TWCS, and I don't understand how claim
>
> "When rows reach their TTL (10 minutes here), they turn into tombstones.
> Our table defines that tombstones can be purged 1 minute after they were
> created. If all rows are created with the same TTL, SSTables will get 100%
> droppable tombstones eventually and perform full SSTable deletion instead
> of purging tombstones through compaction."
>
> goes with
> "Once the major compaction for a time window is completed, no further
> compaction of the data will ever occur."
>
> In above example TTL is 10 minutes, but time window is only one. As far as
> I understand C* never compacts passed bucket. Does it check tombstones
> anyway?
>
>
> On Thursday, January 18, 2018 1:32 PM, Alain RODRIGUEZ 
> wrote:
>
>
> I set  default_time_to_live for existing table. Does it affect existing
> data?
>
>
> No, it sets a default TTL for the future writes (that is no guarantee, as
> it can be overwritten in any specific query).
>
> It seems data to be deleted, but after compaction, I don't see any disk
> space freed as expected
>
>
> Indeed tombstones are responsible for tombstones eviction, yet there are
> some conditions to respect to be able to remove the tombstones (for
> consistency reasons). I detailed this last year, and even though the
> content is a bit old, main principles are still true and the tuning options
> are still relevant.
>
> *About deletes and tombstones: *http://thelastpickle.com/blog/2016/
> 07/27/about-deletes-and-tombstones.html
>
> *tl;dr: *I would give a try to *unchecked_tombstone_compaction: true*.
> Maybe also consider using TWCS because of this "TTL is also ten days on
> one table and 100 days on other.". But I really recommend you to
> understand how this all work to act wisely. My guess can be wrong.
>
> *About TWCS*: http://thelastpickle.com/blog/2016/12/08/TWCS-part1.html
>
> C*heers,
> ---
> Alain Rodriguez - @arodream - al...@thelastpickle.com
> France / Spain
>
> The Last Pickle - Apache Cassandra Consulting
> http://www.thelastpickle.com
>
> 2018-01-18 11:15 GMT+00:00 Vlad :
>
> Hi,
>
> I set  default_time_to_live for existing table. Does it affect existing
> data? It seems data to be deleted, but after compaction, I don't see any
> disk space freed as expected. Database has data for almost year, GC time is
> ten days, and TTL is also ten days on one table and 100 days on other.
>
>  Cassandra version 3.11.0
>
> Thanks.
>
>
>
>
>

Re: ALTER default_time_to_live

[Vlad]

Hi, thanks for answer!
I've read article about TWCS, and I don't understand how claim 

"When rows reach their TTL (10 minutes here), they turn into tombstones. Our 
table defines that tombstones can be purged 1 minute after they were created.If 
all rows are created with the same TTL, SSTables will get 100% droppable 
tombstones eventually and perform full SSTable deletion instead of purging 
tombstones through compaction."

goes with 
"Once the major compaction for a time window is completed, no further 
compaction of the data will ever occur."

In above example TTL is 10 minutes, but time window is only one. As far as I 
understand C* never compacts passed bucket. Does it check tombstones anyway? 

On Thursday, January 18, 2018 1:32 PM, Alain RODRIGUEZ  
wrote:
 

 
I set  default_time_to_live for existing table. Does it affect existing data?

No, it sets a default TTL for the future writes (that is no guarantee, as it 
can be overwritten in any specific query).

It seems data to be deleted, but after compaction, I don't see any disk space 
freed as expected

Indeed tombstones are responsible for tombstones eviction, yet there are some 
conditions to respect to be able to remove the tombstones (for consistency 
reasons). I detailed this last year, and even though the content is a bit old, 
main principles are still true and the tuning options are still relevant.
About deletes and tombstones: 
http://thelastpickle.com/blog/2016/07/27/about-deletes-and-tombstones.html
tl;dr: I would give a try to unchecked_tombstone_compaction: true. Maybe also 
consider using TWCS because of this "TTL is also ten days on one table and 100 
days on other.". But I really recommend you to understand how this all work to 
act wisely. My guess can be wrong.
About TWCS: http://thelastpickle.com/blog/2016/12/08/TWCS-part1.html
C*heers,---Alain Rodriguez - @arodream - 
alain@thelastpickle.comFrance / Spain
The Last Pickle - Apache Cassandra Consultinghttp://www.thelastpickle.com
2018-01-18 11:15 GMT+00:00 Vlad :

Hi,
I set  default_time_to_live for existing table. Does it affect existing data? 
It seems data to be deleted, but after compaction, I don't see any disk space 
freed as expected. Database has data for almost year, GC time is ten days, and 
TTL is also ten days on one table and 100 days on other.
 Cassandra version 3.11.0

Thanks.




   

Re: ALTER default_time_to_live

[Alain RODRIGUEZ]

>
> I set  default_time_to_live for existing table. Does it affect existing
> data?


No, it sets a default TTL for the future writes (that is no guarantee, as
it can be overwritten in any specific query).

It seems data to be deleted, but after compaction, I don't see any disk
> space freed as expected


Indeed tombstones are responsible for tombstones eviction, yet there are
some conditions to respect to be able to remove the tombstones (for
consistency reasons). I detailed this last year, and even though the
content is a bit old, main principles are still true and the tuning options
are still relevant.

*About deletes and tombstones: *
http://thelastpickle.com/blog/2016/07/27/about-deletes-and-tombstones.html

*tl;dr: *I would give a try to *unchecked_tombstone_compaction: true*.
Maybe also consider using TWCS because of this "TTL is also ten days on one
table and 100 days on other.". But I really recommend you to understand how
this all work to act wisely. My guess can be wrong.

*About TWCS*: http://thelastpickle.com/blog/2016/12/08/TWCS-part1.html

C*heers,
---
Alain Rodriguez - @arodream - al...@thelastpickle.com
France / Spain

The Last Pickle - Apache Cassandra Consulting
http://www.thelastpickle.com

2018-01-18 11:15 GMT+00:00 Vlad :

> Hi,
>
> I set  default_time_to_live for existing table. Does it affect existing
> data? It seems data to be deleted, but after compaction, I don't see any
> disk space freed as expected. Database has data for almost year, GC time is
> ten days, and TTL is also ten days on one table and 100 days on other.
>
>  Cassandra version 3.11.0
>
> Thanks.
>