Re: Compaction does not remove tombstones if column has higher TTL

[Ken Hancock]

I agree, that's totally unintuitive.  I would have the same expectations
that compaction is done on a row/column pair instead of simply at the row
level.


On Fri, Feb 28, 2014 at 11:44 AM, Keith Wright kwri...@nanigans.com wrote:

 FYI - I recently filed
 https://issues.apache.org/jira/browse/CASSANDRA-6654 and wanted to let
 everyone know the result as it was not what I expected.   I am using C*
 1.2.12 and found that my droppable tombstone ratio kept increasing on an
 LCS table (currently  .3).  Documentation states that compactions should
 be triggered when that gets above .2 to help cleanup tombstones and in my
 case compactions are definitely not running behind.

 I am setting different TTLs on different columns (this capability was one
 of the things I love about Cassandra) and the result of the ticket is that
 only columns whose write time is less than now + the MAX TTL of columns
 within the row will NOT be removed.  In my case, I was setting some columns
 to 6 months and others to 7 days so this meant that the 7 day data will in
 fact NOT be removed until 6 months!  This results in MUCH wider rows than I
 expected.

 It appears that this was likely fixed in 2.1 but obviously people will not
 be deploying that to production anytime soon.  It appears that I will just
 have to no longer set the 6 month TTL and instead leave it as forever to
 ensure that the smaller TTLs are respected.  This is an acceptable tradeoff
 for me since the 7 day columns are the ones that get much larger (against a
 map column type).

 So be warned, mixing TTLs in a row does not appear to result in the data
 being compacted away.

 Thanks




-- 
*Ken Hancock *| System Architect, Advanced Advertising
SeaChange International
50 Nagog Park
Acton, Massachusetts 01720
ken.hanc...@schange.com | www.schange.com |
NASDAQ:SEAChttp://www.schange.com/en-US/Company/InvestorRelations.aspx

Office: +1 (978) 889-3329 | [image: Google Talk:]
ken.hanc...@schange.com | [image:
Skype:]hancockks | [image: Yahoo IM:]hancockks [image:
LinkedIn]http://www.linkedin.com/in/kenhancock

[image: SeaChange International]
 http://www.schange.com/This e-mail and any attachments may contain
information which is SeaChange International confidential. The information
enclosed is intended only for the addressees herein and may not be copied
or forwarded without permission from SeaChange International.

Compaction does not remove tombstones if column has higher TTL

[Keith Wright]

FYI – I recently filed https://issues.apache.org/jira/browse/CASSANDRA-6654 and 
wanted to let everyone know the result as it was not what I expected.   I am 
using C* 1.2.12 and found that my droppable tombstone ratio kept increasing on 
an LCS table (currently  .3).  Documentation states that compactions should be 
triggered when that gets above .2 to help cleanup tombstones and in my case 
compactions are definitely not running behind.

I am setting different TTLs on different columns (this capability was one of 
the things I love about Cassandra) and the result of the ticket is that only 
columns whose write time is less than now + the MAX TTL of columns within the 
row will NOT be removed.  In my case, I was setting some columns to 6 months 
and others to 7 days so this meant that the 7 day data will in fact NOT be 
removed until 6 months!  This results in MUCH wider rows than I expected.

It appears that this was likely fixed in 2.1 but obviously people will not be 
deploying that to production anytime soon.  It appears that I will just have to 
no longer set the 6 month TTL and instead leave it as forever to ensure that 
the smaller TTLs are respected.  This is an acceptable tradeoff for me since 
the 7 day columns are the ones that get much larger (against a map column type).

So be warned, mixing TTLs in a row does not appear to result in the data being 
compacted away.

Thanks