Re: Cell Level TTL And hfile.format.version

[Anoop John]

Ya as said, there is a warn message while doing the flush.  To check
with every mutation whether it having some tags and hfile version is
<3 would be bit costlier.  Suggest you change the version and things
will work as expected for you.

-Anoop-

On Fri, May 6, 2016 at 10:09 PM, Huaxiang Sun  wrote:
> Hi Saad,
>
>Please see HBASE-14730 
> . With 
> hfile.format.version 2, it will work partially. As long as the cells are in 
> the memstore, the TTL tag will be honored.
>   Once they are flushed into hfiles, ttl tag is lost which means cells will 
> be there forever.
>
>Your db will have cells which are supposed to be cleaned up with ttl. 
> Suggest to change the hfile format to version 3.
>
>Thanks,
>Huaxiang
>
>
>> On May 6, 2016, at 6:43 AM, Saad Mufti  wrote:
>>
>> HI,
>>
>> We're running a  CDH 5.5.2 HBase cluster (HBase Version 1.0.0-cdh5.5.2,
>> revision=Unknown). We are using the per-cell TTL feature (Mutation.setTTL)
>>
>>
>> As I learn more about and read up on HBase, I realized that in our HBase
>> config hfile.format.version was set to 2 (the default, we haven't touched
>> this config yet), and from what I read that version of the HFile format
>> does NOT support cell tags which are needed for cell TTL which use tags.
>>
>> Of course I am in the process of writing a test to check whether our
>> production db is indeed getting filled with cells that should in actuality
>> be expired given their TTL value.
>>
>> We haven't seen any errors at runtime, does this mean our efforts to set a
>> TTL are being silently ignored? Isn't this bad behavior? Even if the
>> hfile.format.version is set to the wrong version, wouldn't it be better to
>> throw an error instead of just silently dropping any tags that are set?
>>
>> Thanks.
>>
>> -
>> Saad
>

Re: Cell Level TTL And hfile.format.version

[Huaxiang Sun]

Hi Saad,

   Please see HBASE-14730 . 
With hfile.format.version 2, it will work partially. As long as the cells are 
in the memstore, the TTL tag will be honored.
  Once they are flushed into hfiles, ttl tag is lost which means cells will be 
there forever.
 
   Your db will have cells which are supposed to be cleaned up with ttl. 
Suggest to change the hfile format to version 3.

   Thanks,
   Huaxiang


> On May 6, 2016, at 6:43 AM, Saad Mufti  wrote:
> 
> HI,
> 
> We're running a  CDH 5.5.2 HBase cluster (HBase Version 1.0.0-cdh5.5.2,
> revision=Unknown). We are using the per-cell TTL feature (Mutation.setTTL)
> 
> 
> As I learn more about and read up on HBase, I realized that in our HBase
> config hfile.format.version was set to 2 (the default, we haven't touched
> this config yet), and from what I read that version of the HFile format
> does NOT support cell tags which are needed for cell TTL which use tags.
> 
> Of course I am in the process of writing a test to check whether our
> production db is indeed getting filled with cells that should in actuality
> be expired given their TTL value.
> 
> We haven't seen any errors at runtime, does this mean our efforts to set a
> TTL are being silently ignored? Isn't this bad behavior? Even if the
> hfile.format.version is set to the wrong version, wouldn't it be better to
> throw an error instead of just silently dropping any tags that are set?
> 
> Thanks.
> 
> -
> Saad

Cell Level TTL And hfile.format.version

[Saad Mufti]

HI,

We're running a  CDH 5.5.2 HBase cluster (HBase Version 1.0.0-cdh5.5.2,
revision=Unknown). We are using the per-cell TTL feature (Mutation.setTTL)


As I learn more about and read up on HBase, I realized that in our HBase
config hfile.format.version was set to 2 (the default, we haven't touched
this config yet), and from what I read that version of the HFile format
does NOT support cell tags which are needed for cell TTL which use tags.

Of course I am in the process of writing a test to check whether our
production db is indeed getting filled with cells that should in actuality
be expired given their TTL value.

We haven't seen any errors at runtime, does this mean our efforts to set a
TTL are being silently ignored? Isn't this bad behavior? Even if the
hfile.format.version is set to the wrong version, wouldn't it be better to
throw an error instead of just silently dropping any tags that are set?

Thanks.

-
Saad