Re: Deflate compressor

2017-07-08 Thread Cogumelos Maravilha 
*"Decrease tombstone_compaction_interval and decrease
tombstone_threshold, or set unchecked_tombstone_compaction to true to
ignore both conditions and collect based /purely on gc grace/."*

Is this actually true for C* version 3.11.0?

AND compaction = {'class': 
'org.apache.cassandra.db.compaction.TimeWindowCompactionStrategy', 
'compaction_window_size': '1', 'compaction_window_unit': 'HOURS', 
'max_threshold': '128', 'min_threshold': 
'2',|*'unchecked_tombstone_compaction': 'true'*|}
AND compression = {'chunk_length_in_kb': '64', 'class': 
'org.apache.cassandra.io.compress.DeflateCompressor'}

Is this approach enough?

Thanks.

On 07/06/2017 06:27 PM, Jeff Jirsa wrote:
>
> On 2017-07-06 01:37 (-0700), Cogumelos Maravilha  
> wrote: 
>> Hi Jeff,
>>
>> Thanks for your reply. But I've already changed from LZ4 to Deflate to 
>> get higher compression level. Can I in the Deflate compressor do the 
>> same, set for a higher level of compression?
>
> Not at this time; if it's important to you, please open a JIRA (as always, 
> patches from the community are welcome)
>
>
>> Another question to the creator of the TimeWindowsCompactionStrategy:
>>
>> AND compaction = {'class': 
>> 'org.apache.cassandra.db.compaction.TimeWindowCompactionStrategy', 
>> 'compaction_window_size': '1', 'compaction_window_unit': 'HOURS', 
>> 'max_threshold': '128', 'min_threshold': '2'}
>> AND compression = {'chunk_length_in_kb': '64', 'class': 
>> 'org.apache.cassandra.io.compress.DeflateCompressor'}
>>
>> There are some days that I have exactly 24 SSTables:
>> ls -alFh *Data*|grep 'Jul  3'|wc
>>   24
>> Others no:
>> ls -alFh *Data*|grep 'Jul  2'|wc
>>   59
>>
>> Is this normal?
> "Maybe", you could use sstablemetadata to get the maxTimestamp from the 
> table, that's what TWCS will use to group data files together. Keep in mind 
> that repaired sstables won't compact with unrepaired (if you're using 
> incremental repair), and that tombstone compaction subproperties (which I 
> don't see in your options, but maybe you had set before) can cause single 
> sstable compactions that change the timestamp of the FILE, but the data 
> within it may continue to be much older.
>
> -
> To unsubscribe, e-mail: user-unsubscr...@cassandra.apache.org
> For additional commands, e-mail: user-h...@cassandra.apache.org
>

Re: Deflate compressor

2017-07-06 Thread Jeff Jirsa 


On 2017-07-06 01:37 (-0700), Cogumelos Maravilha  
wrote: 
> Hi Jeff,
> 
> Thanks for your reply. But I've already changed from LZ4 to Deflate to 
> get higher compression level. Can I in the Deflate compressor do the 
> same, set for a higher level of compression?


Not at this time; if it's important to you, please open a JIRA (as always, 
patches from the community are welcome)


> 
> Another question to the creator of the TimeWindowsCompactionStrategy:
> 
> AND compaction = {'class': 
> 'org.apache.cassandra.db.compaction.TimeWindowCompactionStrategy', 
> 'compaction_window_size': '1', 'compaction_window_unit': 'HOURS', 
> 'max_threshold': '128', 'min_threshold': '2'}
> AND compression = {'chunk_length_in_kb': '64', 'class': 
> 'org.apache.cassandra.io.compress.DeflateCompressor'}
> 
> There are some days that I have exactly 24 SSTables:
> ls -alFh *Data*|grep 'Jul  3'|wc
>   24
> Others no:
> ls -alFh *Data*|grep 'Jul  2'|wc
>   59
> 
> Is this normal?

"Maybe", you could use sstablemetadata to get the maxTimestamp from the table, 
that's what TWCS will use to group data files together. Keep in mind that 
repaired sstables won't compact with unrepaired (if you're using incremental 
repair), and that tombstone compaction subproperties (which I don't see in your 
options, but maybe you had set before) can cause single sstable compactions 
that change the timestamp of the FILE, but the data within it may continue to 
be much older.

-
To unsubscribe, e-mail: user-unsubscr...@cassandra.apache.org
For additional commands, e-mail: user-h...@cassandra.apache.org

Re: Deflate compressor

2017-07-06 Thread Cogumelos Maravilha 

Hi Jeff,

Thanks for your reply. But I've already changed from LZ4 to Deflate to 
get higher compression level. Can I in the Deflate compressor do the 
same, set for a higher level of compression?


Another question to the creator of the TimeWindowsCompactionStrategy:

AND compaction = {'class': 
'org.apache.cassandra.db.compaction.TimeWindowCompactionStrategy', 
'compaction_window_size': '1', 'compaction_window_unit': 'HOURS', 
'max_threshold': '128', 'min_threshold': '2'}
AND compression = {'chunk_length_in_kb': '64', 'class': 
'org.apache.cassandra.io.compress.DeflateCompressor'}


There are some days that I have exactly 24 SSTables:
ls -alFh *Data*|grep 'Jul  3'|wc
 24
Others no:
ls -alFh *Data*|grep 'Jul  2'|wc
 59

Is this normal?
Thanks in advance.


On 06-07-2017 06:30, Jeff Jirsa wrote:


On 2017-07-01 02:50 (-0700), Cogumelos Maravilha  
wrote:

Hi list,

Is there a way to set Deflate level of compression?

Brotli sounds good but unstable.

I just need more compression ratio.

I'm using C* 3.11.0


LZ4 compression levels are configurable as of cassandra 3.6 ( 
https://issues.apache.org/jira/browse/CASSANDRA-11051 ) .

"The attached patch adds the following optional parameters 'lz4_compressor_type' and 
'lz4_high_compressor_level' to the LZ4Compressor."

Glancing at the source: 
https://github.com/apache/cassandra/blob/trunk/src/java/org/apache/cassandra/io/compress/LZ4Compressor.java#L41-L49

lz4_high_compressor_level defaults to 9.
lz4_compressor_type defaults to 'fast' ('high' is also available).




-
To unsubscribe, e-mail: user-unsubscr...@cassandra.apache.org
For additional commands, e-mail: user-h...@cassandra.apache.org




-
To unsubscribe, e-mail: user-unsubscr...@cassandra.apache.org
For additional commands, e-mail: user-h...@cassandra.apache.org

Re: Deflate compressor

2017-07-05 Thread Jeff Jirsa 


On 2017-07-01 02:50 (-0700), Cogumelos Maravilha  
wrote: 
> Hi list,
> 
> Is there a way to set Deflate level of compression?
> 
> Brotli sounds good but unstable.
> 
> I just need more compression ratio.
> 
> I'm using C* 3.11.0
> 

LZ4 compression levels are configurable as of cassandra 3.6 ( 
https://issues.apache.org/jira/browse/CASSANDRA-11051 ) . 

"The attached patch adds the following optional parameters 
'lz4_compressor_type' and 'lz4_high_compressor_level' to the LZ4Compressor."

Glancing at the source: 
https://github.com/apache/cassandra/blob/trunk/src/java/org/apache/cassandra/io/compress/LZ4Compressor.java#L41-L49

lz4_high_compressor_level defaults to 9. 
lz4_compressor_type defaults to 'fast' ('high' is also available).




-
To unsubscribe, e-mail: user-unsubscr...@cassandra.apache.org
For additional commands, e-mail: user-h...@cassandra.apache.org