FYI. It appears that the proper command in CQL3 is the following:
alter table cookie_user_lookup with compaction={'sstable_size_in_mb': '256',
'class': 'LeveledCompactionStrategy'};
I still would have expected my original alter command to at least have thrown
an error. I assume I should open a bug for this?
alter table shard_user_lookup with compaction_strategy_options =
{'sstable_size_in_mb':256};
From: Keith Wright <[email protected]<mailto:[email protected]>>
Reply-To: "[email protected]<mailto:[email protected]>"
<[email protected]<mailto:[email protected]>>
Date: Thursday, July 25, 2013 9:14 AM
To: "[email protected]<mailto:[email protected]>"
<[email protected]<mailto:[email protected]>>, Wei Zhu
<[email protected]<mailto:[email protected]>>
Subject: Re: sstable size change
Unfortunately the table in question is a CQL3 table so cassandra-cli will not
output its describe:
WARNING: CQL3 tables are intentionally omitted from 'describe' output.
See https://issues.apache.org/jira/browse/CASSANDRA-4377 for details.
However, I did figure out that apparently I was not setting the change
properly. I was using the following alter via CQL3:
alter table shard_user_lookup with compaction_strategy_options =
{'sstable_size_in_mb':256};
But when I did a describe, I did not see the change.
CREATE TABLE shard_user_lookup (
shard_user_id int,
shard text,
creation_time timestamp,
status int,
user_id timeuuid,
PRIMARY KEY (shard_user_id, shard)
) WITH
bloom_filter_fp_chance=0.100000 AND
caching='KEYS_ONLY' AND
comment='' AND
dclocal_read_repair_chance=0.000000 AND
gc_grace_seconds=86400 AND
read_repair_chance=0.100000 AND
replicate_on_write='true' AND
populate_io_cache_on_flush='false' AND
compaction={'class': 'LeveledCompactionStrategy'} AND
compression={'chunk_length_kb': '8', 'crc_check_chance': '0.1',
'sstable_compression': 'LZ4Compressor'};
If I then execute the following via CQL2, I do see the property. Seems odd to
me?
alter table shard_user_lookup with
compaction_strategy_options:sstable_size_in_mb=256;
CREATE TABLE shard_user_lookup (
shard_user_id int,
shard text,
creation_time timestamp,
status int,
user_id timeuuid,
PRIMARY KEY (shard_user_id, shard)
) WITH
bloom_filter_fp_chance=0.100000 AND
caching='KEYS_ONLY' AND
comment='' AND
dclocal_read_repair_chance=0.000000 AND
gc_grace_seconds=86400 AND
read_repair_chance=0.100000 AND
replicate_on_write='true' AND
populate_io_cache_on_flush='false' AND
compaction={'sstable_size_in_mb': '256', 'class':
'LeveledCompactionStrategy'} AND
compression={'chunk_length_kb': '8', 'crc_check_chance': '0.1',
'sstable_compression': 'LZ4Compressor'};
From: Wei Zhu <[email protected]<mailto:[email protected]>>
Reply-To: "[email protected]<mailto:[email protected]>"
<[email protected]<mailto:[email protected]>>, Wei Zhu
<[email protected]<mailto:[email protected]>>
Date: Wednesday, July 24, 2013 8:49 PM
To: "[email protected]<mailto:[email protected]>"
<[email protected]<mailto:[email protected]>>
Subject: Re: sstable size change
what is output of show keyspaces from cassandra-cli, did you see the new value?
Compaction Strategy:
org.apache.cassandra.db.compaction.LeveledCompactionStrategy
Compaction Strategy Options:
sstable_size_in_mb: XXX
________________________________
From: Keith Wright <[email protected]<mailto:[email protected]>>
To: "[email protected]<mailto:[email protected]>"
<[email protected]<mailto:[email protected]>>
Sent: Wednesday, July 24, 2013 3:44 PM
Subject: Re: sstable size change
Hi all,
This morning I increased the SSTable size for one of my LCS via an alter
command and saw at least one compaction run (I did not trigger a compaction via
nodetool nor upgrades stables nor removing the .json file). But so far my data
sizes appear the same at the default 5 MB (see below for output of ls –Sal as
well as relevant portion of cfstats). Is this expected? I was hoping to see
at least one file at the new 256 MB size I set.
Thanks
SSTable count: 4965
SSTables in each level: [0, 10, 112/100, 1027/1000, 3816, 0, 0, 0]
Space used (live): 29062393142
Space used (total): 29140547702
Number of Keys (estimate): 195103104
Memtable Columns Count: 441483
Memtable Data Size: 205486218
Memtable Switch Count: 243
Read Count: 154226729
-rw-rw-r-- 1 cassandra cassandra 5247564 Jul 18 01:33
users-shard_user_lookup-ib-97153-Data.db
-rw-rw-r-- 1 cassandra cassandra 5247454 Jul 23 02:59
users-shard_user_lookup-ib-109063-Data.db
-rw-rw-r-- 1 cassandra cassandra 5247421 Jul 20 14:58
users-shard_user_lookup-ib-103127-Data.db
-rw-rw-r-- 1 cassandra cassandra 5247415 Jul 17 13:56
users-shard_user_lookup-ib-95761-Data.db
-rw-rw-r-- 1 cassandra cassandra 5247379 Jul 21 02:44
users-shard_user_lookup-ib-104718-Data.db
-rw-rw-r-- 1 cassandra cassandra 5247346 Jul 21 21:54
users-shard_user_lookup-ib-106280-Data.db
-rw-rw-r-- 1 cassandra cassandra 5247242 Jul 3 19:41
users-shard_user_lookup-ib-66049-Data.db
-rw-rw-r-- 1 cassandra cassandra 5247235 Jul 21 02:44
users-shard_user_lookup-ib-104737-Data.db
-rw-rw-r-- 1 cassandra cassandra 5247233 Jul 20 14:58
users-shard_user_lookup-ib-103169-Data.db
From: sankalp kohli <[email protected]<mailto:[email protected]>>
Reply-To: "[email protected]<mailto:[email protected]>"
<[email protected]<mailto:[email protected]>>
Date: Tuesday, July 23, 2013 3:04 PM
To: "[email protected]<mailto:[email protected]>"
<[email protected]<mailto:[email protected]>>
Subject: Re: sstable size change
"Will Cassandra force any newly compacted files to my new setting as
compactions are naturally triggered"
Yes. Let it compact and increase in size.
On Tue, Jul 23, 2013 at 9:38 AM, Robert Coli
<[email protected]<mailto:[email protected]>> wrote:
On Tue, Jul 23, 2013 at 6:48 AM, Keith Wright
<[email protected]<mailto:[email protected]>> wrote:
Can you elaborate on what you mean by "let it take its own course organically"?
Will Cassandra force any newly compacted files to my new setting as
compactions are naturally triggered?
You see, when two (or more!) SSTables love each other very much, they sometimes
decide they want to compact together..
But seriously, "yes." If you force all existing SSTables to level 0, it is as
if you just flushed them all. Level compaction then does a whole lot of
compaction, using the active table size.
=Rob
