I'm using standalone Bitcask (no Riak at all). It is used as a storage
backend for an in-memory database, so the load is as follows: every 5
minutes I write ~1.2GB of data (~20k keys ~60kb data each) to the same
keys as before, overwriting their contents. Then I call
bitcask:sync(BitcaskRef). The problem is that if I don't call
bitcask:merge(Path) explicitly, Bitcask won't shrink it's size (and it
shrinks if I call it). Here is an example of what I'm talking about:
after ~15 minutes from the start:
total 4.4G
drwxr-xr-x 2 root root 260K Dec 12 22:51 .
drwxr-xr-x 22 root root 4.0K Dec 5 05:16 ..
-rw------- 1 root root 2.0G Dec 12 22:41 1323714971.bitcask.data
-rw-r--r-- 1 root root 2.7M Dec 12 22:41 1323714971.bitcask.hint
-rw------- 1 root root 2.0G Dec 12 22:51 1323715283.bitcask.data
-rw-r--r-- 1 root root 2.7M Dec 12 22:51 1323715283.bitcask.hint
-rw------- 1 root root 366M Dec 12 22:51 1323715882.bitcask.data
-rw-r--r-- 1 root root 286K Dec 12 22:51 1323715882.bitcask.hint
-rw------- 1 root root 45 Dec 12 22:51 bitcask.write.lock
after an hour:
total 19G
drwxr-xr-x 2 root root 260K Dec 12 23:56 .
drwxr-xr-x 22 root root 4.0K Dec 5 05:16 ..
-rw------- 1 root root 2.0G Dec 12 22:41 1323714971.bitcask.data
-rw-r--r-- 1 root root 2.7M Dec 12 22:41 1323714971.bitcask.hint
-rw------- 1 root root 2.0G Dec 12 22:51 1323715283.bitcask.data
-rw-r--r-- 1 root root 2.7M Dec 12 22:51 1323715283.bitcask.hint
-rw------- 1 root root 2.0G Dec 12 23:01 1323715882.bitcask.data
-rw-r--r-- 1 root root 2.7M Dec 12 23:01 1323715882.bitcask.hint
-rw------- 1 root root 2.0G Dec 12 23:11 1323716480.bitcask.data
-rw-r--r-- 1 root root 2.7M Dec 12 23:11 1323716480.bitcask.hint
-rw------- 1 root root 2.0G Dec 12 23:21 1323717078.bitcask.data
-rw-r--r-- 1 root root 2.7M Dec 12 23:21 1323717078.bitcask.hint
-rw------- 1 root root 2.0G Dec 12 23:31 1323717676.bitcask.data
-rw-r--r-- 1 root root 2.3M Dec 12 23:31 1323717676.bitcask.hint
-rw------- 1 root root 2.0G Dec 12 23:36 1323718271.bitcask.data
-rw-r--r-- 1 root root 2.7M Dec 12 23:36 1323718271.bitcask.hint
-rw------- 1 root root 2.0G Dec 12 23:46 1323718583.bitcask.data
-rw-r--r-- 1 root root 2.7M Dec 12 23:46 1323718583.bitcask.hint
-rw------- 1 root root 2.0G Dec 12 23:56 1323719182.bitcask.data
-rw-r--r-- 1 root root 2.7M Dec 12 23:56 1323719182.bitcask.hint
-rw------- 1 root root 541M Dec 12 23:56 1323719779.bitcask.data
-rw-r--r-- 1 root root 422K Dec 12 23:56 1323719779.bitcask.hint
-rw------- 1 root root 45 Dec 12 23:56 bitcask.write.lock
I didn't change any Bitcask settings:
application:get_all_env(bitcask).
[{frag_merge_trigger,60},
{dead_bytes_merge_trigger,536870912},
{frag_threshold,40},
{included_applications,[]},
{open_timeout,4},
{dead_bytes_threshold,134217728},
{max_file_size,2147483648},
{max_fold_puts,0},
{max_fold_age,-1},
{small_file_threshold,10485760},
{sync_strategy,none},
{expiry_secs,-1},
{merge_window,always}]
_______________________________________________
riak-users mailing list
[email protected]
http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
