Hi Lucas. Unfortunately, you have hit a bug in the Bitcask backend that we overlooked. The attached beam file is a patched version of the backend code that should resolve the issue for you. To use it, stop each node, then place riak_kv_bitcask_backend.beam in your basho-patches directory, then restart it. The merges should complete without crashing. I believe in your system the actual location is /usr/lib/riak/lib/basho-patches.
We will add this problem and the patch to the known issues information soon. Please let us know if you come across any other issues with the merges. On Wed, Sep 3, 2014 at 11:25 AM, Lucas Grijander < [email protected]> wrote: > Hi. > > I am new in the list, so i don't know if this is the right place to open > this thread. > > Our riak cluster is composed by 4 nodes. The O.S is Ubuntu 14.04 and the > version of riak is 2.0.0 > > We are getting a lot of errors in the logs of ours riak nodes. The errors > are like this: > > 2014-09-03 13:05:14.212 [error] <0.19152.3672> Failed to merge > {["/var/lib/riak/data/ten_minutes/570899077082383952423314387779798054553098649600/14.bitcask.data","/var/lib/riak/data/ten_minutes/570899077082383952423314387779798054553098649600/13.bitcask.data","/var/lib/riak/data/ten_minutes/570899077082383952423314387779798054553098649600/12.bitcask.data","/var/lib/riak/data/ten_minutes/570899077082383952423314387779798054553098649600/11.bitcask.data","/var/lib/riak/data/ten_minutes/570899077082383952423314387779798054553098649600/10.bitcask.data","/var/lib/riak/data/ten_minutes/570899077082383952423314387779798054553098649600/9.bitcask.data","/var/lib/riak/data/ten_minutes/570899077082383952423314387779798054553098649600/8.bitcask.data","/var/lib/riak/data/ten_minutes/570899077082383952423314387779798054553098649600/7.bitcask.data","/var/lib/riak/data/ten_minutes/570899077082383952423314387779798054553098649600/6.bitcask.data","/var/lib/riak/data/ten_minutes/570899077082383952423314387779798054553098649600/5.bitcask.data","/var/lib/riak/data/ten_minutes/570899077082383952423314387779798054553098649600/4.bitcask.data","/var/lib/riak/data/ten_minutes/570899077082383952423314387779798054553098649600/3.bitcask.data","/var/lib/riak/data/ten_minutes/570899077082383952423314387779798054553098649600/2.bitcask.data","/var/lib/riak/data/ten_minutes/570899077082383952423314387779798054553098649600/1.bitcask.data"],["/var/lib/riak/data/ten_minutes/570899077082383952423314387779798054553098649600/14.bitcask.data","/var/lib/riak/data/ten_minutes/570899077082383952423314387779798054553098649600/13.bitcask.data","/var/lib/riak/data/ten_minutes/570899077082383952423314387779798054553098649600/12.bitcask.data","/var/lib/riak/data/ten_minutes/570899077082383952423314387779798054553098649600/11.bitcask.data","/var/lib/riak/data/ten_minutes/570899077082383952423314387779798054553098649600/10.bitcask.data","/var/lib/riak/data/ten_minutes/570899077082383952423314387779798054553098649600/9.bitcask.data","/var/lib/riak/data/ten_minutes/570899077082383952423314387779798054553...",...]}: > {generic_failure,error,function_clause,[{riak_kv_bitcask_backend,key_transform_to_1,[{tombstone,<<2,0,4,109,116,116,108,50,48,49,52,48,57,48,50,50,50,53,48,50,53,48,56,102,56,102,102,101,54,49,49,97,101,51,99,52,52,55,101,55,55,100,99,50,100,49,52,56,51,57,55,48,50>>}],[{file,"src/riak_kv_bitcask_backend.erl"},{line,99}]},{bitcask,'-expiry_merge/4-fun-0-',7,[{file,"src/bitcask.erl"},{line,1912}]},{bitcask_fileops,fold_hintfile_loop,5,[{file,"src/bitcask_fileops.erl"},{line,660}]},{bitcask_fileops,fold_file_loop,8,[{file,"src/bitcask_fileops.erl"},{line,720}]},{bitcask_fileops,fold_hintfile,3,[{file,"src/bitcask_fileops.erl"},{line,624}]},{bitcask,expiry_merge,4,[{file,"src/bitcask.erl"},{line,1915}]},{bitcask,merge1,4,[{file,"src/bitcask.erl"},{line,686}]},{bitcask,merge,3,[{file,"src/bitcask.erl"},{line,566}]}]} > > It seems riak can not merge data in bitcask data store. > > This is the configuration of bitcask in ten minutes TTL: > > {<<"ten_minutes_ttl">>,riak_kv_bitcask_backend, > [{io_mode,erlang}, > {expiry_grace_time,0}, > {small_file_threshold,5242880}, > {dead_bytes_threshold,4194304}, > {frag_threshold,15}, > {dead_bytes_merge_trigger,4194304}, > {frag_merge_trigger,10}, > {max_file_size,10485760}, > {open_timeout,4}, > {data_root,"/var/lib/riak/data/ten_minutes"}, > {sync_strategy,none}, > {merge_window,always}, > {max_fold_age,-1}, > {max_fold_puts,0}, > {expiry_secs,660}, > {require_hint_crc,true}]} > > > As a result, the amount of used memory(RAM) keeps growing until the server > run out of free memory. > > Could you give me some clue that it can point to the cause of the problem? > > Thanks in advance. > > _______________________________________________ > riak-users mailing list > [email protected] > http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com > >
riak_kv_bitcask_backend.beam
Description: Binary data
_______________________________________________ riak-users mailing list [email protected] http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
