Unfortunately, it doesn’t seem to have made a difference. I did restart the riak-cs process after updating the app.config. Triggering a riak-cs-gc batch did trigger a job, but did not seem to yield any results.
I’m a bit concerned that I’m unable to root cause the issue here. Is there some sort of visibility tools that I’m overlooking? I would assume that I’d be able to query into riak-cs and find “x entries are ready to be garbage collected” or something to provide some visibility. I’m not even sure if the problem is within riak-cs or riak/bitcask. Are riak-cs and riak just black boxes? I can’t imagine this to be the case if people are using these solutions in production. Please tell me I’m just an idiot and not aware of some utility/configuration to make this work. Thanks On Jun 8, 2014, at 7:22 PM, Tyler Flint <[email protected]> wrote: > Yes it is riak 1.4.5 > > Let me see if that helps. > > Thanks > > > >> On Jun 8, 2014, at 7:20 PM, Kota Uenishi <[email protected]> wrote: >> >> Spelling mistake, sorry: >> >> {gc_paginated_indexes, true}, >> >>> On Mon, Jun 9, 2014 at 10:10 AM, Kota Uenishi <[email protected]> wrote: >>> Hi, >>> >>> Are you using 1.4.5? I'm not sure but you may have a log message like this: >>> >>> "Error occurred trying to query from time 0 to ~p in gc key index. Reason: >>> ~p" >>> >>> in console.log. If so, it's even failing not only GC but in listing >>> objects to be collected. Then setting a line >>> >>> {gc_pagenated_indexes, true}, >>> >>> into riak_cs section of your app.config . This is implicit option and >>> its default is false. >>> >>> >>>> On Fri, Jun 6, 2014 at 11:52 PM, Tyler Flint <[email protected]> wrote: >>>> Thank you in advance for any assistance. >>>> >>>> To explore riak-cs, I setup a 2 node cluster with riak and riak-cs on each >>>> node, and put the riak nodes on the same cluster. I then proceeded to use >>>> risk-cs by uploading a 3G file using the multi-part upload protocol. I was >>>> unaware, however, that I needed to finalize the multipart upload and thus >>>> had many in progress multi-part uploads. Through the duration of my >>>> troubleshooting, the storage on the filesystem grew to 224G. Once I >>>> discovered my error, I proceeded to cancel all of the previous uploads. At >>>> this point there are no uploads in progress, and only a single 3G file >>>> available in the bucket. However, it has been 48 hours since the >>>> exploration and the storage use is still 224G on both machines. >>>> >>>> I have attempted multiple strategies in attempt to reclaim space, all >>>> without any effect. >>>> >>>> Within riak-cs: >>>> >>>> - I have shortened the gc leeway duration to 5 minutes and set all of the >>>> gc intervals as low as 5 minutes. >>>> - I have manually triggered the riak-cs-gc batch multiple times. (After >>>> running batch, status shows a gc run and eventually complete) >>>> >>>> Within riak: >>>> >>>> - I have adjusted the bitcask settings in attempt to trigger a merge. >>>> - I have force merged all of the bitcask partitions manually in the erlang >>>> console. >>>> >>>> None of these strategies seem to have any effect whatsoever. >>>> >>>> At this point I’m not even sure where to start, nor do I know if the issue >>>> is within riak-cs or riak (via bitcask). Any suggestions would be >>>> appreciated. >>>> >>>> Thanks >>>> >>>> Tyler >>>> _______________________________________________ >>>> riak-users mailing list >>>> [email protected] >>>> http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com >>> >>> >>> >>> -- >>> Kota UENISHI / @kuenishi >>> Basho Japan KK >> >> >> >> -- >> Kota UENISHI / @kuenishi >> Basho Japan KK _______________________________________________ riak-users mailing list [email protected] http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
