On 21 January 2015 at 15:22, Luke Bakken <[email protected]> wrote: > Hi Toby - > > Are you using the stock bitcask configuration for merging?
Hi Luke, Yes, pretty much. > On Tue, Jan 20, 2015 at 5:07 PM, Toby Corkindale <[email protected]> wrote: >> Hi Kota, >> I had a bit of an off-list chat about this a while ago, plus continued >> to investigate locally, and eventually achieved some faster speeds, >> around 15MByte/sec writes. >> Things that were changed: >> * Adjusted Riak CS GC to be spread out over the cluster much more. >> * Tweaked up the put buffers and concurrency further >> * Moved most of the files out of CS and into Amazon S3+Glacier >> * Switched from nginx to haproxy >> * simplified firewalling for internal clients >> >> Each one of those changes made a small to modest improvement, but >> overall combined to make a quite noticeable improvement. >> >> I did notice something odd though -- despite moving most of the data >> out of the cluster, the disk-space-in-use by Riak is still very large >> compared to the amount stored. I mean, we moved more than 90% of the >> data out of the cluster, yet the actual disk space used only halved. >> For every gigabyte of file stored in CS, dozens of gigabytes are >> actually on disk! >> >> Either the garbage collection algorithm is very, very lazy, or somehow >> something has gone a bit wrong in the past, which might have explained >> part of the performance problems. >> >> We're going to look at redeploying a new, fresh cluster based on Riak >> 2 in the not too distant future, once Riak CS looks like it's approved >> for use there, and maybe that'll clear all of this up. >> >> Toby -- Turning and turning in the widening gyre The falcon cannot hear the falconer Things fall apart; the center cannot hold Mere anarchy is loosed upon the world _______________________________________________ riak-users mailing list [email protected] http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
