Thanks, that's interesting to hear. How have you been finding the stability and reliability to be with leofs, over time?
I still wish I could just get our Riak CS cluster performing better; it just seems so unreasonably slow at the moment, that I suspect there's *something* holding it back. I can build a test cluster on my desktop, and even with five virtual riak nodes on the one machine, I still see 20-40x the performance, so it seems bizarre that dedicated bare-metal servers would be so slow. (Although obviously there's much more network latency between real machines, than a virtual cluster on one desktop; and they have a lot more data in their bitcask databases) However I've tried fiddling with all the Riak and Riak CS options.. ethtool offloads.. mount options.. sysctls.. MTU sizes.. even dropping single nodes out of the cluster one at a time in case they were somehow at fault.. seems like the only performance changes I can make are negative. We're still double the speed of the original poster in this thread, but.. that isn't saying much. Toby On 26 November 2014 at 06:44, Heinz Nikolaus Gies <[email protected]> wrote: > If you’re evaluating RiakCS vs. Ceph you might want to toss LeoFS[1] in the > mix and give it a run. Just as RiakCS it is a dynamo inspired system build > in Erlang and comes with the same advantages and disadvantages. But unlike > RiakCS it is pretty much exclusive a Object Store so can take a few > different optimizations for this kind of work that might not be possible in > a general purpose database as Riak (this is my personal guess not a research > founded conclusion). The team is (much) smaller then bash (obviously) but > they’re a very nice and responsive bunch. I ended up using it as a s3 > backend for Project-FiFo due to it’s performance characteristics. With > current releases I manage to get a sigle file upload speed of ~1.2GB/s using > gof3r[2] (this might be a client limitation but I haven’t had time to > investigate the details). > > [1] http://leo-project.net/leofs/ > [2] https://github.com/rlmcpherson/s3gof3r/tree/master/gof3r > --- > Cheers, > Heinz Nikolaus Gies > [email protected] > > > > On Nov 25, 2014, at 6:08, Toby Corkindale <[email protected]> wrote: > > Hi, > I wondered if you managed to significantly improve your Riak CS > performance, or not? > > I just ask as we've been getting not-dissimilar performance out of > Riak CS too (4-5 mbyte/sec max per client, on bare metal hardware), > for quite a long time. (I swear it was faster originally, when there > was a lot less data in the whole system.) > This is after applying all the tweaks available -- networking stack, > filesystem mount options, assorted Erlang vm.args, and increased put > concurrency/buffer options. > > We put up with it because it's been just-about sufficient enough for > our needs and Riak CS has been reliable and easy to administer -- but > it's becoming more of an issue, and so I'm curious to know if other > people *do* manage to achieve *good* per-client speeds out of Riak CS > or if this is just how things always are? > And we're way off the mark, maybe we can find out why.. > > Details of our setup: > 6 node cluster. RIng size of 64. > Riak 1.4.10 > Riak CS 1.5.2 > (installed from official Basho repos) > > Tests conducted using both multi-part and non-multi-part upload mode; > performance is similar with both. Tested against cluster when very > lightly loaded. > For the sake of testing, a 100M file is being used, that contains > random (hard to compress) data. > > Cheers, > Toby > > On 8 November 2014 at 01:41, David Meekin <[email protected]> > wrote: > > Hi, > I’ve setup a test 4 node RiakCS cluster on HP BL460c hardware and I can’t > seem to get S3 upload speeds above 2MB/s > I’m connecting direct to RiackCS on one of the nodes so there is no load > balancing software in place. > I have also installed s3cmd locally onto one of the nodes and the speeds > locally are the same. > These 4 nodes also run a test CEPH cluster with RadosGW and s3 uploads to > CEPH achieve 125MB/s > Any help would be appreciated as I’m currently evaluating both CEPH and > RiakCS. > > > _______________________________________________ > riak-users mailing list > [email protected] > http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com > > -- Turning and turning in the widening gyre The falcon cannot hear the falconer Things fall apart; the center cannot hold Mere anarchy is loosed upon the world _______________________________________________ riak-users mailing list [email protected] http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
