Oh ok so there is the "spammy" factor to think about with large ring size, thanks all for the helpful info, will go with smaller ring size (will also do a bit of benchmarking to see if there's any difference).
On Thu, Apr 25, 2013 at 6:37 PM, Sean Cribbs <[email protected]> wrote: > Minor correction to Dmitri's calcuation: the size of the covering set of > vnodes is RingSize / NVal, or 64 / 3 ~= 22 for the default configuration. > That said, it is still what one would consider "spammy" and should be done > with caution. > > > On Thu, Apr 25, 2013 at 11:10 PM, Dev Vasantharajan > <[email protected]>wrote: > >> Speaking of ring resizing, this SO answer from Dmitri could also be good >> reference (if you ever wanna go there). >> >> >> http://stackoverflow.com/questions/14685236/migrating-riak-data-when-ring-size-changes >> >> >> On Fri, Apr 26, 2013 at 12:04 AM, Dmitri Zagidulin >> <[email protected]>wrote: >> >>> Tom, >>> >>> Just to emphasize Joe's comment -- 512 should be the _maximum_ you want >>> to use as your ring size with leveldb/multi backend. But you should >>> probably use a smaller size, unless your cluster is going to have several >>> dozen nodes. >>> >>> The recommended rule of thumb with ring size is "~10 vnodes to a >>> physical machine". Meaning, if you have a 5 node cluster (and it's going to >>> stay at 5 nodes and not scale that much), using a ring size of 64 is >>> perfectly fine (since each node is going to have 12 to 13 vnodes on it ( 64 >>> / 5) ). >>> >>> Say you scale that cluster to 10 nodes, and keep the 64 ring size. Here, >>> each machine is going to be running 6 to 7 virtual nodes on it (64/10), and >>> will probably be under-utilized, in terms of hardware resources. >>> Whereas if you select a ring size of 128 for that 10 node cluster, >>> you're back to the 12-13 vnodes per machine range. >>> >>> On the other hand, if you have a 5 node cluster with a ring size of 512, >>> each machine will be responsible for around 100 vnodes, so unless your >>> servers are very powerful, they're going to compete with each other for >>> resources (ram, cpu, etc) and be overwhelmed. Also, keep in mind, that >>> Secondary Index (2i) queries get slower the bigger your ring size is (since >>> 2i uses "covering" queries -- each request has to contact half of all the >>> vnodes, so it matters whether it has to contact around 32 vnodes (for ring >>> size 64) or 256 (for 512)). >>> >>> So in short, most small to medium sized clusters out there use 128 or >>> 256 ring sizes. If you _know_ you're not going to scale past 7-9 nodes, >>> it's safe to go with 64. If you know you're going to scale to a giant >>> cluster of 50 nodes or some such, use 512 (though at that point, you should >>> re-evaluate your use of Secondary Indexes). >>> >>> Last thought - migrating to a different ring size is a big pain, >>> although it's doable. (You basically have to resort to logical backup tools >>> -- export all your data from the old cluster to disk, and then import it >>> all to the new cluster). >>> >>> Dmitri >>> >>> >>> >>> On Mon, Apr 22, 2013 at 5:30 PM, Tom Zeng <[email protected]> wrote: >>> >>>> Ok thanks Joe. We plan to switch to the Multi backend, so will use 512. >>>> >>>> >>>> On Mon, Apr 22, 2013 at 5:17 PM, Joe Caswell <[email protected]>wrote: >>>> >>>>> Tom, >>>>> There is no hard and fast rule for the "best value." The optimal >>>>> value for your situation will need to take into account the physical >>>>> resources available both in your starting cluster as well as your planned >>>>> end-state cluster. If you plan to use secondary indexing, the maximum >>>>> ring_creation_size you should consider is 512. >>>>> There will be a separate concurrent vnode_proxy process for each >>>>> vnode, and a process for each backend for each vnode. Each backend will >>>>> need open file handles and RAM for caching objects. The backend >>>>> configuration section of the docs should help plan your backed settings >>>>> http://docs.basho.com/riak/1.3.1/tutorials/choosing-a-backend/ >>>>> Your planning must also include failure scenarios. If any of your >>>>> nodes crash, the surviving nodes will each start more vnodes to cover the >>>>> missing node(s). The 10 vnodes per node recommendation is to ensure that >>>>> the vnodes from any single failed node can be divided among enough >>>>> surviving nodes to not leave one node handling significantly more load >>>>> than >>>>> the other, but this also is not a hard and fast rule. >>>>> At this time changing number of partitions in the ring does require a >>>>> complete rebuild of the cluster, we do have dynamic ring sizing on the >>>>> product roadmap, but there is no release date set for that feature. >>>>> >>>>> Joe Caswell >>>>> >>>>> From: Tom Zeng <[email protected]> >>>>> Date: Sunday, April 21, 2013 9:38 PM >>>>> To: <[email protected]> >>>>> Subject: the optimal value of the ring_creation_size >>>>> >>>>> Hi, >>>>> >>>>> I am wondering what's the best value for ring_creation_size, the >>>>> default is 64. According to the docs 64 will work for a cluster of no more >>>>> than 6 nodes (64 /10), and ring_creation_size of 128 will allow cluster of >>>>> up to 12 mode. I am wondering what kind of overhead of is associated with >>>>> a large ring_creation_size. Since changing the ring_creation_size will >>>>> result in rebuilding the cluster(destuctive), would a larger value make >>>>> more sense and allow scaling by adding more nodes? >>>>> >>>>> >>>>> Thanks, >>>>> Tom >>>>> >>>>> -- >>>>> Tom Zeng >>>>> Director of Engineering >>>>> Intridea, Inc. | www.intridea.com >>>>> [email protected] >>>>> (o) 888.968.4332 x519 >>>>> (c) 240-643-8728 >>>>> _______________________________________________ riak-users mailing >>>>> list [email protected] >>>>> http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com >>>>> >>>> >>>> >>>> _______________________________________________ >>>> riak-users mailing list >>>> [email protected] >>>> http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com >>>> >>>> >>> >>> _______________________________________________ >>> riak-users mailing list >>> [email protected] >>> http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com >>> >>> >> >> >> -- >> *Dev Kanchen *|* *Solution Architect >> *Sourcebits* >> >> *Bangalore* | San Francisco | Wroclaw | Connecticut | Atlanta >> M: +91 96111 07106 >> GTalk ID: [email protected] | Skype ID: dev.kanchen >> >> Disclaimer:This email and any attachments are sent in strictest >> confidence for the sole use of the addressee and may contain legally >> privileged, confidential, and proprietary data. If you are not the intended >> recipient, please advise the sender by replying promptly to this email and >> then delete and destroy this email and any attachments without any further >> use, copying or forwarding. >> >> _______________________________________________ >> riak-users mailing list >> [email protected] >> http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com >> >> > > > -- > Sean Cribbs <[email protected]> > Software Engineer > Basho Technologies, Inc. > http://basho.com/ > > _______________________________________________ > riak-users mailing list > [email protected] > http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com > > -- Tom Zeng Director of Engineering Intridea, Inc. | www.intridea.com [email protected] (o) 888.968.4332 x519 (c) 240-643-8728
_______________________________________________ riak-users mailing list [email protected] http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
