Not forgotten you Kevin, out of the country at the moment, will get back to you soon as I can
Regards Richard On Tue, Feb 26, 2013 at 4:06 PM, Kevin Burton <[email protected]>wrote: > Here is the config that I am using:**** > > ** ** > > {driver,**** > > basho_bench_driver_2i}.**** > > ** ** > > {operations, [**** > > {get_pb, 1},**** > > {{put_pb, 5}, 20},**** > > {{query_http, 10}, 1},**** > > {{query_mr, 10}, 1},**** > > {{query_pb, 10}, 1}**** > > ]}.**** > > ** ** > > {measurement_driver,**** > > basho_bench_measurement_erlangvm}.**** > > ** ** > > {measurements, [**** > > {memory, 1000},**** > > {cpu, 1000},**** > > {processes, 1000},**** > > {filehandles, 1000}**** > > ]}.**** > > ** ** > > %%% LOAD SETTINGS %%%**** > > ** ** > > {mode, max}.**** > > {duration, 1}.**** > > {concurrent, 3}.**** > > ** ** > > %%% DATA SHAPE %%%**** > > ** ** > > {key_generator, {uniform_int, 1000}}.**** > > {value_generator, {fixed_bin, 1000}}.**** > > ** ** > > {pb_ips, ["172.16.33.107", "172.16.33.138", "172.16.33.221", > "172.16.34.56"]}.**** > > {pb_replies, 1}.**** > > ** ** > > {http_ips, ["172.16.33.107", "172.16.33.138", "172.16.33.221", > "172.16.34.56"]}.**** > > {http_port, 8098}.**** > > ** ** > > %%It can also be configured for devrel!**** > > %%{pb_ips, [{"127.0.0.1", 8081},**** > > %% {"127.0.0.1", 8082},**** > > %% {"127.0.0.1", 8083},**** > > %% {"127.0.0.1", 8084}**** > > %%]}.**** > > %%{pb_replies, 1}.**** > > %%**** > > %%{http_ips, [{"127.0.0.1", 8091},**** > > %% {"127.0.0.1", 8092},**** > > %% {"127.0.0.1", 8093},**** > > %% {"127.0.0.1", 8094}**** > > %%]}.**** > > ** ** > > {rng_seed, {1, 2, 3}}.**** > > ** ** > > %% enforce_keyrange is for use with sequential_int**** > > %% Will error if keys in a range are missing.**** > > %%{enforce_keyrange, 10000}.**** > > ** ** > > %%% MEASUREMENT SETTINGS %%%**** > > ** ** > > {nodes, ['[email protected]', **** > > '[email protected]', **** > > '[email protected]',**** > > '[email protected]']}.**** > > {cookie, riak}.**** > > ** ** > > *From:* Richard Shaw [mailto:[email protected]] > *Sent:* Tuesday, February 26, 2013 5:46 AM > > *To:* Kevin Burton > *Cc:* riak-users > *Subject:* Re: Tuning a Riak cluster.**** > > ** ** > > Hi Kevin,**** > > ** ** > > Yes I have, please can you let me know which basho bench config you're > using and its contents**** > > ** ** > > Thanks**** > > ** ** > > Richard**** > > > **** > > ** ** > > Richard Shaw**** > > Client Services Engineer EMEA > Basho Technologies > +44 (0)7590 507683 > [email protected]**** > > ** ** > > On Mon, Feb 25, 2013 at 10:10 PM, Kevin Burton <[email protected]> > wrote:**** > > Did you have a chance to look at the basho_bench errors?**** > > > On Feb 23, 2013, at 2:35 AM, Richard Shaw <[email protected]> wrote:**** > > ** ** > > Hi Kevin,**** > > ** ** > > I sympathise with R, i'm going to write an installation guide to go on the > site.**** > > ** ** > > In regards to the error, please paste in the contents of you vm.args file > from the target node**** > > ** ** > > Thanks**** > > ** ** > > Richard**** > > ** ** > > On Fri, Feb 22, 2013 at 11:05 PM, <[email protected]> wrote:**** > > Here is the output from one configuration try:**** > > ** ** > > 17:00:08.878 [info] Using http target "172.16.33.138":8098 for worker 2*** > * > > ** ** > > 17:00:08.878 [info] Using pb target "172.16.33.138":8087 for worker 2**** > > ** ** > > 17:00:08.993 [debug] Supervisor basho_bench_sup started > basho_bench_worker:start_link(basho_bench_worker_2, 2) at pid <0.96.0>**** > > ** ** > > 17:00:08.999 [info] Using http target "172.16.33.107":8098 for worker 3*** > * > > ** ** > > 17:00:09.001 [info] Using pb target "172.16.33.107":8087 for worker 3**** > > ** ** > > 17:00:09.006 [debug] Supervisor basho_bench_sup started > basho_bench_worker:start_link(basho_bench_worker_3, 3) at pid <0.99.0>**** > > ** ** > > 17:00:09.022 [debug] Supervisor net_sup started erl_epmd:start_link() at > pid <0.104.0>**** > > ** ** > > 17:00:09.029 [debug] Supervisor net_sup started auth:start_link() at pid > <0.105.0>**** > > ** ** > > 17:00:09.030 [info] Can't set long node name!**** > > Please check your configuration**** > > ** ** > > 17:00:09.030 [error] Failed to start net_kernel for > basho_bench_measurement_erlangvm: > {shutdown,{child,undefined,net_sup_dynamic,{erl_distribution,start_link,[[basho_bench,longnames]]},permanent,1000,supervisor,[erl_distribution]}} > **** > > ** ** > > There is alot more output but this is the tail end of the output.**** > > ** ** > > On Fri, Feb 22, 2013 at 4:34 PM, Richard Shaw wrote:**** > > ** ** > > Hi Kevin, **** > > ** ** > > Always good practise to reply to the mailing list so others can benefit > from the content **** > > ** ** > > basho bench is the best place to start, If you have a working cluster or > even a working single node, follow the benchmarking guide[0] **** > > ** ** > > Please let me see some of the specific errors you're having and we can > help resolve them. **** > > ** ** > > Outside of basho bench, I recommend benchmarking the raw performance of > your VM, you can do this using a variety of tools like iostat, iperf or a > test suite like Phoronix test suite, a very comprehensive FOSS tool[1] look > at the `aio` test **** > > ** ** > > I'd also recommend benchmarking each time you make a configuration change > to really understand how effective that change was **** > > ** ** > > [0] http://docs.basho.com/riak/latest/cookbooks/Benchmarking/**** > > [1] http://www.phoronix-test-suite.com/**** > > ** ** > > Kind Regards **** > > ** ** > > Richard **** > > ** ** > > On Fri, Feb 22, 2013 at 10:14 PM, Kevin Burton < *[email protected] > *> wrote: **** > > That was my next question. I have tried to get basho_bench to work but so > far have been unsuccessful. If you have a simple "hello world" sample > config I would greatly appreciate it. I have tried a bunch of configs in > the examples directory but get met with one error or another. **** > > Other than that I have just timed various client driver calls. Mostly > random reads some writes. **** > > ** ** > > On Feb 22, 2013, at 3:48 PM, Richard Shaw < *[email protected]*> wrote: ** > ** > > ** ** > > Hi Kevin, **** > > What kind of benchmarking have you done on your VMs and what did you use ? > **** > > ** ** > > Kind Regards **** > > ** ** > > Richard **** > > ** ** > > On Fri, Feb 22, 2013 at 9:49 AM, Kevin Burton < *[email protected]*> > wrote: **** > > Ok I suspected as much. This will give me more ammo when requesting more > resources. Thank you. **** > > Now, back to the original question. Given that the physical hardware is > taken care of. What parameters are most important when tuning a cluster. > Again for arguments sake assume the same 4 node cluster with a read-write > ratio of about 75 percent. **** > > ** ** > > On Feb 21, 2013, at 8:26 PM, Alexander Sicular < *[email protected]*> > wrote: **** > > ** ** > > It can't be said enough times but the number one thing you can do to > ensure that you are getting true performance (not to mention redundancy) is > to use different physical hardware for each of your nodes. Under no > circumstances should you have more than one VM (one logical node in a Riak > cluster) on the same physical hardware. Also, use multiple > connections/threads/parallelism/whatever on client side and be sure to hit > all the nodes in the cluster haproxy roundrobin-esque when writing to Riak. > Everything else is in the noise. **** > > ** ** > > -Alexander Sicular **** > > ** ** > > @siculars **** > > ** ** > > On Feb 21, 2013, at 9:04 PM, Kevin Burton < *[email protected]*> > wrote: **** > > ** ** > > **** > > There each has about 20-30GB of disk space. They each are a VM so I am > not sure how to specify the CPU. They all seem to be 64 bit Intel > processors but I could tell you the clock speed. The network is 1Gb > Ethernet. **** > > **** > > *From: * Sean Carey [mailto: *carey@*basho.com] **** > > *Sent: * Thursday, February 21, 2013 7:59 PM **** > > *To: * Kevin Burton **** > > *Cc: * *[email protected]***** > > *Subject: * Re: Tuning a Riak cluster. **** > > **** > > Kevin, **** > > Disk and CPU, and Network? **** > > **** > > ** ** > > Sean Carey **** > > @densone **** > > On Thursday, February 21, 2013 at 20:31, Kevin Burton wrote: **** > > **** > > I have a cluster of 4 machines (4 Linux VM machines each allocated about 1 > Gb of memory – yea I know it isn’t a lot). I would like to get some > pointers on getting the fastest query time possible given these meager > resources. Thank you. **** > > _______________________________________________ **** > > riak-users mailing list **** > > *[email protected]***** > > http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com**** > > **** > > _______________________________________________ **** > > riak-users mailing list **** > > *[email protected]***** > > http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com**** > > ** ** > > _______________________________________________ **** > > riak-users mailing list **** > > *[email protected]***** > > http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com**** > > > > **** > > ** ** > > ** ** >
_______________________________________________ riak-users mailing list [email protected] http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
