This is very little information that you provided. Based on that nobody can draw conclusions. It is better to use the system performance monitoring tools and find out what is saturated and change the settings based on that.
Linux performance guide: http://www.brendangregg.com/USEmethod/use-linux.html LevelDB configuration: http://docs.basho.com/riak/latest/ops/advanced/backends/leveldb/#Parameter-Planning 20 ops/s is very low, you can use basho_bench to benchmark riak and get a better understanding what is going on with pretty graphs and useful p values. basho_bench: https://github.com/basho/basho_bench Regards, Istvan On Mon, Apr 14, 2014 at 7:10 PM, Jie Lu <[email protected]> wrote: > > I also have a problem in performance test of Riak Cluster. > ~~~~~~~~~~~~ > Riak version: 1.4.7 > OS: openSUSE 11.3 > RAM: 4G > ring size is 64 > backend: leveldb > Nodes in cluster: 6 nodes > > ~~~~~~~~~~~~~ > > I write a key/value with value is 1K bytes, and 25 concurrent threads on > one client nodes. The test result only 20 ops/s performance. > > Is there any performance benchmark to compare with? > > > > > > > On Tue, Apr 15, 2014 at 6:26 AM, Luke Bakken <[email protected]> wrote: >> >> Hi Matthew - >> >> Some suggestions: >> >> * Upgrade to Riak 1.4.8 >> >> * Test with a ring size of 64 >> >> * Use staggered merge windows in your cluster >> (http://docs.basho.com/riak/latest/ops/advanced/backends/bitcask/) >> >> * Since you're on dedicated hardware RAID, use the noop scheduler for your >> Riak data volumes: >> >> cat /sys/block/sd*/queue/scheduler >> noop anticipatory deadline [cfq] >> >> * Increase +zdbbl in /etc/riak/vm.args to 96000 >> >> Thanks >> -- >> Luke Bakken >> CSE >> [email protected] >> >> >> On Mon, Apr 14, 2014 at 2:33 PM, Matthew MacClary >> <[email protected]> wrote: >>> >>> I have a persistent issue I am trying to diagnose. In our use of Riak we >>> have multiple data creators writing into a 7 node cluster. The value size is >>> a bit large at around 2MB. The behavior I am seeing is that if I delete all >>> data out of bitcask, then test performance I get fast writes. As I keep >>> doing the same work of writing to the cluster, then the Riak write times >>> will start tailing off and getting really bad. >>> >>> Initial write times seen by my application: 0.5 seconds for 100MB worth >>> of values (~200MB/s) >>> Subsequent write times: 11 seconds for 100MB worth of values (~9MB/s) >>> >>> This slow down can happen over roughly 20-40 minutes of writing or about >>> 200GB worth of key/value pairs written. >>> >>> I can reset the cluster to get fast performance again by stopping Riak >>> and deleting the bitcask directories, then starting Riak again. This step is >>> not feasible for production, but during testing at least the write speed >>> goes up by 20x. >>> >>> Watching iostat I see that every few seconds the disk io jumps to ~11%. >>> It doesn't seem that highly loaded from my cursory look. Watching top I see >>> that beam.smp runs at around 100 for CPU% or less when heavily loaded. I am >>> not sure how to tell what it is doing though :-) >>> >>> Thanks for any suggestions!! >>> >>> -Matt >>> >>> >>> >>> ================ >>> System Description >>> >>> >>> avg value size = 2MB >>> Riak version = 1.4.1 >>> n_val = 2 >>> client threads total = 105 >>> backend = bitcask >>> ring_creation_size = 128 >>> node count = 7 >>> node OS = RHEL 6.2 >>> server RAM = 128GB >>> RAID = RAID0 across 8 SAS drives >>> FS = ext4 >>> FS options = /dev/mapper/vg0-lv0 / ext4 >>> rw,noatime,barrier=0,stripe=512,data=ordered 0 0 >>> bitcask size on one server = 133GB >>> AAE = off >>> interface = protobuf >>> client library = riak java client >>> file-max = 65536 >>> >>> _______________________________________________ >>> riak-users mailing list >>> [email protected] >>> http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com >>> >> >> >> _______________________________________________ >> riak-users mailing list >> [email protected] >> http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com >> > > > > -- > Best Regards. > Lu Jie > > > > -- > Best Regards. > Lu Jie > > _______________________________________________ > riak-users mailing list > [email protected] > http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com > -- the sun shines for all _______________________________________________ riak-users mailing list [email protected] http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
