Jie, A colleague points out that I typed "40" in one place and "50" in another. "40" is the appropriate setting.
Matthew On Apr 15, 2014, at 8:41 AM, Matthew Von-Maszewski <[email protected]> wrote: > Jie, > > 4G ram is small for your ring size and node count. My first recommendation > is that you reduce your ring size to 32 {ring_creation_size, 32}. Then > change the max_open_files setting of eleveldb to 40 {max_open_files, 50} and > block_size to 32768 {sst_block_size, 32768}. These three settings should > quickly improve performance … but require you start over with your cluster. > > Active Anti-Entropy has a bug in 1.4.7. You might as well disable it too: > {anti_entropy, {off, []}}. The recommendation is that you upgrade to 1.4.8. > > Matthew > > > > > On Apr 14, 2014, at 10:10 PM, Jie Lu <[email protected]> wrote: > >> >> I also have a problem in performance test of Riak Cluster. >> ~~~~~~~~~~~~ >> Riak version: 1.4.7 >> OS: openSUSE 11.3 >> RAM: 4G >> ring size is 64 >> backend: leveldb >> Nodes in cluster: 6 nodes >> >> ~~~~~~~~~~~~~ >> >> I write a key/value with value is 1K bytes, and 25 concurrent threads on >> one client nodes. The test result only 20 ops/s performance. >> >> Is there any performance benchmark to compare with? >> >> >> >> >> >> >> On Tue, Apr 15, 2014 at 6:26 AM, Luke Bakken <[email protected]> wrote: >> Hi Matthew - >> >> Some suggestions: >> >> * Upgrade to Riak 1.4.8 >> >> * Test with a ring size of 64 >> >> * Use staggered merge windows in your cluster >> (http://docs.basho.com/riak/latest/ops/advanced/backends/bitcask/) >> >> * Since you're on dedicated hardware RAID, use the noop scheduler for your >> Riak data volumes: >> >> cat /sys/block/sd*/queue/scheduler >> noop anticipatory deadline [cfq] >> >> * Increase +zdbbl in /etc/riak/vm.args to 96000 >> >> Thanks >> -- >> Luke Bakken >> CSE >> [email protected] >> >> >> On Mon, Apr 14, 2014 at 2:33 PM, Matthew MacClary >> <[email protected]> wrote: >> I have a persistent issue I am trying to diagnose. In our use of Riak we >> have multiple data creators writing into a 7 node cluster. The value size is >> a bit large at around 2MB. The behavior I am seeing is that if I delete all >> data out of bitcask, then test performance I get fast writes. As I keep >> doing the same work of writing to the cluster, then the Riak write times >> will start tailing off and getting really bad. >> >> Initial write times seen by my application: 0.5 seconds for 100MB worth of >> values (~200MB/s) >> Subsequent write times: 11 seconds for 100MB worth of values (~9MB/s) >> >> This slow down can happen over roughly 20-40 minutes of writing or about >> 200GB worth of key/value pairs written. >> >> I can reset the cluster to get fast performance again by stopping Riak and >> deleting the bitcask directories, then starting Riak again. This step is not >> feasible for production, but during testing at least the write speed goes up >> by 20x. >> >> Watching iostat I see that every few seconds the disk io jumps to ~11%. It >> doesn't seem that highly loaded from my cursory look. Watching top I see >> that beam.smp runs at around 100 for CPU% or less when heavily loaded. I am >> not sure how to tell what it is doing though :-) >> >> Thanks for any suggestions!! >> >> -Matt >> >> >> >> ================ >> System Description >> >> >> avg value size = 2MB >> Riak version = 1.4.1 >> n_val = 2 >> client threads total = 105 >> backend = bitcask >> ring_creation_size = 128 >> node count = 7 >> node OS = RHEL 6.2 >> server RAM = 128GB >> RAID = RAID0 across 8 SAS drives >> FS = ext4 >> FS options = /dev/mapper/vg0-lv0 / ext4 >> rw,noatime,barrier=0,stripe=512,data=ordered 0 0 >> bitcask size on one server = 133GB >> AAE = off >> interface = protobuf >> client library = riak java client >> file-max = 65536 >> >> _______________________________________________ >> riak-users mailing list >> [email protected] >> http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com >> >> >> >> _______________________________________________ >> riak-users mailing list >> [email protected] >> http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com >> >> >> >> >> -- >> Best Regards. >> Lu Jie >> >> >> >> -- >> Best Regards. >> Lu Jie >> _______________________________________________ >> riak-users mailing list >> [email protected] >> http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com >
_______________________________________________ riak-users mailing list [email protected] http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
