Hi Dave, This is very cool. Do you have the results and the scripts you used to benchmarch CouchDB?
Cheers Garren On Thu, Oct 22, 2015 at 3:13 PM, Dave Amies <[email protected]> wrote: > Hi All, > > I'm sure by now most of you will have read at least some parts of this > guide: > > http://guide.couchdb.org/draft/performance.html > > I was reading it the other day and noticed the "Call to Arms" section at > the bottom of the page. I don't know if there are already any benchmarking > tools out there, but I decided to try writing one. Hopefully the one I have > written will be useful. > > About my background, for my day job i am a performance tester, usually > specialising in Loadrunner, so this project was something to keep my mind > occupied while waiting for my test system to be rebuilt. Given this I have > only spent a few hours on it and so there is probably still room for > improvement, this email is about finding out if there is interest or if > this will be useful to the CouchDB community, so really should I continue > developing this tool, or am I wasting my time? > > In designing this benchmarking utility I reflected on all the systems I > have tested and tried to come up with some common areas where database > systems suffer in performance. Then bearing in mind the fundamental > differences between traditional databases and NoSQL databases (particularly > CouchDB) I tried to construct some some common database usage scenarios. > > The 3 scenarios I came up with are: > > 1. Write heavy (each user performs 12 writes, 6 reads and 3 searches / > index queries) > 2. Index / Query / Search heavy (each user performs 1 write, 2 reads and > 6 searches / index queries) > 3. Read Heavy (each user performs 1 writes, 10 reads and 3 searches / > index queries) > > I have tried out my benchmarking tool on a couple of machines so far, in > these tests I managed to cause CouchDB to encounter the following > situations: > > 1. Performance degradation due to being Disk IO bound > 2. Performance degradation due to being Memory bound > 3. Performance degradation due to being CPU bound > 4. Couch DB crashed > 5. Benchmarking completed successfully and produce a performance score > > Based on these results I believe I have created an effective tool for > benchmarking, so I decided the best next step was to release the tool as an > open source project, so I created a github project which can be found here: > https://github.com/damies13/kvbench. Here you will the readme file > describes the 3 scenarios in more detail, the benchmark definition or > design and also the pre benchmark data priming. You will also find here the > python script that is the benchmarking tool and some instructions for > setting up a couch db database for the benchmarking process. > > As this is getting long i'll wrap up by noting that I deliberately did not > use the python couchdb libraries but instead I used the requests library > (standard http) and json library because I wanted to keep the code as > generic as possible, the intention is that this benchmarking tool should be > able to be used to benchmarking any key / value store, whether that be a > document based NoSQL, and Key Value based NoSQL database or some other Rest > API / engine (e.g. backed by a traditional database). > > I look forward to some feed back, hopefully I have created something > useful. > > Sincerely, > > Dave. >
