Hi Garren, Thanks for your kind words.
I will post each test result separately rather than one enormous email, Unfortunately I lost the logs from the Couch DB crash. I tried to reproduce it but instead the benchmark completed successfully, so good news in a way. Dave. On Tue, Oct 27, 2015 at 8:18 PM, Garren Smith <[email protected]> wrote: > Hi Dave, > > This is very cool. Do you have the results and the scripts you used to > benchmarch CouchDB? > > Cheers > Garren > > On Thu, Oct 22, 2015 at 3:13 PM, Dave Amies <[email protected]> wrote: > > > Hi All, > > > > I'm sure by now most of you will have read at least some parts of this > > guide: > > > > http://guide.couchdb.org/draft/performance.html > > > > I was reading it the other day and noticed the "Call to Arms" section at > > the bottom of the page. I don't know if there are already any > benchmarking > > tools out there, but I decided to try writing one. Hopefully the one I > have > > written will be useful. > > > > About my background, for my day job i am a performance tester, usually > > specialising in Loadrunner, so this project was something to keep my mind > > occupied while waiting for my test system to be rebuilt. Given this I > have > > only spent a few hours on it and so there is probably still room for > > improvement, this email is about finding out if there is interest or if > > this will be useful to the CouchDB community, so really should I continue > > developing this tool, or am I wasting my time? > > > > In designing this benchmarking utility I reflected on all the systems I > > have tested and tried to come up with some common areas where database > > systems suffer in performance. Then bearing in mind the fundamental > > differences between traditional databases and NoSQL databases > (particularly > > CouchDB) I tried to construct some some common database usage scenarios. > > > > The 3 scenarios I came up with are: > > > > 1. Write heavy (each user performs 12 writes, 6 reads and 3 searches / > > index queries) > > 2. Index / Query / Search heavy (each user performs 1 write, 2 reads > and > > 6 searches / index queries) > > 3. Read Heavy (each user performs 1 writes, 10 reads and 3 searches / > > index queries) > > > > I have tried out my benchmarking tool on a couple of machines so far, in > > these tests I managed to cause CouchDB to encounter the following > > situations: > > > > 1. Performance degradation due to being Disk IO bound > > 2. Performance degradation due to being Memory bound > > 3. Performance degradation due to being CPU bound > > 4. Couch DB crashed > > 5. Benchmarking completed successfully and produce a performance score > > > > Based on these results I believe I have created an effective tool for > > benchmarking, so I decided the best next step was to release the tool as > an > > open source project, so I created a github project which can be found > here: > > https://github.com/damies13/kvbench. Here you will the readme file > > describes the 3 scenarios in more detail, the benchmark definition or > > design and also the pre benchmark data priming. You will also find here > the > > python script that is the benchmarking tool and some instructions for > > setting up a couch db database for the benchmarking process. > > > > As this is getting long i'll wrap up by noting that I deliberately did > not > > use the python couchdb libraries but instead I used the requests library > > (standard http) and json library because I wanted to keep the code as > > generic as possible, the intention is that this benchmarking tool should > be > > able to be used to benchmarking any key / value store, whether that be a > > document based NoSQL, and Key Value based NoSQL database or some other > Rest > > API / engine (e.g. backed by a traditional database). > > > > I look forward to some feed back, hopefully I have created something > > useful. > > > > Sincerely, > > > > Dave. > > >
