5. Benchmarking completed successfully and produce a performance score
Again the benchmark script and CouchDB were running on the same machine.
This machine is a Mid 2011 27" iMac (i7 and 8GB Ram), despite it's age
still quite a reasonable machine. During this Benchmark the machine was
never CPU, Memory or Disk IO bound. as there was not much else running (1
browser window with futon open and the terminal to run the python benchmark
script) CouchDB was able make use of the ram for disk cache, this allowed
the entire database to remain in Memory (~1.1GB by the end of the test )
The system info:
------------------------------------------------------------------
------------------------------------------------------------------
----------------------
Model Name: iMac
Model Identifier: iMac12,2
Processor Name: Intel Core i7
Processor Speed: 3.4 GHz
Number of Processors: 1
Total Number of Cores: 4
L2 Cache (per Core): 256 KB
L3 Cache: 8 MB
Memory: 8 GB
Boot ROM Version: IM121.0047.B21
SMC Version (system): 1.72f2
------------------------------------------------------------------
------------------------------------------------------------------
----------------------
and the benchmark test output:
------------------------------------------------------------------
------------------------------------------------------------------
----------------------
iMac:KVBench dave$ python KVBench.py
Started DataPrime(100) 10/21/15 22:36:08 AEST
Finished DataPrime(100) 10/21/15 22:44:16 AEST
DataPrime took 487.661973 Seconds
Started Phase 1(50) 10/21/15 22:44:16 AEST
.....................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................
Finished Phase 1(50) 10/21/15 22:59:08 AEST
Phase 1 took 891.991589069 Seconds
Started Phase 2(50) 10/21/15 22:59:08 AEST
.....................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................
Finished Phase 2(50) 10/21/15 23:31:23 AEST
Phase 2 took 1934.88140798 Seconds
Started Phase 3(50) 10/21/15 23:31:23 AEST
.....................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................
Finished Phase 3(50) 10/21/15 23:45:08 AEST
Phase 3 took 825.528186083 Seconds
Phase 1
All : 4.14037817873
Query 3.1 : 22.3941755424
Query 3.2 : 50.2415282245
Query 3.3 : 0.855944446087
Read : 0.072270228068
Write : 0.0681956933339
Phase 2
All : 20.4774304217
Query 3.1 : 10.1777444873
Query 3.2 : 54.5941499936
Query 3.3 : 0.0601243863106
Read : 0.0342762567997
Write : 0.0302579402924
Phase 3
All : 5.02951330086
Query 3.1 : 9.73035276413
Query 3.2 : 60.2849013572
Query 3.3 : 0.0723288316727
Read : 0.0303373830795
Write : 0.0222294282913
KV Bench v1.0.3 score: 9.88244063377
iMac:KVBench dave$
------------------------------------------------------------------
------------------------------------------------------------------
----------------------
On Tue, Oct 27, 2015 at 8:18 PM, Garren Smith <[email protected]> wrote:
> Hi Dave,
>
> This is very cool. Do you have the results and the scripts you used to
> benchmarch CouchDB?
>
> Cheers
> Garren
>
> On Thu, Oct 22, 2015 at 3:13 PM, Dave Amies <[email protected]> wrote:
>
> > Hi All,
> >
> > I'm sure by now most of you will have read at least some parts of this
> > guide:
> >
> > http://guide.couchdb.org/draft/performance.html
> >
> > I was reading it the other day and noticed the "Call to Arms" section at
> > the bottom of the page. I don't know if there are already any
> benchmarking
> > tools out there, but I decided to try writing one. Hopefully the one I
> have
> > written will be useful.
> >
> > About my background, for my day job i am a performance tester, usually
> > specialising in Loadrunner, so this project was something to keep my mind
> > occupied while waiting for my test system to be rebuilt. Given this I
> have
> > only spent a few hours on it and so there is probably still room for
> > improvement, this email is about finding out if there is interest or if
> > this will be useful to the CouchDB community, so really should I continue
> > developing this tool, or am I wasting my time?
> >
> > In designing this benchmarking utility I reflected on all the systems I
> > have tested and tried to come up with some common areas where database
> > systems suffer in performance. Then bearing in mind the fundamental
> > differences between traditional databases and NoSQL databases
> (particularly
> > CouchDB) I tried to construct some some common database usage scenarios.
> >
> > The 3 scenarios I came up with are:
> >
> > 1. Write heavy (each user performs 12 writes, 6 reads and 3 searches /
> > index queries)
> > 2. Index / Query / Search heavy (each user performs 1 write, 2 reads
> and
> > 6 searches / index queries)
> > 3. Read Heavy (each user performs 1 writes, 10 reads and 3 searches /
> > index queries)
> >
> > I have tried out my benchmarking tool on a couple of machines so far, in
> > these tests I managed to cause CouchDB to encounter the following
> > situations:
> >
> > 1. Performance degradation due to being Disk IO bound
> > 2. Performance degradation due to being Memory bound
> > 3. Performance degradation due to being CPU bound
> > 4. Couch DB crashed
> > 5. Benchmarking completed successfully and produce a performance score
> >
> > Based on these results I believe I have created an effective tool for
> > benchmarking, so I decided the best next step was to release the tool as
> an
> > open source project, so I created a github project which can be found
> here:
> > https://github.com/damies13/kvbench. Here you will the readme file
> > describes the 3 scenarios in more detail, the benchmark definition or
> > design and also the pre benchmark data priming. You will also find here
> the
> > python script that is the benchmarking tool and some instructions for
> > setting up a couch db database for the benchmarking process.
> >
> > As this is getting long i'll wrap up by noting that I deliberately did
> not
> > use the python couchdb libraries but instead I used the requests library
> > (standard http) and json library because I wanted to keep the code as
> > generic as possible, the intention is that this benchmarking tool should
> be
> > able to be used to benchmarking any key / value store, whether that be a
> > document based NoSQL, and Key Value based NoSQL database or some other
> Rest
> > API / engine (e.g. backed by a traditional database).
> >
> > I look forward to some feed back, hopefully I have created something
> > useful.
> >
> > Sincerely,
> >
> > Dave.
> >
>
