# tl;dr: bench_R14B04_1.1.1_default_doc.tpl.log 0m18.749s bench_R14B04_1.2.x_default_doc.tpl.log 0m16.304s bench_R15B_1.1.1_default_doc.tpl.log 0m12.946s bench_R15B_1.2.x_default_doc.tpl.log 0m13.616s
bench_R14B04_1.1.1_nested_6k.tpl.log 1m27.267s bench_R14B04_1.2.x_nested_6k.tpl.log 0m37.910s bench_R15B_1.1.1_nested_6k.tpl.log 0m46.963s bench_R15B_1.2.x_nested_6k.tpl.log 0m33.011s bench_R14B04_1.1.1_small_doc.tpl.log 1m17.212s bench_R14B04_1.2.x_small_doc.tpl.log 1m41.383s bench_R15B_1.1.1_small_doc.tpl.log 0m52.858s bench_R15B_1.2.x_small_doc.tpl.log 1m9.043s bench_R14B04_1.1.1_wow.tpl.log 0m29.842s bench_R14B04_1.2.x_wow.tpl.log 0m24.178s bench_R15B_1.1.1_wow.tpl.log 0m20.493s bench_R15B_1.2.x_wow.tpl.log 0m19.584s (Full logs at [5]) # Description All of these are on Mac OS X 10.7.3 on an SSD. I'll be running the same set on spinning disk and then Robert N asked me to populate the DBs not using builk docs. Since that's gonna take a while, I'll probably run this overnight. All of the results are generated by my fork of Jason's slow_couchdb[1] and Filipe's seatoncouch[2]. The changes I've made is have the small_doc test run with 500k instead of 50k docs, added .view files to match the tpl files in seatoncouch/templates/* so we can have similar views use the different doc structures. I also added two scripts to orchestrate the above testing in a more automated fashion. It also allows you to run the full matrix yourself. All you need is set up homebrew allow `brew switch erlang R14B04` and R15B (which is controlled in matrix.sh[3]) and have a git checkout of the CouchDB sources that allow you to do `git checkout 1.1.1` or `1.2.x` (which is controlled in runner.sh[4], adjust the path to the git checkout there as well). matrix.sh also allows you to specify which docs to run. Please shout if you need any more info about this test run or how to run this yourself. # Analysis Inconclusive, I'l like to run this on larger dbs in general to see if there are more differences that shake out and I've yet have to run this on a spinning disk let alone another OS* or more complex view functions or larger design docs (like the one Stefan had). * It shouldn't be too much work to port slow_couchdb to other OSs, I'll definitely be looking into that, but we can do with every bit of help :) So far, I'm happy to conclude that while there are definitely provable differences, that we can live with them. Cheers Jan -- [1]: https://github.com/janl/slow_couchdb [2]: https://github.com/janl/seatoncouch [3]: https://github.com/janl/slow_couchdb/blob/master/matrix.sh [4]: https://github.com/janl/slow_couchdb/blob/master/runner.sh [5]: http://jan.prima.de/slow_couch/ssd/ On Feb 28, 2012, at 18:53 , Filipe David Manana wrote: > Jason, repeated my last test with the 1Kb docs ( > https://gist.github.com/1930804, map function > http://friendpaste.com/5C99aqXocN6N6H1BAYIigs ) to cover branch 1.1.x > as well. Here are the full results (also in > https://gist.github.com/1930807): > > > Before COUCHDB-1186 > > fdmanana 23:21:05 ~/git/hub/slow_couchdb (master)> docs=500000 > batch=5000 ./bench.sh wow.tpl > Server: CouchDB/1.2.0a-a68a792-git (Erlang OTP/R14B03) > {"couchdb":"Welcome","version":"1.2.0a-a68a792-git"} > > [INFO] Created DB named `db1' > [INFO] Uploaded 5000 documents via _bulk_docs > (....) > [INFO] Uploaded 5000 documents via _bulk_docs > Building view. > {"total_rows":500000,"offset":0,"rows":[ > {"id":"00051ef7-d735-48d7-9ba8-5a21a86e8d57","key":["dwarf","assassin"],"value":[{"x":31227.35,"y":31529.73},{"x":116667.85,"y":82008.25},{"x":224.11,"y":36652.41},{"x":128565.95,"y":6780.2},{"x":165230.43,"y":176208.63}]} > ]} > > real 5m6.676s > user 0m0.009s > sys 0m0.010s > > > After COUCHDB-1186 > > fdmanana 23:50:07 ~/git/hub/slow_couchdb (master)> docs=500000 > batch=5000 ./bench.sh wow.tpl > Server: CouchDB/1.2.0a-f023052-git (Erlang OTP/R14B03) > {"couchdb":"Welcome","version":"1.2.0a-f023052-git"} > > [INFO] Created DB named `db1' > [INFO] Uploaded 5000 documents via _bulk_docs > (....) > [INFO] Uploaded 5000 documents via _bulk_docs > Building view. > {"total_rows":500000,"offset":0,"rows":[ > {"id":"00051ef7-d735-48d7-9ba8-5a21a86e8d57","key":["dwarf","assassin"],"value":[{"x":31227.35,"y":31529.73},{"x":116667.85,"y":82008.25},{"x":224.11,"y":36652.41},{"x":128565.95,"y":6780.2},{"x":165230.43,"y":176208.63}]} > ]} > > real 5m1.395s > user 0m0.008s > sys 0m0.010s > > > After COUCHDB-1186 + better queueing patch > (http://friendpaste.com/178nPFgfyyeGf2vtNRpL0w) > > fdmanana 00:14:25 ~/git/hub/slow_couchdb (master)> docs=500000 > batch=5000 ./bench.sh wow.tpl > Server: CouchDB/1.2.0a-f023052-git (Erlang OTP/R14B03) > {"couchdb":"Welcome","version":"1.2.0a-f023052-git"} > > [INFO] Created DB named `db1' > [INFO] Uploaded 5000 documents via _bulk_docs > (....) > [INFO] Uploaded 5000 documents via _bulk_docs > Building view. > {"total_rows":500000,"offset":0,"rows":[ > {"id":"00051ef7-d735-48d7-9ba8-5a21a86e8d57","key":["dwarf","assassin"],"value":[{"x":31227.35,"y":31529.73},{"x":116667.85,"y":82008.25},{"x":224.11,"y":36652.41},{"x":128565.95,"y":6780.2},{"x":165230.43,"y":176208.63}]} > ]} > > real 4m48.175s > user 0m0.008s > sys 0m0.009s > > > CouchDB branch 1.1.x > > fdmanana 08:16:58 ~/git/hub/slow_couchdb (master)> docs=500000 > batch=5000 ./bench.sh wow.tpl > Server: CouchDB/1.1.2a785d32f-git (Erlang OTP/R14B03) > {"couchdb":"Welcome","version":"1.1.2a785d32f-git"} > > [INFO] Created DB named `db1' > [INFO] Uploaded 5000 documents via _bulk_docs > (....) > [INFO] Uploaded 5000 documents via _bulk_docs > Building view. > {"total_rows":500000,"offset":0,"rows":[ > {"id":"0001c0a1-edcb-4dbc-aa9d-533c73d980cb","key":["dwarf","assassin"],"value":[{"x":62038.32,"y":105825.29},{"x":90713.13,"y":128570.97},{"x":43836.37,"y":80517.12},{"x":71610.97,"y":143739.99},{"x":86038.39,"y":84731.8}]} > ]} > > real 5m44.374s > user 0m0.008s > sys 0m0.010s > > > Disk model APPLE SSD TS128C, quad core machine, 8Gb of ram. > > > > On Tue, Feb 28, 2012 at 5:17 AM, Jason Smith <[email protected]> wrote: >> Forgive the clean new thread. Hopefully it will not remain so. >> >> If you can, would you please clone https://github.com/jhs/slow_couchdb >> >> And build whatever Erlangs and CouchDB checkouts you see fit, and run >> the test. For example: >> >> docs=500000 ./bench.sh small_doc.tpl >> >> That should run the test and, God willing, upload the results to a >> couch in the cloud. We should be able to use that information to >> identify who you are, whether you are on SSD, what Erlang and Couch >> build, and how fast it ran. Modulo bugs. > > > > -- > Filipe David Manana, > > "Reasonable men adapt themselves to the world. > Unreasonable men adapt the world to themselves. > That's why all progress depends on unreasonable men."
