nickva commented on PR #5625: URL: https://github.com/apache/couchdb/pull/5625#issuecomment-3208291841
I found a 12 node cluster (80 cpus, 128GB ram per node) and ran a high concurrency benchmark with 10k virtual users reading, updating and creating documents. To generate the load used k6 with this script: https://gist.github.com/nickva/d5312248635534315e76d25b31a86faa - starts with a 1M docs - then 10k users connect to the cluster and each do: * read a random document from the initial 1M * update the document * insert a new document (with a random uuid) - run for 10 minutes after initial setup and report stats Results for main: ``` HTTP http_req_duration..............: avg=1.1s min=5.01ms med=1.09s max=3.81s p(90)=2.32s p(95)=2.52s { expected_response:true }...: avg=1.1s min=5.01ms med=1.09s max=3.81s p(90)=2.32s p(95)=2.52s { name:bulk_docs }...........: avg=188.58ms min=124.78ms med=188.34ms max=517.28ms p(90)=204.56ms p(95)=208.42ms { name:get_doc }.............: avg=40.97ms min=5.01ms med=39.47ms max=3.49s p(90)=52.12ms p(95)=55.7ms { name:post_doc }............: avg=1.34s min=29.05ms med=1.25s max=3.81s p(90)=2.13s p(95)=2.35s { name:put_doc }.............: avg=1.92s min=75.03ms med=2.01s max=3.81s p(90)=2.58s p(95)=2.73s http_req_failed................: 0.13% 5617 out of 4183555 http_reqs......................: 4183555 4969.270526/s EXECUTION iteration_duration.............: avg=4.31s min=1.31s med=4.34s max=8.35s p(90)=4.94s p(95)=5.12s iterations.....................: 1394184 1656.026384/s vus............................: 390 min=0 max=10000 vus_max........................: 10000 min=10000 max=10000 NETWORK data_received..................: 2.6 GB 3.1 MB/s data_sent......................: 721 MB 856 kB/s running (14m01.9s), 00000/10000 VUs, 1394184 complete and 0 interrupted iterations ``` Results with the bt cache ``` HTTP http_req_duration..............: avg=743.25ms min=3.62ms med=896.05ms max=3.4s p(90)=1.41s p(95)=1.56s { expected_response:true }...: avg=742.56ms min=3.62ms med=895.55ms max=3.03s p(90)=1.41s p(95)=1.56s { name:bulk_docs }...........: avg=112.65ms min=93.67ms med=109.18ms max=424.51ms p(90)=117.06ms p(95)=119.81ms { name:get_doc }.............: avg=13.6ms min=3.62ms med=11.68ms max=2.35s p(90)=15.95ms p(95)=17.6ms { name:post_doc }............: avg=1.09s min=16.85ms med=1.06s max=2.98s p(90)=1.49s p(95)=1.64s { name:put_doc }.............: avg=1.11s min=117.82ms med=1.08s max=3.4s p(90)=1.51s p(95)=1.65s http_req_failed................: 0.09% 5148 out of 5580982 http_reqs......................: 5580982 7284.775804/s EXECUTION iteration_duration.............: avg=3.23s min=1.63s med=3.19s max=6.92s p(90)=3.8s p(95)=3.99s iterations.....................: 1859993 2427.822201/s vus............................: 959 min=0 max=10000 vus_max........................: 10000 min=10000 max=10000 NETWORK data_received..................: 3.5 GB 4.5 MB/s data_sent......................: 927 MB 1.2 MB/s running (12m46.1s), 00000/10000 VUs, 1859993 complete and 0 interrupted iterations ``` <img width="792" height="1279" alt="btree_bench" src="https://github.com/user-attachments/assets/89bebf02-114d-4b67-84b1-e1f65a142bc8"; /> Based on the metrics: * We can do 9.6k vs http 7.3k requests / sec in favor of the btree cache. * In the read/write metrics the first grey bump is the initial bulk load. We can do 18k vs 12k inserts /sec in favor of the cache. During the main part of the benchmark 10k db reads vs 8k db reads also in favor of the btree-cache. * In the latency graph all latencies, but especially the higher ones p90, p99 are improved (1.4 sec vs 2.2 sec). * Memory usage is a bit better with the cache probably because we pile up more client processes. After the benchmark ran I queried the stats and got: ``` curl http://$node/_stats/couchdb/bt_engine_cache { "full": { "value": 0, "type": "counter", "desc": "number of times bt_engine cache was full" }, "hits": { "value": 24371805, "type": "counter", "desc": "number of bt_engine cache hits" }, "misses": { "value": 374470, "type": "counter", "desc": "number of bt_engine cache misses" } } ``` We saved 24M btree block reads and didn't hit the full limit/ -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: notifications-unsubscr...@couchdb.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
