eiri commented on issue #512: Couchdb 3298 optimize writing btree nodes URL: https://github.com/apache/couchdb/pull/512#issuecomment-300536273 I've ran a separate test comparing view disk size difference between master and this PR. My setup is a single view index emitting pseudo-random value of fixed size 32k with pre-generated sequential and (on a second run) random keys. I've inserted 10000 documents in one-by-one fashion, triggering view reindexing after each insert to maximize number of changes for view's btree. Index sizes were taken after each 10 inserts as reported by `_info` endpoint. The results are pretty impressive. I see 129% of disk size improvement for sequential keys and 51% disk size improvement for random keys. Obviously this is an extreme setup done to maximize "garbage" and emphasize effect of this patch, in real life the numbers are going to be more humble, but still this is a great improvement. [Here is the complete jupiter's notebook with results](https://cdn.rawgit.com/eiri/d6a5abc7dbf180eabf965fe54cf5a215/raw/bc4704c634b82301d6077dc5308bad3c4c20fa3b/Compare%20view%20sizes%20for%20COUCHDB-3298.html) Out of curiosity I've ran cfcheck's btree analysis on final view index for each case, to calculate number of kp and kv nodes and max and min length of the nodes per branch/leaf. Here the results: Sequential keys: View's btree before: `{ "depth": 4, "kp_nodes": { "count": 209, "min": 8, "max": 25 }, "kv_nodes": { "count": 5000, "min": 2, "max": 25 } }` View's btree after the patch: `{ "depth": 5, "kp_nodes": { "count": 419, "min": 3, "max": 13 }, "kv_nodes": { "count": 5001, "min": 1, "max": 13 } }` Random keys: View's btree before: `{ "depth": 6, "kp_nodes": { "count": 865, "min": 2, "max": 18 }, "kv_nodes": { "count": 4286, "min": 2, "max": 18 } }` View's btree after the patch: `{ "depth": 5, "kp_nodes": { "count": 833, "min": 1, "max": 16 }, "kv_nodes": { "count": 5480, "min": 1, "max": 16 } }` I second Nick, this is a great work! +1 ---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected]
With regards, Apache Git Services
