pgj opened a new pull request, #4512: URL: https://github.com/apache/couchdb/pull/4512
This pull request aims to implement the first phase of the covering indexes work (_"keys-only covering indexes"_) as it was described in the corresponding RFC document by @mikerhodes, see #4410 and #4413 for the details. Ideally, end users should not notice any change in the behavior but speed-ups for certain kind of queries. Performance results obtained by [k6](https://k6.io/) on my MacBook Pro 2021 M1 Max, with 10,000 documents, compared against `a9bce2f598edc8ef843baa9412c60d22157eeabf` with 3 nodes: - 1-field index covers the query: - before: `min=0.60ms max=3223.65ms p(95)=2113.95ms p(98)=2354.79ms p(99)=2505.30ms` - after: `min=0.50ms max=615.87ms p(95)=95.89ms p(98)=140.80ms p(99)=171.90ms` - 2-field index covers the query: - before: `min=73.09ms max=6245.72ms p(95)=3835.45ms p(98)=4376.79ms p(99)=5022.26ms` - after: `min=2.65ms max=579.62ms p(95)=211.18ms p(98)=264.66ms p(99)=305.37ms` - generic case: - before: `min=0.80ms max=14991.06ms p(95)=13171.23ms p(98)=13975.51ms p(99)=14456.11ms` - after: `min=0.87ms max=15379.69ms p(95)=10748.65ms p(98)=11998.88ms p(99)=12171.89ms` Please find the details of the performance evaluation in [`pgj/couchdb-k6/mango/couchdb_4482`](https://github.com/pgj/couchdb-k6/tree/main/mango/couchdb_4482). Unit test coverage increased significantly, see before: ```console $ make eunit apps=mango ==> mango (compile) ==> rel (compile) ==> couchdb (compile) ==> couchdb (setup_eunit) ==> mango (eunit) ======================== EUnit ======================== [..] module 'mango_idx_view' module 'mango_cursor_view' mango_cursor_view: does_not_refetch_doc_with_value_test...ok mango_cursor_view: doc_member_and_extract_fields_test...ok mango_cursor_view: match_and_extract_doc_match_test...ok mango_cursor_view: match_and_extract_doc_matchextract_test...ok mango_cursor_view: match_and_extract_doc_nomatch_test...ok mango_cursor_view: match_and_extract_doc_nomatch_fields_test...ok mango_cursor_view: choose_best_index_with_singleton_test...ok mango_cursor_view: choose_best_index_lowest_difference_test...ok mango_cursor_view: choose_best_index_least_number_of_fields_test...ok mango_cursor_view: choose_best_index_lowest_index_triple_test...ok [done in 0.030 s] [..] ======================================================= All 81 tests passed. [..] Code Coverage: mango_app : 0% mango_crud : 0% mango_cursor : 0% mango_cursor_special : 0% mango_cursor_text : 0% mango_cursor_view : 33% mango_doc : 3% mango_epi : 100% mango_error : 0% mango_execution_stats : 0% mango_fields : 50% mango_httpd : 0% mango_httpd_handlers : 0% mango_idx : 18% mango_idx_special : 0% mango_idx_text : 63% mango_idx_view : 1% mango_json : 10% mango_json_bookmark : 0% mango_native_proc : 5% mango_opts : 25% mango_selector : 62% mango_selector_text : 86% mango_sort : 0% mango_sup : 0% mango_util : 38% Total : 31% ==> rel (eunit) ==> couchdb (eunit) ``` and then after: ```console ==> mango (compile) ==> rel (compile) ==> couchdb (compile) ==> couchdb (setup_eunit) ==> mango (eunit) ======================== EUnit ======================== [..] module 'mango_idx_view' mango_idx_view: indexable_fields_empty_test...ok mango_idx_view: indexable_fields_and_test...ok mango_idx_view: indexable_fields_or_test...ok mango_idx_view: indexable_fields_nor_test...ok mango_idx_view: indexable_fields_all_test...ok mango_idx_view: indexable_fields_elemMatch_test...ok mango_idx_view: indexable_fields_allMatch_test...ok mango_idx_view: indexable_fields_keyMapMatch_test...ok mango_idx_view: indexable_fields_in_test...ok mango_idx_view: indexable_fields_nin_test...ok mango_idx_view: indexable_fields_not_test...ok mango_idx_view: indexable_fields_lt_test...ok mango_idx_view: indexable_fields_lte_test...ok mango_idx_view: indexable_fields_eq_test...ok mango_idx_view: indexable_fields_ne_test...ok mango_idx_view: indexable_fields_gte_test...ok mango_idx_view: indexable_fields_gt_test...ok mango_idx_view: indexable_fields_mod_test...ok mango_idx_view: indexable_fields_regex_test...ok mango_idx_view: indexable_fields_exists_test...ok mango_idx_view: indexable_fields_type_test...ok mango_idx_view: indexable_fields_size_test...ok mango_idx_view: covers_all_fields_test...ok mango_idx_view: covers_all_docs_test...ok mango_idx_view: covers_empty_index_test...ok mango_idx_view: covers_regular_index_test...ok [done in 0.081 s] module 'mango_cursor_view' mango_cursor_view: viewcbargs_test...ok mango_cursor_view: maybe_replace_max_json_test...ok mango_cursor_view: base_opts_test...ok mango_cursor_view: apply_opts_empty_test...ok mango_cursor_view: apply_opts_r_test...ok mango_cursor_view: apply_opts_conflicts_test...ok mango_cursor_view: apply_opts_sort_test...ok mango_cursor_view: apply_opts_stale_test...ok mango_cursor_view: apply_opts_stable_test...ok mango_cursor_view: apply_opts_update_test...ok mango_cursor_view: apply_opts_partition_test...[0.020 s] ok mango_cursor_view: consider_index_coverage_positive_test...ok mango_cursor_view: consider_index_coverage_negative_test...ok mango_cursor_view: derive_doc_from_index_test...ok mango_cursor_view: composite_indexes_test...ok mango_cursor_view: create_test...ok mango_cursor_view: explain_test...ok mango_cursor_view: execute_empty_test...[0.122 s] ok mango_cursor_view: execute_ok_all_docs_test...[0.316 s] ok mango_cursor_view: execute_ok_query_view_test...[0.297 s] ok mango_cursor_view: execute_error_test...[0.152 s] ok mango_cursor_view: view_cb_meta_test...[0.046 s] ok mango_cursor_view: view_cb_row_matching_regular_doc_test...[0.055 s] ok mango_cursor_view: view_cb_row_non_matching_regular_doc_test...[0.065 s] ok mango_cursor_view: view_cb_row_null_doc_test...[0.053 s] ok mango_cursor_view: view_cb_row_missing_doc_triggers_quorum_fetch_test...[0.056 s] ok mango_cursor_view: view_cb_row_matching_covered_doc_test...[0.145 s] ok mango_cursor_view: view_cb_row_non_matching_covered_doc_test...[0.052 s] ok mango_cursor_view: view_cb_row_backwards_compatible_test...[0.060 s] ok mango_cursor_view: view_cb_complete_test...[0.054 s] ok mango_cursor_view: view_cb_ok_test...[0.047 s] ok mango_cursor_view: maybe_send_mango_ping_nop_test...[0.054 s] ok mango_cursor_view: maybe_send_mango_ping_happens_test...[0.048 s] ok mango_cursor_view: ddocid_test...ok mango_cursor_view: is_design_doc_test...ok mango_cursor_view: handle_message_meta_test...ok mango_cursor_view: handle_message_row_ok_above_limit_test...[0.223 s] ok mango_cursor_view: handle_message_row_ok_at_limit_test...ok mango_cursor_view: handle_message_row_ok_skip_test...ok mango_cursor_view: handle_message_row_ok_triggers_quorum_fetch_match_test...[1.424 s] ok mango_cursor_view: handle_message_row_ok_triggers_quorum_fetch_no_match_test...ok mango_cursor_view: handle_message_row_no_match_test...ok mango_cursor_view: handle_message_row_error_test...[0.047 s] ok mango_cursor_view: handle_message_execution_stats_test...ok mango_cursor_view: handle_message_complete_test...ok mango_cursor_view: handle_message_error_test...ok mango_cursor_view: handle_all_docs_message_ddoc_test...ok mango_cursor_view: handle_all_docs_message_row_test...ok mango_cursor_view: handle_all_docs_message_regular_test...ok mango_cursor_view: does_not_refetch_doc_with_value_test...ok mango_cursor_view: doc_member_and_extract_fields_test...ok mango_cursor_view: match_and_extract_doc_match_test...ok mango_cursor_view: match_and_extract_doc_matchextract_test...ok mango_cursor_view: match_and_extract_doc_nomatch_test...ok mango_cursor_view: match_and_extract_doc_nomatch_fields_test...ok mango_cursor_view: choose_best_index_with_singleton_test...ok mango_cursor_view: choose_best_index_lowest_difference_test...ok mango_cursor_view: choose_best_index_least_number_of_fields_test...ok mango_cursor_view: choose_best_index_lowest_index_triple_test...ok mango_cursor_view: update_bookmark_keys_test...ok [done in 3.526 s] [..] ======================================================= All 157 tests passed. [..] Code Coverage: mango_app : 0% mango_crud : 0% mango_cursor : 20% mango_cursor_special : 0% mango_cursor_text : 0% mango_cursor_view : 100% mango_doc : 4% mango_epi : 100% mango_error : 0% mango_execution_stats : 63% mango_fields : 57% mango_httpd : 0% mango_httpd_handlers : 0% mango_idx : 21% mango_idx_special : 0% mango_idx_text : 63% mango_idx_view : 39% mango_json : 12% mango_json_bookmark : 25% mango_native_proc : 5% mango_opts : 27% mango_selector : 63% mango_selector_text : 86% mango_sort : 5% mango_sup : 0% Total : 48% ==> rel (eunit) ==> couchdb (eunit) ``` Tasks: - [x] Code is written and works correctly - [x] Changes are covered by tests - [x] Documentation changes were made in the `src/docs` folder -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
