pgj commented on code in PR #4662:
URL: https://github.com/apache/couchdb/pull/4662#discussion_r1263044804
##########
src/docs/src/api/database/find.rst:
##########
@@ -1508,22 +1514,151 @@ it easier to take advantage of future improvements to
query planning
"title"
],
"partitioned": false
+ "index_candidates": [
+ {
+ "index": {
+ "ddoc": null,
+ "name": "_all_docs",
+ "type": "special",
+ "def": {
+ "fields": [
+ {
+ "_id": "asc"
+ }
+ ]
+ }
+ },
+ "usable": true,
+ "reason": "unfavored_type",
+ "ranking": 1
+ }
+ ],
+ "selector_hints": {
+ "indexable_fields": {
+ "json": [
+ "year"
+ ]
+ },
+ "unindexable_fields": {
+ "json": []
+ }
+ }
}
+.. _find/index_selection:
+
Index selection
===============
:ref:`_find <api/db/_find>` chooses which index to use for responding
-to a query, unless you specify an index at query time.
+to a query, unless you specify an index at query time. In this
+section, a brief overview of the index selection process is presented.
-The query planner looks at the selector section and finds the index with the
-closest match to operators and fields used in the query. If there are two
-or more json type indexes that match, the index with the smallest
-number of fields in the index is preferred.
-If there are still two or more candidate indexes,
-the index with the first alphabetical name is chosen.
+.. note::
+
+ It is good practice to specify indexes explicitly in your
+ queries. This prevents existing queries being affected by new
+ indexes that might get added in a production environment.
.. note::
- It is good practice to specify indexes explicitly in your queries. This
- prevents existing queries being affected by new indexes that might get
added
- in a production environment.
+
+ Both the :ref:`_explain <api/db/find/explain>` and :ref:`_find
+ <api/db/_find>` endpoints rely on the same index selection logic.
+ But :ref:`_explain <api/db/find/explain>` is a bit more elaborate,
+ therefore it could be used for simulation and exploration. In the
+ output, reasons for discarding indexes are placed in the ``reason``
+ field of the JSON objects under ``index_candidates``. Each reason
+ has a specific code, which will be mentioned at the relevant
+ subsections below.
+
+The index selection happens in multiple rounds.
+
+First, all the indexes for the database are collected. Whenever
+possible, the result includes the special index called `all docs`
+which is a trivial index on the ``_id`` field. This is reserved as a
+catch-all answer when no other suitable indexes could be found, but
+its use of discouraged for performance reasons.
+
+In the next round, :ref:`partial indexes <find/partial_indexes>` are
+eliminated unless they are not requested explicitly, they are not
+considered automatically. When not specified, partial indexes are
+pushed back to the list of candidate indexes with the ``is_partial``
+code.
+
+After that, indexes that are different from the partition for which
+the query was issued are removed. That is, if the query applies to
+specific partition, global indexes and indexes for every other
+partition are discarded because they would not be able to support the
+execution, and vice versa. Such indexes are associated with the
+``scope_mismatch`` reason code.
+
+The remaining indexes are filtered by a series of usability checks.
+These checks depend on the type of index.
+
+- ``"special"``: The selector must reference the ``_id`` field,
+ sorting could be requested on ``_id`` only.
+
+- ``"json"``: The selector must not request a free-form text search
+ via the ``$text`` operator, and all the columns that are restricted
+ by the selector must exist in the index. Sort fields are required
+ to exist as well.
+
+- ``"text"``: The selector must contain references to fields that are
+ indexable. Sometimes it cannot be pre-determined if the given field
+ would exist in the text index, hence it will not be supported.
Review Comment:
I agree that using the concept "indexable" does not help here. The
corresponding code has a relatively complicated logic to determine whether a
field in the selector is indexable, I might be failing to translate it properly.
Mango queries first translated to some abstract intermediate data structure
which is then translated further to its counterpart in Lucene. These are the
specific Erlang function alternatives where a field is excluded, with the
respective comments to explain (from `mango_idx_text`):
```erlang
% forces "$exists" : false to use _all_docs
indexable_fields(_, {op_not, {_, false}}) ->
[];
% fieldname.[]:length is not a user[-]defined field.
indexable_fields(Fields, {op_field, {[_, <<":length">>], _}}) ->
Fields;
% ...
% In this particular case, the [L]ucene index is doing a field_exists query
% so it is looking at all sorts of combinations of field:* and field.*
% We don't add the field because we cannot predetermine what field will
exist.
% Hence we just return Fields and make it less restrictive.
indexable_fields(Fields, {op_fieldname, {_, _}}) ->
Fields;
% Similar idea to op_fieldname but with fieldname:null
indexable_fields(Fields, {op_null, {_, _}}) ->
Fields;
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
