parkertimmins commented on PR #16669: URL: https://github.com/apache/lucene/pull/16669#issuecomment-5701885764
Nice PR! I recently opened a [PR](https://github.com/apache/lucene/pull/16568) which did something similar, but in a less general way (It only applied to non-scoring queries). This is a better approach. That PR was to speed up not-equals queries, where the excluded filter matches relatively few values. It includes a benchmark showing that `must_not` performs poorly when it's excluded query is selective, as compared to `filter`, which perform well when it's included query is selective. The benchmark uses a single range query and varies it's selectivity. I ran the benchmark on this PR and got good results! | Inner query selectivity | Filter default | MUST_NOT default | Filter Panama | MUST_NOT Panama | |-------------|---------------|-----------------|---------------|-----------------| | 0.01 | 219.2 | 218.9 | 492.5 | 494.5 | | 0.1 | 191.4 | 189.2 | 327.6 | 331.3 | | 0.5 | 187.7 | 204.0 | 429.0 | 430.8 | | 0.9 | 540.4 | 513.1 | 472.7 | 459.1 | | 0.99 | 690.7 | 695.9 | 487.4 | 468.4 | For comparison, these are the baseline results from a few weeks ago: | Inner query selectivity | Filter default | MUST_NOT default | Filter Panama | MUST_NOT Panama | |-------------|---------------|-----------------|---------------|-----------------| | 0.01 | 236.786 ± 5.855 | 42.162 ± 5.477 | 545.464 ± 15.000 | 44.303 ± 2.776 | | 0.10 | 200.210 ± 21.402 | 43.665 ± 4.309 | 337.842 ± 5.168 | 44.643 ± 0.734 | | 0.50 | 202.529 ± 1.772 | 56.723 ± 5.886 | 470.476 ± 24.148 | 56.038 ± 0.550 | | 0.90 | 549.659 ± 69.162 | 130.545 ± 1.347 | 517.654 ± 24.159 | 136.902 ± 8.890 | | 0.99 | 778.650 ± 75.649 | 205.053 ± 21.086 | 523.284 ± 8.663 | 204.756 ± 14.010 | -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
