matanper commented on issue #70414: URL: https://github.com/apache/doris/issues/70414#issuecomment-6761046139
**Update after measuring on a real workload.** The workload: a selective `ARRAY_CONTAINS` term lookup ANDed with VARIANT string equalities and broad BKD equalities, about 5,100 segments per day, with a cold file cache. - **Most of the gain comes from evaluation order (item 1).** It also needs an exit once the row bitmap is empty, applied between phases and to pushed-down conjuncts too, which is what #67171 adds on master. With the array term lookup evaluated first, it emptied about 99% of segments, and the remaining index predicates on those segments were skipped. Index searcher lookups fell from 5.9 to 1.1 per segment. - **The small-candidate BKD skip (item 2) never fired on this workload.** The empty-bitmap exit handled almost every segment first. It might help when a narrow filter leaves a few rows rather than none, but the effect we measured from it was small, so I'm narrowing this proposal to item 1, plus backporting #67171 to 4.1.x. We have a patch for item 1 on 4.1.x and can open a PR. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
