guptashivam04 commented on PR #69438: URL: https://github.com/apache/doris/pull/69438#issuecomment-6397760070
Thanks for the detailed review, @airborne12! **Delete-sign options** and **SNII norm sums** were introduced by this PR, so I'll fix both here: - The statistics will count delete-sign rows when the scan doesn't filter `__DORIS_DELETE_SIGN__`, so `skip_delete_sign=true` and `show_hidden_columns=true` score with statistics that match the returned rows. - SNII norm sums will be cached per segment (like CLucene's), so repeated queries at the same read version read no norms. I'll measure the repeated-query cost before and after. This also covers the bot's P2 on the SNII selected-row buffer: the remaining reads will be batched and tracked. - I'll also address the bot's other P2: the statistics' CLucene norm reads will carry the query's I/O context without keeping it on the cached searcher. - I'll add regression tests for the delete-sign options across V2/V3/SNII. **MoW light deletes** were already counted in the statistics before this PR. Supporting them means evaluating delete predicates inside the statistics (reading the predicate columns of each older segment, with caching), which is a sizeable change of its own. Since light delete is off by default for MoW tables, could I handle it, with its V2/V3/SNII regression tests, in a follow-up PR right after this one? This PR fixes a production issue for us (replicas returning different `score()` values with CDC upserts and deletes), so getting the default paths merged first would help a lot. Does this plan work for you? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
