raghavyadav01 opened a new issue, #20083: URL: https://github.com/apache/pinot/issues/20083
TEXT_SCORE builds a map of every document in the segment that matches the text query (about 10 bytes per match) before the projection reads the scores, no matter how selective the rest of the filter is. A broad term under a narrow filter (for example `TEXT_SCORE(title, 'usb')` with `WHERE brand = 'x'`) scores the whole segment to return a handful of rows. Proposal: when the filter is selective, score lazily for the doc ids in each block, using Lucene's `Scorer` advanced to those ids, or pass the filter's bitmap into the search as a filter query so only allowed docs are scored. Keep the current eager map when most matches survive the filter. Measure heap and latency on a broad-term, narrow-filter query before and after. Follows #20075. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
