deepthi912 opened a new pull request, #16832: URL: https://github.com/apache/pinot/pull/16832
In cases where the number of docs in a segment exceed far beyond 10K, and the cardinality percent still remains low even if the cardinality of the dictionary exceeds 10k. So having a configurable parameter for the user to switch between RAW scan and DICTIONARY based scan is necessary. This PR introduces this switch in 2 ways: 1) Always use Dictionary scan using query option - `useDictForRegexpLikePredicate` **Ex:** SELECT COUNT(*) FROM mytable WHERE REGEXP_LIKE(NewAddedSVJSONDimension, '.*') OPTION(useDictForRegexpLikePredicate=true) 2) If the `useDictForRegexpLikePredicate` is not set to true, then check the threshold limit on the dict usage from `regexpDictCardinalityThreshold`. By default this is set to 10K. **Ex:** SELECT COUNT(*) FROM mytable WHERE REGEXP_LIKE(NewAddedSVJSONDimension, '.*') OPTION(regexpDictCardinalityThreshold=50000) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
