somu-imply opened a new pull request #12195:
URL: https://github.com/apache/druid/pull/12195
An existing issue is when users execute a query with a heavy load of numeric
IN filters. Internally these are converted to bound filters one per condition
which leads to queries stalling historicals. To preempt and catch this we keep
the max limit for numeric IN filters to a query to 10K. This can be configured
by user through query context as `"maxNumericInFilters": value`
Current behavior now is:
```
SELECT *
FROM wikipedia
WHERE metroCode IN (609,517,817,101,102,103)
```
This runs as the default is set to 10,000.
Now if we change the query context as
```
{
"maxNumericInFilters": 4
}
```
The following query now works as
```
SELECT *
FROM wikipedia
WHERE metroCode IN (609,517,817,101,102,103)
Error: Unknown exception
Filters for column [metroCode] exceeds configured filter limit of [4]. Cast
[6] values to String
org.apache.druid.sql.calcite.rel.CannotBuildQueryException
```
If the query is now corrected, the same query goes through InFilters and not
BoundFilters and the error is not shown and the query passes. An example is
```
SELECT *
FROM wikipedia
WHERE metroCode IN ('609','517','817','101','102','103')
```
The value through query context cannot be set to anything below 1 and would
throw an error in such a case
This PR has:
- [ ] been self-reviewed.
- [ ] using the [concurrency
checklist](https://github.com/apache/druid/blob/master/dev/code-review/concurrency.md)
(Remove this item if the PR doesn't have any relation to concurrency.)
- [ ] added documentation for new or modified features or behaviors.
- [ ] added Javadocs for most classes and all non-trivial methods. Linked
related entities via Javadoc links.
- [ ] added or updated version, license, or notice information in
[licenses.yaml](https://github.com/apache/druid/blob/master/dev/license.md)
- [ ] added comments explaining the "why" and the intent of the code
wherever would not be obvious for an unfamiliar reader.
- [ ] added unit tests or modified existing tests to cover new code paths,
ensuring the threshold for [code
coverage](https://github.com/apache/druid/blob/master/dev/code-review/code-coverage.md)
is met.
- [ ] added integration tests.
- [ ] been tested in a test Druid cluster.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
