clintropolis opened a new pull request, #14096: URL: https://github.com/apache/druid/pull/14096
### Description This PR fixes a bug when trying to filter `COMPLEX<json>` columns where omission of setting the `isFilterable` flag of `ColumnCapabilities` when reading the column was incorrectly causing the column to be treated as if it were all null values. The deserializers have been updated to set `isFilterable` to true, allow `COMPLEX<json>` columns to be used in expression filters, which can often be generated by the SQL planner for queries like the added SQL test. I'm not really certain how useful this behavior is, and it doesn't seem very intuitive, so we might want to consider changing this in the future so that `COMPLEX` columns produce some form of error when used with filters which do not support the type. It might have made sense in the past when there was not really a possible way for complex types to participate in filtering, but since expressions now support complex types it becomes relatively easy to imagine using these columns in filters when used with expressions which can extract meaningful data from the complex types (such as get the estimate from an HLL sketch, or .. extract a value from a json blob as in this case). #### Release note Fixed a bug when using expression filters for filtering `COMPLEX<json>` columns or values extracted from them using `JSON_VALUE` or any other nested column function, where the column would be incorrectly treated as all null values. <hr> This PR has: - [x] been self-reviewed. - [x] a release note entry in the PR description. - [x] added comments explaining the "why" and the intent of the code wherever would not be obvious for an unfamiliar reader. - [x] added unit tests or modified existing tests to cover new code paths, ensuring the threshold for [code coverage](https://github.com/apache/druid/blob/master/dev/code-review/code-coverage.md) is met. - [x] been tested in a test Druid cluster. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
