clintropolis opened a new pull request, #15503: URL: https://github.com/apache/druid/pull/15503
### Description The equality filter allows automatic coercion of scalar typed columns to array columns to help assist schema migration between mvds and arrays, as well as scalars to arrays, where the scalar inputs are treated as single element arrays for the purposes of matching. However, this had an unintended side-effect of allowing the reverse that impacts only native queries, allowing using an equality filter with a scalar match value to match single element arrays of array columns, which is pretty strange (and json_value using SQL). This PR modifies the equality filter (and array indexes) to no longer allow matching single element arrays in columns with scalar values, while preserving the opposite, where scalar columns can match single element arrays. This has a side-effect that strictly impacts native query equality filter on mixed type 'auto' columns which contain arrays, that they must now be interacted with as their presenting type, so if any rows are arrays, e.g. the segment metadata and information_schema reports the type as some array type, then the native queries must also filter as if they are some array type. This does not impact SQL, which already has this limitation due to how the type presents itself, and again only impacts mixed type 'auto' columns. #### Release note Native query `equals` filter on mixed type 'auto' columns which contain arrays must now be filtered as their presenting type, so if any rows are arrays (e.g. the segment metadata and information_schema reports the type as some array type), then the native queries must also filter as if they are some array type. This does not impact SQL, which already has this limitation due to how the type presents itself, and again only impacts mixed type 'auto' columns which contain both scalars and arrays. <hr> This PR has: - [ ] been self-reviewed. - [ ] added documentation for new or modified features or behaviors. - [x] a release note entry in the PR description. - [x] added comments explaining the "why" and the intent of the code wherever would not be obvious for an unfamiliar reader. - [x] added unit tests or modified existing tests to cover new code paths, ensuring the threshold for [code coverage](https://github.com/apache/druid/blob/master/dev/code-review/code-coverage.md) is met. - [x] been tested in a test Druid cluster. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
