gortiz opened a new pull request, #16658: URL: https://github.com/apache/pinot/pull/16658
We detected an issue in MSE when using [basic null support](https://docs.pinot.apache.org/developers/advanced/null-value-support#basic-null-support). Specifically, queries like: ```sql SELECT 1 FROM table WHERE salary IS NOT NULL AND salary <> 0 ``` Are transformed into: ```sql SELECT 1 FROM table WHERE salary <> 0 ``` This transformation is correct in terms of SQL given `salary <> 0` implies that `salary IS NOT NULL`. But when Pinot runs in basic null handling mode, we don't fulfill that SQL property. Instead, `salary IS NOT NULL` should be interpreted as `salary <> -2147483648` (or the equivalent default null value), which shouldn't be removed from the query. Calcite applies the optimization at the simplification level, which is something we cannot configure. Therefore, the way I fixed the issue is by adding 2 PinotOperatorTable instances: one when null handling is enabled and one when it is disabled: 1. When null handling is enabled, the PinotOperatorTable used is the same as the one used before this patch 2. When null handling is disabled, the PinotOperatorTable used includes a custom signature for the IS NULL and IS NOT NULL operators. These new operators are very similar to the default ones, but return SqlKind.OTHER_FUNCTION instead of SqlKind.IS_NULL and SqlKind.IS_NOT_NULL. This is enough to skip the simplification at Calcite level. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
