cloud-fan commented on issue #26811: [SPARK-29600][SQL] array_contains built in function is not backward compatible in 3.0 URL: https://github.com/apache/spark/pull/26811#issuecomment-564526877 #26485 is accepted because it just makes the type coercion consistent between In and InSubquery. I think it's a wrong design that, we have many type coercion rules for different operators, and have a `DecimalPrecision` rule to handle the decimal cases for all operators. This means that an operator needs to be dealt with twice: once in the normal type coercion rule, once in the `DecimalPrecision`. For `array_contains`, it calls `findTightestCommonType` which skips decimal type. We should also deal with `array_contains` in `DecimalPrecision`. This is not ideal but should be the right fix for now. In the future, we should really refactor this part. We should either separate type coercion rules by operators, or implement type coercion inside operators.
---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected] With regards, Apache Git Services --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
