Github user cloud-fan commented on the issue:
https://github.com/apache/spark/pull/14389
currently we have 3 different semantics for finding a wider type:
1. `findTightestCommonType`: try to find a wider type, but skip decimal
types
2. `findTightestCommonTypeToString`: similar to rule 1, but if one side is
string, it can promote the other side to string
3. `findWiderCommonType`: similar to rule 2, but handles decimal type, and
truncate it if necessary.
It makes sense to have 2 different semantics for string promotion, however,
I don't think we need 2 different semantics for decimal types. There is no such
a function that need its arguments to be same type, but can't accept precision
loss for decimal type. I think it's better to run the query and log a warning
message about the truncation, rather than fail the query.
We only need 2 semantics:
1. `findWiderType`: try to find the wider type, including decimal type, and
truncate if necessary
2. `findWiderTypeAndPromoteToString`: similar to rule 1, but handles string
promotion.
We also need to add some checks before applying the type widen rules, to
avoid conflicting with `DecimalPrecision`, which defines some special rules for
binary arithmetic about decimal type.
@petermaxlee in your PR, the string promotion semantic is hidden in
`findWiderType`, and makes `greatest` and `least` accept string promotion,
which is not expected. What do you think?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
