paleolimbot commented on issue #22079:
URL: https://github.com/apache/datafusion/issues/22079#issuecomment-4671316688
Sorry, I got hung up because the existing behaviour was tricky to untangle
and then forgot about it 😬 . I was also a little bit demoralized from having
fixed this once before and not being sure that anybody agrees with me about
what should happen here. I'm happy to take another stab this week...I think
something like this will work:
- Logical casts always propagate source metadata and nullability, except for
the extension name and extension metadata key which are overwritten from the
logical cast target field.
- The planner refuses to plan a cast where the target field is an extension
type. The other PR can do that once this is sorted (introduces
`CastExtension`), or an optimizer rule does just fine for this.
- Physical casts no longer have a target field...replaced with `target:
DataType, nullability: NullabilityCast`, where `enum NullabilityCast {
PropagateSource, ForceFalse, ForceTrue }`, or something like that.
The issue with the physical cast is that the sentinel field is
confusing...two different operations (aligning nullability, performing a
computation to align a bare arrow data type) are getting forced on one
operator. I'm a little worried that the approach could still end up with a
metadata mismatch for the use of the physical cast to align a target schema,
and I would still prefer a cast to drop metadata completely (easier to get
right, more consistent with scalar functions).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
