jiangxt2 commented on PR #58029: URL: https://github.com/apache/spark/pull/58029#issuecomment-5658558519
> forceNullable(DecimalType, TimestampType) is now true for types that can overflow (DECIMAL(20,0), DECIMAL(13,0), …). That is the right Cast model, but it is a second user-facing change that should be pointed out properly: > > * MAP<DECIMAL(20,0), T> → MAP<TIMESTAMP, T> is now rejected (map keys cannot be null). > * ARRAY/STRUCT casts into a non-nullable timestamp element/field are now rejected. > > Safe types such as DECIMAL(10,0) and DECIMAL(12,0) stay non-nullable and keep those complex casts. The tests cover the map-key case; the migration guide only mentions overflow NULL / CAST_OVERFLOW. A brief sentence on the analysis failure would help, for example, something like: > > > Casting a decimal type that can overflow to timestamp is now treated as nullable. Casts into map keys, or into non-nullable array elements / struct fields, that used to analyze successfully may now fail type checking. Thanks, agreed. I updated the migration guide to document that, for non-ANSI CAST and TRY_CAST, decimal-to-timestamp casts that may overflow are nullable; casts into map keys or non-nullable array elements and struct fields may now fail analysis, while safe decimal types retain the previous behavior. I also added Array/Struct nullability coverage. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
