kumarUjjawal commented on PR #25023: URL: https://github.com/apache/datafusion/pull/25023#issuecomment-5572699236
> > The documentation .. is no longer accurate. It says structs without overlapping field names are rejected, but this change allows an empty source struct to cast to an empty or fully nullable target struct. Could we update this sentence to document the empty source exception? > > I reviewed the documentation and I'm now wondering if this function should actually be even more flexible. It states > > > structs should always be allowed to cast to other structs > > but then requires at least one overlapping field by name. Should we just drop the overlap check and let the nullability check that comes after it deal with the case where mapping is not possible instead? I think it was introduced in [PR #19955](https://github.com/apache/datafusion/pull/19955) and was DataFusion 53 breaking change because unrelated structs could otherwise silently lose data. For example, casting {left: 1} to {alpha: INT NULL} would discard left and produce {alpha: NULL}. The nullability check only protects nonnullable target fields, so it cannot replace the overlap check. The empty source case is different because there are no source fields to discard. I suggest keeping the current exception and updating the documentation around [nested_struct.rs lines 39 to 45](https://github.com/apache/datafusion/blob/7127cf6c8956c641cb3f3dbfaf907c6b5c4e882b/datafusion/common/src/nested_struct.rs#L39-L45) to say that a nonempty source requires at least one overlapping field name, while an empty source can cast to an empty or fully nullable target. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
