johanl-db commented on PR #56370: URL: https://github.com/apache/spark/pull/56370#issuecomment-4657314238
> it looks good, but should we add some more detail? > > * Only works with DSV2 data sources supporting this feature > * Should we document more specifics like type widening, add new column, and missing source column handling? > * And also the behavior for by position vs by name +1, this should call out that it allows automatically evolving schema to add new columns and change data types based on the connector capability. I wouldn't necessarily explicitly call out it requires DSv2 (since users probably don't know / care about DSv1 vs. DSv2) but just let that fall under 'connector capability' Also call that columns and nested fields are matched by position between the input and target schema, unless `BY NAME` is specified in which case columns and fields are matched by name. We can leave more detailed info about e.g. missing columns undocumented for now imo -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
