cloud-fan commented on PR #53229: URL: https://github.com/apache/spark/pull/53229#issuecomment-3584372833
to share some initial thoughts: - The behavior of `SET target.struct_col = source.struct_col` is quite intuitive: assignment means we fully assign the right side value to the left side variable. After schema evolution or coercion, these two structs should have the same schema, and we simply write the source struct to the table, no matter it's null or it has null fields. - If we don't want this "full assignment" behavior, but retain the original values of some target struct fields that are not present in the source struct, we can follow `SET *` and support this syntax: `SET target.struct_col.* = source.struct_col.*`. - Let's say if the struct is deeply nested, and we want to control how deep we expand the fields during assignment, we can make the syntax more flexible. For example, `SET target.struct_col.*.* = source.struct_col.*.*` means expand two levels, `.**` means expands all levels until hit the leaf field. - Table row is similar to a struct, so the syntax should be consistent. e.g. `UPDATE SET *` only expand one level, which means assign top-level columns from source to target. `UPDATE SET *.*` means two levels and `UPDATE SET **` means all levels. For full assignment, it's like `UPDATE SET target_table = source_table`, but we likely don't want it as it's very weird. - In reality, we likely don't need to be so flexible, having `*` and `**` should be good enough for expanding one level and all levels. @szehon-ho let's keep the existing code if they may still be useful to implement the new syntaxes. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
