Brijesh-Thakkar commented on code in PR #22640:
URL: https://github.com/apache/datafusion/pull/22640#discussion_r3378300518
##########
datafusion/expr/src/logical_plan/plan.rs:
##########
@@ -705,20 +705,76 @@ impl LogicalPlan {
}))
}
LogicalPlan::Union(Union { inputs, schema }) => {
- let first_input_schema = inputs[0].schema();
- if schema.fields().len() == first_input_schema.fields().len() {
- // If inputs are not pruned do not change schema
- Ok(LogicalPlan::Union(Union { inputs, schema }))
- } else {
- // A note on `Union`s constructed via `try_new_by_name`:
- //
- // At this point, the schema for each input should have
- // the same width. Thus, we do not need to save whether a
- // `Union` was created `BY NAME`, and can safely rely on
the
- // `try_new` initializer to derive the new schema based on
- // column positions.
- Ok(LogicalPlan::Union(Union::try_new(inputs)?))
+ // Fast path: if all inputs structurally match the cached
schema
+ // (field count, types, names, qualifiers, nullability) then no
+ // recomputation is needed and we avoid any allocation.
+ let schemas_match = inputs.iter().all(|input| {
+ let input_schema = input.schema();
+ schema.fields().len() == input_schema.fields().len()
+ && schema.iter().zip(input_schema.iter()).all(
+ |((q1, f1), (q2, f2))| {
+ q1 == q2
+ && f1.name() == f2.name()
Review Comment:
For position-based Union (try_new), schema names come exclusively from
inputs[0], so checking names/qualifiers against non-first inputs was overly
strict.
Fixed: names and qualifiers are now only checked against inputs[0]; data
types and nullability are checked across all inputs.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
