Dandandan commented on a change in pull request #1088:
URL: https://github.com/apache/arrow-datafusion/pull/1088#discussion_r725249253
##########
File path: datafusion/src/optimizer/projection_push_down.rs
##########
@@ -417,9 +416,15 @@ fn optimize_plan(
})
.collect::<Result<Vec<_>>>()?;
+ let mut new_schema = Vec::new();
+ for df_field in schema.fields() {
+ if union_required_fields.contains(df_field.field()) {
Review comment:
This has quadratic complexity, which can be problematic when we have
many fields but I think we have more areas in the code base with this issue, so
not really needed to solve now.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
