alamb commented on a change in pull request #619:
URL: https://github.com/apache/arrow-datafusion/pull/619#discussion_r658862205
##########
File path: datafusion/src/optimizer/projection_push_down.rs
##########
@@ -75,9 +78,12 @@ fn get_projected_schema(
//
// we discard non-existing columns because some column names are not part
of the schema,
// e.g. when the column derives from an aggregation
- let mut projection: Vec<usize> = required_columns
+ //
+ // Use BTreeSet to remove potential duplicates (e.g. union) as
+ // well as to sort the projection to ensure deterministic behavior
+ let mut projection: BTreeSet<usize> = required_columns
Review comment:
When I did not use a `BTreeSet` one of the `UNION ALL` tests failed due
to a duplicate column being projected.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
