mustafasrepo commented on code in PR #7566:
URL: https://github.com/apache/arrow-datafusion/pull/7566#discussion_r1328429794
##########
datafusion/core/src/physical_plan/filter.rs:
##########
@@ -153,7 +154,19 @@ impl ExecutionPlan for FilterExec {
}
fn ordering_equivalence_properties(&self) -> OrderingEquivalenceProperties
{
- self.input.ordering_equivalence_properties()
+ let stats = self.statistics();
+ // Add the columns that have only one value (singleton) after
filtering to constants.
Review Comment:
> In other words, if the filter has a predicate like column = 5 shouldn't
column be added to list of constants even if the column had more than one value
in the statistics?
In this case, statistics should be able to determine `column` will have
single value(5) onwards. Hence I presumed, using statistics is sufficient.
However, if there are cases, where it is obvious from the predicate value is
constant, but statistics fail to resolve it. We can change this implementation
I think
##########
datafusion/core/src/physical_plan/filter.rs:
##########
@@ -153,7 +154,19 @@ impl ExecutionPlan for FilterExec {
}
fn ordering_equivalence_properties(&self) -> OrderingEquivalenceProperties
{
- self.input.ordering_equivalence_properties()
+ let stats = self.statistics();
+ // Add the columns that have only one value (singleton) after
filtering to constants.
Review Comment:
> In other words, if the filter has a predicate like column = 5 shouldn't
column be added to list of constants even if the column had more than one value
in the statistics?
In this case, statistics should be able to determine `column` will have
single value(5) onwards. Hence I presumed, using statistics is sufficient.
However, if there are cases, where it is obvious from the predicate value is
constant, but statistics fail to resolve it. We can change this implementation
I think
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
