yordan-pavlov commented on a change in pull request #9704:
URL: https://github.com/apache/arrow/pull/9704#discussion_r593961778
##########
File path: rust/datafusion/src/physical_plan/parquet.rs
##########
@@ -645,8 +654,29 @@ fn build_predicate_expression(
use crate::logical_plan;
// predicate expression can only be a binary expression
let (left, op, right) = match expr {
- Expr::BinaryExpr { left, op, right } => (left, *op, right),
+ Expr::BinaryExpr { left, op, right } => (left.clone(), *op,
right.clone()),
+ Expr::Between {
+ expr,
+ negated,
+ low,
+ high,
+ } if !negated // NOT BETWEEN not supported
+ => {
+ let left = Box::new(Expr::BinaryExpr {
+ left: expr.clone(),
+ op: Operator::GtEq,
+ right: low.clone(),
+ });
+ let right = Box::new(Expr::BinaryExpr {
+ left: expr.clone(),
+ op: Operator::LtEq,
+ right: high.clone(),
+ });
+
+ (left, Operator::And, right)
+ }
_ => {
+ debug!("Filter expression not supported in ParquetExec {:?}",
expr);
Review comment:
thank you for adding some `debug!` logs, I have been thinking how
predicate push-down into parquet could be displayed into `explain` output for
easier diagnostics, but lately I have been preoccupied with rethinking of
reading of arrow arrays from parquet files
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
