westonpace commented on code in PR #14194:
URL: https://github.com/apache/datafusion/pull/14194#discussion_r1975966228
##########
datafusion/substrait/src/logical_plan/producer.rs:
##########
@@ -559,12 +559,31 @@ pub fn from_table_scan(
let table_schema = scan.source.schema().to_dfschema_ref()?;
let base_schema = to_substrait_named_struct(&table_schema)?;
+ let best_effort_filter_option = if !scan.filters.is_empty() {
+ let table_schema_qualified = Arc::new(
+ DFSchema::try_from_qualified_schema(
+ scan.table_name.clone(),
+ &(scan.source.schema()),
+ )
+ .unwrap(),
+ );
+ let mut combined_expr = scan.filters[0].clone();
+ for i in 1..scan.filters.len() {
+ combined_expr = combined_expr.and(scan.filters[i].clone());
+ }
+ let best_effort_filter_expr =
+ producer.handle_expr(&combined_expr, &table_schema_qualified)?;
+ Some(Box::new(best_effort_filter_expr))
+ } else {
+ None
+ };
+
Ok(Box::new(Rel {
rel_type: Some(RelType::Read(Box::new(ReadRel {
common: None,
base_schema: Some(base_schema),
filter: None,
- best_effort_filter: None,
+ best_effort_filter: best_effort_filter_option,
Review Comment:
> Whether a filter is exact or inexact can be determined by calling
TableScan::source's supports_filter_pushdown function.
Agreed, but that is not part of the logical plan.
Let's try a different approach. I can do this...
```
let session_context: SessionContext = make_session_context();
let (some_table_provider, some_table_schema): (
Arc<dyn TableProvider>,
Arc<DFSchema>,
) = make_table_provider();
let table_provider_source = provider_as_source(some_table_provider);
let plan = LogicalPlan::TableScan(TableScan {
table_name: TableReference::bare("weston"),
source: table_provider_source,
projection: None,
projected_schema: some_table_schema,
filters: vec![datafusion_expr::col("a").eq(datafusion_expr::lit(1))],
fetch: None,
});
let results = ctx.execute_logical_plan(plan).await?;
```
Can `results` contain values where `a != 1`? Does it depend on the table
provider?
I don't think it does it matters what the table provider is. I think the
results will never contain values where `a != 1`.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscr...@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscr...@datafusion.apache.org
For additional commands, e-mail: github-h...@datafusion.apache.org
