alamb commented on code in PR #19130:
URL: https://github.com/apache/datafusion/pull/19130#discussion_r2596253893
##########
datafusion/core/tests/schema_adapter/schema_adapter_integration_tests.rs:
##########
@@ -564,29 +564,36 @@ async fn test_parquet_missing_column() -> Result<()> {
.push_down_filters(false)
.execute()
.await?;
- // There will be data: the filter is (null) is not null or a = 24.
- // Statistics pruning doesn't handle `null is not null` so it resolves to
`true or a = 24` -> `true` so no row groups are pruned
+ // There should be zero batches
+ assert_eq!(batches.len(), 0);
+ // Check another filter: `b = 'foo' and a = 24` should also prune data
with only statistics-based pushdown
+ let filter = col("b").eq(lit("foo")).and(col("a").eq(lit(24)));
+ let batches = test_case
+ .clone()
+ .with_predicate(filter)
+ .push_down_filters(false)
+ .execute()
+ .await?;
+ // There should be zero batches
+ assert_eq!(batches.len(), 0);
+ // On the other hand `b is null and a = 2` should prune only the second
row group with stats only pruning
Review Comment:
This is pruned because now `b is null` gets folded to `null is null AND a =
2` --> `a = 2`?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
