zhuqi-lucas commented on PR #13788:
URL: https://github.com/apache/datafusion/pull/13788#issuecomment-2544356695
> Thank you @zhuqi-lucas and @korowa
>
> I tried to verify the test covers the code by reverting the code change:
>
> ```diff
> andrewlamb@Andrews-MacBook-Pro-2:~/Software/datafusion2$ git diff
> diff --git a/datafusion/core/src/datasource/listing/table.rs
b/datafusion/core/src/datasource/listing/table.rs
> index b12f37ed7..ffe49dd2b 100644
> --- a/datafusion/core/src/datasource/listing/table.rs
> +++ b/datafusion/core/src/datasource/listing/table.rs
> @@ -843,13 +843,8 @@ impl TableProvider for ListingTable {
> });
> // TODO (https://github.com/apache/datafusion/issues/11600)
remove downcast_ref from here?
> let session_state =
state.as_any().downcast_ref::<SessionState>().unwrap();
> -
> - // We should not limit the number of partitioned files to scan if
there are filters and limit
> - // at the same time. This is because the limit should be applied
after the filters are applied.
> - let statistic_file_limit = if filters.is_empty() { limit } else {
None };
> -
> let (mut partitioned_file_lists, statistics) = self
> - .list_files_for_scan(session_state, &partition_filters,
statistic_file_limit)
> + .list_files_for_scan(session_state, &partition_filters, limit)
> .await?;
>
> // if no files need to be read, return an `EmptyExec`
> ```
>
> And running the test:
>
> ```shell
> andrewlamb@Andrews-MacBook-Pro-2:~/Software/datafusion2$ cargo test --test
sqllogictests -- pushdown
> Finished `test` profile [unoptimized + debuginfo] target(s) in 0.15s
> Running bin/sqllogictests.rs
(target/debug/deps/sqllogictests-b3c91103e28d27e7)
> Executed "parquet_filter_pushdown.slt". Took 24.915792ms
> andrewlamb@Andrews-MacBook-Pro-2:~/Software/datafusion2$
> ```
>
> However, the test seems to pass without the code changes 🤔
Thank you @alamb for review, i will try to change the testing, it may not
hit the test case.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
