Re: [PR] WIP: Test DataFusion with experimental parquet pushdown [datafusion]

2025-05-30 Thread via GitHub
Dandandan commented on code in PR #16208: URL: https://github.com/apache/datafusion/pull/16208#discussion_r2115799584 ## datafusion/physical-plan/src/filter.rs: ## @@ -689,6 +702,46 @@ fn filter_and_project( }) } +impl FilterExecStream { +/// Evaluates the predic

Re: [PR] WIP: Test DataFusion with experimental parquet pushdown [datafusion]

2025-05-30 Thread via GitHub
alamb commented on code in PR #16208: URL: https://github.com/apache/datafusion/pull/16208#discussion_r2115668031 ## datafusion/physical-plan/src/filter.rs: ## @@ -689,6 +702,46 @@ fn filter_and_project( }) } +impl FilterExecStream { +/// Evaluates the predicate

Re: [PR] WIP: Test DataFusion with experimental parquet pushdown [datafusion]

2025-05-30 Thread via GitHub
Dandandan commented on code in PR #16208: URL: https://github.com/apache/datafusion/pull/16208#discussion_r2115388263 ## datafusion/physical-plan/src/filter.rs: ## @@ -689,6 +702,46 @@ fn filter_and_project( }) } +impl FilterExecStream { +/// Evaluates the predic

Re: [PR] WIP: Test DataFusion with experimental parquet pushdown [datafusion]

2025-05-30 Thread via GitHub
Dandandan commented on code in PR #16208: URL: https://github.com/apache/datafusion/pull/16208#discussion_r2115388263 ## datafusion/physical-plan/src/filter.rs: ## @@ -689,6 +702,46 @@ fn filter_and_project( }) } +impl FilterExecStream { +/// Evaluates the predic

Re: [PR] WIP: Test DataFusion with experimental parquet pushdown [datafusion]

2025-05-30 Thread via GitHub
Dandandan commented on code in PR #16208: URL: https://github.com/apache/datafusion/pull/16208#discussion_r2115400750 ## datafusion/physical-plan/src/filter.rs: ## @@ -689,6 +702,46 @@ fn filter_and_project( }) } +impl FilterExecStream { +/// Evaluates the predic

Re: [PR] WIP: Test DataFusion with experimental parquet pushdown [datafusion]

2025-05-29 Thread via GitHub
zhuqi-lucas commented on PR #16208: URL: https://github.com/apache/datafusion/pull/16208#issuecomment-2921147300 Interesting, i am confused why it's different from arrow-rs clickbench result. -- This is an automated message from the Apache Git Service. To respond to the message, please lo

Re: [PR] WIP: Test DataFusion with experimental parquet pushdown [datafusion]

2025-05-29 Thread via GitHub
zhuqi-lucas commented on PR #16208: URL: https://github.com/apache/datafusion/pull/16208#issuecomment-2921150637 Ok, i misunderstood this testing is not for parquet filter pushdown enabled... -- This is an automated message from the Apache Git Service. To respond to the message, please lo

Re: [PR] WIP: Test DataFusion with experimental parquet pushdown [datafusion]

2025-05-29 Thread via GitHub
alamb commented on PR #16208: URL: https://github.com/apache/datafusion/pull/16208#issuecomment-2920064834 > 🤖: Benchmark completed Hmm that is somewhat depressing. I will investigate tomorrow -- This is an automated message from the Apache Git Service. To respond to the message, pl

Re: [PR] WIP: Test DataFusion with experimental parquet pushdown [datafusion]

2025-05-29 Thread via GitHub
alamb commented on PR #16208: URL: https://github.com/apache/datafusion/pull/16208#issuecomment-2920053295 🤖: Benchmark completed Details ``` Comparing HEAD and alamb_test_filter_pushdown Benchmark clickbench_extended.json ---

Re: [PR] WIP: Test DataFusion with experimental parquet pushdown [datafusion]

2025-05-29 Thread via GitHub
alamb commented on PR #16208: URL: https://github.com/apache/datafusion/pull/16208#issuecomment-2919955178 🤖 `./gh_compare_branch.sh` [Benchmark Script](https://github.com/alamb/datafusion-benchmarking/blob/main/gh_compare_branch.sh) Running Linux aal-dev 6.11.0-1013-gcp #13~24.04.1-Ubun

[PR] WIP: Test DataFusion with experimental parquet pushdown [datafusion]

2025-05-29 Thread via GitHub
alamb opened a new pull request, #16208: URL: https://github.com/apache/datafusion/pull/16208 This PR is for testing DataFusion with the code in the following PR - https://github.com/apache/arrow-rs/pull/7513 I want to run 2 experiments: 1. Does using `IncremntalRecordBatchBuilde