lvheyang commented on issue #723: URL: https://github.com/apache/arrow-datafusion/issues/723#issuecomment-882086751
@alamb sorry for reply late, it really cost me some time to get clear of how the parquet table works. I proposed a [pull request](https://github.com/apache/arrow-datafusion/pull/749) (WIP) and aimed to fulfill the first step in your plan: > 1. A config option (so we can ensure correct results) here is what I have done 1. Put the `parquet_pruning` option in ExecutionConfig, put the `enable_pruning` option in ParquetTable, default value of them are both enabled (true). 2. Added a new construction function `try_new_with_pruning_config` in ParquetTable. Kept the original `try_new` function and kept the function's behavior (enable pruning) 3. When initializing ParquetExec in `ParquetTable::scan`, we will build the predicate based on the pruning option. However, I still have some concerns, let's discuss them on the pull request page. Thanks a lot for your help ~ -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
