Lordworms commented on PR #13054: URL: https://github.com/apache/datafusion/pull/13054#issuecomment-2461159231
Another weird thing I found after rebasing main is that the ParquetExec time seems to be wrong without optimization <img width="1413" alt="Screenshot 2024-11-06 at 5 50 47 PM" src="https://github.com/user-attachments/assets/6207c16e-69e2-4565-898b-70a9e0e3e806";> with optimization <img width="1418" alt="Screenshot 2024-11-06 at 5 51 02 PM" src="https://github.com/user-attachments/assets/64b87eb4-3a2c-4df7-9e64-b5627c7984f7";> The total time with optimzation is 30s, while it is 19s without processing, which means adding the dynamic filter added 10s on ParquetExec. Seems like there's something wrong with parquet filter pushdown -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: github-unsubscr...@datafusion.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org --------------------------------------------------------------------- To unsubscribe, e-mail: github-unsubscr...@datafusion.apache.org For additional commands, e-mail: github-h...@datafusion.apache.org
