GitHub user NoahKus added a comment to the discussion: I tried benchmarking TPC-DS for Spark vs Datafusion Comet on AWS Glue Catalog Iceberg Tables and Spark was faster.
Are there configs I should particularly focus on to get iceberg comet performant? https://datafusion.apache.org/comet/user-guide/0.13/configs.html I can't tell which makes the greatest difference, and flipping configs on / off and tuning batch-sizes and thread counts can become complicated pretty quickly. GitHub link: https://github.com/apache/datafusion-comet/discussions/3199#discussioncomment-15654144 ---- This is an automatically sent email for [email protected]. To unsubscribe, please send an email to: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
