alamb commented on issue #18411: URL: https://github.com/apache/datafusion/issues/18411#issuecomment-3474037162
BTW DuckDB is also 2x faster with the tPCH SF=100 dataset (which is much faster to generate) Create dataset: ```shell nice tpchgen-cli --tables=lineitem --format=parquet --scale-factor 100 ``` ``` andrewlamb@Andrews-MacBook-Pro-3:~/Downloads$ time duckdb -c "select l_returnflag,l_linestatus, count(*) as count_order from 'lineitem.parquet' group by l_returnflag, l_linestatus;" ┌──────────────┬──────────────┬─────────────┐ │ l_returnflag │ l_linestatus │ count_order │ │ varchar │ varchar │ int64 │ ├──────────────┼──────────────┼─────────────┤ │ R │ F │ 148067261 │ │ N │ O │ 300058170 │ │ N │ F │ 3864590 │ │ A │ F │ 148047881 │ └──────────────┴──────────────┴─────────────┘ real 0m0.910s user 0m9.005s sys 0m0.666s andrewlamb@Andrews-MacBook-Pro-3:~/Downloads$ time datafusion-cli -c "select l_returnflag,l_linestatus, count(*) as count_order from 'lineitem.parquet' group by l_returnflag, l_linestatus;" DataFusion CLI v50.3.0 +--------------+--------------+-------------+ | l_returnflag | l_linestatus | count_order | +--------------+--------------+-------------+ | N | O | 300058170 | | R | F | 148067261 | | A | F | 148047881 | | N | F | 3864590 | +--------------+--------------+-------------+ 4 row(s) fetched. Elapsed 1.291 seconds. real 0m1.359s user 0m16.846s sys 0m0.708s ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
