mnorfolk03 commented on PR #13085: URL: https://github.com/apache/datafusion/pull/13085#issuecomment-2435902070
@alamb I've run cargo fmt. > Thanks @mnorfolk03 -- this is a nice improvement > > I ran these benchmarks and they look good. > > ```sql > cargo bench --bench sql_planner -- physical_join_distinct > ``` > > I also made some flamegraphs to see what might be going on which was quite instructive: > > ```shell > sudo flamegraph -- target/release/deps/sql_planner-65bd83b445d4016c --bench physical_join_distinct > ``` > >  > >  > > ```shell > sudo flamegraph -- target/release/deps/sql_planner-65bd83b445d4016c --bench physical_many_self_joins > ``` > >  > >  > > So TLDR is I think these are a significant improvement to our coverage. Thank you 🙏 > > One thing that might be helpful to improve more would be adding a ParquetExec as well as queries that have sortedness > > So the equivalent of planning against tables like this (docs here): https://datafusion.apache.org/user-guide/sql/ddl.html#create-external-table > > ```sql > CREATE EXTERNAL TABLE foo STORED AS PARQUET LOCATION '..' > ``` > > ```sql > CREATE EXTERNAL TABLE test ( > c1 VARCHAR NOT NULL, > c2 INT NOT NULL, > c3 SMALLINT NOT NULL, > c4 SMALLINT NOT NULL, > c5 INT NOT NULL, > c6 BIGINT NOT NULL, > c7 SMALLINT NOT NULL, > c8 INT NOT NULL, > c9 BIGINT NOT NULL, > c10 VARCHAR NOT NULL, > c11 FLOAT NOT NULL, > c12 DOUBLE NOT NULL, > c13 VARCHAR NOT NULL > ) > STORED AS CSV > WITH ORDER (c2 ASC, c5 + c8 DESC NULL FIRST) > LOCATION '/path/to/aggregate_test_100.csv' > OPTIONS ('has_header' 'true'); > ``` I'll leave this for a future PR since I've run out of free time this week. Should I open an issue for this in that case? Thanks! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: github-unsubscr...@datafusion.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org --------------------------------------------------------------------- To unsubscribe, e-mail: github-unsubscr...@datafusion.apache.org For additional commands, e-mail: github-h...@datafusion.apache.org
