mustafasrepo commented on code in PR #5655: URL: https://github.com/apache/arrow-datafusion/pull/5655#discussion_r1145785176
########## benchmarks/README.md: ########## @@ -76,12 +76,54 @@ cargo run --release --bin tpch -- convert --input ./data --output /mnt/tpch-parq Or if you want to verify and run all the queries in the benchmark, you can just run `cargo test`. -### Machine readable benchmark summary +### Comparing results between runs Any `tpch` execution with `-o <dir>` argument will produce a summary file right under the `<dir>` directory. It is a JSON serialized form of all the runs that happened as well as the runtime metadata (number of cores, DataFusion version, etc.). +```shell +$ git checkout main +# generate an output script in /tmp/output_main +$ cargo run --release --bin tpch -- benchmark datafusion --iterations 5 --path /data --format parquet -o /tmp/output_main Review Comment: Also when I run this script unless `/tmp/output_main` already exists. I receive `IO Error`. Is this expected?. If so, I think we should `mkdir /tmp/output_main` above this line. ########## benchmarks/README.md: ########## @@ -76,12 +76,54 @@ cargo run --release --bin tpch -- convert --input ./data --output /mnt/tpch-parq Or if you want to verify and run all the queries in the benchmark, you can just run `cargo test`. -### Machine readable benchmark summary +### Comparing results between runs Any `tpch` execution with `-o <dir>` argument will produce a summary file right under the `<dir>` directory. It is a JSON serialized form of all the runs that happened as well as the runtime metadata (number of cores, DataFusion version, etc.). +```shell +$ git checkout main +# generate an output script in /tmp/output_main +$ cargo run --release --bin tpch -- benchmark datafusion --iterations 5 --path /data --format parquet -o /tmp/output_main Review Comment: Also when I run this script unless `/tmp/output_main` already exists. I receive `IO Error`. Is this expected?. If so, I think we should add `mkdir /tmp/output_main` above this line. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
