Till Westmann has posted comments on this change. Change subject: Add cluster performance testing scripts. ......................................................................
Patch Set 4: (12 comments) Looks generally good, I just have a few questions (and I basically don't understand the ansible scripts ..): 1) Reference results were removed from the benchmarks package. Do we still have (and use) TPC-H reference results for regression testing? 2) The TPC-H benchmark parameterizes some queries based on the scale factor. Should we file an issue to support this as well (doesn't seem to be very urgent)? https://asterix-gerrit.ics.uci.edu/#/c/1504/4/asterixdb/asterix-benchmark/src/main/resources/ansible/runquery.yml File asterixdb/asterix-benchmark/src/main/resources/ansible/runquery.yml: PS4, Line 61: report Can we remove those empty lines at the end? https://asterix-gerrit.ics.uci.edu/#/c/1504/4/asterixdb/asterix-benchmark/src/main/resources/benchmarks/tpch/queries/q18.sqlpp.dis File asterixdb/asterix-benchmark/src/main/resources/benchmarks/tpch/queries/q18.sqlpp.dis: PS4, Line 20: Error Do we use this information or can we remove it from the file? https://asterix-gerrit.ics.uci.edu/#/c/1504/4/asterixdb/asterix-benchmark/src/main/resources/benchmarks/tpch/queries/q2.sqlpp.dis File asterixdb/asterix-benchmark/src/main/resources/benchmarks/tpch/queries/q2.sqlpp.dis: PS4, Line 20: Error Do we use this information or can we remove it from the file? https://asterix-gerrit.ics.uci.edu/#/c/1504/4/asterixdb/asterix-benchmark/src/main/resources/benchmarks/tpch/queries/q20.sqlpp.dis File asterixdb/asterix-benchmark/src/main/resources/benchmarks/tpch/queries/q20.sqlpp.dis: PS4, Line 20: Error Do we use this information or can we remove it from the file? https://asterix-gerrit.ics.uci.edu/#/c/1504/4/asterixdb/asterix-benchmark/src/main/resources/benchmarks/tpch/queries/q21.sqlpp.dis File asterixdb/asterix-benchmark/src/main/resources/benchmarks/tpch/queries/q21.sqlpp.dis: PS4, Line 20: Error Do we use this information or can we remove it from the file? https://asterix-gerrit.ics.uci.edu/#/c/1504/4/asterixdb/asterix-benchmark/src/main/resources/benchmarks/tpch/queries/q22.sqlpp.dis File asterixdb/asterix-benchmark/src/main/resources/benchmarks/tpch/queries/q22.sqlpp.dis: PS4, Line 20: Error Do we use this information or can we remove it from the file? https://asterix-gerrit.ics.uci.edu/#/c/1504/4/asterixdb/asterix-benchmark/src/main/resources/benchmarks/tpch/queries/q5.sqlpp.dis File asterixdb/asterix-benchmark/src/main/resources/benchmarks/tpch/queries/q5.sqlpp.dis: PS4, Line 20: Error Do we use this information or can we remove it from the file? https://asterix-gerrit.ics.uci.edu/#/c/1504/4/asterixdb/asterix-benchmark/src/main/resources/benchmarks/tpch/queries/q7.sqlpp.dis File asterixdb/asterix-benchmark/src/main/resources/benchmarks/tpch/queries/q7.sqlpp.dis: PS4, Line 20: Error Do we use this information or can we remove it from the file? https://asterix-gerrit.ics.uci.edu/#/c/1504/4/asterixdb/asterix-benchmark/src/main/resources/benchmarks/tpch/queries/q8.sqlpp.dis File asterixdb/asterix-benchmark/src/main/resources/benchmarks/tpch/queries/q8.sqlpp.dis: PS4, Line 20: Error Do we use this information or can we remove it from the file? https://asterix-gerrit.ics.uci.edu/#/c/1504/4/asterixdb/asterix-benchmark/src/main/resources/benchmarks/tpch/queries/q9.sqlpp.dis File asterixdb/asterix-benchmark/src/main/resources/benchmarks/tpch/queries/q9.sqlpp.dis: PS4, Line 20: Error Do we use this information or can we remove it from the file? https://asterix-gerrit.ics.uci.edu/#/c/1504/4/asterixdb/asterix-benchmark/src/main/resources/bin/runall.sh File asterixdb/asterix-benchmark/src/main/resources/bin/runall.sh: PS4, Line 33: INVENTORY Should we validate the inventory before we start running? Or will the ansible playbooks be NOOPs if the inventory is not set? https://asterix-gerrit.ics.uci.edu/#/c/1504/4/asterixdb/asterix-benchmark/src/main/resources/conf/benchmark_setting.yml File asterixdb/asterix-benchmark/src/main/resources/conf/benchmark_setting.yml: PS4, Line 33: unhygienix This might need to change when Ian get's a new CNAME for the machine, right? -- To view, visit https://asterix-gerrit.ics.uci.edu/1504 To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings Gerrit-MessageType: comment Gerrit-Change-Id: Ie17f20dc87694a865aebc561d8eded064c66f5bb Gerrit-PatchSet: 4 Gerrit-Project: asterixdb Gerrit-Branch: master Gerrit-Owner: Yingyi Bu <[email protected]> Gerrit-Reviewer: Ian Maxon <[email protected]> Gerrit-Reviewer: Jenkins <[email protected]> Gerrit-Reviewer: Michael Blow <[email protected]> Gerrit-Reviewer: Till Westmann <[email protected]> Gerrit-Reviewer: Yingyi Bu <[email protected]> Gerrit-HasComments: Yes
