Dimitris Tsirogiannis has posted comments on this change. Change subject: IMPALA-3739: Enable stress tests on Kudu ......................................................................
Patch Set 3: (8 comments) http://gerrit.cloudera.org:8080/#/c/4327/3/testdata/bin/load-tpc-kudu.py File testdata/bin/load-tpc-kudu.py: PS3, Line 51: tbls_to_clean = tpch_tables if workload.lower() == 'tpch' else tpcds_tables > Maybe use the cursor to get the list of tables? That way you don't have to The change to enable drop db cascade for Kudu is in review. If you're strongly against simply doing a drop db when this is in, I'll implement your proposal. PS3, Line 81: sql_file_path = "%s/testdata/datasets/%s/%s_kudu_template.sql" > Use os.path.join() here. Done http://gerrit.cloudera.org:8080/#/c/4327/3/testdata/datasets/tpcds/tpcds_kudu_template.sql File testdata/datasets/tpcds/tpcds_kudu_template.sql: PS3, Line 39: 'kudu.key_columns' = 'ss_sold_date_sk,ss_ticket_number, ss_item_sk' > For my education, I looked at http://www.tpc.org/tpc_documents_current_vers No intention here, I just got this wrong :) I actually found a few other inconsistencies. http://gerrit.cloudera.org:8080/#/c/4327/3/testdata/workloads/tpcds/queries/tpcds-kudu-q19.test File testdata/workloads/tpcds/queries/tpcds-kudu-q19.test: Line 39: ==== > I noticed none of the TPC-DS Kudu queries have RESULTS. Why? (I searched fo The TPC-DS workload is not currently enabled for Kudu in our regular (non-stress) testing framework. It requires some extra work which I left for another patch. Once this is done, these queries will also get proper RESULTS and TYPES sections. For now it seemed kind of pointless to add these sections. Besides the stress test has its own result verification mechanism. http://gerrit.cloudera.org:8080/#/c/4327/3/testdata/workloads/tpcds/queries/tpcds-kudu-q47.test File testdata/workloads/tpcds/queries/tpcds-kudu-q47.test: PS3, Line 33: ,round(v1_lead.sum_sales, 2) nsum > Nit: tab character. Done http://gerrit.cloudera.org:8080/#/c/4327/3/testdata/workloads/tpcds/queries/tpcds-kudu-q65.test File testdata/workloads/tpcds/queries/tpcds-kudu-q65.test: PS3, Line 55: order by : s_store_name, : i_item_desc, : sc.revenue, : i_current_price, : i_wholesale_cost, : i_brand > The ORDER BY has more columns than the TPC-DS-for-HDFS counterpart. Any rea I had trouble getting consistent results across multiple invocations of this query unless I included these additional columns here. Added a comment. http://gerrit.cloudera.org:8080/#/c/4327/3/tests/stress/concurrent_select.py File tests/stress/concurrent_select.py: PS3, Line 1463: tpch_kudu_queries = load_tpc_queries("tpch", "kudu") > Change "kudu" to load_in_kudu=True Good catch, thanks. Done PS3, Line 1468: tpcds_kudu_queries = load_tpc_queries("tpcds", "kudu") > Change "kudu" to load_in_kudu=True Done -- To view, visit http://gerrit.cloudera.org:8080/4327 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-MessageType: comment Gerrit-Change-Id: I3c9fc3dae24b761f031ee8e014bd611a49029d34 Gerrit-PatchSet: 3 Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-Owner: Dimitris Tsirogiannis <dtsirogian...@cloudera.com> Gerrit-Reviewer: Dimitris Tsirogiannis <dtsirogian...@cloudera.com> Gerrit-Reviewer: Matthew Jacobs <m...@cloudera.com> Gerrit-Reviewer: Michael Brown <mi...@cloudera.com> Gerrit-HasComments: Yes
