[
https://issues.apache.org/jira/browse/IMPALA-14491?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=18032947#comment-18032947
]
ASF subversion and git services commented on IMPALA-14491:
----------------------------------------------------------
Commit 3398f20afe3157f7977d77ef2c51b224b7d5c40a in impala's branch
refs/heads/master from Joe McDonnell
[ https://gitbox.apache.org/repos/asf?p=impala.git;h=3398f20af ]
IMPALA-14491: Fix run-workload.py's handling of HS2's exec summary
Recently, we switched bin/run-workload.py to use HS2. It turns
out that the HS2 client code is not producing the same data
structure for the exec summary. report_benchmark_results.py
relies on that data structure and fails for HS2.
This changes the HS2 client code to use the same representation
as the beeswax. There is already a function that does this
conversion (build_summary_table_from_thrift) for our regular
tests, so this reuses that function.
Testing:
- Ran bin/run-workload.py twice to produce json files and
processed them with report_benchmark_results.py. This
failed before the change and passed afterward.
Change-Id: I0a041bdebe748b6b3a05b552584e0ca2327cff67
Reviewed-on: http://gerrit.cloudera.org:8080/23597
Reviewed-by: Riza Suminto <[email protected]>
Reviewed-by: Michael Smith <[email protected]>
Tested-by: Impala Public Jenkins <[email protected]>
> benchmark/report_benchmark_results.py fails to process exec summary
> -------------------------------------------------------------------
>
> Key: IMPALA-14491
> URL: https://issues.apache.org/jira/browse/IMPALA-14491
> Project: IMPALA
> Issue Type: Bug
> Components: Infrastructure
> Affects Versions: Impala 5.0.0
> Reporter: Joe McDonnell
> Assignee: Joe McDonnell
> Priority: Critical
>
> After running the queries, the perf-AB-test job is failing when generating
> the performance difference report:
> {noformat}
> 20:58:02 Traceback (most recent call last):
> 20:58:02 File
> "/home/ubuntu/Impala/tests/benchmark/report_benchmark_results.py", line 1157,
> in <module>
> 20:58:02 report = Report(grouped, ref_grouped)
> 20:58:02 File
> "/home/ubuntu/Impala/tests/benchmark/report_benchmark_results.py", line 509,
> in __init__
> 20:58:02 self.__analyze()
> 20:58:02 File
> "/home/ubuntu/Impala/tests/benchmark/report_benchmark_results.py", line 532,
> in __analyze
> 20:58:02 query_variability_row = Report.QueryVariabilityRow(results,
> ref_results)
> 20:58:02 File
> "/home/ubuntu/Impala/tests/benchmark/report_benchmark_results.py", line 494,
> in __init__
> 20:58:02 self.exec_summary_str = build_exec_summary_str(
> 20:58:02 File
> "/home/ubuntu/Impala/tests/benchmark/report_benchmark_results.py", line 1104,
> in build_exec_summary_str
> 20:58:02 combined_summary = CombinedExecSummaries(exec_summaries)
> 20:58:02 File
> "/home/ubuntu/Impala/tests/benchmark/report_benchmark_results.py", line 649,
> in __init__
> 20:58:02 ok, err_str = self.__check_exec_summary_schema(exec_summaries)
> 20:58:02 File
> "/home/ubuntu/Impala/tests/benchmark/report_benchmark_results.py", line 773,
> in __check_exec_summary_schema
> 20:58:02 if row[OPERATOR] != comp_row[OPERATOR]:
> 20:58:02 TypeError: string indices must be integers{noformat}
> https://jenkins.impala.io/job/perf-AB-test-ub2004/325/
> It is trying to process the exec summary, but it doesn't have the same
> structure as it expects. This could be related to the switch from beeswax to
> HS2.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
