Joe McDonnell has submitted this change and it was merged. ( http://gerrit.cloudera.org:8080/20918 )
Change subject: IMPALA-12038: Switch report_benchmark_results.py to python 3 ...................................................................... IMPALA-12038: Switch report_benchmark_results.py to python 3 When using TPC-DS with a large number of iterations, the results JSON files are enormous. Using Python2, report_benchmark_results.py runs out of memory and fails to produce the report. Python 3 is more efficient in how it processes Unicode inputs (see Python PEP-0393), so it's memory usage is much lower. It is able to handle generating reports that Python 2 cannot. As a general cleanup, this fixes all the flake8 issues for this file. Testing: - Processed very large JSON results (4+GB each for both baseline result and new result). Python 3 completes successfully when Python 2 failed. Change-Id: Idbde17f720b18d38dc2c2104ecf3fec807c1839d Reviewed-on: http://gerrit.cloudera.org:8080/20918 Reviewed-by: Riza Suminto <[email protected]> Tested-by: Impala Public Jenkins <[email protected]> --- M tests/benchmark/report_benchmark_results.py 1 file changed, 66 insertions(+), 56 deletions(-) Approvals: Riza Suminto: Looks good to me, approved Impala Public Jenkins: Verified -- To view, visit http://gerrit.cloudera.org:8080/20918 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: merged Gerrit-Change-Id: Idbde17f720b18d38dc2c2104ecf3fec807c1839d Gerrit-Change-Number: 20918 Gerrit-PatchSet: 3 Gerrit-Owner: Joe McDonnell <[email protected]> Gerrit-Reviewer: Impala Public Jenkins <[email protected]> Gerrit-Reviewer: Joe McDonnell <[email protected]> Gerrit-Reviewer: Riza Suminto <[email protected]>
