Matthew Mulder has uploaded a new patch set (#3). Change subject: IMPALA-5625: stress test: write profile when queries fail ......................................................................
IMPALA-5625: stress test: write profile when queries fail This change writes query profiles as text files for all of the major query failure reasons in the concurrent_select stress test. 1) Change the --result-hash-log-dir command-line option to --results-dir and update the help text. 2) Introduce two new directories under the directory given by the --results-dir command-line argument: profiles result_hashes 3) Move results into the result_hashes directory. 4) Write the query profile to the profiles directory when a query times out or gets an error or incorrect results. 5) Remove the query profile from the log output for unexpected mem limit exceeded exceptions. Instead, write those to the profiles directory as well. Testing: Ran the stress test with a driver that changes the hashes of some of the query results in the runtime info json file to inject incorrect result failures. Set tight bounds on the mem limit and timeout to ensure there would be timeouts and exceeded memory limit failures. Restarted the NameNode mid test run to induce a query failure. That covers the 4 cases for which an exception is thrown and profile is written for query failures. Verified that the profiles were written for each kind of query failure. Change-Id: I1dbdf5fcf97d6c5681c9fc8fb9eb448bc459b3b0 --- M tests/stress/concurrent_select.py 1 file changed, 92 insertions(+), 60 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/76/7376/3 -- To view, visit http://gerrit.cloudera.org:8080/7376 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-MessageType: newpatchset Gerrit-Change-Id: I1dbdf5fcf97d6c5681c9fc8fb9eb448bc459b3b0 Gerrit-PatchSet: 3 Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-Owner: Matthew Mulder <mmul...@cloudera.com> Gerrit-Reviewer: David Knupp <dkn...@cloudera.com> Gerrit-Reviewer: Matthew Mulder <mmul...@cloudera.com> Gerrit-Reviewer: Michael Brown <mi...@cloudera.com> Gerrit-Reviewer: Tim Wood <tw...@cloudera.com>