[Impala-ASF-CR] Fix stress test tracking of past metrics values
Impala Public Jenkins has submitted this change and it was merged. ( http://gerrit.cloudera.org:8080/12903 ) Change subject: Fix stress test tracking of past metrics values .. Fix stress test tracking of past metrics values The stress test keeps track of various metrics about queries that it has already run, such as the number of completed queries and the number of errors. The stress test also forks off several processes to handle running of queries. The intention is that the metrics should be shared across all of the processes. However, a recent patch (IMPALA-6662) changed these metrics to be stored in a dict and in the process caused them to no longer be shared. This patch solves the problem by storing the value in the dict as 'Value' objects, which are shared. Synchronization of access to these values is handled by taking '_query_runners_lock'. Testing: - Ran the stress test for 1000 queries on tpch 500 on a 10 node cluster. Previously, this would reach a point where there were 0 'Active' queries and hang. Change-Id: I03b5e24b41fff1d33f9bb64b14a6d0b7a23a56e2 Reviewed-on: http://gerrit.cloudera.org:8080/12903 Reviewed-by: Tim Armstrong Tested-by: Impala Public Jenkins --- M tests/stress/concurrent_select.py 1 file changed, 6 insertions(+), 4 deletions(-) Approvals: Tim Armstrong: Looks good to me, approved Impala Public Jenkins: Verified -- To view, visit http://gerrit.cloudera.org:8080/12903 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: merged Gerrit-Change-Id: I03b5e24b41fff1d33f9bb64b14a6d0b7a23a56e2 Gerrit-Change-Number: 12903 Gerrit-PatchSet: 2 Gerrit-Owner: Thomas Marshall Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Tim Armstrong
[Impala-ASF-CR] Fix stress test tracking of past metrics values
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/12903 ) Change subject: Fix stress test tracking of past metrics values .. Patch Set 1: Verified+1 -- To view, visit http://gerrit.cloudera.org:8080/12903 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I03b5e24b41fff1d33f9bb64b14a6d0b7a23a56e2 Gerrit-Change-Number: 12903 Gerrit-PatchSet: 1 Gerrit-Owner: Thomas Marshall Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Tim Armstrong Gerrit-Comment-Date: Mon, 01 Apr 2019 22:40:14 + Gerrit-HasComments: No
[Impala-ASF-CR] Fix stress test tracking of past metrics values
Tim Armstrong has posted comments on this change. ( http://gerrit.cloudera.org:8080/12903 ) Change subject: Fix stress test tracking of past metrics values .. Patch Set 1: Code-Review+2 -- To view, visit http://gerrit.cloudera.org:8080/12903 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I03b5e24b41fff1d33f9bb64b14a6d0b7a23a56e2 Gerrit-Change-Number: 12903 Gerrit-PatchSet: 1 Gerrit-Owner: Thomas Marshall Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Tim Armstrong Gerrit-Comment-Date: Mon, 01 Apr 2019 17:54:02 + Gerrit-HasComments: No
[Impala-ASF-CR] Fix stress test tracking of past metrics values
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/12903 ) Change subject: Fix stress test tracking of past metrics values .. Patch Set 1: Build started: https://jenkins.impala.io/job/gerrit-verify-dryrun/3970/ DRY_RUN=false -- To view, visit http://gerrit.cloudera.org:8080/12903 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I03b5e24b41fff1d33f9bb64b14a6d0b7a23a56e2 Gerrit-Change-Number: 12903 Gerrit-PatchSet: 1 Gerrit-Owner: Thomas Marshall Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Tim Armstrong Gerrit-Comment-Date: Mon, 01 Apr 2019 18:02:23 + Gerrit-HasComments: No
[Impala-ASF-CR] Fix stress test tracking of past metrics values
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/12903 ) Change subject: Fix stress test tracking of past metrics values .. Patch Set 1: Build Successful https://jenkins.impala.io/job/gerrit-code-review-checks/2606/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests. -- To view, visit http://gerrit.cloudera.org:8080/12903 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I03b5e24b41fff1d33f9bb64b14a6d0b7a23a56e2 Gerrit-Change-Number: 12903 Gerrit-PatchSet: 1 Gerrit-Owner: Thomas Marshall Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Tim Armstrong Gerrit-Comment-Date: Mon, 01 Apr 2019 17:16:50 + Gerrit-HasComments: No
[Impala-ASF-CR] Fix stress test tracking of past metrics values
Thomas Marshall has uploaded this change for review. ( http://gerrit.cloudera.org:8080/12903 Change subject: Fix stress test tracking of past metrics values .. Fix stress test tracking of past metrics values The stress test keeps track of various metrics about queries that it has already run, such as the number of completed queries and the number of errors. The stress test also forks off several processes to handle running of queries. The intention is that the metrics should be shared across all of the processes. However, a recent patch (IMPALA-6662) changed these metrics to be stored in a dict and in the process caused them to no longer be shared. This patch solves the problem by storing the value in the dict as 'Value' objects, which are shared. Synchronization of access to these values is handled by taking '_query_runners_lock'. Testing: - Ran the stress test for 1000 queries on tpch 500 on a 10 node cluster. Previously, this would reach a point where there were 0 'Active' queries and hang. Change-Id: I03b5e24b41fff1d33f9bb64b14a6d0b7a23a56e2 --- M tests/stress/concurrent_select.py 1 file changed, 6 insertions(+), 4 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/03/12903/1 -- To view, visit http://gerrit.cloudera.org:8080/12903 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newchange Gerrit-Change-Id: I03b5e24b41fff1d33f9bb64b14a6d0b7a23a56e2 Gerrit-Change-Number: 12903 Gerrit-PatchSet: 1 Gerrit-Owner: Thomas Marshall
