[Impala-ASF-CR] IMPALA-6326: part 2: remove fetch thread in stress test
Impala Public Jenkins has submitted this change and it was merged. ( http://gerrit.cloudera.org:8080/12681 ) Change subject: IMPALA-6326: part 2: remove fetch thread in stress test .. IMPALA-6326: part 2: remove fetch thread in stress test Ensure cursor is only accessed from a single thread. The means reworking the code so that we check the time limit between fetch calls. Use EXEC_TIME_LIMIT_S as an alternative to the previous multi-threaded cancellation logic - it allows queries to be cancelled even when the client is blocked or slow. This is implemented with the concept of a CancelMechanism that determines *how* a query should be cancelled. Query timeouts (where we want to cancel queries that run longer than expected) are implemented using both cancel mechanisms, in case the client is stuck in fetch or similar. Expected cancellations are implemented with a random mechanism so that both code paths get covered. Testing: Ran a cluster stress test. Ran a couple of single-node stress tests with TPC-H and random queries. Change-Id: If9afd74e1408823a9e5c0f2628ec9f8aafdcec69 Reviewed-on: http://gerrit.cloudera.org:8080/12681 Reviewed-by: Impala Public Jenkins Tested-by: Impala Public Jenkins --- M tests/stress/concurrent_select.py M tests/stress/query_runner.py 2 files changed, 128 insertions(+), 90 deletions(-) Approvals: Impala Public Jenkins: Looks good to me, approved; Verified -- To view, visit http://gerrit.cloudera.org:8080/12681 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: merged Gerrit-Change-Id: If9afd74e1408823a9e5c0f2628ec9f8aafdcec69 Gerrit-Change-Number: 12681 Gerrit-PatchSet: 9 Gerrit-Owner: Tim Armstrong Gerrit-Reviewer: David Knupp Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Thomas Marshall Gerrit-Reviewer: Tim Armstrong
[Impala-ASF-CR] IMPALA-6326: part 2: remove fetch thread in stress test
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/12681 ) Change subject: IMPALA-6326: part 2: remove fetch thread in stress test .. Patch Set 8: Verified+1 -- To view, visit http://gerrit.cloudera.org:8080/12681 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: If9afd74e1408823a9e5c0f2628ec9f8aafdcec69 Gerrit-Change-Number: 12681 Gerrit-PatchSet: 8 Gerrit-Owner: Tim Armstrong Gerrit-Reviewer: David Knupp Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Thomas Marshall Gerrit-Reviewer: Tim Armstrong Gerrit-Comment-Date: Thu, 28 Mar 2019 07:18:49 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-6326: part 2: remove fetch thread in stress test
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/12681 ) Change subject: IMPALA-6326: part 2: remove fetch thread in stress test .. Patch Set 7: Build Successful https://jenkins.impala.io/job/gerrit-code-review-checks/2570/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests. -- To view, visit http://gerrit.cloudera.org:8080/12681 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: If9afd74e1408823a9e5c0f2628ec9f8aafdcec69 Gerrit-Change-Number: 12681 Gerrit-PatchSet: 7 Gerrit-Owner: Tim Armstrong Gerrit-Reviewer: David Knupp Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Thomas Marshall Gerrit-Reviewer: Tim Armstrong Gerrit-Comment-Date: Thu, 28 Mar 2019 03:11:35 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-6326: part 2: remove fetch thread in stress test
Hello Thomas Marshall, David Knupp, Impala Public Jenkins, I'd like you to reexamine a change. Please visit http://gerrit.cloudera.org:8080/12681 to look at the new patch set (#7). Change subject: IMPALA-6326: part 2: remove fetch thread in stress test .. IMPALA-6326: part 2: remove fetch thread in stress test Ensure cursor is only accessed from a single thread. The means reworking the code so that we check the time limit between fetch calls. Use EXEC_TIME_LIMIT_S as an alternative to the previous multi-threaded cancellation logic - it allows queries to be cancelled even when the client is blocked or slow. This is implemented with the concept of a CancelMechanism that determines *how* a query should be cancelled. Query timeouts (where we want to cancel queries that run longer than expected) are implemented using both cancel mechanisms, in case the client is stuck in fetch or similar. Expected cancellations are implemented with a random mechanism so that both code paths get covered. Testing: Ran a cluster stress test. Ran a couple of single-node stress tests with TPC-H and random queries. Change-Id: If9afd74e1408823a9e5c0f2628ec9f8aafdcec69 --- M tests/stress/concurrent_select.py M tests/stress/query_runner.py 2 files changed, 128 insertions(+), 90 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/81/12681/7 -- To view, visit http://gerrit.cloudera.org:8080/12681 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: If9afd74e1408823a9e5c0f2628ec9f8aafdcec69 Gerrit-Change-Number: 12681 Gerrit-PatchSet: 7 Gerrit-Owner: Tim Armstrong Gerrit-Reviewer: David Knupp Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Thomas Marshall Gerrit-Reviewer: Tim Armstrong
[Impala-ASF-CR] IMPALA-6326: part 2: remove fetch thread in stress test
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/12681 ) Change subject: IMPALA-6326: part 2: remove fetch thread in stress test .. Patch Set 8: Code-Review+2 -- To view, visit http://gerrit.cloudera.org:8080/12681 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: If9afd74e1408823a9e5c0f2628ec9f8aafdcec69 Gerrit-Change-Number: 12681 Gerrit-PatchSet: 8 Gerrit-Owner: Tim Armstrong Gerrit-Reviewer: David Knupp Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Thomas Marshall Gerrit-Reviewer: Tim Armstrong Gerrit-Comment-Date: Thu, 28 Mar 2019 02:36:50 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-6326: part 2: remove fetch thread in stress test
Tim Armstrong has posted comments on this change. ( http://gerrit.cloudera.org:8080/12681 ) Change subject: IMPALA-6326: part 2: remove fetch thread in stress test .. Patch Set 6: (2 comments) http://gerrit.cloudera.org:8080/#/c/12681/6/tests/stress/query_runner.py File tests/stress/query_runner.py: http://gerrit.cloudera.org:8080/#/c/12681/6/tests/stress/query_runner.py@201 PS6, Line 201: IMPALA-8288 > this JIRA has already been resolved Removed workaround http://gerrit.cloudera.org:8080/#/c/12681/6/tests/stress/query_runner.py@253 PS6, Line 253: from > to? Done -- To view, visit http://gerrit.cloudera.org:8080/12681 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: If9afd74e1408823a9e5c0f2628ec9f8aafdcec69 Gerrit-Change-Number: 12681 Gerrit-PatchSet: 6 Gerrit-Owner: Tim Armstrong Gerrit-Reviewer: David Knupp Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Thomas Marshall Gerrit-Reviewer: Tim Armstrong Gerrit-Comment-Date: Thu, 28 Mar 2019 02:35:52 + Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-6326: part 2: remove fetch thread in stress test
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/12681 ) Change subject: IMPALA-6326: part 2: remove fetch thread in stress test .. Patch Set 8: Build started: https://jenkins.impala.io/job/gerrit-verify-dryrun/3963/ DRY_RUN=false -- To view, visit http://gerrit.cloudera.org:8080/12681 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: If9afd74e1408823a9e5c0f2628ec9f8aafdcec69 Gerrit-Change-Number: 12681 Gerrit-PatchSet: 8 Gerrit-Owner: Tim Armstrong Gerrit-Reviewer: David Knupp Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Thomas Marshall Gerrit-Reviewer: Tim Armstrong Gerrit-Comment-Date: Thu, 28 Mar 2019 02:36:51 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-6326: part 2: remove fetch thread in stress test
Thomas Marshall has posted comments on this change. ( http://gerrit.cloudera.org:8080/12681 ) Change subject: IMPALA-6326: part 2: remove fetch thread in stress test .. Patch Set 6: Code-Review+2 (2 comments) http://gerrit.cloudera.org:8080/#/c/12681/6/tests/stress/query_runner.py File tests/stress/query_runner.py: http://gerrit.cloudera.org:8080/#/c/12681/6/tests/stress/query_runner.py@201 PS6, Line 201: IMPALA-8288 this JIRA has already been resolved http://gerrit.cloudera.org:8080/#/c/12681/6/tests/stress/query_runner.py@253 PS6, Line 253: from to? -- To view, visit http://gerrit.cloudera.org:8080/12681 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: If9afd74e1408823a9e5c0f2628ec9f8aafdcec69 Gerrit-Change-Number: 12681 Gerrit-PatchSet: 6 Gerrit-Owner: Tim Armstrong Gerrit-Reviewer: David Knupp Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Thomas Marshall Gerrit-Reviewer: Tim Armstrong Gerrit-Comment-Date: Wed, 27 Mar 2019 17:12:08 + Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-6326: part 2: remove fetch thread in stress test
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/12681 ) Change subject: IMPALA-6326: part 2: remove fetch thread in stress test .. Patch Set 6: Build Successful https://jenkins.impala.io/job/gerrit-code-review-checks/2493/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests. -- To view, visit http://gerrit.cloudera.org:8080/12681 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: If9afd74e1408823a9e5c0f2628ec9f8aafdcec69 Gerrit-Change-Number: 12681 Gerrit-PatchSet: 6 Gerrit-Owner: Tim Armstrong Gerrit-Reviewer: David Knupp Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Thomas Marshall Gerrit-Reviewer: Tim Armstrong Gerrit-Comment-Date: Wed, 20 Mar 2019 23:00:27 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-6326: part 2: remove fetch thread in stress test
Tim Armstrong has posted comments on this change. ( http://gerrit.cloudera.org:8080/12681 ) Change subject: IMPALA-6326: part 2: remove fetch thread in stress test .. Patch Set 6: Rebased and resolved a conflict with the merged version of part 1 -- To view, visit http://gerrit.cloudera.org:8080/12681 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: If9afd74e1408823a9e5c0f2628ec9f8aafdcec69 Gerrit-Change-Number: 12681 Gerrit-PatchSet: 6 Gerrit-Owner: Tim Armstrong Gerrit-Reviewer: David Knupp Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Thomas Marshall Gerrit-Reviewer: Tim Armstrong Gerrit-Comment-Date: Wed, 20 Mar 2019 22:33:48 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-6326: part 2: remove fetch thread in stress test
Hello Thomas Marshall, David Knupp, Impala Public Jenkins, I'd like you to reexamine a change. Please visit http://gerrit.cloudera.org:8080/12681 to look at the new patch set (#6). Change subject: IMPALA-6326: part 2: remove fetch thread in stress test .. IMPALA-6326: part 2: remove fetch thread in stress test Ensure cursor is only accessed from a single thread. The means reworking the code so that we check the time limit between fetch calls. Use EXEC_TIME_LIMIT_S as an alternative to the previous multi-threaded cancellation logic - it allows queries to be cancelled even when the client is blocked or slow. This is implemented with the concept of a CancelMechanism that determines *how* a query should be cancelled. Query timeouts (where we want to cancel queries that run longer than expected) are implemented using both cancel mechanisms, in case the client is stuck in fetch or similar. Expected cancellations are implemented with a random mechanism so that both code paths get covered. Testing: Ran a cluster stress test. Ran a couple of single-node stress tests with TPC-H and random queries. Change-Id: If9afd74e1408823a9e5c0f2628ec9f8aafdcec69 --- M tests/stress/concurrent_select.py M tests/stress/query_runner.py 2 files changed, 129 insertions(+), 90 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/81/12681/6 -- To view, visit http://gerrit.cloudera.org:8080/12681 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: If9afd74e1408823a9e5c0f2628ec9f8aafdcec69 Gerrit-Change-Number: 12681 Gerrit-PatchSet: 6 Gerrit-Owner: Tim Armstrong Gerrit-Reviewer: David Knupp Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Thomas Marshall
[Impala-ASF-CR] IMPALA-6326: part 2: remove fetch thread in stress test
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/12681 ) Change subject: IMPALA-6326: part 2: remove fetch thread in stress test .. Patch Set 3: Build Successful https://jenkins.impala.io/job/gerrit-code-review-checks/2382/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests. -- To view, visit http://gerrit.cloudera.org:8080/12681 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: If9afd74e1408823a9e5c0f2628ec9f8aafdcec69 Gerrit-Change-Number: 12681 Gerrit-PatchSet: 3 Gerrit-Owner: Tim Armstrong Gerrit-Reviewer: Impala Public Jenkins Gerrit-Comment-Date: Thu, 07 Mar 2019 02:38:34 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-6326: part 2: remove fetch thread in stress test
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/12681 ) Change subject: IMPALA-6326: part 2: remove fetch thread in stress test .. Patch Set 2: Build Successful https://jenkins.impala.io/job/gerrit-code-review-checks/2381/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests. -- To view, visit http://gerrit.cloudera.org:8080/12681 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: If9afd74e1408823a9e5c0f2628ec9f8aafdcec69 Gerrit-Change-Number: 12681 Gerrit-PatchSet: 2 Gerrit-Owner: Tim Armstrong Gerrit-Reviewer: Impala Public Jenkins Gerrit-Comment-Date: Thu, 07 Mar 2019 02:18:29 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-6326: part 2: remove fetch thread in stress test
Tim Armstrong has uploaded a new patch set (#3). ( http://gerrit.cloudera.org:8080/12681 ) Change subject: IMPALA-6326: part 2: remove fetch thread in stress test .. IMPALA-6326: part 2: remove fetch thread in stress test Ensure cursor is only accessed from a single thread. The means reworking the code so that we check the time limit between fetch calls. Use EXEC_TIME_LIMIT_S as an alternative to the previous multi-threaded cancellation logic - it allows queries to be cancelled even when the client is blocked or slow. This is implemented with the concept of a CancelMechanism that determines *how* a query should be cancelled. Query timeouts (where we want to cancel queries that run longer than expected) are implemented using both cancel mechanisms, in case the client is stuck in fetch or similar. Expected cancellations are implemented with a random mechanism so that both code paths get covered. Testing: Ran a cluster stress test. Ran a couple of single-node stress tests with TPC-H and random queries. Change-Id: If9afd74e1408823a9e5c0f2628ec9f8aafdcec69 --- M tests/stress/concurrent_select.py M tests/stress/query_runner.py 2 files changed, 127 insertions(+), 88 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/81/12681/3 -- To view, visit http://gerrit.cloudera.org:8080/12681 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: If9afd74e1408823a9e5c0f2628ec9f8aafdcec69 Gerrit-Change-Number: 12681 Gerrit-PatchSet: 3 Gerrit-Owner: Tim Armstrong
[Impala-ASF-CR] IMPALA-6326: part 2: remove fetch thread in stress test
Tim Armstrong has uploaded a new patch set (#2). ( http://gerrit.cloudera.org:8080/12681 ) Change subject: IMPALA-6326: part 2: remove fetch thread in stress test .. IMPALA-6326: part 2: remove fetch thread in stress test Ensure cursor is only accessed from a single thread. The means reworking the code so that we check the time limit between fetch calls. Use EXEC_TIME_LIMIT_S as an alternative to the previous multi-threaded cancellation logic - it allows queries to be cancelled even when the client is blocked or slow. This is implemented with the concept of a CancelMechanism that determines *how* a query should be cancelled. Query timeouts (where we want to cancel queries that run longer than expected) are implemented using both cancel mechanisms, in case the client is stuck in fetch or similar. Expected cancellations are implemented with a random mechanism so that both code paths get covered. Testing: Ran a cluster stress test. Ran a couple of single-node stress tests with TPC-H and random queries. Change-Id: If9afd74e1408823a9e5c0f2628ec9f8aafdcec69 --- M tests/stress/concurrent_select.py M tests/stress/query_runner.py 2 files changed, 129 insertions(+), 88 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/81/12681/2 -- To view, visit http://gerrit.cloudera.org:8080/12681 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: If9afd74e1408823a9e5c0f2628ec9f8aafdcec69 Gerrit-Change-Number: 12681 Gerrit-PatchSet: 2 Gerrit-Owner: Tim Armstrong
