[Impala-ASF-CR] IMPALA-9355: TestExchangeMemUsage.test exchange mem usage scaling doesn't hit the memory limit
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/17586 ) Change subject: IMPALA-9355: TestExchangeMemUsage.test_exchange_mem_usage_scaling doesn't hit the memory limit .. Patch Set 2: Verified+1 -- To view, visit http://gerrit.cloudera.org:8080/17586 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ia4ad4508028645b67de419cfdfa2327d2847cfc4 Gerrit-Change-Number: 17586 Gerrit-PatchSet: 2 Gerrit-Owner: Qifan Chen Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Quanlong Huang Gerrit-Reviewer: Riza Suminto Gerrit-Reviewer: Wenzhe Zhou Gerrit-Comment-Date: Mon, 14 Jun 2021 17:13:24 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-9355: TestExchangeMemUsage.test exchange mem usage scaling doesn't hit the memory limit
Impala Public Jenkins has submitted this change and it was merged. ( http://gerrit.cloudera.org:8080/17586 ) Change subject: IMPALA-9355: TestExchangeMemUsage.test_exchange_mem_usage_scaling doesn't hit the memory limit .. IMPALA-9355: TestExchangeMemUsage.test_exchange_mem_usage_scaling doesn't hit the memory limit This patch restores the previous testing enviroment in which exchange node 5 in the test query shown below reliablly reports memory exceeding limit error. set mem_limit=171520k; set num_scanner_threads=1; select * from tpch_parquet.lineitem l1 join tpch_parquet.lineitem l2 on l1.l_orderkey = l2.l_orderkey and l1.l_partkey = l2.l_partkey and l1.l_suppkey = l2.l_suppkey and l1.l_linenumber = l2.l_linenumber order by l1.l_orderkey desc, l1.l_partkey, l1.l_suppkey, l1.l_linenumber limit 5 In the patch, the memory limit for the entire query is lowered to 164Mb, less than 171520k, the minimum for the query to be accepted after the result spooling is enabled by default. This is achieved by disabling result spooling by setting spool_query_results to false. Testing: 1). Ran exchange-mem-scaling.test Change-Id: Ia4ad4508028645b67de419cfdfa2327d2847cfc4 Reviewed-on: http://gerrit.cloudera.org:8080/17586 Reviewed-by: Impala Public Jenkins Tested-by: Impala Public Jenkins --- M testdata/workloads/functional-query/queries/QueryTest/exchange-mem-scaling.test 1 file changed, 7 insertions(+), 1 deletion(-) Approvals: Impala Public Jenkins: Looks good to me, approved; Verified -- To view, visit http://gerrit.cloudera.org:8080/17586 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: merged Gerrit-Change-Id: Ia4ad4508028645b67de419cfdfa2327d2847cfc4 Gerrit-Change-Number: 17586 Gerrit-PatchSet: 3 Gerrit-Owner: Qifan Chen Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Quanlong Huang Gerrit-Reviewer: Riza Suminto Gerrit-Reviewer: Wenzhe Zhou
[Impala-ASF-CR] IMPALA-9355: TestExchangeMemUsage.test exchange mem usage scaling doesn't hit the memory limit
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/17586 ) Change subject: IMPALA-9355: TestExchangeMemUsage.test_exchange_mem_usage_scaling doesn't hit the memory limit .. Patch Set 2: Code-Review+2 -- To view, visit http://gerrit.cloudera.org:8080/17586 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ia4ad4508028645b67de419cfdfa2327d2847cfc4 Gerrit-Change-Number: 17586 Gerrit-PatchSet: 2 Gerrit-Owner: Qifan Chen Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Quanlong Huang Gerrit-Reviewer: Riza Suminto Gerrit-Reviewer: Wenzhe Zhou Gerrit-Comment-Date: Mon, 14 Jun 2021 11:15:03 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-9355: TestExchangeMemUsage.test exchange mem usage scaling doesn't hit the memory limit
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/17586 ) Change subject: IMPALA-9355: TestExchangeMemUsage.test_exchange_mem_usage_scaling doesn't hit the memory limit .. Patch Set 2: Build started: https://jenkins.impala.io/job/gerrit-verify-dryrun/7224/ DRY_RUN=false -- To view, visit http://gerrit.cloudera.org:8080/17586 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ia4ad4508028645b67de419cfdfa2327d2847cfc4 Gerrit-Change-Number: 17586 Gerrit-PatchSet: 2 Gerrit-Owner: Qifan Chen Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Quanlong Huang Gerrit-Reviewer: Riza Suminto Gerrit-Reviewer: Wenzhe Zhou Gerrit-Comment-Date: Mon, 14 Jun 2021 11:15:04 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-9355: TestExchangeMemUsage.test exchange mem usage scaling doesn't hit the memory limit
Quanlong Huang has posted comments on this change. ( http://gerrit.cloudera.org:8080/17586 ) Change subject: IMPALA-9355: TestExchangeMemUsage.test_exchange_mem_usage_scaling doesn't hit the memory limit .. Patch Set 1: Code-Review+2 Thank Qifan for fixing this! -- To view, visit http://gerrit.cloudera.org:8080/17586 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ia4ad4508028645b67de419cfdfa2327d2847cfc4 Gerrit-Change-Number: 17586 Gerrit-PatchSet: 1 Gerrit-Owner: Qifan Chen Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Quanlong Huang Gerrit-Reviewer: Riza Suminto Gerrit-Reviewer: Wenzhe Zhou Gerrit-Comment-Date: Sun, 13 Jun 2021 00:00:03 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-9355: TestExchangeMemUsage.test exchange mem usage scaling doesn't hit the memory limit
Riza Suminto has posted comments on this change. ( http://gerrit.cloudera.org:8080/17586 ) Change subject: IMPALA-9355: TestExchangeMemUsage.test_exchange_mem_usage_scaling doesn't hit the memory limit .. Patch Set 1: Code-Review+1 -- To view, visit http://gerrit.cloudera.org:8080/17586 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ia4ad4508028645b67de419cfdfa2327d2847cfc4 Gerrit-Change-Number: 17586 Gerrit-PatchSet: 1 Gerrit-Owner: Qifan Chen Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Riza Suminto Gerrit-Reviewer: Wenzhe Zhou Gerrit-Comment-Date: Fri, 11 Jun 2021 23:22:44 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-9355: TestExchangeMemUsage.test exchange mem usage scaling doesn't hit the memory limit
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/17586 ) Change subject: IMPALA-9355: TestExchangeMemUsage.test_exchange_mem_usage_scaling doesn't hit the memory limit .. Patch Set 1: Build Successful https://jenkins.impala.io/job/gerrit-code-review-checks/8899/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests. -- To view, visit http://gerrit.cloudera.org:8080/17586 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ia4ad4508028645b67de419cfdfa2327d2847cfc4 Gerrit-Change-Number: 17586 Gerrit-PatchSet: 1 Gerrit-Owner: Qifan Chen Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Wenzhe Zhou Gerrit-Comment-Date: Fri, 11 Jun 2021 18:52:12 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-9355: TestExchangeMemUsage.test exchange mem usage scaling doesn't hit the memory limit
Wenzhe Zhou has posted comments on this change. ( http://gerrit.cloudera.org:8080/17586 ) Change subject: IMPALA-9355: TestExchangeMemUsage.test_exchange_mem_usage_scaling doesn't hit the memory limit .. Patch Set 1: Code-Review+1 -- To view, visit http://gerrit.cloudera.org:8080/17586 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ia4ad4508028645b67de419cfdfa2327d2847cfc4 Gerrit-Change-Number: 17586 Gerrit-PatchSet: 1 Gerrit-Owner: Qifan Chen Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Wenzhe Zhou Gerrit-Comment-Date: Fri, 11 Jun 2021 18:44:37 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-9355: TestExchangeMemUsage.test exchange mem usage scaling doesn't hit the memory limit
Qifan Chen has uploaded this change for review. ( http://gerrit.cloudera.org:8080/17586 Change subject: IMPALA-9355: TestExchangeMemUsage.test_exchange_mem_usage_scaling doesn't hit the memory limit .. IMPALA-9355: TestExchangeMemUsage.test_exchange_mem_usage_scaling doesn't hit the memory limit This patch restores the previous testing enviroment in which exchange node 5 in the test query shown below reliablly reports memory exceeding limit error. set mem_limit=171520k; set num_scanner_threads=1; select * from tpch_parquet.lineitem l1 join tpch_parquet.lineitem l2 on l1.l_orderkey = l2.l_orderkey and l1.l_partkey = l2.l_partkey and l1.l_suppkey = l2.l_suppkey and l1.l_linenumber = l2.l_linenumber order by l1.l_orderkey desc, l1.l_partkey, l1.l_suppkey, l1.l_linenumber limit 5 In the patch, the memory limit for the entire query is lowered to 164Mb, less than 171520k, the minimum for the query to be accepted after the result spooling is enabled by default. This is achieved by disabling result spooling by setting spool_query_results to false. Testing: 1). Ran exchange-mem-scaling.test Change-Id: Ia4ad4508028645b67de419cfdfa2327d2847cfc4 --- M testdata/workloads/functional-query/queries/QueryTest/exchange-mem-scaling.test 1 file changed, 7 insertions(+), 1 deletion(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/86/17586/1 -- To view, visit http://gerrit.cloudera.org:8080/17586 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newchange Gerrit-Change-Id: Ia4ad4508028645b67de419cfdfa2327d2847cfc4 Gerrit-Change-Number: 17586 Gerrit-PatchSet: 1 Gerrit-Owner: Qifan Chen
[Impala-ASF-CR] IMPALA-9355: TestExchangeMemUsage.test exchange mem usage scaling doesn't hit the memory limit
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/16791 ) Change subject: IMPALA-9355: TestExchangeMemUsage.test_exchange_mem_usage_scaling doesn't hit the memory limit .. Patch Set 1: Verified+1 -- To view, visit http://gerrit.cloudera.org:8080/16791 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Id945d7e37fac07beb7808e6ccf8530e667cbaad4 Gerrit-Change-Number: 16791 Gerrit-PatchSet: 1 Gerrit-Owner: Qifan Chen Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Joe McDonnell Gerrit-Reviewer: Tim Armstrong Gerrit-Comment-Date: Mon, 30 Nov 2020 22:33:56 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-9355: TestExchangeMemUsage.test exchange mem usage scaling doesn't hit the memory limit
Impala Public Jenkins has submitted this change and it was merged. ( http://gerrit.cloudera.org:8080/16791 ) Change subject: IMPALA-9355: TestExchangeMemUsage.test_exchange_mem_usage_scaling doesn't hit the memory limit .. IMPALA-9355: TestExchangeMemUsage.test_exchange_mem_usage_scaling doesn't hit the memory limit This patch reduces the memory limit for the following query in test_exchange_mem_usage_scaling test from 170MB to 164MB to reduce the chance of not detecting a memory allocation failure. set mem_limit= set num_scanner_threads=1; select * from tpch_parquet.lineitem l1 join tpch_parquet.lineitem l2 on l1.l_orderkey = l2.l_orderkey and l1.l_partkey = l2.l_partkey and l1.l_suppkey = l2.l_suppkey and l1.l_linenumber = l2.l_linenumber order by l1.l_orderkey desc, l1.l_partkey, l1.l_suppkey, l1.l_linenumber limit 5; In a test with 500 executions of the above query with the memory limit set to 164MB, there were 500 memory allocation failures in total (one in each execution), and a total of 266 of them from Exchange Node #4. Testing: Ran the query in question individually; Ran TestExchangeMemUsage.test_exchange_mem_usage_scaling test; Ran core tests. Change-Id: Id945d7e37fac07beb7808e6ccf8530e667cbaad4 Reviewed-on: http://gerrit.cloudera.org:8080/16791 Reviewed-by: Tim Armstrong Tested-by: Impala Public Jenkins --- M testdata/workloads/functional-query/queries/QueryTest/exchange-mem-scaling.test 1 file changed, 1 insertion(+), 1 deletion(-) Approvals: Tim Armstrong: Looks good to me, approved Impala Public Jenkins: Verified -- To view, visit http://gerrit.cloudera.org:8080/16791 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: merged Gerrit-Change-Id: Id945d7e37fac07beb7808e6ccf8530e667cbaad4 Gerrit-Change-Number: 16791 Gerrit-PatchSet: 2 Gerrit-Owner: Qifan Chen Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Joe McDonnell Gerrit-Reviewer: Tim Armstrong
[Impala-ASF-CR] IMPALA-9355: TestExchangeMemUsage.test exchange mem usage scaling doesn't hit the memory limit
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/16791 ) Change subject: IMPALA-9355: TestExchangeMemUsage.test_exchange_mem_usage_scaling doesn't hit the memory limit .. Patch Set 1: Build started: https://jenkins.impala.io/job/gerrit-verify-dryrun/6719/ DRY_RUN=false -- To view, visit http://gerrit.cloudera.org:8080/16791 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Id945d7e37fac07beb7808e6ccf8530e667cbaad4 Gerrit-Change-Number: 16791 Gerrit-PatchSet: 1 Gerrit-Owner: Qifan Chen Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Joe McDonnell Gerrit-Reviewer: Tim Armstrong Gerrit-Comment-Date: Mon, 30 Nov 2020 17:10:29 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-9355: TestExchangeMemUsage.test exchange mem usage scaling doesn't hit the memory limit
Tim Armstrong has posted comments on this change. ( http://gerrit.cloudera.org:8080/16791 ) Change subject: IMPALA-9355: TestExchangeMemUsage.test_exchange_mem_usage_scaling doesn't hit the memory limit .. Patch Set 1: Code-Review+2 -- To view, visit http://gerrit.cloudera.org:8080/16791 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Id945d7e37fac07beb7808e6ccf8530e667cbaad4 Gerrit-Change-Number: 16791 Gerrit-PatchSet: 1 Gerrit-Owner: Qifan Chen Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Joe McDonnell Gerrit-Reviewer: Tim Armstrong Gerrit-Comment-Date: Mon, 30 Nov 2020 17:09:42 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-9355: TestExchangeMemUsage.test exchange mem usage scaling doesn't hit the memory limit
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/16791 ) Change subject: IMPALA-9355: TestExchangeMemUsage.test_exchange_mem_usage_scaling doesn't hit the memory limit .. Patch Set 1: Build Successful https://jenkins.impala.io/job/gerrit-code-review-checks/7746/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests. -- To view, visit http://gerrit.cloudera.org:8080/16791 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Id945d7e37fac07beb7808e6ccf8530e667cbaad4 Gerrit-Change-Number: 16791 Gerrit-PatchSet: 1 Gerrit-Owner: Qifan Chen Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Joe McDonnell Gerrit-Reviewer: Tim Armstrong Gerrit-Comment-Date: Sun, 29 Nov 2020 21:19:03 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-9355: TestExchangeMemUsage.test exchange mem usage scaling doesn't hit the memory limit
Qifan Chen has uploaded this change for review. ( http://gerrit.cloudera.org:8080/16791 Change subject: IMPALA-9355: TestExchangeMemUsage.test_exchange_mem_usage_scaling doesn't hit the memory limit .. IMPALA-9355: TestExchangeMemUsage.test_exchange_mem_usage_scaling doesn't hit the memory limit This patch reduces the memory limit for the following query in test_exchange_mem_usage_scaling test from 170MB to 164MB to reduce the chance of not detecting a memory allocation failure. set mem_limit= set num_scanner_threads=1; select * from tpch_parquet.lineitem l1 join tpch_parquet.lineitem l2 on l1.l_orderkey = l2.l_orderkey and l1.l_partkey = l2.l_partkey and l1.l_suppkey = l2.l_suppkey and l1.l_linenumber = l2.l_linenumber order by l1.l_orderkey desc, l1.l_partkey, l1.l_suppkey, l1.l_linenumber limit 5; In a test with 500 executions of the above query with the memory limit set to 164MB, there were 500 memory allocation failures in total (one in each execution), and a total of 266 of them from Exchange Node #4. Testing: Ran the query in question individually; Ran TestExchangeMemUsage.test_exchange_mem_usage_scaling test; Ran core tests. Change-Id: Id945d7e37fac07beb7808e6ccf8530e667cbaad4 --- M testdata/workloads/functional-query/queries/QueryTest/exchange-mem-scaling.test 1 file changed, 1 insertion(+), 1 deletion(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/91/16791/1 -- To view, visit http://gerrit.cloudera.org:8080/16791 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newchange Gerrit-Change-Id: Id945d7e37fac07beb7808e6ccf8530e667cbaad4 Gerrit-Change-Number: 16791 Gerrit-PatchSet: 1 Gerrit-Owner: Qifan Chen
[Impala-ASF-CR] IMPALA-9355: TestExchangeMemUsage.test exchange mem usage scaling doesn't hit the memory limit
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/16676 ) Change subject: IMPALA-9355: TestExchangeMemUsage.test_exchange_mem_usage_scaling doesn't hit the memory limit .. Patch Set 3: Build Successful https://jenkins.impala.io/job/gerrit-code-review-checks/7609/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests. -- To view, visit http://gerrit.cloudera.org:8080/16676 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I3c33e3b254ce75054bf6cee0eff8b7206fa0c122 Gerrit-Change-Number: 16676 Gerrit-PatchSet: 3 Gerrit-Owner: Qifan Chen Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Joe McDonnell Gerrit-Reviewer: Qifan Chen Gerrit-Comment-Date: Tue, 03 Nov 2020 17:53:08 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-9355: TestExchangeMemUsage.test exchange mem usage scaling doesn't hit the memory limit
Joe McDonnell has posted comments on this change. ( http://gerrit.cloudera.org:8080/16676 ) Change subject: IMPALA-9355: TestExchangeMemUsage.test_exchange_mem_usage_scaling doesn't hit the memory limit .. Patch Set 3: Code-Review+2 Thanks for fixing this! -- To view, visit http://gerrit.cloudera.org:8080/16676 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I3c33e3b254ce75054bf6cee0eff8b7206fa0c122 Gerrit-Change-Number: 16676 Gerrit-PatchSet: 3 Gerrit-Owner: Qifan Chen Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Joe McDonnell Gerrit-Reviewer: Qifan Chen Gerrit-Comment-Date: Tue, 03 Nov 2020 17:43:10 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-9355: TestExchangeMemUsage.test exchange mem usage scaling doesn't hit the memory limit
Qifan Chen has posted comments on this change. ( http://gerrit.cloudera.org:8080/16676 ) Change subject: IMPALA-9355: TestExchangeMemUsage.test_exchange_mem_usage_scaling doesn't hit the memory limit .. Patch Set 3: (1 comment) http://gerrit.cloudera.org:8080/#/c/16676/2/testdata/workloads/functional-query/queries/QueryTest/exchange-mem-scaling.test File testdata/workloads/functional-query/queries/QueryTest/exchange-mem-scaling.test: http://gerrit.cloudera.org:8080/#/c/16676/2/testdata/workloads/functional-query/queries/QueryTest/exchange-mem-scaling.test@7 PS2, Line 7: set mem_limit=168m; : set num_scanner_threads=1; : select * : from tpch_parquet.lin > From a purely style note, I think this comment is not necessary. The commen Done -- To view, visit http://gerrit.cloudera.org:8080/16676 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I3c33e3b254ce75054bf6cee0eff8b7206fa0c122 Gerrit-Change-Number: 16676 Gerrit-PatchSet: 3 Gerrit-Owner: Qifan Chen Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Joe McDonnell Gerrit-Reviewer: Qifan Chen Gerrit-Comment-Date: Tue, 03 Nov 2020 17:31:34 + Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-9355: TestExchangeMemUsage.test exchange mem usage scaling doesn't hit the memory limit
Qifan Chen has uploaded a new patch set (#3). ( http://gerrit.cloudera.org:8080/16676 ) Change subject: IMPALA-9355: TestExchangeMemUsage.test_exchange_mem_usage_scaling doesn't hit the memory limit .. IMPALA-9355: TestExchangeMemUsage.test_exchange_mem_usage_scaling doesn't hit the memory limit This fix improved the robustness of the test by lowering the memory limit from 170MB to 168MB. In 10 local tests, the memory allocation shortage at EXCHANGE_NODE was as low as 8KB without exceeding the limit of 170MB. Testing: 1. Unit testing with 200 test iterations. Change-Id: I3c33e3b254ce75054bf6cee0eff8b7206fa0c122 --- M testdata/workloads/functional-query/queries/QueryTest/exchange-mem-scaling.test 1 file changed, 1 insertion(+), 1 deletion(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/76/16676/3 -- To view, visit http://gerrit.cloudera.org:8080/16676 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: I3c33e3b254ce75054bf6cee0eff8b7206fa0c122 Gerrit-Change-Number: 16676 Gerrit-PatchSet: 3 Gerrit-Owner: Qifan Chen Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Joe McDonnell
[Impala-ASF-CR] IMPALA-9355: TestExchangeMemUsage.test exchange mem usage scaling doesn't hit the memory limit
Joe McDonnell has posted comments on this change. ( http://gerrit.cloudera.org:8080/16676 ) Change subject: IMPALA-9355: TestExchangeMemUsage.test_exchange_mem_usage_scaling doesn't hit the memory limit .. Patch Set 2: (1 comment) The change looks good from a content point of view. I have one small style point. http://gerrit.cloudera.org:8080/#/c/16676/2/testdata/workloads/functional-query/queries/QueryTest/exchange-mem-scaling.test File testdata/workloads/functional-query/queries/QueryTest/exchange-mem-scaling.test: http://gerrit.cloudera.org:8080/#/c/16676/2/testdata/workloads/functional-query/queries/QueryTest/exchange-mem-scaling.test@7 PS2, Line 7: # IMPALA-9355: TestExchangeMemUsage.test_exchange_mem_usage_scaling doesn't hit : # the memory limit. The mem_limit is reduced from 170MB to 168MB since the memory : # shortage observed at exchange nodes can be as small as 8KB without exceeding : # the limit of 170MB. >From a purely style note, I think this comment is not necessary. The comment >about IMPALA-5485 is describing why this test exists and is useful to a >reader. The details about adjustments to the memory limit to avoid flakiness >are important for the history, but it is usually not something a reader of >this file needs to know about. All the history is accessible by git blame, so >they can find the commit that set the mem_limit=168m if they need. So, I would >lean toward not including a comment for this. -- To view, visit http://gerrit.cloudera.org:8080/16676 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I3c33e3b254ce75054bf6cee0eff8b7206fa0c122 Gerrit-Change-Number: 16676 Gerrit-PatchSet: 2 Gerrit-Owner: Qifan Chen Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Joe McDonnell Gerrit-Comment-Date: Tue, 03 Nov 2020 01:30:07 + Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-9355: TestExchangeMemUsage.test exchange mem usage scaling doesn't hit the memory limit
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/16676 ) Change subject: IMPALA-9355: TestExchangeMemUsage.test_exchange_mem_usage_scaling doesn't hit the memory limit .. Patch Set 2: Build Successful https://jenkins.impala.io/job/gerrit-code-review-checks/7578/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests. -- To view, visit http://gerrit.cloudera.org:8080/16676 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I3c33e3b254ce75054bf6cee0eff8b7206fa0c122 Gerrit-Change-Number: 16676 Gerrit-PatchSet: 2 Gerrit-Owner: Qifan Chen Gerrit-Reviewer: Impala Public Jenkins Gerrit-Comment-Date: Thu, 29 Oct 2020 16:57:41 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-9355: TestExchangeMemUsage.test exchange mem usage scaling doesn't hit the memory limit
Qifan Chen has uploaded this change for review. ( http://gerrit.cloudera.org:8080/16676 Change subject: IMPALA-9355: TestExchangeMemUsage.test_exchange_mem_usage_scaling doesn't hit the memory limit .. IMPALA-9355: TestExchangeMemUsage.test_exchange_mem_usage_scaling doesn't hit the memory limit This fix improved the robustness of the test by lowering the memory limit from 170MB to 168MB. In 10 local tests, the memory allocation shortage at EXCHANGE_NODE was as low as 8KB without exceeding the limit of 170MB. Testing: 1. Unit testing with 200 test iterations. Change-Id: I3c33e3b254ce75054bf6cee0eff8b7206fa0c122 --- M testdata/workloads/functional-query/queries/QueryTest/exchange-mem-scaling.test 1 file changed, 5 insertions(+), 1 deletion(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/76/16676/2 -- To view, visit http://gerrit.cloudera.org:8080/16676 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newchange Gerrit-Change-Id: I3c33e3b254ce75054bf6cee0eff8b7206fa0c122 Gerrit-Change-Number: 16676 Gerrit-PatchSet: 2 Gerrit-Owner: Qifan Chen
