[Impala-ASF-CR] IMPALA-9889: Fixed flaky test runtime filters on Kudu table
Impala Public Jenkins has submitted this change and it was merged. ( http://gerrit.cloudera.org:8080/16191 ) Change subject: IMPALA-9889: Fixed flaky test_runtime_filters on Kudu table .. IMPALA-9889: Fixed flaky test_runtime_filters on Kudu table Test cases in test_runtime_filters failed occasionally in ASAN builds due to runtime filters not arriving scan nodes in time. Query profiles showed that codegen took 2 to 4 minutes for one fragment when this issue happened. This caused hash join nodes waiting long time to generate and publish runtime filters, hence arrival delay on scan nodes. To avoid the delay, turn on ASYNC_CODEGEN for test_runtime_filters when test runs for slow build like ASAN, TSAN, UBSAN, etc. Testing: - Passed core test for regular debug build and ASAN build. Change-Id: I94a08e272f0870c04c96563fa614e3416fb5379b Reviewed-on: http://gerrit.cloudera.org:8080/16191 Reviewed-by: Thomas Tauber-Marshall Tested-by: Impala Public Jenkins --- M tests/query_test/test_runtime_filters.py 1 file changed, 23 insertions(+), 0 deletions(-) Approvals: Thomas Tauber-Marshall: Looks good to me, approved Impala Public Jenkins: Verified -- To view, visit http://gerrit.cloudera.org:8080/16191 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: merged Gerrit-Change-Id: I94a08e272f0870c04c96563fa614e3416fb5379b Gerrit-Change-Number: 16191 Gerrit-PatchSet: 6 Gerrit-Owner: Wenzhe Zhou Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Thomas Tauber-Marshall Gerrit-Reviewer: Wenzhe Zhou
[Impala-ASF-CR] IMPALA-9889: Fixed flaky test runtime filters on Kudu table
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/16191 ) Change subject: IMPALA-9889: Fixed flaky test_runtime_filters on Kudu table .. Patch Set 5: Verified+1 -- To view, visit http://gerrit.cloudera.org:8080/16191 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I94a08e272f0870c04c96563fa614e3416fb5379b Gerrit-Change-Number: 16191 Gerrit-PatchSet: 5 Gerrit-Owner: Wenzhe Zhou Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Thomas Tauber-Marshall Gerrit-Reviewer: Wenzhe Zhou Gerrit-Comment-Date: Tue, 21 Jul 2020 03:05:29 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-9889: Fixed flaky test runtime filters on Kudu table
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/16191 ) Change subject: IMPALA-9889: Fixed flaky test_runtime_filters on Kudu table .. Patch Set 5: Build started: https://jenkins.impala.io/job/gerrit-verify-dryrun/6149/ DRY_RUN=false -- To view, visit http://gerrit.cloudera.org:8080/16191 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I94a08e272f0870c04c96563fa614e3416fb5379b Gerrit-Change-Number: 16191 Gerrit-PatchSet: 5 Gerrit-Owner: Wenzhe Zhou Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Thomas Tauber-Marshall Gerrit-Reviewer: Wenzhe Zhou Gerrit-Comment-Date: Mon, 20 Jul 2020 21:56:16 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-9889: Fixed flaky test runtime filters on Kudu table
Thomas Tauber-Marshall has posted comments on this change. ( http://gerrit.cloudera.org:8080/16191 ) Change subject: IMPALA-9889: Fixed flaky test_runtime_filters on Kudu table .. Patch Set 5: Code-Review+2 -- To view, visit http://gerrit.cloudera.org:8080/16191 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I94a08e272f0870c04c96563fa614e3416fb5379b Gerrit-Change-Number: 16191 Gerrit-PatchSet: 5 Gerrit-Owner: Wenzhe Zhou Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Thomas Tauber-Marshall Gerrit-Reviewer: Wenzhe Zhou Gerrit-Comment-Date: Mon, 20 Jul 2020 21:41:00 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-9889: Fixed flaky test runtime filters on Kudu table
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/16191 ) Change subject: IMPALA-9889: Fixed flaky test_runtime_filters on Kudu table .. Patch Set 5: Build Successful https://jenkins.impala.io/job/gerrit-code-review-checks/6659/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests. -- To view, visit http://gerrit.cloudera.org:8080/16191 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I94a08e272f0870c04c96563fa614e3416fb5379b Gerrit-Change-Number: 16191 Gerrit-PatchSet: 5 Gerrit-Owner: Wenzhe Zhou Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Thomas Tauber-Marshall Gerrit-Reviewer: Wenzhe Zhou Gerrit-Comment-Date: Mon, 20 Jul 2020 19:48:33 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-9889: Fixed flaky test runtime filters on Kudu table
Wenzhe Zhou has uploaded a new patch set (#5). ( http://gerrit.cloudera.org:8080/16191 ) Change subject: IMPALA-9889: Fixed flaky test_runtime_filters on Kudu table .. IMPALA-9889: Fixed flaky test_runtime_filters on Kudu table Test cases in test_runtime_filters failed occasionally in ASAN builds due to runtime filters not arriving scan nodes in time. Query profiles showed that codegen took 2 to 4 minutes for one fragment when this issue happened. This caused hash join nodes waiting long time to generate and publish runtime filters, hence arrival delay on scan nodes. To avoid the delay, turn on ASYNC_CODEGEN for test_runtime_filters when test runs for slow build like ASAN, TSAN, UBSAN, etc. Testing: - Passed core test for regular debug build and ASAN build. Change-Id: I94a08e272f0870c04c96563fa614e3416fb5379b --- M tests/query_test/test_runtime_filters.py 1 file changed, 23 insertions(+), 0 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/91/16191/5 -- To view, visit http://gerrit.cloudera.org:8080/16191 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: I94a08e272f0870c04c96563fa614e3416fb5379b Gerrit-Change-Number: 16191 Gerrit-PatchSet: 5 Gerrit-Owner: Wenzhe Zhou Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Thomas Tauber-Marshall Gerrit-Reviewer: Wenzhe Zhou
[Impala-ASF-CR] IMPALA-9889: Fixed flaky test runtime filters on Kudu table
Wenzhe Zhou has posted comments on this change. (
http://gerrit.cloudera.org:8080/16191 )
Change subject: IMPALA-9889: Fixed flaky test_runtime_filters on Kudu table
..
Patch Set 4:
(2 comments)
http://gerrit.cloudera.org:8080/#/c/16191/4/tests/query_test/test_runtime_filters.py
File tests/query_test/test_runtime_filters.py:
http://gerrit.cloudera.org:8080/#/c/16191/4/tests/query_test/test_runtime_filters.py@41
PS4, Line 41: is_asan
> Might be good to use runs_slowly() here instead, to ensure we don't see thi
Agree. Will apply the change for all slow build.
http://gerrit.cloudera.org:8080/#/c/16191/4/tests/query_test/test_runtime_filters.py@76
PS4, Line 76: self.execute_query("SET ASYNC_CODEGEN=TRUE")
> I think rather than doing this for each test case, you can use 'add_exec_op
Will fix it as suggested.
--
To view, visit http://gerrit.cloudera.org:8080/16191
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I94a08e272f0870c04c96563fa614e3416fb5379b
Gerrit-Change-Number: 16191
Gerrit-PatchSet: 4
Gerrit-Owner: Wenzhe Zhou
Gerrit-Reviewer: Impala Public Jenkins
Gerrit-Reviewer: Thomas Tauber-Marshall
Gerrit-Reviewer: Wenzhe Zhou
Gerrit-Comment-Date: Mon, 20 Jul 2020 19:15:37 +
Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-9889: Fixed flaky test runtime filters on Kudu table
Thomas Tauber-Marshall has posted comments on this change. (
http://gerrit.cloudera.org:8080/16191 )
Change subject: IMPALA-9889: Fixed flaky test_runtime_filters on Kudu table
..
Patch Set 4:
(2 comments)
http://gerrit.cloudera.org:8080/#/c/16191/4/tests/query_test/test_runtime_filters.py
File tests/query_test/test_runtime_filters.py:
http://gerrit.cloudera.org:8080/#/c/16191/4/tests/query_test/test_runtime_filters.py@41
PS4, Line 41: is_asan
Might be good to use runs_slowly() here instead, to ensure we don't see this
same flakiness on ubsan, tsan, etc.
http://gerrit.cloudera.org:8080/#/c/16191/4/tests/query_test/test_runtime_filters.py@76
PS4, Line 76: self.execute_query("SET ASYNC_CODEGEN=TRUE")
I think rather than doing this for each test case, you can use
'add_exec_option_dimension' in add_test_dimensions above, which will ensure
every test in the class is covered.
--
To view, visit http://gerrit.cloudera.org:8080/16191
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I94a08e272f0870c04c96563fa614e3416fb5379b
Gerrit-Change-Number: 16191
Gerrit-PatchSet: 4
Gerrit-Owner: Wenzhe Zhou
Gerrit-Reviewer: Impala Public Jenkins
Gerrit-Reviewer: Thomas Tauber-Marshall
Gerrit-Comment-Date: Mon, 20 Jul 2020 15:33:19 +
Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-9889: Fixed flaky test runtime filters on Kudu table
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/16191 ) Change subject: IMPALA-9889: Fixed flaky test_runtime_filters on Kudu table .. Patch Set 4: Build Successful https://jenkins.impala.io/job/gerrit-code-review-checks/6617/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests. -- To view, visit http://gerrit.cloudera.org:8080/16191 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I94a08e272f0870c04c96563fa614e3416fb5379b Gerrit-Change-Number: 16191 Gerrit-PatchSet: 4 Gerrit-Owner: Wenzhe Zhou Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Thomas Tauber-Marshall Gerrit-Comment-Date: Thu, 16 Jul 2020 16:15:40 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-9889: Fixed flaky test runtime filters on Kudu table
Wenzhe Zhou has uploaded a new patch set (#4). ( http://gerrit.cloudera.org:8080/16191 ) Change subject: IMPALA-9889: Fixed flaky test_runtime_filters on Kudu table .. IMPALA-9889: Fixed flaky test_runtime_filters on Kudu table Test cases in test_runtime_filters failed occasionally in ASAN builds due to runtime filters not arriving scan nodes in time. Query profiles showed that codegen took 2 to 4 minutes for one fragment when this issue happened. This caused hash join nodes waiting long time to generate and publish runtime filters, hence arrival delay on scan nodes. To avoid the delay, turn on ASYNC_CODEGEN for test_runtime_filters agaiest Kudu table when test runs for ASAN build. Testing: - Passed core test for regular debug build and ASAN build. Change-Id: I94a08e272f0870c04c96563fa614e3416fb5379b --- M tests/query_test/test_runtime_filters.py 1 file changed, 19 insertions(+), 0 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/91/16191/4 -- To view, visit http://gerrit.cloudera.org:8080/16191 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: I94a08e272f0870c04c96563fa614e3416fb5379b Gerrit-Change-Number: 16191 Gerrit-PatchSet: 4 Gerrit-Owner: Wenzhe Zhou Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Thomas Tauber-Marshall
