[Impala-ASF-CR] IMPALA-9382: part 2/3: aggregate profiles sent to coordinator
Tim Armstrong has posted comments on this change. ( http://gerrit.cloudera.org:8080/16057 ) Change subject: IMPALA-9382: part 2/3: aggregate profiles sent to coordinator .. Patch Set 17: Looks like I missed updating a test - https://jenkins.impala.io/job/ubuntu-16.04-from-scratch/12743/testReport/junit/custom_cluster.test_runtime_profile/TestRuntimeProfile/test_runtime_profile_aggregated_protocol__beeswax___exec_optionbatch_size___0___num_nodes___0___disable_codegen_rows_threshold___0___disable_codegen___False___abort_on_error___1___exec_single_node_rows_threshold___0table_format__text_none_/ -- To view, visit http://gerrit.cloudera.org:8080/16057 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ic680cbfe94c939c2a8fad9d0943034ed058c6bca Gerrit-Change-Number: 16057 Gerrit-PatchSet: 17 Gerrit-Owner: Tim Armstrong Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Joe McDonnell Gerrit-Reviewer: Tim Armstrong Gerrit-Comment-Date: Wed, 25 Nov 2020 05:07:21 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-10216: make TestWriteErrorBlacklist deterministic
Impala Public Jenkins has submitted this change and it was merged. ( http://gerrit.cloudera.org:8080/16782 ) Change subject: IMPALA-10216: make TestWriteErrorBlacklist deterministic .. IMPALA-10216: make TestWriteErrorBlacklist deterministic There is a subtle bug in the test where it does a BufferPool::Pin() call followed immediately by a BufferPool::Unpin() call. This is meant to ensure that a new scratch range is allocated for the file, but does not guarantee that because the Pin() is asynchronous and there is a short-circuit case in buffer pool that cancels the Pin() if it the page is unpinned before the pin completes. We can force the pin to complete by requesting the actual buffer for the page (and verifying the data for good measure). Testing: I was never able to reproduce this race locally, but the fix is pretty safe and I looped the modified test for a while. Change-Id: I158dbc1ac60c8fefd53a138aa3e41cced9c4d674 Reviewed-on: http://gerrit.cloudera.org:8080/16782 Reviewed-by: Csaba Ringhofer Tested-by: Impala Public Jenkins --- M be/src/runtime/bufferpool/buffer-pool-test.cc 1 file changed, 2 insertions(+), 0 deletions(-) Approvals: Csaba Ringhofer: Looks good to me, approved Impala Public Jenkins: Verified -- To view, visit http://gerrit.cloudera.org:8080/16782 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: merged Gerrit-Change-Id: I158dbc1ac60c8fefd53a138aa3e41cced9c4d674 Gerrit-Change-Number: 16782 Gerrit-PatchSet: 2 Gerrit-Owner: Tim Armstrong Gerrit-Reviewer: Csaba Ringhofer Gerrit-Reviewer: Impala Public Jenkins
[Impala-ASF-CR] IMPALA-10216: make TestWriteErrorBlacklist deterministic
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/16782 ) Change subject: IMPALA-10216: make TestWriteErrorBlacklist deterministic .. Patch Set 1: Verified+1 -- To view, visit http://gerrit.cloudera.org:8080/16782 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I158dbc1ac60c8fefd53a138aa3e41cced9c4d674 Gerrit-Change-Number: 16782 Gerrit-PatchSet: 1 Gerrit-Owner: Tim Armstrong Gerrit-Reviewer: Csaba Ringhofer Gerrit-Reviewer: Impala Public Jenkins Gerrit-Comment-Date: Wed, 25 Nov 2020 03:20:02 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-10351,IMPALA-9812: enable mt dop for DML by default
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/16775 ) Change subject: IMPALA-10351,IMPALA-9812: enable mt_dop for DML by default .. Patch Set 4: Verified+1 -- To view, visit http://gerrit.cloudera.org:8080/16775 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I66331481260fe4b69d9e95b0200029b14d230ade Gerrit-Change-Number: 16775 Gerrit-PatchSet: 4 Gerrit-Owner: Tim Armstrong Gerrit-Reviewer: Bikramjeet Vig Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Joe McDonnell Gerrit-Comment-Date: Wed, 25 Nov 2020 03:14:36 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-10351,IMPALA-9812: enable mt dop for DML by default
Impala Public Jenkins has submitted this change and it was merged. ( http://gerrit.cloudera.org:8080/16775 ) Change subject: IMPALA-10351,IMPALA-9812: enable mt_dop for DML by default .. IMPALA-10351,IMPALA-9812: enable mt_dop for DML by default This allows setting mt_dop for any query with any configuration. Before this patch it was not supported for DML. --unlock_mt_dop and --mt_dop_auto_fallback are now ignored. Testing: * Updated tests to reflect new behaviour. * Removed irrelevant tests for fallback/validation. * Ran exhaustive tests. Change-Id: I66331481260fe4b69d9e95b0200029b14d230ade Reviewed-on: http://gerrit.cloudera.org:8080/16775 Reviewed-by: Impala Public Jenkins Tested-by: Impala Public Jenkins --- M be/src/common/global-flags.cc M be/src/util/backend-gflag-util.cc M bin/run-all-tests.sh M common/thrift/BackendGflags.thrift M fe/src/main/java/org/apache/impala/common/RuntimeEnv.java M fe/src/main/java/org/apache/impala/planner/Planner.java M fe/src/main/java/org/apache/impala/service/BackendConfig.java M fe/src/test/java/org/apache/impala/planner/PlannerTest.java M testdata/workloads/functional-planner/queries/PlannerTest/mt-dop-validation.test D testdata/workloads/functional-query/queries/QueryTest/mt-dop-auto-fallback.test M tests/custom_cluster/test_mt_dop.py 11 files changed, 69 insertions(+), 132 deletions(-) Approvals: Impala Public Jenkins: Looks good to me, approved; Verified -- To view, visit http://gerrit.cloudera.org:8080/16775 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: merged Gerrit-Change-Id: I66331481260fe4b69d9e95b0200029b14d230ade Gerrit-Change-Number: 16775 Gerrit-PatchSet: 5 Gerrit-Owner: Tim Armstrong Gerrit-Reviewer: Bikramjeet Vig Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Joe McDonnell
[Impala-ASF-CR] IMPALA-10330 Fix build error on aarch64 platform
zhaoren...@hotmail.com has abandoned this change. ( http://gerrit.cloudera.org:8080/16780 ) Change subject: IMPALA-10330 Fix build error on aarch64 platform .. Abandoned -- To view, visit http://gerrit.cloudera.org:8080/16780 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: abandon Gerrit-Change-Id: I11887169d717f8e1f2661a50d188154526898b93 Gerrit-Change-Number: 16780 Gerrit-PatchSet: 1 Gerrit-Owner: Anonymous Coward Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Tim Armstrong
[Impala-ASF-CR] IMPALA-9382: part 2/3: aggregate profiles sent to coordinator
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/16057 ) Change subject: IMPALA-9382: part 2/3: aggregate profiles sent to coordinator .. Patch Set 17: Verified-1 Build failed: https://jenkins.impala.io/job/gerrit-verify-dryrun/6708/ -- To view, visit http://gerrit.cloudera.org:8080/16057 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ic680cbfe94c939c2a8fad9d0943034ed058c6bca Gerrit-Change-Number: 16057 Gerrit-PatchSet: 17 Gerrit-Owner: Tim Armstrong Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Joe McDonnell Gerrit-Reviewer: Tim Armstrong Gerrit-Comment-Date: Wed, 25 Nov 2020 01:31:33 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-8990: Fix flakiness in test set request pool
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/16749 ) Change subject: IMPALA-8990: Fix flakiness in test_set_request_pool .. Patch Set 4: Verified-1 Build failed: https://jenkins.impala.io/job/gerrit-verify-dryrun/6709/ -- To view, visit http://gerrit.cloudera.org:8080/16749 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ife06509e936443579ca60780013ce01352c8932e Gerrit-Change-Number: 16749 Gerrit-PatchSet: 4 Gerrit-Owner: Bikramjeet Vig Gerrit-Reviewer: Bikramjeet Vig Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Tim Armstrong Gerrit-Comment-Date: Wed, 25 Nov 2020 01:28:53 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-10314: Optimize planning time for simple limits
Tim Armstrong has posted comments on this change. ( http://gerrit.cloudera.org:8080/16723 ) Change subject: IMPALA-10314: Optimize planning time for simple limits .. Patch Set 12: Code-Review+2 -- To view, visit http://gerrit.cloudera.org:8080/16723 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I9d6a79263bc092e0f3e9a1d72da5618f3cc35574 Gerrit-Change-Number: 16723 Gerrit-PatchSet: 12 Gerrit-Owner: Aman Sinha Gerrit-Reviewer: Aman Sinha Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Qifan Chen Gerrit-Reviewer: Shant Hovsepian Gerrit-Reviewer: Tim Armstrong Gerrit-Comment-Date: Wed, 25 Nov 2020 01:10:31 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-10314: Optimize planning time for simple limits
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/16723 ) Change subject: IMPALA-10314: Optimize planning time for simple limits .. Patch Set 12: Build Successful https://jenkins.impala.io/job/gerrit-code-review-checks/7732/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests. -- To view, visit http://gerrit.cloudera.org:8080/16723 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I9d6a79263bc092e0f3e9a1d72da5618f3cc35574 Gerrit-Change-Number: 16723 Gerrit-PatchSet: 12 Gerrit-Owner: Aman Sinha Gerrit-Reviewer: Aman Sinha Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Qifan Chen Gerrit-Reviewer: Shant Hovsepian Gerrit-Reviewer: Tim Armstrong Gerrit-Comment-Date: Wed, 25 Nov 2020 00:47:39 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-10314: Optimize planning time for simple limits
Aman Sinha has posted comments on this change. ( http://gerrit.cloudera.org:8080/16723 ) Change subject: IMPALA-10314: Optimize planning time for simple limits .. Patch Set 12: > Uploaded patch set 12. Additional testing found a bug that when the WHERE clause hint was within a view, it was not being recognized by the simple limit optimization. The reason was the SQL generation was not propagating this additional hint and HMS definition for the view did not have the hint. PS 12 fixes this. I have done manual testing with such views. However, I haven't been able to add a test with a hint in a view. I tried creating a view with hints in the functional_schema_template.sql and the view was created but doing a 'DESCRIBE EXTENDED' shows hint was missing. This needs further exploration on the test infrastructure. -- To view, visit http://gerrit.cloudera.org:8080/16723 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I9d6a79263bc092e0f3e9a1d72da5618f3cc35574 Gerrit-Change-Number: 16723 Gerrit-PatchSet: 12 Gerrit-Owner: Aman Sinha Gerrit-Reviewer: Aman Sinha Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Qifan Chen Gerrit-Reviewer: Shant Hovsepian Gerrit-Reviewer: Tim Armstrong Gerrit-Comment-Date: Wed, 25 Nov 2020 00:33:39 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-9856: Enable result spooling by default.
Bikramjeet Vig has posted comments on this change. ( http://gerrit.cloudera.org:8080/16755 ) Change subject: IMPALA-9856: Enable result spooling by default. .. Patch Set 1: (2 comments) Looks good. Like I pointed out in the 2 places below can you mention wherever else required as to why it needed to be disabled. I had to fish around for the specific reason or sometimes it was not apparent. It would be great to have that reason briefly mentioned in the comment itself. http://gerrit.cloudera.org:8080/#/c/16755/1/tests/custom_cluster/test_admission_controller.py File tests/custom_cluster/test_admission_controller.py: http://gerrit.cloudera.org:8080/#/c/16755/1/tests/custom_cluster/test_admission_controller.py@337 PS1, Line 337: assertion nit: mention briefly what we are trying to assert, that way we know from this comment why disabling it was necessary. Same below http://gerrit.cloudera.org:8080/#/c/16755/1/tests/custom_cluster/test_observability.py File tests/custom_cluster/test_observability.py: http://gerrit.cloudera.org:8080/#/c/16755/1/tests/custom_cluster/test_observability.py@37 PS1, Line 37: to avoid crash. nit: mention why will this cause a crash -- To view, visit http://gerrit.cloudera.org:8080/16755 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I9e360c1428676d8f3fab5d95efee18aca085eba4 Gerrit-Change-Number: 16755 Gerrit-PatchSet: 1 Gerrit-Owner: Riza Suminto Gerrit-Reviewer: Bikramjeet Vig Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Quanlong Huang Gerrit-Reviewer: Riza Suminto Gerrit-Comment-Date: Wed, 25 Nov 2020 00:27:48 + Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-10314: Optimize planning time for simple limits
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/16723 ) Change subject: IMPALA-10314: Optimize planning time for simple limits .. Patch Set 12: (1 comment) http://gerrit.cloudera.org:8080/#/c/16723/12/fe/src/main/java/org/apache/impala/analysis/SelectStmt.java File fe/src/main/java/org/apache/impala/analysis/SelectStmt.java: http://gerrit.cloudera.org:8080/#/c/16723/12/fe/src/main/java/org/apache/impala/analysis/SelectStmt.java@1211 PS12, Line 1211: strBuilder.append(ToSqlUtils.getPlanHintsSql(options, whereClause_.getPredicateHints())) line too long (96 > 90) -- To view, visit http://gerrit.cloudera.org:8080/16723 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I9d6a79263bc092e0f3e9a1d72da5618f3cc35574 Gerrit-Change-Number: 16723 Gerrit-PatchSet: 12 Gerrit-Owner: Aman Sinha Gerrit-Reviewer: Aman Sinha Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Qifan Chen Gerrit-Reviewer: Shant Hovsepian Gerrit-Reviewer: Tim Armstrong Gerrit-Comment-Date: Wed, 25 Nov 2020 00:27:45 + Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-10314: Optimize planning time for simple limits
Hello Qifan Chen, Shant Hovsepian, Tim Armstrong, Impala Public Jenkins, I'd like you to reexamine a change. Please visit http://gerrit.cloudera.org:8080/16723 to look at the new patch set (#12). Change subject: IMPALA-10314: Optimize planning time for simple limits .. IMPALA-10314: Optimize planning time for simple limits This patch optimizes the planning time for simple limit queries by only considering a minimal set of partitions whose file descriptors add up to N (the specified limit). Each file is conservatively estimated to contain 1 row. This reduces the number of partitions processed by HdfsScanNode.computeScanRangeLocations() which, according to query profiling, has been the main contributor to the planning time especially for large number of partitions. Further, within each partition, we only consider the number of non-empty files that brings the total to N. This is an opt-in optimization. A new planner option OPTIMIZE_SIMPLE_LIMIT enables this optimization. Further, if there's a WHERE clause, it must have an 'always_true' hint in order for the optimization to be considered. For example: set optimize_simple_limit = true; SELECT * FROM T WHERE /* +always_true */ LIMIT 10; If there are too many empty files in the partitions, it is possible that the query may produce fewer rows although those are still valid rows. Testing: - Added planner tests for the optimization - Ran query_test.py tests by enabling the optimize_simple_limit - Added an e2e test. Since result rows are non-deterministic, only simple count(*) query on top of subquery with limit was added. Change-Id: I9d6a79263bc092e0f3e9a1d72da5618f3cc35574 --- M be/src/service/query-options.cc M be/src/service/query-options.h M common/thrift/ImpalaInternalService.thrift M common/thrift/ImpalaService.thrift M fe/src/main/cup/sql-parser.cup M fe/src/main/java/org/apache/impala/analysis/Analyzer.java M fe/src/main/java/org/apache/impala/analysis/Expr.java M fe/src/main/java/org/apache/impala/analysis/PartitionSet.java M fe/src/main/java/org/apache/impala/analysis/Predicate.java M fe/src/main/java/org/apache/impala/analysis/SelectStmt.java M fe/src/main/java/org/apache/impala/planner/HdfsPartitionPruner.java M fe/src/main/java/org/apache/impala/planner/HdfsScanNode.java M fe/src/main/java/org/apache/impala/planner/SingleNodePlanner.java M fe/src/test/java/org/apache/impala/analysis/ParserTest.java M fe/src/test/java/org/apache/impala/planner/PlannerTest.java A testdata/workloads/functional-planner/queries/PlannerTest/optimize-simple-limit.test M testdata/workloads/functional-query/queries/QueryTest/range-constant-propagation.test 17 files changed, 514 insertions(+), 20 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/23/16723/12 -- To view, visit http://gerrit.cloudera.org:8080/16723 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: I9d6a79263bc092e0f3e9a1d72da5618f3cc35574 Gerrit-Change-Number: 16723 Gerrit-PatchSet: 12 Gerrit-Owner: Aman Sinha Gerrit-Reviewer: Aman Sinha Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Qifan Chen Gerrit-Reviewer: Shant Hovsepian Gerrit-Reviewer: Tim Armstrong
[Impala-ASF-CR] IMPALA-9930 (part 2): Introduce new admission control rpc service
Tim Armstrong has posted comments on this change. ( http://gerrit.cloudera.org:8080/16412 ) Change subject: IMPALA-9930 (part 2): Introduce new admission control rpc service .. Patch Set 9: Code-Review+2 (4 comments) Looks good to merge modulo a few minor things http://gerrit.cloudera.org:8080/#/c/16412/9/be/src/scheduling/admission-control-service.cc File be/src/scheduling/admission-control-service.cc: http://gerrit.cloudera.org:8080/#/c/16412/9/be/src/scheduling/admission-control-service.cc@159 PS9, Line 159: admission_state->admission_done = true; Can we file a JIRA to rework this so that this doesn't block the RPC thread (or if there's already one, link it here). At larger scale I think this could eat up all the RPC threads over time. http://gerrit.cloudera.org:8080/#/c/16412/9/be/src/scheduling/admission-control-service.cc@236 PS9, Line 236: LOG(ERROR) << s; When would this error happen? Is this the appropriate log level? http://gerrit.cloudera.org:8080/#/c/16412/9/be/src/scheduling/admission-controller.h File be/src/scheduling/admission-controller.h: http://gerrit.cloudera.org:8080/#/c/16412/9/be/src/scheduling/admission-controller.h@887 PS9, Line 887: QueueNode* queue_node, bool& coordinator_resource_limited); nit: should use pointer for output argument here and below. http://gerrit.cloudera.org:8080/#/c/16412/9/be/src/scheduling/remote-admission-control-client.cc File be/src/scheduling/remote-admission-control-client.cc: http://gerrit.cloudera.org:8080/#/c/16412/9/be/src/scheduling/remote-admission-control-client.cc@189 PS9, Line 189: const int num_retries = 3; Should we move these constants into a shared place? -- To view, visit http://gerrit.cloudera.org:8080/16412 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I594fc593a27b24b6952e381a9bc1a9a5c6b757ae Gerrit-Change-Number: 16412 Gerrit-PatchSet: 9 Gerrit-Owner: Thomas Tauber-Marshall Gerrit-Reviewer: Bikramjeet Vig Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Joe McDonnell Gerrit-Reviewer: Sahil Takiar Gerrit-Reviewer: Thomas Tauber-Marshall Gerrit-Reviewer: Tim Armstrong Gerrit-Comment-Date: Tue, 24 Nov 2020 22:13:02 + Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-10216: make TestWriteErrorBlacklist deterministic
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/16782 ) Change subject: IMPALA-10216: make TestWriteErrorBlacklist deterministic .. Patch Set 1: Build started: https://jenkins.impala.io/job/gerrit-verify-dryrun/6711/ DRY_RUN=false -- To view, visit http://gerrit.cloudera.org:8080/16782 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I158dbc1ac60c8fefd53a138aa3e41cced9c4d674 Gerrit-Change-Number: 16782 Gerrit-PatchSet: 1 Gerrit-Owner: Tim Armstrong Gerrit-Reviewer: Csaba Ringhofer Gerrit-Reviewer: Impala Public Jenkins Gerrit-Comment-Date: Tue, 24 Nov 2020 21:48:07 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-10351,IMPALA-9812: enable mt dop for DML by default
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/16775 ) Change subject: IMPALA-10351,IMPALA-9812: enable mt_dop for DML by default .. Patch Set 4: Build started: https://jenkins.impala.io/job/gerrit-verify-dryrun/6710/ DRY_RUN=false -- To view, visit http://gerrit.cloudera.org:8080/16775 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I66331481260fe4b69d9e95b0200029b14d230ade Gerrit-Change-Number: 16775 Gerrit-PatchSet: 4 Gerrit-Owner: Tim Armstrong Gerrit-Reviewer: Bikramjeet Vig Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Joe McDonnell Gerrit-Comment-Date: Tue, 24 Nov 2020 21:34:59 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-10351,IMPALA-9812: enable mt dop for DML by default
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/16775 ) Change subject: IMPALA-10351,IMPALA-9812: enable mt_dop for DML by default .. Patch Set 4: Code-Review+2 -- To view, visit http://gerrit.cloudera.org:8080/16775 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I66331481260fe4b69d9e95b0200029b14d230ade Gerrit-Change-Number: 16775 Gerrit-PatchSet: 4 Gerrit-Owner: Tim Armstrong Gerrit-Reviewer: Bikramjeet Vig Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Joe McDonnell Gerrit-Comment-Date: Tue, 24 Nov 2020 21:34:58 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-10216: make TestWriteErrorBlacklist deterministic
Csaba Ringhofer has posted comments on this change. ( http://gerrit.cloudera.org:8080/16782 ) Change subject: IMPALA-10216: make TestWriteErrorBlacklist deterministic .. Patch Set 1: Code-Review+2 -- To view, visit http://gerrit.cloudera.org:8080/16782 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I158dbc1ac60c8fefd53a138aa3e41cced9c4d674 Gerrit-Change-Number: 16782 Gerrit-PatchSet: 1 Gerrit-Owner: Tim Armstrong Gerrit-Reviewer: Csaba Ringhofer Gerrit-Reviewer: Impala Public Jenkins Gerrit-Comment-Date: Tue, 24 Nov 2020 21:34:36 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-10351,IMPALA-9812: enable mt dop for DML by default
Bikramjeet Vig has posted comments on this change. ( http://gerrit.cloudera.org:8080/16775 ) Change subject: IMPALA-10351,IMPALA-9812: enable mt_dop for DML by default .. Patch Set 3: Code-Review+2 -- To view, visit http://gerrit.cloudera.org:8080/16775 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I66331481260fe4b69d9e95b0200029b14d230ade Gerrit-Change-Number: 16775 Gerrit-PatchSet: 3 Gerrit-Owner: Tim Armstrong Gerrit-Reviewer: Bikramjeet Vig Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Joe McDonnell Gerrit-Comment-Date: Tue, 24 Nov 2020 21:34:12 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-10337: Consider MAX ROW SIZE when computing max reservation
Bikramjeet Vig has posted comments on this change. ( http://gerrit.cloudera.org:8080/16765 ) Change subject: IMPALA-10337: Consider MAX_ROW_SIZE when computing max reservation .. Patch Set 1: (2 comments) http://gerrit.cloudera.org:8080/#/c/16765/1/fe/src/main/java/org/apache/impala/planner/PlanRootSink.java File fe/src/main/java/org/apache/impala/planner/PlanRootSink.java: http://gerrit.cloudera.org:8080/#/c/16765/1/fe/src/main/java/org/apache/impala/planner/PlanRootSink.java@87 PS1, Line 87: getDefault_spillable_buffer_size > My understanding is that DEFAULT_SPILLABLE_BUFFER_SIZE control the minimal got it, thanks for the explanation http://gerrit.cloudera.org:8080/#/c/16765/1/tests/custom_cluster/test_query_retries.py File tests/custom_cluster/test_query_retries.py: http://gerrit.cloudera.org:8080/#/c/16765/1/tests/custom_cluster/test_query_retries.py@589 PS1, Line 589: 'max_row_size': 4 * 1024, > It intended to keep the test the same, keeping the maxMemReservationBytes t can you add a test that confirms that changing this will have an effect. Something like have a test that would fail without using this and then setting this would let it run. Or maybe something that you can confirm, a metric perhaps that changes when you set this -- To view, visit http://gerrit.cloudera.org:8080/16765 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Id7138e1e034ea5d1cd15cf8de399690e52a9d726 Gerrit-Change-Number: 16765 Gerrit-PatchSet: 1 Gerrit-Owner: Riza Suminto Gerrit-Reviewer: Bikramjeet Vig Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Quanlong Huang Gerrit-Reviewer: Riza Suminto Gerrit-Comment-Date: Tue, 24 Nov 2020 21:11:07 + Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-10216: make TestWriteErrorBlacklist deterministic
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/16782 ) Change subject: IMPALA-10216: make TestWriteErrorBlacklist deterministic .. Patch Set 1: Build Successful https://jenkins.impala.io/job/gerrit-code-review-checks/7731/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests. -- To view, visit http://gerrit.cloudera.org:8080/16782 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I158dbc1ac60c8fefd53a138aa3e41cced9c4d674 Gerrit-Change-Number: 16782 Gerrit-PatchSet: 1 Gerrit-Owner: Tim Armstrong Gerrit-Reviewer: Csaba Ringhofer Gerrit-Reviewer: Impala Public Jenkins Gerrit-Comment-Date: Tue, 24 Nov 2020 20:44:02 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-10258, IMPALA-10109: Fixed flaky test in test query retries.py
Thomas Tauber-Marshall has posted comments on this change. ( http://gerrit.cloudera.org:8080/16763 ) Change subject: IMPALA-10258, IMPALA-10109: Fixed flaky test in test_query_retries.py .. Patch Set 2: (3 comments) http://gerrit.cloudera.org:8080/#/c/16763/2//COMMIT_MSG Commit Message: http://gerrit.cloudera.org:8080/#/c/16763/2//COMMIT_MSG@7 PS2, Line 7: IMPALA-10258, IMPALA-10109 since these issues are basically unrelated, could you separate them out into two reviews? http://gerrit.cloudera.org:8080/#/c/16763/2//COMMIT_MSG@9 PS2, Line 9: When TestQueryRetries.test_original_query_cancel was ran on s3 I'm not sure I understand what you're saying the issue is: According to the JIRA, the test was waiting for the query to reach state "RUNNING", but it was already at state "EXCEPTION" (QueryState = 5, see beeswax.thrift). At that point in the test, the query shouldn't have failed, since the impalad hasn't been killed yet, so really not sure what could have happened, and unfortunately it doesn't look like we have the logs for it. http://gerrit.cloudera.org:8080/#/c/16763/2//COMMIT_MSG@16 PS2, Line 16: For IMPALA-10109, test_retries_from_cancellation_pool did not I'm not sure I understand what you're saying the issue is: According to the JIRA, the query timed out after ~784s, which is a lot longer than the default statestore time-to-detect-failure of heartbeat_frequency x max_missed = 1000ms x 10 = 10s. So it seems like the coordinator should have had plenty of time to get the statestore message, even under the old values. Looking through the logs, I'm a little confused by what I see - the coordinator says the query was only scheduled on 2 backends, but I think the test assumes that it gets scheduled on all 3 backends in the minicluster (see __kill_random_impalad()). I also see a reference to CancelFromThreadPool in QueryExecMgr on impalad_node1, but that should be hit unless the coordinator is killed, which it shouldn't have been. -- To view, visit http://gerrit.cloudera.org:8080/16763 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ib89f7b01a0f2a66a97f312e779a4ab04f4f347f3 Gerrit-Change-Number: 16763 Gerrit-PatchSet: 2 Gerrit-Owner: Wenzhe Zhou Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Thomas Tauber-Marshall Gerrit-Comment-Date: Tue, 24 Nov 2020 20:36:46 + Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-10216: make TestWriteErrorBlacklist deterministic
Tim Armstrong has uploaded this change for review. ( http://gerrit.cloudera.org:8080/16782 Change subject: IMPALA-10216: make TestWriteErrorBlacklist deterministic .. IMPALA-10216: make TestWriteErrorBlacklist deterministic There is a subtle bug in the test where it does a BufferPool::Pin() call followed immediately by a BufferPool::Unpin() call. This is meant to ensure that a new scratch range is allocated for the file, but does not guarantee that because the Pin() is asynchronous and there is a short-circuit case in buffer pool that cancels the Pin() if it the page is unpinned before the pin completes. We can force the pin to complete by requesting the actual buffer for the page (and verifying the data for good measure). Testing: I was never able to reproduce this race locally, but the fix is pretty safe and I looped the modified test for a while. Change-Id: I158dbc1ac60c8fefd53a138aa3e41cced9c4d674 --- M be/src/runtime/bufferpool/buffer-pool-test.cc 1 file changed, 2 insertions(+), 0 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/82/16782/1 -- To view, visit http://gerrit.cloudera.org:8080/16782 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newchange Gerrit-Change-Id: I158dbc1ac60c8fefd53a138aa3e41cced9c4d674 Gerrit-Change-Number: 16782 Gerrit-PatchSet: 1 Gerrit-Owner: Tim Armstrong
[Impala-ASF-CR] IMPALA-9382: part 2/3: aggregate profiles sent to coordinator
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/16057 ) Change subject: IMPALA-9382: part 2/3: aggregate profiles sent to coordinator .. Patch Set 17: Build Successful https://jenkins.impala.io/job/gerrit-code-review-checks/7730/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests. -- To view, visit http://gerrit.cloudera.org:8080/16057 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ic680cbfe94c939c2a8fad9d0943034ed058c6bca Gerrit-Change-Number: 16057 Gerrit-PatchSet: 17 Gerrit-Owner: Tim Armstrong Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Joe McDonnell Gerrit-Reviewer: Tim Armstrong Gerrit-Comment-Date: Tue, 24 Nov 2020 20:12:41 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-10325: Parquet scan should use min/max statistics to skip pages based on equi-join predicate
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/16720 ) Change subject: IMPALA-10325: Parquet scan should use min/max statistics to skip pages based on equi-join predicate .. Patch Set 16: Build Failed https://jenkins.impala.io/job/gerrit-code-review-checks/7729/ : Initial code review checks failed. See linked job for details on the failure. -- To view, visit http://gerrit.cloudera.org:8080/16720 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I379405ee75b14929df7d6b5d20dabc6f51375691 Gerrit-Change-Number: 16720 Gerrit-PatchSet: 16 Gerrit-Owner: Qifan Chen Gerrit-Reviewer: Csaba Ringhofer Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Qifan Chen Gerrit-Reviewer: Tim Armstrong Gerrit-Reviewer: Zoltan Borok-Nagy Gerrit-Comment-Date: Tue, 24 Nov 2020 19:57:03 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-8990: Fix flakiness in test set request pool
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/16749 ) Change subject: IMPALA-8990: Fix flakiness in test_set_request_pool .. Patch Set 4: Build started: https://jenkins.impala.io/job/gerrit-verify-dryrun/6709/ DRY_RUN=false -- To view, visit http://gerrit.cloudera.org:8080/16749 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ife06509e936443579ca60780013ce01352c8932e Gerrit-Change-Number: 16749 Gerrit-PatchSet: 4 Gerrit-Owner: Bikramjeet Vig Gerrit-Reviewer: Bikramjeet Vig Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Tim Armstrong Gerrit-Comment-Date: Tue, 24 Nov 2020 19:58:19 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-8990: Fix flakiness in test set request pool
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/16749 ) Change subject: IMPALA-8990: Fix flakiness in test_set_request_pool .. Patch Set 4: Code-Review+2 -- To view, visit http://gerrit.cloudera.org:8080/16749 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ife06509e936443579ca60780013ce01352c8932e Gerrit-Change-Number: 16749 Gerrit-PatchSet: 4 Gerrit-Owner: Bikramjeet Vig Gerrit-Reviewer: Bikramjeet Vig Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Tim Armstrong Gerrit-Comment-Date: Tue, 24 Nov 2020 19:58:19 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-8990: Fix flakiness in test set request pool
Bikramjeet Vig has posted comments on this change. ( http://gerrit.cloudera.org:8080/16749 ) Change subject: IMPALA-8990: Fix flakiness in test_set_request_pool .. Patch Set 3: flaky data loading issue hit -- To view, visit http://gerrit.cloudera.org:8080/16749 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ife06509e936443579ca60780013ce01352c8932e Gerrit-Change-Number: 16749 Gerrit-PatchSet: 3 Gerrit-Owner: Bikramjeet Vig Gerrit-Reviewer: Bikramjeet Vig Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Tim Armstrong Gerrit-Comment-Date: Tue, 24 Nov 2020 19:57:27 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-9382: part 2/3: aggregate profiles sent to coordinator
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/16057 ) Change subject: IMPALA-9382: part 2/3: aggregate profiles sent to coordinator .. Patch Set 16: Build Successful https://jenkins.impala.io/job/gerrit-code-review-checks/7728/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests. -- To view, visit http://gerrit.cloudera.org:8080/16057 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ic680cbfe94c939c2a8fad9d0943034ed058c6bca Gerrit-Change-Number: 16057 Gerrit-PatchSet: 16 Gerrit-Owner: Tim Armstrong Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Joe McDonnell Gerrit-Reviewer: Tim Armstrong Gerrit-Comment-Date: Tue, 24 Nov 2020 19:53:17 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-9382: part 2/3: aggregate profiles sent to coordinator
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/16057 ) Change subject: IMPALA-9382: part 2/3: aggregate profiles sent to coordinator .. Patch Set 17: Build started: https://jenkins.impala.io/job/gerrit-verify-dryrun/6708/ DRY_RUN=false -- To view, visit http://gerrit.cloudera.org:8080/16057 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ic680cbfe94c939c2a8fad9d0943034ed058c6bca Gerrit-Change-Number: 16057 Gerrit-PatchSet: 17 Gerrit-Owner: Tim Armstrong Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Joe McDonnell Gerrit-Reviewer: Tim Armstrong Gerrit-Comment-Date: Tue, 24 Nov 2020 19:52:42 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-9382: part 2/3: aggregate profiles sent to coordinator
Tim Armstrong has posted comments on this change. ( http://gerrit.cloudera.org:8080/16057 ) Change subject: IMPALA-9382: part 2/3: aggregate profiles sent to coordinator .. Patch Set 17: Code-Review+2 -- To view, visit http://gerrit.cloudera.org:8080/16057 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ic680cbfe94c939c2a8fad9d0943034ed058c6bca Gerrit-Change-Number: 16057 Gerrit-PatchSet: 17 Gerrit-Owner: Tim Armstrong Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Joe McDonnell Gerrit-Reviewer: Tim Armstrong Gerrit-Comment-Date: Tue, 24 Nov 2020 19:52:29 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-9382: part 2/3: aggregate profiles sent to coordinator
Hello Joe McDonnell, Impala Public Jenkins, I'd like you to reexamine a change. Please visit http://gerrit.cloudera.org:8080/16057 to look at the new patch set (#17). Change subject: IMPALA-9382: part 2/3: aggregate profiles sent to coordinator .. IMPALA-9382: part 2/3: aggregate profiles sent to coordinator This reworks the status reporting so that serialized AggregatedRuntimeProfile objects are sent from executors to coordinators. These profiles are substantially denser and faster to process for higher mt_dop values. The aggregation is also done in a single step, merging the aggregated thrift profile from the executor directly into the final aggregated profile, instead of converting it to an unaggregated profile first. The changes required were: * A new Update() method for AggregatedRuntimeProfile that updates the profile from a serialised AggregateRuntimeProfile for a subset of the instances. The code is generalized from the existing InitFromThrift() code path. * Per-fragment reports included in the status report protobuf when --gen_experimental_profile=true. * Logic on the coordinator that either consumes serialized AggregatedRuntimeProfile per fragment, when --gen_experimental_profile=true, or consumes a serialized RuntimeProfile per finstance otherwise. This also adds support for event sequences and time series in the aggregated profile, so the amount of information in the aggregated profile is now on par with the basic profile. We also finish off support for JSON profile. The JSON profile is more stripped down because we do not need to round-trip profiles via JSON and it is a much less dense profile representation. Part 3 will clean up and improve the display of the profile. Testing: * Add sanity tests for aggregated runtime profile. * Add unit tests to exercise aggregation of the various counter types * Ran core tests. Change-Id: Ic680cbfe94c939c2a8fad9d0943034ed058c6bca --- M be/src/runtime/coordinator-backend-state.cc M be/src/runtime/coordinator-backend-state.h M be/src/runtime/fragment-instance-state.cc M be/src/runtime/fragment-instance-state.h M be/src/runtime/fragment-state.cc M be/src/runtime/fragment-state.h M be/src/runtime/query-state.cc M be/src/runtime/query-state.h M be/src/service/impala-server.cc M be/src/util/runtime-profile-counters.h M be/src/util/runtime-profile-test.cc M be/src/util/runtime-profile.cc M be/src/util/runtime-profile.h M common/protobuf/control_service.proto M common/thrift/ImpalaInternalService.thrift M common/thrift/RuntimeProfile.thrift A testdata/workloads/functional-query/queries/QueryTest/runtime-profile-aggregated.test A tests/custom_cluster/test_runtime_profile.py 18 files changed, 1,344 insertions(+), 264 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/57/16057/17 -- To view, visit http://gerrit.cloudera.org:8080/16057 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: Ic680cbfe94c939c2a8fad9d0943034ed058c6bca Gerrit-Change-Number: 16057 Gerrit-PatchSet: 17 Gerrit-Owner: Tim Armstrong Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Joe McDonnell Gerrit-Reviewer: Tim Armstrong
[Impala-ASF-CR] IMPALA-10325: Parquet scan should use min/max statistics to skip pages based on equi-join predicate
Qifan Chen has uploaded a new patch set (#16). ( http://gerrit.cloudera.org:8080/16720 ) Change subject: IMPALA-10325: Parquet scan should use min/max statistics to skip pages based on equi-join predicate .. IMPALA-10325: Parquet scan should use min/max statistics to skip pages based on equi-join predicate This patch adds the logic to utilize min/max stats for Parquet row groups or pages to skip these entities when they don't qualify an equi-join predicate. A new class of predicates called overlap predicates is introduced to aid in the determination of whether a Parquet row group or a page overlap with a range computed from the hash join. If not, then the entire Parquet row group or the page are skipped. The new class of predicates co-exist with the existing min/max conjuncts that are introduced based on the local scan predicates. Both classes of predicates can work individually or together with each other. The overlap predicates are evaluted after the existing min/max conjuncts. To be done: 1. Handle STRING, DATE, TIME and DECIMAL data tyes; 2. Unit/performance testing; 3. Core testing. Change-Id: I379405ee75b14929df7d6b5d20dabc6f51375691 --- M be/src/exec/exec-node.h M be/src/exec/hdfs-scan-node-base.cc M be/src/exec/hdfs-scan-node-base.h M be/src/exec/parquet/hdfs-parquet-scanner.cc M be/src/exec/parquet/hdfs-parquet-scanner.h M be/src/exec/parquet/parquet-column-stats.cc M be/src/exec/parquet/parquet-column-stats.h M be/src/exec/partitioned-hash-join-builder.cc M be/src/exec/scan-node.cc M be/src/runtime/coordinator.cc M be/src/util/min-max-filter.cc M be/src/util/min-max-filter.h M common/thrift/PlanNodes.thrift M fe/src/main/java/org/apache/impala/analysis/TupleDescriptor.java M fe/src/main/java/org/apache/impala/planner/HdfsScanNode.java M fe/src/main/java/org/apache/impala/planner/RuntimeFilterGenerator.java 16 files changed, 525 insertions(+), 69 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/20/16720/16 -- To view, visit http://gerrit.cloudera.org:8080/16720 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: I379405ee75b14929df7d6b5d20dabc6f51375691 Gerrit-Change-Number: 16720 Gerrit-PatchSet: 16 Gerrit-Owner: Qifan Chen Gerrit-Reviewer: Csaba Ringhofer Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Qifan Chen Gerrit-Reviewer: Tim Armstrong Gerrit-Reviewer: Zoltan Borok-Nagy
[Impala-ASF-CR] IMPALA-9382: part 2/3: aggregate profiles sent to coordinator
Joe McDonnell has posted comments on this change. ( http://gerrit.cloudera.org:8080/16057 ) Change subject: IMPALA-9382: part 2/3: aggregate profiles sent to coordinator .. Patch Set 15: (1 comment) http://gerrit.cloudera.org:8080/#/c/16057/15/be/src/util/runtime-profile-test.cc File be/src/util/runtime-profile-test.cc: http://gerrit.cloudera.org:8080/#/c/16057/15/be/src/util/runtime-profile-test.cc@771 PS15, Line 771: static void VerifyThriftInfoStrings( > For info strings I think we're validating this already, since we're compari Ah, right, good point. -- To view, visit http://gerrit.cloudera.org:8080/16057 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ic680cbfe94c939c2a8fad9d0943034ed058c6bca Gerrit-Change-Number: 16057 Gerrit-PatchSet: 15 Gerrit-Owner: Tim Armstrong Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Joe McDonnell Gerrit-Reviewer: Tim Armstrong Gerrit-Comment-Date: Tue, 24 Nov 2020 19:35:05 + Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-9382: part 2/3: aggregate profiles sent to coordinator
Hello Joe McDonnell, Impala Public Jenkins, I'd like you to reexamine a change. Please visit http://gerrit.cloudera.org:8080/16057 to look at the new patch set (#16). Change subject: IMPALA-9382: part 2/3: aggregate profiles sent to coordinator .. IMPALA-9382: part 2/3: aggregate profiles sent to coordinator This reworks the status reporting so that serialized AggregatedRuntimeProfile objects are sent from executors to coordinators. These profiles are substantially denser and faster to process for higher mt_dop values. The aggregation is also done in a single step, merging the aggregated thrift profile from the executor directly into the final aggregated profile, instead of converting it to an unaggregated profile first. The changes required were: * A new Update() method for AggregatedRuntimeProfile that updates the profile from a serialised AggregateRuntimeProfile for a subset of the instances. The code is generalized from the existing InitFromThrift() code path. * Per-fragment reports included in the status report protobuf when --gen_experimental_profile=true. * Logic on the coordinator that either consumes serialized AggregatedRuntimeProfile per fragment, when --gen_experimental_profile=true, or consumes a serialized RuntimeProfile per finstance otherwise. This also adds support for event sequences and time series in the aggregated profile, so the amount of information in the aggregated profile is now on par with the basic profile. We also finish off support for JSON profile. The JSON profile is more stripped down because we do not need to round-trip profiles via JSON and it is a much less dense profile representation. Part 3 will clean up and improve the display of the profile. Testing: * Add sanity tests for aggregated runtime profile. * Add unit tests to exercise aggregation of the various counter types * Ran core tests. Change-Id: Ic680cbfe94c939c2a8fad9d0943034ed058c6bca --- M be/src/runtime/coordinator-backend-state.cc M be/src/runtime/coordinator-backend-state.h M be/src/runtime/fragment-instance-state.cc M be/src/runtime/fragment-instance-state.h M be/src/runtime/fragment-state.cc M be/src/runtime/fragment-state.h M be/src/runtime/query-state.cc M be/src/runtime/query-state.h M be/src/service/impala-server.cc M be/src/util/runtime-profile-counters.h M be/src/util/runtime-profile-test.cc M be/src/util/runtime-profile.cc M be/src/util/runtime-profile.h M common/protobuf/control_service.proto M common/thrift/ImpalaInternalService.thrift M common/thrift/RuntimeProfile.thrift A testdata/workloads/functional-query/queries/QueryTest/runtime-profile-aggregated.test A tests/custom_cluster/test_runtime_profile.py 18 files changed, 1,344 insertions(+), 264 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/57/16057/16 -- To view, visit http://gerrit.cloudera.org:8080/16057 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: Ic680cbfe94c939c2a8fad9d0943034ed058c6bca Gerrit-Change-Number: 16057 Gerrit-PatchSet: 16 Gerrit-Owner: Tim Armstrong Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Joe McDonnell Gerrit-Reviewer: Tim Armstrong
[Impala-ASF-CR] IMPALA-9382: part 2/3: aggregate profiles sent to coordinator
Tim Armstrong has posted comments on this change. ( http://gerrit.cloudera.org:8080/16057 ) Change subject: IMPALA-9382: part 2/3: aggregate profiles sent to coordinator .. Patch Set 16: Code-Review+2 Carry +2 -- To view, visit http://gerrit.cloudera.org:8080/16057 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ic680cbfe94c939c2a8fad9d0943034ed058c6bca Gerrit-Change-Number: 16057 Gerrit-PatchSet: 16 Gerrit-Owner: Tim Armstrong Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Joe McDonnell Gerrit-Reviewer: Tim Armstrong Gerrit-Comment-Date: Tue, 24 Nov 2020 19:30:57 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-9382: part 2/3: aggregate profiles sent to coordinator
Tim Armstrong has posted comments on this change. (
http://gerrit.cloudera.org:8080/16057 )
Change subject: IMPALA-9382: part 2/3: aggregate profiles sent to coordinator
..
Patch Set 15:
(4 comments)
http://gerrit.cloudera.org:8080/#/c/16057/15/be/src/util/runtime-profile-test.cc
File be/src/util/runtime-profile-test.cc:
http://gerrit.cloudera.org:8080/#/c/16057/15/be/src/util/runtime-profile-test.cc@562
PS15, Line 562: for (int i = 0; i < offset + NUM_PROFILES; ++i) {
: if (i < offset || i >= offset + NUM_PROFILES) {
: EXPECT_FALSE(tcounter.has_value[i]);
: continue;
: }
> Due to the loop condition, i < offset + NUM_PROFILES, so it won't test the
Done
http://gerrit.cloudera.org:8080/#/c/16057/15/be/src/util/runtime-profile-test.cc@771
PS15, Line 771: static void VerifyThriftInfoStrings(
> Nit: It would be nice to verify that the entries outside of offset, offset
For info strings I think we're validating this already, since we're comparing
the full vectors of indices.
http://gerrit.cloudera.org:8080/#/c/16057/15/be/src/util/runtime-profile-test.cc@962
PS15, Line 962: static void VerifyThriftEventSequences(
> Nit: It would be nice to verify that the entries outside of offset, offset
Done
http://gerrit.cloudera.org:8080/#/c/16057/15/be/src/util/runtime-profile-test.cc@1374
PS15, Line 1374: static void VerifyThriftTimeSeries(
> Nit: It would be nice to verify that the entries outside of offset, offset
Done
--
To view, visit http://gerrit.cloudera.org:8080/16057
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Ic680cbfe94c939c2a8fad9d0943034ed058c6bca
Gerrit-Change-Number: 16057
Gerrit-PatchSet: 15
Gerrit-Owner: Tim Armstrong
Gerrit-Reviewer: Impala Public Jenkins
Gerrit-Reviewer: Joe McDonnell
Gerrit-Reviewer: Tim Armstrong
Gerrit-Comment-Date: Tue, 24 Nov 2020 19:30:49 +
Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-10258, IMPALA-10109: Fixed flaky test in test query retries.py
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/16763 ) Change subject: IMPALA-10258, IMPALA-10109: Fixed flaky test in test_query_retries.py .. Patch Set 2: Build Successful https://jenkins.impala.io/job/gerrit-code-review-checks/7727/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests. -- To view, visit http://gerrit.cloudera.org:8080/16763 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ib89f7b01a0f2a66a97f312e779a4ab04f4f347f3 Gerrit-Change-Number: 16763 Gerrit-PatchSet: 2 Gerrit-Owner: Wenzhe Zhou Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Thomas Tauber-Marshall Gerrit-Comment-Date: Tue, 24 Nov 2020 18:46:28 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-10345: Impala hits DCHECK in parquet-column-stats.inline.h
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/16771 ) Change subject: IMPALA-10345: Impala hits DCHECK in parquet-column-stats.inline.h .. Patch Set 5: Verified+1 -- To view, visit http://gerrit.cloudera.org:8080/16771 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I4289bd743e951cc4c607d5a5ea75d27825a1c12b Gerrit-Change-Number: 16771 Gerrit-PatchSet: 5 Gerrit-Owner: Zoltan Borok-Nagy Gerrit-Reviewer: Csaba Ringhofer Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Zoltan Borok-Nagy Gerrit-Comment-Date: Tue, 24 Nov 2020 18:35:07 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-10345: Impala hits DCHECK in parquet-column-stats.inline.h
Impala Public Jenkins has submitted this change and it was merged. ( http://gerrit.cloudera.org:8080/16771 ) Change subject: IMPALA-10345: Impala hits DCHECK in parquet-column-stats.inline.h .. IMPALA-10345: Impala hits DCHECK in parquet-column-stats.inline.h During Parquet file writing, a DCHECK checks if row group stats have copied the min/max string values into their internal buffers. This check is at the finalization of each page. The copying of the string values happened at the end of each row batch. Thus, if a row batch spans over multiple pages then the min/max string values don't get copied by the end of the page. Since the memory is attached to the row batch this isn't really an error. As a workaround this commit also copies the min/max string values at the end of the page if they haven't been copied yet. Testing * Added e2e test Change-Id: I4289bd743e951cc4c607d5a5ea75d27825a1c12b Reviewed-on: http://gerrit.cloudera.org:8080/16771 Reviewed-by: Impala Public Jenkins Tested-by: Impala Public Jenkins --- M be/src/exec/parquet/hdfs-parquet-table-writer.cc M testdata/workloads/functional-query/queries/QueryTest/parquet-page-index.test M tests/query_test/test_parquet_stats.py 3 files changed, 23 insertions(+), 1 deletion(-) Approvals: Impala Public Jenkins: Looks good to me, approved; Verified -- To view, visit http://gerrit.cloudera.org:8080/16771 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: merged Gerrit-Change-Id: I4289bd743e951cc4c607d5a5ea75d27825a1c12b Gerrit-Change-Number: 16771 Gerrit-PatchSet: 6 Gerrit-Owner: Zoltan Borok-Nagy Gerrit-Reviewer: Csaba Ringhofer Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Zoltan Borok-Nagy
[Impala-ASF-CR] IMPALA-10258, IMPALA-10109: Fixed flaky test in test query retries.py
Wenzhe Zhou has uploaded a new patch set (#2). ( http://gerrit.cloudera.org:8080/16763 ) Change subject: IMPALA-10258, IMPALA-10109: Fixed flaky test in test_query_retries.py .. IMPALA-10258, IMPALA-10109: Fixed flaky test in test_query_retries.py When TestQueryRetries.test_original_query_cancel was ran on s3 with query option spool_query_results enabled, the query was timeout before reaching the expected state. This patch double the timeout for the query when the test is running on S3 and double the timeout for query to reaching "FINISHED" state. For IMPALA-10109, test_retries_from_cancellation_pool did not trigger query-retry when one of impalad was killed. It seems that membership updating message was not received and processed by coordinator before reaching terminated state, hence the query-retry was not triggered. This patch reduce the heartbeat_frequency and max_missed_heartbeats so that statestore will take much less time to update membership when one impalad was killed so that coordinator could start query-retry. Testing: - Ran the two tests in a loop for more than 3 hours. The test failures did not happen. Change-Id: Ib89f7b01a0f2a66a97f312e779a4ab04f4f347f3 --- M tests/custom_cluster/test_query_retries.py 1 file changed, 12 insertions(+), 2 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/63/16763/2 -- To view, visit http://gerrit.cloudera.org:8080/16763 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: Ib89f7b01a0f2a66a97f312e779a4ab04f4f347f3 Gerrit-Change-Number: 16763 Gerrit-PatchSet: 2 Gerrit-Owner: Wenzhe Zhou Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Thomas Tauber-Marshall
[Impala-ASF-CR] IMPALA-9382: part 2/3: aggregate profiles sent to coordinator
Joe McDonnell has posted comments on this change. (
http://gerrit.cloudera.org:8080/16057 )
Change subject: IMPALA-9382: part 2/3: aggregate profiles sent to coordinator
..
Patch Set 15: Code-Review+2
(4 comments)
Thanks for the patience, this looks good to me. Good to see the new tests and
cleanup.
Couple minor points, but I'm going to go ahead and +2.
http://gerrit.cloudera.org:8080/#/c/16057/15/be/src/util/runtime-profile-test.cc
File be/src/util/runtime-profile-test.cc:
http://gerrit.cloudera.org:8080/#/c/16057/15/be/src/util/runtime-profile-test.cc@562
PS15, Line 562: for (int i = 0; i < offset + NUM_PROFILES; ++i) {
: if (i < offset || i >= offset + NUM_PROFILES) {
: EXPECT_FALSE(tcounter.has_value[i]);
: continue;
: }
Due to the loop condition, i < offset + NUM_PROFILES, so it won't test the i >=
offset + NUM_PROFILES part. For the case of the AggregatedRuntimeProfile with 5
entries, it will test the first four. We probably need to detect the total size
to be able to iterate over the whole thing.
I like that this is testing the entries that are not supposed to be set.
http://gerrit.cloudera.org:8080/#/c/16057/15/be/src/util/runtime-profile-test.cc@771
PS15, Line 771: static void VerifyThriftInfoStrings(
Nit: It would be nice to verify that the entries outside of offset, offset +
NUM_PROFILES are not touched like we do in VerifyThriftSummaryStats().
http://gerrit.cloudera.org:8080/#/c/16057/15/be/src/util/runtime-profile-test.cc@962
PS15, Line 962: static void VerifyThriftEventSequences(
Nit: It would be nice to verify that the entries outside of offset, offset +
NUM_PROFILES are not touched like we do in VerifyThriftSummaryStats().
http://gerrit.cloudera.org:8080/#/c/16057/15/be/src/util/runtime-profile-test.cc@1374
PS15, Line 1374: static void VerifyThriftTimeSeries(
Nit: It would be nice to verify that the entries outside of offset, offset +
NUM_PROFILES are not touched like we do in VerifyThriftSummaryStats().
--
To view, visit http://gerrit.cloudera.org:8080/16057
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Ic680cbfe94c939c2a8fad9d0943034ed058c6bca
Gerrit-Change-Number: 16057
Gerrit-PatchSet: 15
Gerrit-Owner: Tim Armstrong
Gerrit-Reviewer: Impala Public Jenkins
Gerrit-Reviewer: Joe McDonnell
Gerrit-Reviewer: Tim Armstrong
Gerrit-Comment-Date: Tue, 24 Nov 2020 18:00:23 +
Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-10351,IMPALA-9812: enable mt dop for DML by default
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/16775 ) Change subject: IMPALA-10351,IMPALA-9812: enable mt_dop for DML by default .. Patch Set 3: Build Successful https://jenkins.impala.io/job/gerrit-code-review-checks/7724/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests. -- To view, visit http://gerrit.cloudera.org:8080/16775 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I66331481260fe4b69d9e95b0200029b14d230ade Gerrit-Change-Number: 16775 Gerrit-PatchSet: 3 Gerrit-Owner: Tim Armstrong Gerrit-Reviewer: Bikramjeet Vig Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Joe McDonnell Gerrit-Comment-Date: Tue, 24 Nov 2020 17:48:12 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-10351,IMPALA-9812: enable mt dop for DML by default
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/16775 ) Change subject: IMPALA-10351,IMPALA-9812: enable mt_dop for DML by default .. Patch Set 1: Build Successful https://jenkins.impala.io/job/gerrit-code-review-checks/7726/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests. -- To view, visit http://gerrit.cloudera.org:8080/16775 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I66331481260fe4b69d9e95b0200029b14d230ade Gerrit-Change-Number: 16775 Gerrit-PatchSet: 1 Gerrit-Owner: Tim Armstrong Gerrit-Reviewer: Bikramjeet Vig Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Joe McDonnell Gerrit-Comment-Date: Tue, 24 Nov 2020 17:48:40 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-10351,IMPALA-9812: enable mt dop for DML by default
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/16775 ) Change subject: IMPALA-10351,IMPALA-9812: enable mt_dop for DML by default .. Patch Set 2: Build Successful https://jenkins.impala.io/job/gerrit-code-review-checks/7725/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests. -- To view, visit http://gerrit.cloudera.org:8080/16775 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I66331481260fe4b69d9e95b0200029b14d230ade Gerrit-Change-Number: 16775 Gerrit-PatchSet: 2 Gerrit-Owner: Tim Armstrong Gerrit-Reviewer: Bikramjeet Vig Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Joe McDonnell Gerrit-Comment-Date: Tue, 24 Nov 2020 17:48:27 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-10351,IMPALA-9812: enable mt dop for DML by default
Tim Armstrong has uploaded a new patch set (#1). ( http://gerrit.cloudera.org:8080/16775 ) Change subject: IMPALA-10351,IMPALA-9812: enable mt_dop for DML by default .. IMPALA-10351,IMPALA-9812: enable mt_dop for DML by default This allows setting mt_dop for any query with any configuration. Before this patch it was not supported for DML. TODO: * exhaustive tests * precommit tests Change-Id: I66331481260fe4b69d9e95b0200029b14d230ade --- M be/src/common/global-flags.cc M be/src/util/backend-gflag-util.cc M bin/run-all-tests.sh M common/thrift/BackendGflags.thrift M fe/src/main/java/org/apache/impala/common/RuntimeEnv.java M fe/src/main/java/org/apache/impala/planner/Planner.java M fe/src/main/java/org/apache/impala/service/BackendConfig.java M fe/src/test/java/org/apache/impala/planner/PlannerTest.java M testdata/workloads/functional-planner/queries/PlannerTest/mt-dop-validation.test D testdata/workloads/functional-query/queries/QueryTest/mt-dop-auto-fallback.test M tests/custom_cluster/test_mt_dop.py 11 files changed, 69 insertions(+), 132 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/75/16775/1 -- To view, visit http://gerrit.cloudera.org:8080/16775 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: I66331481260fe4b69d9e95b0200029b14d230ade Gerrit-Change-Number: 16775 Gerrit-PatchSet: 1 Gerrit-Owner: Tim Armstrong Gerrit-Reviewer: Impala Public Jenkins
[Impala-ASF-CR] IMPALA-10351,IMPALA-9812: enable mt dop for DML by default
Hello Impala Public Jenkins, I'd like you to reexamine a change. Please visit http://gerrit.cloudera.org:8080/16775 to look at the new patch set (#3). Change subject: IMPALA-10351,IMPALA-9812: enable mt_dop for DML by default .. IMPALA-10351,IMPALA-9812: enable mt_dop for DML by default This allows setting mt_dop for any query with any configuration. Before this patch it was not supported for DML. --unlock_mt_dop and --mt_dop_auto_fallback are now ignored. Testing: * Updated tests to reflect new behaviour. * Removed irrelevant tests for fallback/validation. * Ran exhaustive tests. Change-Id: I66331481260fe4b69d9e95b0200029b14d230ade --- M be/src/common/global-flags.cc M be/src/util/backend-gflag-util.cc M bin/run-all-tests.sh M common/thrift/BackendGflags.thrift M fe/src/main/java/org/apache/impala/common/RuntimeEnv.java M fe/src/main/java/org/apache/impala/planner/Planner.java M fe/src/main/java/org/apache/impala/service/BackendConfig.java M fe/src/test/java/org/apache/impala/planner/PlannerTest.java M testdata/workloads/functional-planner/queries/PlannerTest/mt-dop-validation.test D testdata/workloads/functional-query/queries/QueryTest/mt-dop-auto-fallback.test M tests/custom_cluster/test_mt_dop.py 11 files changed, 69 insertions(+), 132 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/75/16775/3 -- To view, visit http://gerrit.cloudera.org:8080/16775 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: I66331481260fe4b69d9e95b0200029b14d230ade Gerrit-Change-Number: 16775 Gerrit-PatchSet: 3 Gerrit-Owner: Tim Armstrong Gerrit-Reviewer: Impala Public Jenkins
[Impala-ASF-CR] IMPALA-10351,IMPALA-9812: enable mt dop for DML by default
Tim Armstrong has uploaded a new patch set (#2). ( http://gerrit.cloudera.org:8080/16775 ) Change subject: IMPALA-10351,IMPALA-9812: enable mt_dop for DML by default .. IMPALA-10351,IMPALA-9812: enable mt_dop for DML by default This allows setting mt_dop for any query with any configuration. Before this patch it was not supported for DML. TODO: * exhaustive tests * precommit tests Change-Id: I66331481260fe4b69d9e95b0200029b14d230ade --- M be/src/common/global-flags.cc M be/src/util/backend-gflag-util.cc M bin/run-all-tests.sh M common/thrift/BackendGflags.thrift M fe/src/main/java/org/apache/impala/common/RuntimeEnv.java M fe/src/main/java/org/apache/impala/planner/Planner.java M fe/src/main/java/org/apache/impala/service/BackendConfig.java M fe/src/test/java/org/apache/impala/planner/PlannerTest.java M testdata/workloads/functional-planner/queries/PlannerTest/mt-dop-validation.test D testdata/workloads/functional-query/queries/QueryTest/mt-dop-auto-fallback.test M tests/custom_cluster/test_mt_dop.py 11 files changed, 69 insertions(+), 132 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/75/16775/2 -- To view, visit http://gerrit.cloudera.org:8080/16775 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: I66331481260fe4b69d9e95b0200029b14d230ade Gerrit-Change-Number: 16775 Gerrit-PatchSet: 2 Gerrit-Owner: Tim Armstrong Gerrit-Reviewer: Impala Public Jenkins
[Impala-ASF-CR] IMPALA-10330 Fix build error on aarch64 platform
Tim Armstrong has posted comments on this change. ( http://gerrit.cloudera.org:8080/16780 ) Change subject: IMPALA-10330 Fix build error on aarch64 platform .. Patch Set 1: I already did this here - https://gerrit.cloudera.org/#/c/16735/3/bin/impala-config.sh -- To view, visit http://gerrit.cloudera.org:8080/16780 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I11887169d717f8e1f2661a50d188154526898b93 Gerrit-Change-Number: 16780 Gerrit-PatchSet: 1 Gerrit-Owner: Anonymous Coward Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Tim Armstrong Gerrit-Comment-Date: Tue, 24 Nov 2020 17:03:01 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-10330: Bump toolchain build id for new Kudu
Tim Armstrong has submitted this change and it was merged. ( http://gerrit.cloudera.org:8080/16735 ) Change subject: IMPALA-10330: Bump toolchain build id for new Kudu .. IMPALA-10330: Bump toolchain build id for new Kudu Change-Id: I1d351720bb5322a7ae3c038256f5385372ba6c6b Reviewed-on: http://gerrit.cloudera.org:8080/16735 Tested-by: Impala Public Jenkins Reviewed-by: Joe McDonnell --- M bin/impala-config.sh 1 file changed, 2 insertions(+), 2 deletions(-) Approvals: Impala Public Jenkins: Verified Joe McDonnell: Looks good to me, approved -- To view, visit http://gerrit.cloudera.org:8080/16735 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: merged Gerrit-Change-Id: I1d351720bb5322a7ae3c038256f5385372ba6c6b Gerrit-Change-Number: 16735 Gerrit-PatchSet: 3 Gerrit-Owner: Tim Armstrong Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Joe McDonnell Gerrit-Reviewer: Tim Armstrong
[Impala-ASF-CR] Prototype to try out one coord exec and two executors
Joe McDonnell has abandoned this change. ( http://gerrit.cloudera.org:8080/16637 ) Change subject: Prototype to try out one coord_exec and two executors .. Abandoned -- To view, visit http://gerrit.cloudera.org:8080/16637 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: abandon Gerrit-Change-Id: Ib523044813f01f74e9b530192faa12346b3162dd Gerrit-Change-Number: 16637 Gerrit-PatchSet: 1 Gerrit-Owner: Joe McDonnell Gerrit-Reviewer: Impala Public Jenkins
[Impala-ASF-CR] PROTOTYPE: Produce minidumps if test metrics are zero fails
Joe McDonnell has abandoned this change. ( http://gerrit.cloudera.org:8080/16084 ) Change subject: PROTOTYPE: Produce minidumps if test_metrics_are_zero fails .. Abandoned -- To view, visit http://gerrit.cloudera.org:8080/16084 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: abandon Gerrit-Change-Id: I909611642c9eb2884fc9a89a9b2b000158e62c3a Gerrit-Change-Number: 16084 Gerrit-PatchSet: 1 Gerrit-Owner: Joe McDonnell Gerrit-Reviewer: Impala Public Jenkins
[Impala-ASF-CR] Fix for startup crash in scheduler-benchmark.
Joe McDonnell has posted comments on this change. ( http://gerrit.cloudera.org:8080/16778 ) Change subject: Fix for startup crash in scheduler-benchmark. .. Patch Set 2: Code-Review+2 This looks good to me. If you are looking into scheduler performance, one related JIRA is IMPALA-9221. -- To view, visit http://gerrit.cloudera.org:8080/16778 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ib9fba3b97f102e41f2024a2bfaacbf0568bd4c68 Gerrit-Change-Number: 16778 Gerrit-PatchSet: 2 Gerrit-Owner: Shant Hovsepian Gerrit-Reviewer: Aman Sinha Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Joe McDonnell Gerrit-Reviewer: Shant Hovsepian Gerrit-Reviewer: Tim Armstrong Gerrit-Comment-Date: Tue, 24 Nov 2020 16:34:48 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-10330: Bump toolchain build id for new Kudu
Joe McDonnell has posted comments on this change. ( http://gerrit.cloudera.org:8080/16735 ) Change subject: IMPALA-10330: Bump toolchain build id for new Kudu .. Patch Set 2: Code-Review+2 -- To view, visit http://gerrit.cloudera.org:8080/16735 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I1d351720bb5322a7ae3c038256f5385372ba6c6b Gerrit-Change-Number: 16735 Gerrit-PatchSet: 2 Gerrit-Owner: Tim Armstrong Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Joe McDonnell Gerrit-Comment-Date: Tue, 24 Nov 2020 16:14:11 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-10345: Impala hits DCHECK in parquet-column-stats.inline.h
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/16771 ) Change subject: IMPALA-10345: Impala hits DCHECK in parquet-column-stats.inline.h .. Patch Set 4: Build Successful https://jenkins.impala.io/job/gerrit-code-review-checks/7723/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests. -- To view, visit http://gerrit.cloudera.org:8080/16771 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I4289bd743e951cc4c607d5a5ea75d27825a1c12b Gerrit-Change-Number: 16771 Gerrit-PatchSet: 4 Gerrit-Owner: Zoltan Borok-Nagy Gerrit-Reviewer: Csaba Ringhofer Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Zoltan Borok-Nagy Gerrit-Comment-Date: Tue, 24 Nov 2020 13:28:53 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-10345: Impala hits DCHECK in parquet-column-stats.inline.h
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/16771 ) Change subject: IMPALA-10345: Impala hits DCHECK in parquet-column-stats.inline.h .. Patch Set 5: Build started: https://jenkins.impala.io/job/gerrit-verify-dryrun/6707/ DRY_RUN=false -- To view, visit http://gerrit.cloudera.org:8080/16771 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I4289bd743e951cc4c607d5a5ea75d27825a1c12b Gerrit-Change-Number: 16771 Gerrit-PatchSet: 5 Gerrit-Owner: Zoltan Borok-Nagy Gerrit-Reviewer: Csaba Ringhofer Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Zoltan Borok-Nagy Gerrit-Comment-Date: Tue, 24 Nov 2020 13:08:37 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-10345: Impala hits DCHECK in parquet-column-stats.inline.h
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/16771 ) Change subject: IMPALA-10345: Impala hits DCHECK in parquet-column-stats.inline.h .. Patch Set 5: Code-Review+2 -- To view, visit http://gerrit.cloudera.org:8080/16771 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I4289bd743e951cc4c607d5a5ea75d27825a1c12b Gerrit-Change-Number: 16771 Gerrit-PatchSet: 5 Gerrit-Owner: Zoltan Borok-Nagy Gerrit-Reviewer: Csaba Ringhofer Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Zoltan Borok-Nagy Gerrit-Comment-Date: Tue, 24 Nov 2020 13:08:36 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-10345: Impala hits DCHECK in parquet-column-stats.inline.h
Zoltan Borok-Nagy has posted comments on this change. ( http://gerrit.cloudera.org:8080/16771 ) Change subject: IMPALA-10345: Impala hits DCHECK in parquet-column-stats.inline.h .. Patch Set 4: Code-Review+2 Carry +2 -- To view, visit http://gerrit.cloudera.org:8080/16771 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I4289bd743e951cc4c607d5a5ea75d27825a1c12b Gerrit-Change-Number: 16771 Gerrit-PatchSet: 4 Gerrit-Owner: Zoltan Borok-Nagy Gerrit-Reviewer: Csaba Ringhofer Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Zoltan Borok-Nagy Gerrit-Comment-Date: Tue, 24 Nov 2020 13:08:18 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-10345: Impala hits DCHECK in parquet-column-stats.inline.h
Hello Csaba Ringhofer, Impala Public Jenkins, I'd like you to reexamine a change. Please visit http://gerrit.cloudera.org:8080/16771 to look at the new patch set (#4). Change subject: IMPALA-10345: Impala hits DCHECK in parquet-column-stats.inline.h .. IMPALA-10345: Impala hits DCHECK in parquet-column-stats.inline.h During Parquet file writing, a DCHECK checks if row group stats have copied the min/max string values into their internal buffers. This check is at the finalization of each page. The copying of the string values happened at the end of each row batch. Thus, if a row batch spans over multiple pages then the min/max string values don't get copied by the end of the page. Since the memory is attached to the row batch this isn't really an error. As a workaround this commit also copies the min/max string values at the end of the page if they haven't been copied yet. Testing * Added e2e test Change-Id: I4289bd743e951cc4c607d5a5ea75d27825a1c12b --- M be/src/exec/parquet/hdfs-parquet-table-writer.cc M testdata/workloads/functional-query/queries/QueryTest/parquet-page-index.test M tests/query_test/test_parquet_stats.py 3 files changed, 23 insertions(+), 1 deletion(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/71/16771/4 -- To view, visit http://gerrit.cloudera.org:8080/16771 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: I4289bd743e951cc4c607d5a5ea75d27825a1c12b Gerrit-Change-Number: 16771 Gerrit-PatchSet: 4 Gerrit-Owner: Zoltan Borok-Nagy Gerrit-Reviewer: Csaba Ringhofer Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Zoltan Borok-Nagy
[Impala-ASF-CR] IMPALA-10323: use gutil to cast int to string
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/16719 ) Change subject: IMPALA-10323: use gutil to cast int to string .. Patch Set 7: Build Successful https://jenkins.impala.io/job/gerrit-code-review-checks/7722/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests. -- To view, visit http://gerrit.cloudera.org:8080/16719 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ief1d6b2c2cb5961c1d6cee1b0eac86ab61509768 Gerrit-Change-Number: 16719 Gerrit-PatchSet: 7 Gerrit-Owner: wesleydeng Gerrit-Reviewer: Csaba Ringhofer Gerrit-Reviewer: Gabor Kaszab Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Qifan Chen Gerrit-Reviewer: Tim Armstrong Gerrit-Reviewer: abeltian Gerrit-Reviewer: wesleydeng Gerrit-Comment-Date: Tue, 24 Nov 2020 12:07:08 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-10323: use gutil to cast int to string
Csaba Ringhofer has submitted this change and it was merged. ( http://gerrit.cloudera.org:8080/16719 ) Change subject: IMPALA-10323: use gutil to cast int to string .. IMPALA-10323: use gutil to cast int to string Using gutil instead of lexical_cast when casting int type value to string, performance can be improved significantly in multi-thread mode. In contrast, performance using lexical_cast decreased seriously in multi-thread mode. For example: when mt_dop=8, this change can get 20x performance improvement; when mt_dop=0, this change can get 2x performance improvement; Change-Id: Ief1d6b2c2cb5961c1d6cee1b0eac86ab61509768 Reviewed-on: http://gerrit.cloudera.org:8080/16719 Reviewed-by: Impala Public Jenkins Tested-by: Impala Public Jenkins Reviewed-by: Csaba Ringhofer --- M be/src/exprs/cast-functions-ir.cc 1 file changed, 34 insertions(+), 16 deletions(-) Approvals: Impala Public Jenkins: Looks good to me, approved; Verified Csaba Ringhofer: Looks good to me, approved -- To view, visit http://gerrit.cloudera.org:8080/16719 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: merged Gerrit-Change-Id: Ief1d6b2c2cb5961c1d6cee1b0eac86ab61509768 Gerrit-Change-Number: 16719 Gerrit-PatchSet: 8 Gerrit-Owner: wesleydeng Gerrit-Reviewer: Csaba Ringhofer Gerrit-Reviewer: Gabor Kaszab Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Qifan Chen Gerrit-Reviewer: Tim Armstrong Gerrit-Reviewer: abeltian Gerrit-Reviewer: wesleydeng
[Impala-ASF-CR] IMPALA-10323: use gutil to cast int to string
wesleydeng has posted comments on this change. ( http://gerrit.cloudera.org:8080/16719 ) Change subject: IMPALA-10323: use gutil to cast int to string .. Patch Set 7: (3 comments) > wesleydeng, can you publish the change as non-draft? The later > patches are drafts, which is why the Jenkins job didn't merge it. done http://gerrit.cloudera.org:8080/#/c/16719/3/be/src/exprs/cast-functions-ir.cc File be/src/exprs/cast-functions-ir.cc: http://gerrit.cloudera.org:8080/#/c/16719/3/be/src/exprs/cast-functions-ir.cc@134 PS3, Line 134: > I'd prefer if you used a constant here, i.e. MAX_FLOAT_CHARS and maybe add Actually max_char_size is a constant here which will be replace by MAX_BIGINT_CHARS. But to defend others to reuse this macro by passing a variable, I will make this modification as you suggest. I has some problem to connect gerrit now, I will push new patch on Monday. http://gerrit.cloudera.org:8080/#/c/16719/3/be/src/exprs/cast-functions-ir.cc@140 PS3, Line 140: char char_buffer[MAX_EXACT_NUMERIC_CHARS];\ : char* end_position = to_string_method(val.val, char_buffer); \ : size_t len = end_position - char_buffer; \ : DCH > I wonder if we can skip this check, as a StringVal object with not sufficie I keep this check because I'm not sure. Other sugguestion are applied to new patch. http://gerrit.cloudera.org:8080/#/c/16719/4/be/src/exprs/cast-functions-ir.cc File be/src/exprs/cast-functions-ir.cc: http://gerrit.cloudera.org:8080/#/c/16719/4/be/src/exprs/cast-functions-ir.cc@27 PS4, Line 27: #include > nit: we generally include headers sorted alphabetically, so numbers.h comes fixed -- To view, visit http://gerrit.cloudera.org:8080/16719 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ief1d6b2c2cb5961c1d6cee1b0eac86ab61509768 Gerrit-Change-Number: 16719 Gerrit-PatchSet: 7 Gerrit-Owner: wesleydeng Gerrit-Reviewer: Csaba Ringhofer Gerrit-Reviewer: Gabor Kaszab Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Qifan Chen Gerrit-Reviewer: Tim Armstrong Gerrit-Reviewer: abeltian Gerrit-Reviewer: wesleydeng Gerrit-Comment-Date: Tue, 24 Nov 2020 11:45:42 + Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-10323: use gutil to cast int to string
wesleydeng has uploaded a new patch set (#7). ( http://gerrit.cloudera.org:8080/16719 ) Change subject: IMPALA-10323: use gutil to cast int to string .. IMPALA-10323: use gutil to cast int to string Using gutil instead of lexical_cast when casting int type value to string, performance can be improved significantly in multi-thread mode. In contrast, performance using lexical_cast decreased seriously in multi-thread mode. For example: when mt_dop=8, this change can get 20x performance improvement; when mt_dop=0, this change can get 2x performance improvement; Change-Id: Ief1d6b2c2cb5961c1d6cee1b0eac86ab61509768 --- M be/src/exprs/cast-functions-ir.cc 1 file changed, 34 insertions(+), 16 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/19/16719/7 -- To view, visit http://gerrit.cloudera.org:8080/16719 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: Ief1d6b2c2cb5961c1d6cee1b0eac86ab61509768 Gerrit-Change-Number: 16719 Gerrit-PatchSet: 7 Gerrit-Owner: wesleydeng Gerrit-Reviewer: Csaba Ringhofer Gerrit-Reviewer: Gabor Kaszab Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Qifan Chen Gerrit-Reviewer: Tim Armstrong Gerrit-Reviewer: abeltian Gerrit-Reviewer: wesleydeng
[Impala-ASF-CR] IMPALA-10330 Fix build error on aarch64 platform
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/16780 ) Change subject: IMPALA-10330 Fix build error on aarch64 platform .. Patch Set 1: Build Failed https://jenkins.impala.io/job/gerrit-code-review-checks/7721/ : Initial code review checks failed. See linked job for details on the failure. -- To view, visit http://gerrit.cloudera.org:8080/16780 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I11887169d717f8e1f2661a50d188154526898b93 Gerrit-Change-Number: 16780 Gerrit-PatchSet: 1 Gerrit-Owner: Anonymous Coward Gerrit-Reviewer: Impala Public Jenkins Gerrit-Comment-Date: Tue, 24 Nov 2020 09:45:26 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-10330 Fix build error on aarch64 platform
zhaoren...@hotmail.com has uploaded this change for review. ( http://gerrit.cloudera.org:8080/16780 Change subject: IMPALA-10330 Fix build error on aarch64 platform .. IMPALA-10330 Fix build error on aarch64 platform Recently, impala was built failed on aarch64 platform because of Kudu cannot be built successfully on aarch64. So this commit will change kudu version to 6a7cadc74 which can be built successfully on aarch64. Change-Id: I11887169d717f8e1f2661a50d188154526898b93 --- M bin/impala-config.sh 1 file changed, 1 insertion(+), 1 deletion(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/80/16780/1 -- To view, visit http://gerrit.cloudera.org:8080/16780 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newchange Gerrit-Change-Id: I11887169d717f8e1f2661a50d188154526898b93 Gerrit-Change-Number: 16780 Gerrit-PatchSet: 1 Gerrit-Owner: Anonymous Coward
