[Impala-ASF-CR] IMPALA-7961: Avoid adding unmodified objects to DDL response
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/12428 ) Change subject: IMPALA-7961: Avoid adding unmodified objects to DDL response .. Patch Set 5: Build Successful https://jenkins.impala.io/job/gerrit-code-review-checks/2095/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests. -- To view, visit http://gerrit.cloudera.org:8080/12428 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: If3e914b70ba796c9b224e9dea559b8c40aa25d83 Gerrit-Change-Number: 12428 Gerrit-PatchSet: 5 Gerrit-Owner: Bharath Vissapragada Gerrit-Reviewer: Bharath Vissapragada Gerrit-Reviewer: Fredy Wijaya Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Paul Rogers Gerrit-Comment-Date: Wed, 13 Feb 2019 05:08:30 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-6503: Support reading complex types from ORC format files
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/12168 ) Change subject: IMPALA-6503: Support reading complex types from ORC format files .. Patch Set 14: Verified+1 -- To view, visit http://gerrit.cloudera.org:8080/12168 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I244dc9d2b3e425393f90e45632cb8cdbea6cf790 Gerrit-Change-Number: 12168 Gerrit-PatchSet: 14 Gerrit-Owner: Quanlong Huang Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Joe McDonnell Gerrit-Reviewer: Quanlong Huang Gerrit-Reviewer: Tim Armstrong Gerrit-Reviewer: Zoltan Borok-Nagy Gerrit-Comment-Date: Wed, 13 Feb 2019 05:06:49 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-7961: Avoid adding unmodified objects to DDL response
Bharath Vissapragada has posted comments on this change. ( http://gerrit.cloudera.org:8080/12428 ) Change subject: IMPALA-7961: Avoid adding unmodified objects to DDL response .. Patch Set 5: Redid the patch after discussing with Paul. The issue with the older implementation is that it can break the SYNC_DDL semantics if the table already exists. The newer patch takes a more conservative approach by bumping up the table version number and then waiting for it to be propagated. This can cause some unnecessary invalidates but it is more in-line with the user's expectations. -- To view, visit http://gerrit.cloudera.org:8080/12428 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: If3e914b70ba796c9b224e9dea559b8c40aa25d83 Gerrit-Change-Number: 12428 Gerrit-PatchSet: 5 Gerrit-Owner: Bharath Vissapragada Gerrit-Reviewer: Bharath Vissapragada Gerrit-Reviewer: Fredy Wijaya Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Paul Rogers Gerrit-Comment-Date: Wed, 13 Feb 2019 04:50:43 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-7961: Avoid adding unmodified objects to DDL response
Hello Paul Rogers, Fredy Wijaya, Impala Public Jenkins, I'd like you to reexamine a change. Please visit http://gerrit.cloudera.org:8080/12428 to look at the new patch set (#5). Change subject: IMPALA-7961: Avoid adding unmodified objects to DDL response .. IMPALA-7961: Avoid adding unmodified objects to DDL response When a DDL is processed, we typically add the affected (added/removed) objects to the response TCatalogUpdateResult struct. This response is processed on the coordinator and the changes are applied locally. When SYNC_DDL is enabled, the Catalog server also includes a topic version number that should include all the affected objects so that the coordinator can wait for that miniumum topic version to be applied on all other coordinators before returning the control back to the user. This covering topic version is calculated by looking at the topic update log, which contains all the in-flight updates (and to an extent past updates) that are perodically GC'ed. Bug: In certain cases like CREATE TBL IF NOT EXISTS, we could end up adding objects to the DDL response which haven't been modified in a while (> TOPIC_UPDATE_LOG_GC_FREQUENCY) and hence could be potentially GC'ed from the TopicUpdateLog. This means that the Catalog server wouldn't be able to find a covering topic update version and eventually gives up throwing an error as described in the jira. Fix: Bumps the version of any objects that already exists when IF EXISTS is used in conjunction with SYNC_DDL. This makes sure that the object is included in the upcoming topic updates and waitForSyncDdlVersion() can find a covering topic update that includes this object. This is a hack and could cause false-positive invalidations, but definitely better than breaking SYNC_DDL semantics. Also added some additional diagnostic logging that could've simplified debugging an issue like this. Testing: Since this is a racy bug, I could only repro it by forcing frequent topic update log GCs along with a specific sequence of actions. Couldn't reproduce it with the patch. Change-Id: If3e914b70ba796c9b224e9dea559b8c40aa25d83 --- M fe/src/main/java/org/apache/impala/catalog/CatalogServiceCatalog.java M fe/src/main/java/org/apache/impala/catalog/FeCatalogUtils.java M fe/src/main/java/org/apache/impala/catalog/TopicUpdateLog.java M fe/src/main/java/org/apache/impala/service/CatalogOpExecutor.java 4 files changed, 92 insertions(+), 31 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/28/12428/5 -- To view, visit http://gerrit.cloudera.org:8080/12428 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: If3e914b70ba796c9b224e9dea559b8c40aa25d83 Gerrit-Change-Number: 12428 Gerrit-PatchSet: 5 Gerrit-Owner: Bharath Vissapragada Gerrit-Reviewer: Bharath Vissapragada Gerrit-Reviewer: Fredy Wijaya Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Paul Rogers
[Impala-ASF-CR] IMPALA-8183: fix test reportexecstatus retry flakiness
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/12461 ) Change subject: IMPALA-8183: fix test_reportexecstatus_retry flakiness .. Patch Set 3: Verified+1 -- To view, visit http://gerrit.cloudera.org:8080/12461 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I7027a6e099c543705e5845ee0e5268f1f9a3fb05 Gerrit-Change-Number: 12461 Gerrit-PatchSet: 3 Gerrit-Owner: Thomas Marshall Gerrit-Reviewer: Andrew Sherman Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Michael Ho Gerrit-Reviewer: Thomas Marshall Gerrit-Comment-Date: Wed, 13 Feb 2019 03:20:54 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-8183: fix test reportexecstatus retry flakiness
Impala Public Jenkins has submitted this change and it was merged. ( http://gerrit.cloudera.org:8080/12461 ) Change subject: IMPALA-8183: fix test_reportexecstatus_retry flakiness .. IMPALA-8183: fix test_reportexecstatus_retry flakiness The test is designed to cause ReportExecStatus() rpcs to fail by backing up the control service queue. Previously, after a failed ReportExecStatus() we would wait 'report_status_retry_interval_ms' between retries, which was 100ms by default and wasn't touched by the test. That 100ms was right on the edge of being enough time for the coordinator to keep up with processing the reports, so that some would fail but most would succeed. It was always possible that we could hit IMPALA-2990 in this setup, but it was unlikely. Now, with IMPALA-4555 'report_status_retry_interval_ms' was removed and we instead wait 'status_report_interval_ms' between retries. By default, this is 5000ms, so it should give the coordinator even more time and make these issues less likely. However, the test sets 'status_report_interval_ms' to 10ms, which isn't nearly enough time for the coordinator to do its processing, causing lots of the ReportExecStatus() rpcs to fail and making us hit IMPALA-2990 pretty often. The solution is to set 'status_report_interval_ms' to 100ms in the test, which roughly achieves the same retry frequency as before. The same change is made to a similar test test_reportexecstatus_timeout. Testing: - Ran test_reportexecstatus_retry in a loop 400 times without seeing a failure. It previously repro-ed for me about once per 50 runs. - Manually verified that both tests are still hitting the error paths that they are supposed to be testing. Change-Id: I7027a6e099c543705e5845ee0e5268f1f9a3fb05 Reviewed-on: http://gerrit.cloudera.org:8080/12461 Reviewed-by: Impala Public Jenkins Tested-by: Impala Public Jenkins --- M tests/custom_cluster/test_rpc_timeout.py 1 file changed, 2 insertions(+), 2 deletions(-) Approvals: Impala Public Jenkins: Looks good to me, approved; Verified -- To view, visit http://gerrit.cloudera.org:8080/12461 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: merged Gerrit-Change-Id: I7027a6e099c543705e5845ee0e5268f1f9a3fb05 Gerrit-Change-Number: 12461 Gerrit-PatchSet: 4 Gerrit-Owner: Thomas Marshall Gerrit-Reviewer: Andrew Sherman Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Michael Ho Gerrit-Reviewer: Thomas Marshall
[Impala-ASF-CR] Update toolchain to support ubuntu 18.04
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/12421 ) Change subject: Update toolchain to support ubuntu 18.04 .. Patch Set 2: Build Successful https://jenkins.impala.io/job/gerrit-code-review-checks/2094/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests. -- To view, visit http://gerrit.cloudera.org:8080/12421 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ie25c8cb129c6817a2e116f31853ae64c5a8acfe9 Gerrit-Change-Number: 12421 Gerrit-PatchSet: 2 Gerrit-Owner: hector.aco...@cloudera.com Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Philip Zeyliger Gerrit-Comment-Date: Wed, 13 Feb 2019 02:50:32 + Gerrit-HasComments: No
[Impala-ASF-CR] Update toolchain to support ubuntu 18.04
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/12465 ) Change subject: Update toolchain to support ubuntu 18.04 .. Patch Set 1: Build Successful https://jenkins.impala.io/job/gerrit-code-review-checks/2093/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests. -- To view, visit http://gerrit.cloudera.org:8080/12465 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I4132e8a4e58bb100c9f21d61b50c80b687c4b0d1 Gerrit-Change-Number: 12465 Gerrit-PatchSet: 1 Gerrit-Owner: hector.aco...@cloudera.com Gerrit-Reviewer: Impala Public Jenkins Gerrit-Comment-Date: Wed, 13 Feb 2019 02:34:39 + Gerrit-HasComments: No
[native-toolchain-CR] Enable publishing on the docker-based build
hector.aco...@cloudera.com has uploaded a new patch set (#2). ( http://gerrit.cloudera.org:8080/12423 ) Change subject: Enable publishing on the docker-based build .. Enable publishing on the docker-based build In order to enable publishing from each container this commit: * Adds maven and aws commands * Modifies the upload function to conditionally upload to artifactory/s3 * Generate TOOLCHAIN_ID (if not set) and share it with all the containers. This is done in the Makefile so that all containers get the same TOOLCHAIN_ID. * Generate the BUILD_TARGET_LABEL based on the docker image name. Change-Id: I32966bf5b5ad32a3caaf2aee0430ea3a67ecb5d6 --- M Makefile M docker/all/assert-dependencies-present.py M docker/debian7.df M docker/debian8.df M docker/redhat6.df M docker/redhat7.df M docker/sles12.df M docker/ubuntu1204.df M docker/ubuntu1404.df M docker/ubuntu1604.df M docker/ubuntu1804.df M functions.sh M in-docker.py M init.sh 14 files changed, 181 insertions(+), 23 deletions(-) git pull ssh://gerrit.cloudera.org:29418/native-toolchain refs/changes/23/12423/2 -- To view, visit http://gerrit.cloudera.org:8080/12423 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: native-toolchain Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: I32966bf5b5ad32a3caaf2aee0430ea3a67ecb5d6 Gerrit-Change-Number: 12423 Gerrit-PatchSet: 2 Gerrit-Owner: hector.aco...@cloudera.com
[Impala-ASF-CR] Update toolchain to support ubuntu 18.04
Hello Philip Zeyliger, Impala Public Jenkins, I'd like you to reexamine a change. Please visit http://gerrit.cloudera.org:8080/12421 to look at the new patch set (#2). Change subject: Update toolchain to support ubuntu 18.04 .. Update toolchain to support ubuntu 18.04 Change-Id: Ie25c8cb129c6817a2e116f31853ae64c5a8acfe9 --- M bin/bootstrap_toolchain.py M bin/impala-config.sh 2 files changed, 10 insertions(+), 8 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/21/12421/2 -- To view, visit http://gerrit.cloudera.org:8080/12421 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: Ie25c8cb129c6817a2e116f31853ae64c5a8acfe9 Gerrit-Change-Number: 12421 Gerrit-PatchSet: 2 Gerrit-Owner: hector.aco...@cloudera.com Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Philip Zeyliger
[Impala-ASF-CR] Update toolchain to support ubuntu 18.04
hector.aco...@cloudera.com has uploaded this change for review. ( http://gerrit.cloudera.org:8080/12465 Change subject: Update toolchain to support ubuntu 18.04 .. Update toolchain to support ubuntu 18.04 Change-Id: I4132e8a4e58bb100c9f21d61b50c80b687c4b0d1 --- M bin/bootstrap_toolchain.py M bin/impala-config.sh 2 files changed, 10 insertions(+), 8 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/65/12465/1 -- To view, visit http://gerrit.cloudera.org:8080/12465 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newchange Gerrit-Change-Id: I4132e8a4e58bb100c9f21d61b50c80b687c4b0d1 Gerrit-Change-Number: 12465 Gerrit-PatchSet: 1 Gerrit-Owner: hector.aco...@cloudera.com
[Impala-ASF-CR] Update toolchain to support ubuntu 18.04
hector.aco...@cloudera.com has abandoned this change. ( http://gerrit.cloudera.org:8080/12465 ) Change subject: Update toolchain to support ubuntu 18.04 .. Abandoned -- To view, visit http://gerrit.cloudera.org:8080/12465 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: abandon Gerrit-Change-Id: I4132e8a4e58bb100c9f21d61b50c80b687c4b0d1 Gerrit-Change-Number: 12465 Gerrit-PatchSet: 1 Gerrit-Owner: hector.aco...@cloudera.com
[Impala-ASF-CR] IMPALA-6503: Support reading complex types from ORC format files
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/12168 ) Change subject: IMPALA-6503: Support reading complex types from ORC format files .. Patch Set 14: Build started: https://jenkins.impala.io/job/gerrit-verify-dryrun/3768/ DRY_RUN=false -- To view, visit http://gerrit.cloudera.org:8080/12168 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I244dc9d2b3e425393f90e45632cb8cdbea6cf790 Gerrit-Change-Number: 12168 Gerrit-PatchSet: 14 Gerrit-Owner: Quanlong Huang Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Joe McDonnell Gerrit-Reviewer: Quanlong Huang Gerrit-Reviewer: Tim Armstrong Gerrit-Reviewer: Zoltan Borok-Nagy Gerrit-Comment-Date: Wed, 13 Feb 2019 01:00:38 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-7935: Disable /catalog object in local catalog mode.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/12443 ) Change subject: IMPALA-7935: Disable /catalog_object in local catalog mode. .. Patch Set 4: Build Successful https://jenkins.impala.io/job/gerrit-code-review-checks/2092/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests. -- To view, visit http://gerrit.cloudera.org:8080/12443 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ia04797b32964c2edaa2e860dcf510d6f9cccd81c Gerrit-Change-Number: 12443 Gerrit-PatchSet: 4 Gerrit-Owner: Anurag Mantripragada Gerrit-Reviewer: Anurag Mantripragada Gerrit-Reviewer: Bharath Vissapragada Gerrit-Reviewer: Impala Public Jenkins Gerrit-Comment-Date: Wed, 13 Feb 2019 01:00:58 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-8187: UDF samples hide symbols by default
Zoram Thanga has posted comments on this change. (
http://gerrit.cloudera.org:8080/12451 )
Change subject: IMPALA-8187: UDF samples hide symbols by default
..
Patch Set 3: Code-Review+1
(1 comment)
Thanks for doing this! LGTM.
http://gerrit.cloudera.org:8080/#/c/12451/3/tests/query_test/test_udfs.py
File tests/query_test/test_udfs.py:
http://gerrit.cloudera.org:8080/#/c/12451/3/tests/query_test/test_udfs.py@440
PS3, Line 440: create function `{0}`.unexported() returns BIGINT
LOCATION '{1}'
Should this be unexported_symbol()?
--
To view, visit http://gerrit.cloudera.org:8080/12451
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Ie17b74b9bce9cc6962393017879b65409ce23b28
Gerrit-Change-Number: 12451
Gerrit-PatchSet: 3
Gerrit-Owner: Tim Armstrong
Gerrit-Reviewer: Impala Public Jenkins
Gerrit-Reviewer: Michael Ho
Gerrit-Reviewer: Zoram Thanga
Gerrit-Comment-Date: Wed, 13 Feb 2019 00:33:24 +
Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-6503: Support reading complex types from ORC format files
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/12168 ) Change subject: IMPALA-6503: Support reading complex types from ORC format files .. Patch Set 13: Build Successful https://jenkins.impala.io/job/gerrit-code-review-checks/2091/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests. -- To view, visit http://gerrit.cloudera.org:8080/12168 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I244dc9d2b3e425393f90e45632cb8cdbea6cf790 Gerrit-Change-Number: 12168 Gerrit-PatchSet: 13 Gerrit-Owner: Quanlong Huang Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Joe McDonnell Gerrit-Reviewer: Quanlong Huang Gerrit-Reviewer: Tim Armstrong Gerrit-Reviewer: Zoltan Borok-Nagy Gerrit-Comment-Date: Wed, 13 Feb 2019 00:33:11 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-8187: UDF samples hide symbols by default
Michael Ho has posted comments on this change. (
http://gerrit.cloudera.org:8080/12451 )
Change subject: IMPALA-8187: UDF samples hide symbols by default
..
Patch Set 3:
(1 comment)
http://gerrit.cloudera.org:8080/#/c/12451/3/be/src/udf_samples/CMakeLists.txt
File be/src/udf_samples/CMakeLists.txt:
http://gerrit.cloudera.org:8080/#/c/12451/3/be/src/udf_samples/CMakeLists.txt@27
PS3, Line 27: set(IR_COMPILE_FLAGS "-emit-llvm" "-O3" "-std=c++14" "-c" "-I../"
${CLANG_BASE_FLAGS})
> Do we care about this case as well ?
Actually, since it's emitting IR, it's irrelevant.
--
To view, visit http://gerrit.cloudera.org:8080/12451
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Ie17b74b9bce9cc6962393017879b65409ce23b28
Gerrit-Change-Number: 12451
Gerrit-PatchSet: 3
Gerrit-Owner: Tim Armstrong
Gerrit-Reviewer: Impala Public Jenkins
Gerrit-Reviewer: Michael Ho
Gerrit-Reviewer: Zoram Thanga
Gerrit-Comment-Date: Wed, 13 Feb 2019 00:23:42 +
Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-7935: Disable /catalog object in local catalog mode.
Anurag Mantripragada has uploaded a new patch set (#4). ( http://gerrit.cloudera.org:8080/12443 ) Change subject: IMPALA-7935: Disable /catalog_object in local catalog mode. .. IMPALA-7935: Disable /catalog_object in local catalog mode. getTCatalogObject() is not supported in local catalog mode since metadata is partially fetched on demand. Removed hyperlinks to the /catalog_object endpoints when local_catalog_mode is enabled. Testing: 1. Added a positive test in test_local_catalog to verify /catalog_mode endpoint is disabled. 2. Added a negative test to test_web_pages.py to check /catalog_object is working when local_catalog mode is not set. Change-Id: Ia04797b32964c2edaa2e860dcf510d6f9cccd81c --- M be/src/service/impala-http-handler.cc M tests/custom_cluster/test_local_catalog.py M tests/custom_cluster/test_web_pages.py M www/catalog.tmpl 4 files changed, 62 insertions(+), 14 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/43/12443/4 -- To view, visit http://gerrit.cloudera.org:8080/12443 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: Ia04797b32964c2edaa2e860dcf510d6f9cccd81c Gerrit-Change-Number: 12443 Gerrit-PatchSet: 4 Gerrit-Owner: Anurag Mantripragada Gerrit-Reviewer: Anurag Mantripragada Gerrit-Reviewer: Bharath Vissapragada Gerrit-Reviewer: Impala Public Jenkins
[Impala-ASF-CR] IMPALA-8187: UDF samples hide symbols by default
Michael Ho has posted comments on this change. ( http://gerrit.cloudera.org:8080/12451 ) Change subject: IMPALA-8187: UDF samples hide symbols by default .. Patch Set 3: Code-Review+1 -- To view, visit http://gerrit.cloudera.org:8080/12451 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ie17b74b9bce9cc6962393017879b65409ce23b28 Gerrit-Change-Number: 12451 Gerrit-PatchSet: 3 Gerrit-Owner: Tim Armstrong Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Michael Ho Gerrit-Reviewer: Zoram Thanga Gerrit-Comment-Date: Wed, 13 Feb 2019 00:10:18 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-8187: UDF samples hide symbols by default
Michael Ho has posted comments on this change. ( http://gerrit.cloudera.org:8080/12451 ) Change subject: IMPALA-8187: UDF samples hide symbols by default .. Patch Set 3: Zoram, can you please also take a look ? -- To view, visit http://gerrit.cloudera.org:8080/12451 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ie17b74b9bce9cc6962393017879b65409ce23b28 Gerrit-Change-Number: 12451 Gerrit-PatchSet: 3 Gerrit-Owner: Tim Armstrong Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Michael Ho Gerrit-Reviewer: Zoram Thanga Gerrit-Comment-Date: Wed, 13 Feb 2019 00:10:34 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-8187: UDF samples hide symbols by default
Michael Ho has posted comments on this change. (
http://gerrit.cloudera.org:8080/12451 )
Change subject: IMPALA-8187: UDF samples hide symbols by default
..
Patch Set 3:
(2 comments)
http://gerrit.cloudera.org:8080/#/c/12451/3/be/src/testutil/test-udas.cc
File be/src/testutil/test-udas.cc:
http://gerrit.cloudera.org:8080/#/c/12451/3/be/src/testutil/test-udas.cc@343
PS3, Line 343: }
nit: Please keep a blank line between functions
http://gerrit.cloudera.org:8080/#/c/12451/3/be/src/udf_samples/CMakeLists.txt
File be/src/udf_samples/CMakeLists.txt:
http://gerrit.cloudera.org:8080/#/c/12451/3/be/src/udf_samples/CMakeLists.txt@27
PS3, Line 27: set(IR_COMPILE_FLAGS "-emit-llvm" "-O3" "-std=c++14" "-c" "-I../"
${CLANG_BASE_FLAGS})
Do we care about this case as well ?
--
To view, visit http://gerrit.cloudera.org:8080/12451
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Ie17b74b9bce9cc6962393017879b65409ce23b28
Gerrit-Change-Number: 12451
Gerrit-PatchSet: 3
Gerrit-Owner: Tim Armstrong
Gerrit-Reviewer: Impala Public Jenkins
Gerrit-Reviewer: Michael Ho
Gerrit-Reviewer: Zoram Thanga
Gerrit-Comment-Date: Wed, 13 Feb 2019 00:10:00 +
Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-6503: Support reading complex types from ORC format files
Quanlong Huang has posted comments on this change. (
http://gerrit.cloudera.org:8080/12168 )
Change subject: IMPALA-6503: Support reading complex types from ORC format files
..
Patch Set 13:
(15 comments)
Thanks for your new feedbacks! Let's keep reviewing it so things could be
simpler:)
http://gerrit.cloudera.org:8080/#/c/12168/9/be/src/exec/hdfs-orc-scanner.cc
File be/src/exec/hdfs-orc-scanner.cc:
http://gerrit.cloudera.org:8080/#/c/12168/9/be/src/exec/hdfs-orc-scanner.cc@363
PS9, Line 363: // We only deal with collection columns (ARRAY/MAP) and
primitive columns here.
> Yep that's the rule - use braces if the whole thing doesn't fit on a single
Sure
http://gerrit.cloudera.org:8080/#/c/12168/12/be/src/exec/hdfs-orc-scanner.cc
File be/src/exec/hdfs-orc-scanner.cc:
http://gerrit.cloudera.org:8080/#/c/12168/12/be/src/exec/hdfs-orc-scanner.cc@198
PS12, Line 198: niqu
> nit:std:: not needed since we import it into the namespace with common/name
Done
http://gerrit.cloudera.org:8080/#/c/12168/12/be/src/exec/hdfs-orc-scanner.cc@316
PS12, Line 316: VLOG(3) << "Add ORC column " << node->getMaximumColumnId()
<< " for empty tuple "
> Can we remove these VLOG_FILE statements here and below? It could get quite
Yes, they're mainly used for debugging. VLOG(3) is equal to VLOG_ROW but this
function will only be called once for an orc-scanner, not for each row or
RowBatch. Isn't it a VLOG(2) level info?
However, I still change it to VLOG(3) to respect your opinions :)
http://gerrit.cloudera.org:8080/#/c/12168/12/be/src/exec/hdfs-orc-scanner.cc@380
PS12, Line 380: const list& selected_type_ids) {
> Can this argument be const?
Yes. Should be const.
http://gerrit.cloudera.org:8080/#/c/12168/12/be/src/exec/hdfs-orc-scanner.cc@593
PS12, Line 593:
> How do we know that we're finished processing the previous batch? What happ
If we come here, the previous batch should have been drained. The relative
codes are in GetNextInternal:
// Transfer remaining values in current orc batches tracked by readers.
if (!orc_root_reader_->EndOfBatch()) {
assemble_rows_timer_.Start();
RETURN_IF_ERROR(TransferTuples(orc_root_reader_, row_batch));
assemble_rows_timer_.Stop();
if (row_batch->AtCapacity()) return Status::OK();
DCHECK(orc_root_reader_->EndOfBatch());
}
If we exited the while() loop because row_batch() was at capacity, the
remaining rows will be processed in the above codes. Since the batch size
equals to the capacity of RowBatch, the remaining rows should no more than the
capacity of RowBatch. Thus calling TransferTuples once can drain the original
batch.
http://gerrit.cloudera.org:8080/#/c/12168/12/be/src/exec/hdfs-orc-scanner.cc@638
PS12, Line 638:
> typo: materialization
Done
http://gerrit.cloudera.org:8080/#/c/12168/12/be/src/exec/hdfs-orc-scanner.cc@638
PS12, Line 638: alizeTuple()) {
> grammar: "that are not materializing tuples"
Done
http://gerrit.cloudera.org:8080/#/c/12168/12/be/src/exec/hdfs-orc-scanner.cc@688
PS12, Line 688:
> Pass in complex_col_reader by pointer since it's mutable.
It can be const. We don't change any states of it in this function. I just add
'const' for it and add const for functions we called here.
http://gerrit.cloudera.org:8080/#/c/12168/12/be/src/exec/orc-column-readers.cc
File be/src/exec/orc-column-readers.cc:
http://gerrit.cloudera.org:8080/#/c/12168/12/be/src/exec/orc-column-readers.cc@133
PS12, Line 133: }
> VLOG_FILE is also probably too verbose here - VLOG(3) would be more appropr
Done
http://gerrit.cloudera.org:8080/#/c/12168/12/be/src/exec/orc-column-readers.cc@219
PS12, Line 219: materiali
> same comment about logging
Done
http://gerrit.cloudera.org:8080/#/c/12168/12/be/src/runtime/descriptors.cc
File be/src/runtime/descriptors.cc:
http://gerrit.cloudera.org:8080/#/c/12168/12/be/src/runtime/descriptors.cc@65
PS12, Line 65: const int SchemaPathConstants::ARRAY_ITEM;
> I think it's better to define the constant value in the header and just ins
Done
http://gerrit.cloudera.org:8080/#/c/12168/12/testdata/bin/create-load-data.sh
File testdata/bin/create-load-data.sh:
http://gerrit.cloudera.org:8080/#/c/12168/12/testdata/bin/create-load-data.sh@581
PS12, Line 581: run-step "Loading nested parquet data" load-nested.log \
: ${IMPALA_HOME}/testdata/bin/load_nested.py \
: -t tpch_nested_parquet -f parquet/none ${LOAD_NESTED_ARGS:-}
: run-step "Loading nested orc data" load-nested.log \
: ${IMPALA_HOME}/testdata/bin/load_nested.py \
> I think it would make sense to specify the arguments for the call to run th
Sure. It's clearer.
http://gerrit.cloudera.org:8080/#/c/12168/12/testdata/bin/load_nested.py
File testdata/bin/load_nested.py:
http://gerrit.cloudera.org:8080/#/c/12168/12/testdata/bin/load_nested.py@286
PS12, Line 286: hive.execute("""
:
[Impala-ASF-CR] IMPALA-6503: Support reading complex types from ORC format files
Hello Zoltan Borok-Nagy, Tim Armstrong, Joe McDonnell, Impala Public Jenkins, I'd like you to reexamine a change. Please visit http://gerrit.cloudera.org:8080/12168 to look at the new patch set (#13). Change subject: IMPALA-6503: Support reading complex types from ORC format files .. IMPALA-6503: Support reading complex types from ORC format files We've supported reading primitive types from ORC files (IMPALA-5717). In this patch we add support for complex types (struct/array/map). In IMPALA-5717, we depend on the ORC lib to read ORC binaries. The ORC lib can materialize ORC column binaries into its representation (orc::ColumnVectorBatch), so we don't need to do anything about decoding/decompression in hdfs-orc-scanner. Since it already supports complex types, we'll still depend on it. What we need to add in IMPALA-6503 are two things: 1. Specify which nested columns we need to the ORC lib 2. Transform outputs of ORC lib (nested orc::ColumnVectorBatch) into Impala's representation To format the materialization, we implement several ORC column readers in hdfs-orc-scanner. Each kind of reader treats a column type. The ORC column readers differ from the Parquet readers (used in hdfs-parquet-scanner) which materializes Parquet column binaries into tuple values directly. They just need to transform outputs of the ORC lib into tuple/slot values. Tests: * Enable existing tests for complex types (test_nested_types.py, test_tpch_nested_queries.py) for ORC. Change-Id: I244dc9d2b3e425393f90e45632cb8cdbea6cf790 --- M be/src/exec/CMakeLists.txt M be/src/exec/hdfs-orc-scanner.cc M be/src/exec/hdfs-orc-scanner.h A be/src/exec/orc-column-readers.cc A be/src/exec/orc-column-readers.h A be/src/exec/orc-metadata-utils.cc A be/src/exec/orc-metadata-utils.h M be/src/runtime/descriptors.cc M fe/src/main/java/org/apache/impala/catalog/HdfsFileFormat.java A testdata/ComplexTypesTbl/README A testdata/ComplexTypesTbl/nonnullable.orc A testdata/ComplexTypesTbl/nullable.orc M testdata/bin/create-load-data.sh M testdata/bin/load_nested.py M testdata/datasets/functional/functional_schema_template.sql M testdata/datasets/functional/schema_constraints.csv M testdata/workloads/functional-planner/queries/PlannerTest/complex-types-file-formats.test M testdata/workloads/functional-query/queries/QueryTest/max-nesting-depth.test M testdata/workloads/functional-query/queries/QueryTest/nested-types-subplan.test M testdata/workloads/functional-query/queries/QueryTest/nested-types-tpch-limit.test M testdata/workloads/functional-query/queries/QueryTest/nested-types-tpch-mem-limit-single-node.test M testdata/workloads/functional-query/queries/QueryTest/nested-types-tpch-mem-limit.test M testdata/workloads/functional-query/queries/QueryTest/nested-types-tpch.test M testdata/workloads/tpch_nested/tpch_nested_core.csv M testdata/workloads/tpch_nested/tpch_nested_dimensions.csv M testdata/workloads/tpch_nested/tpch_nested_exhaustive.csv M testdata/workloads/tpch_nested/tpch_nested_pairwise.csv M tests/query_test/test_nested_types.py M tests/query_test/test_tpch_nested_queries.py 29 files changed, 1,864 insertions(+), 461 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/68/12168/13 -- To view, visit http://gerrit.cloudera.org:8080/12168 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: I244dc9d2b3e425393f90e45632cb8cdbea6cf790 Gerrit-Change-Number: 12168 Gerrit-PatchSet: 13 Gerrit-Owner: Quanlong Huang Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Joe McDonnell Gerrit-Reviewer: Quanlong Huang Gerrit-Reviewer: Tim Armstrong Gerrit-Reviewer: Zoltan Borok-Nagy
[Impala-ASF-CR] IMPALA-8183: fix test reportexecstatus retry flakiness
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/12461 ) Change subject: IMPALA-8183: fix test_reportexecstatus_retry flakiness .. Patch Set 1: Build Successful https://jenkins.impala.io/job/gerrit-code-review-checks/2090/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests. -- To view, visit http://gerrit.cloudera.org:8080/12461 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I7027a6e099c543705e5845ee0e5268f1f9a3fb05 Gerrit-Change-Number: 12461 Gerrit-PatchSet: 1 Gerrit-Owner: Thomas Marshall Gerrit-Reviewer: Andrew Sherman Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Michael Ho Gerrit-Reviewer: Thomas Marshall Gerrit-Comment-Date: Tue, 12 Feb 2019 23:36:51 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-8183: fix test reportexecstatus retry flakiness
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/12461 ) Change subject: IMPALA-8183: fix test_reportexecstatus_retry flakiness .. Patch Set 3: Code-Review+2 -- To view, visit http://gerrit.cloudera.org:8080/12461 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I7027a6e099c543705e5845ee0e5268f1f9a3fb05 Gerrit-Change-Number: 12461 Gerrit-PatchSet: 3 Gerrit-Owner: Thomas Marshall Gerrit-Reviewer: Andrew Sherman Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Michael Ho Gerrit-Reviewer: Thomas Marshall Gerrit-Comment-Date: Tue, 12 Feb 2019 23:21:00 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-8183: fix test reportexecstatus retry flakiness
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/12461 ) Change subject: IMPALA-8183: fix test_reportexecstatus_retry flakiness .. Patch Set 3: Build started: https://jenkins.impala.io/job/gerrit-verify-dryrun/3767/ DRY_RUN=false -- To view, visit http://gerrit.cloudera.org:8080/12461 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I7027a6e099c543705e5845ee0e5268f1f9a3fb05 Gerrit-Change-Number: 12461 Gerrit-PatchSet: 3 Gerrit-Owner: Thomas Marshall Gerrit-Reviewer: Andrew Sherman Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Michael Ho Gerrit-Reviewer: Thomas Marshall Gerrit-Comment-Date: Tue, 12 Feb 2019 23:21:01 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-8183: fix test reportexecstatus retry flakiness
Thomas Marshall has posted comments on this change. ( http://gerrit.cloudera.org:8080/12461 ) Change subject: IMPALA-8183: fix test_reportexecstatus_retry flakiness .. Patch Set 2: (4 comments) http://gerrit.cloudera.org:8080/#/c/12461/1//COMMIT_MSG Commit Message: http://gerrit.cloudera.org:8080/#/c/12461/1//COMMIT_MSG@12 PS1, Line 12: between retries, which was 100ms b > Please also mention that this flag is removed after IMPALA-4555. Done http://gerrit.cloudera.org:8080/#/c/12461/1//COMMIT_MSG@16 PS1, Line 16: IMPALA-2990 in this setup, but it was unlikely. > I had to think for a while, if you say "IMPALA-2990" it would be clearer Done http://gerrit.cloudera.org:8080/#/c/12461/1//COMMIT_MSG@19 PS1, Line 19: and we instead wait 'status_report_interval_ms' between retries. By > between retries Done http://gerrit.cloudera.org:8080/#/c/12461/1//COMMIT_MSG@24 PS1, Line 24: ReportExecStatus() rpcs to fail and making us hit IMPALA-2990 pretty > IMPALA-2990 Done -- To view, visit http://gerrit.cloudera.org:8080/12461 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I7027a6e099c543705e5845ee0e5268f1f9a3fb05 Gerrit-Change-Number: 12461 Gerrit-PatchSet: 2 Gerrit-Owner: Thomas Marshall Gerrit-Reviewer: Andrew Sherman Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Michael Ho Gerrit-Reviewer: Thomas Marshall Gerrit-Comment-Date: Tue, 12 Feb 2019 23:20:20 + Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-8183: fix test reportexecstatus retry flakiness
Hello Michael Ho, Andrew Sherman, Impala Public Jenkins, I'd like you to reexamine a change. Please visit http://gerrit.cloudera.org:8080/12461 to look at the new patch set (#2). Change subject: IMPALA-8183: fix test_reportexecstatus_retry flakiness .. IMPALA-8183: fix test_reportexecstatus_retry flakiness The test is designed to cause ReportExecStatus() rpcs to fail by backing up the control service queue. Previously, after a failed ReportExecStatus() we would wait 'report_status_retry_interval_ms' between retries, which was 100ms by default and wasn't touched by the test. That 100ms was right on the edge of being enough time for the coordinator to keep up with processing the reports, so that some would fail but most would succeed. It was always possible that we could hit IMPALA-2990 in this setup, but it was unlikely. Now, with IMPALA-4555 'report_status_retry_interval_ms' was removed and we instead wait 'status_report_interval_ms' between retries. By default, this is 5000ms, so it should give the coordinator even more time and make these issues less likely. However, the test sets 'status_report_interval_ms' to 10ms, which isn't nearly enough time for the coordinator to do its processing, causing lots of the ReportExecStatus() rpcs to fail and making us hit IMPALA-2990 pretty often. The solution is to set 'status_report_interval_ms' to 100ms in the test, which roughly achieves the same retry frequency as before. The same change is made to a similar test test_reportexecstatus_timeout. Testing: - Ran test_reportexecstatus_retry in a loop 400 times without seeing a failure. It previously repro-ed for me about once per 50 runs. - Manually verified that both tests are still hitting the error paths that they are supposed to be testing. Change-Id: I7027a6e099c543705e5845ee0e5268f1f9a3fb05 --- M tests/custom_cluster/test_rpc_timeout.py 1 file changed, 2 insertions(+), 2 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/61/12461/2 -- To view, visit http://gerrit.cloudera.org:8080/12461 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: I7027a6e099c543705e5845ee0e5268f1f9a3fb05 Gerrit-Change-Number: 12461 Gerrit-PatchSet: 2 Gerrit-Owner: Thomas Marshall Gerrit-Reviewer: Andrew Sherman Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Michael Ho
[Impala-ASF-CR] IMPALA-8183: fix test reportexecstatus retry flakiness
Michael Ho has posted comments on this change. ( http://gerrit.cloudera.org:8080/12461 ) Change subject: IMPALA-8183: fix test_reportexecstatus_retry flakiness .. Patch Set 1: Code-Review+2 (2 comments) http://gerrit.cloudera.org:8080/#/c/12461/1//COMMIT_MSG Commit Message: http://gerrit.cloudera.org:8080/#/c/12461/1//COMMIT_MSG@12 PS1, Line 12: 'report_status_retry_interval_ms' Please also mention that this flag is removed after IMPALA-4555. http://gerrit.cloudera.org:8080/#/c/12461/1//COMMIT_MSG@19 PS1, Line 19: Now, we wait 'status_report_interval_ms'. By default, this is 5000ms, between retries -- To view, visit http://gerrit.cloudera.org:8080/12461 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I7027a6e099c543705e5845ee0e5268f1f9a3fb05 Gerrit-Change-Number: 12461 Gerrit-PatchSet: 1 Gerrit-Owner: Thomas Marshall Gerrit-Reviewer: Andrew Sherman Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Michael Ho Gerrit-Comment-Date: Tue, 12 Feb 2019 23:15:56 + Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-8183: fix test reportexecstatus retry flakiness
Andrew Sherman has posted comments on this change. ( http://gerrit.cloudera.org:8080/12461 ) Change subject: IMPALA-8183: fix test_reportexecstatus_retry flakiness .. Patch Set 1: Code-Review+1 (2 comments) LGTM http://gerrit.cloudera.org:8080/#/c/12461/1//COMMIT_MSG Commit Message: http://gerrit.cloudera.org:8080/#/c/12461/1//COMMIT_MSG@16 PS1, Line 16: succeed. It was always possible that we could hit 2990 in this setup, I had to think for a while, if you say "IMPALA-2990" it would be clearer http://gerrit.cloudera.org:8080/#/c/12461/1//COMMIT_MSG@24 PS1, Line 24: making us hit 2990 pretty often. IMPALA-2990 -- To view, visit http://gerrit.cloudera.org:8080/12461 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I7027a6e099c543705e5845ee0e5268f1f9a3fb05 Gerrit-Change-Number: 12461 Gerrit-PatchSet: 1 Gerrit-Owner: Thomas Marshall Gerrit-Reviewer: Andrew Sherman Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Michael Ho Gerrit-Comment-Date: Tue, 12 Feb 2019 23:00:38 + Gerrit-HasComments: Yes
[Impala-ASF-CR] Add support for compiling using OpenSSL 1.1
Impala Public Jenkins has submitted this change and it was merged. ( http://gerrit.cloudera.org:8080/12420 ) Change subject: Add support for compiling using OpenSSL 1.1 .. Add support for compiling using OpenSSL 1.1 Change-Id: Iaccf1b2dedf0d957a2665df8f9afca4139754264 Reviewed-on: http://gerrit.cloudera.org:8080/12420 Reviewed-by: Impala Public Jenkins Tested-by: Impala Public Jenkins --- M be/src/util/openssl-util.cc 1 file changed, 32 insertions(+), 13 deletions(-) Approvals: Impala Public Jenkins: Looks good to me, approved; Verified -- To view, visit http://gerrit.cloudera.org:8080/12420 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: merged Gerrit-Change-Id: Iaccf1b2dedf0d957a2665df8f9afca4139754264 Gerrit-Change-Number: 12420 Gerrit-PatchSet: 5 Gerrit-Owner: hector.aco...@cloudera.com Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Michael Ho Gerrit-Reviewer: Tim Armstrong Gerrit-Reviewer: hector.aco...@cloudera.com
[Impala-ASF-CR] IMPALA-7214: [DOCS] More on decoupling impala and DataNodes
Alex Rodoni has submitted this change and it was merged. ( http://gerrit.cloudera.org:8080/12457 ) Change subject: IMPALA-7214: [DOCS] More on decoupling impala and DataNodes .. IMPALA-7214: [DOCS] More on decoupling impala and DataNodes Change-Id: I4b6f1c704c1e328af9f0beec73f8b6b61fba992e Reviewed-on: http://gerrit.cloudera.org:8080/12457 Tested-by: Impala Public Jenkins Reviewed-by: Tim Armstrong --- M docs/topics/impala_processes.xml M docs/topics/impala_troubleshooting.xml 2 files changed, 23 insertions(+), 26 deletions(-) Approvals: Impala Public Jenkins: Verified Tim Armstrong: Looks good to me, approved -- To view, visit http://gerrit.cloudera.org:8080/12457 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: merged Gerrit-Change-Id: I4b6f1c704c1e328af9f0beec73f8b6b61fba992e Gerrit-Change-Number: 12457 Gerrit-PatchSet: 2 Gerrit-Owner: Alex Rodoni Gerrit-Reviewer: Alex Rodoni Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Tim Armstrong
[Impala-ASF-CR] IMPALA-7976 : Add a flag to disable sync using events at a table level
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/12365 ) Change subject: IMPALA-7976 : Add a flag to disable sync using events at a table level .. Patch Set 8: Build Successful https://jenkins.impala.io/job/gerrit-code-review-checks/2089/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests. -- To view, visit http://gerrit.cloudera.org:8080/12365 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: If116edba51bae139bc003be858b3214c3f0104e3 Gerrit-Change-Number: 12365 Gerrit-PatchSet: 8 Gerrit-Owner: Vihang Karajgaonkar Gerrit-Reviewer: Bharath Krishna Gerrit-Reviewer: Bharath Vissapragada Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Paul Rogers Gerrit-Reviewer: Vihang Karajgaonkar Gerrit-Comment-Date: Tue, 12 Feb 2019 20:58:57 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-8186: script to configure docker network
Tim Armstrong has posted comments on this change. ( http://gerrit.cloudera.org:8080/12452 ) Change subject: IMPALA-8186: script to configure docker network .. Patch Set 3: Verified+1 This script isn't exercised by the precommit jobs and the RAT Check, etc passed. -- To view, visit http://gerrit.cloudera.org:8080/12452 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Icb4854aa951bcad7087a9653845b22ffd862057d Gerrit-Change-Number: 12452 Gerrit-PatchSet: 3 Gerrit-Owner: Tim Armstrong Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Joe McDonnell Gerrit-Reviewer: Philip Zeyliger Gerrit-Reviewer: Tim Armstrong Gerrit-Comment-Date: Tue, 12 Feb 2019 21:03:27 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-8186: script to configure docker network
Tim Armstrong has submitted this change and it was merged. ( http://gerrit.cloudera.org:8080/12452 ) Change subject: IMPALA-8186: script to configure docker network .. IMPALA-8186: script to configure docker network This automates the network setup that I did manually in http://gerrit.cloudera.org:8080/12189 After running the script it should be possible to run "./buildall.sh -format -testdata" to load test data with the right hostnames, then "start-impala-cluster.py --docker_network=network-name" to run a dockerised minicluster. Change-Id: Icb4854aa951bcad7087a9653845b22ffd862057d Reviewed-on: http://gerrit.cloudera.org:8080/12452 Reviewed-by: Philip Zeyliger Tested-by: Tim Armstrong --- A docker/configure_test_network.sh 1 file changed, 53 insertions(+), 0 deletions(-) Approvals: Philip Zeyliger: Looks good to me, approved Tim Armstrong: Verified -- To view, visit http://gerrit.cloudera.org:8080/12452 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: merged Gerrit-Change-Id: Icb4854aa951bcad7087a9653845b22ffd862057d Gerrit-Change-Number: 12452 Gerrit-PatchSet: 4 Gerrit-Owner: Tim Armstrong Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Joe McDonnell Gerrit-Reviewer: Philip Zeyliger Gerrit-Reviewer: Tim Armstrong
[Impala-ASF-CR] IMPALA-8186: script to configure docker network
Tim Armstrong has posted comments on this change. (
http://gerrit.cloudera.org:8080/12452 )
Change subject: IMPALA-8186: script to configure docker network
..
Patch Set 2:
(1 comment)
http://gerrit.cloudera.org:8080/#/c/12452/2/docker/configure_test_network.sh
File docker/configure_test_network.sh:
http://gerrit.cloudera.org:8080/#/c/12452/2/docker/configure_test_network.sh@50
PS2, Line 50: echo "export INTERNAL_LISTEN_HOST=${GATEWAY}" >>
bin/impala-config-local.sh
> There's some sort of "--bip" flag that I've used to make Docker not knock m
Locally I've passed in --gateway and --subnet to "docker network create" to
force it to use particular IPs. I'm going to think if just exposing those flags
is the right way to approach this or if there's a nicer way to handle it.
--
To view, visit http://gerrit.cloudera.org:8080/12452
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Icb4854aa951bcad7087a9653845b22ffd862057d
Gerrit-Change-Number: 12452
Gerrit-PatchSet: 2
Gerrit-Owner: Tim Armstrong
Gerrit-Reviewer: Impala Public Jenkins
Gerrit-Reviewer: Joe McDonnell
Gerrit-Reviewer: Philip Zeyliger
Gerrit-Reviewer: Tim Armstrong
Gerrit-Comment-Date: Tue, 12 Feb 2019 21:02:35 +
Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-7214: [DOCS] More on decoupling impala and DataNodes
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/12457 ) Change subject: IMPALA-7214: [DOCS] More on decoupling impala and DataNodes .. Patch Set 1: Verified+1 Build Successful https://jenkins.impala.io/job/gerrit-docs-auto-test/217/ : Doc tests passed. -- To view, visit http://gerrit.cloudera.org:8080/12457 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I4b6f1c704c1e328af9f0beec73f8b6b61fba992e Gerrit-Change-Number: 12457 Gerrit-PatchSet: 1 Gerrit-Owner: Alex Rodoni Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Tim Armstrong Gerrit-Comment-Date: Tue, 12 Feb 2019 20:50:22 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-8186: script to configure docker network
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/12452 ) Change subject: IMPALA-8186: script to configure docker network .. Patch Set 3: Build Successful https://jenkins.impala.io/job/gerrit-code-review-checks/2088/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests. -- To view, visit http://gerrit.cloudera.org:8080/12452 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Icb4854aa951bcad7087a9653845b22ffd862057d Gerrit-Change-Number: 12452 Gerrit-PatchSet: 3 Gerrit-Owner: Tim Armstrong Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Joe McDonnell Gerrit-Reviewer: Philip Zeyliger Gerrit-Reviewer: Tim Armstrong Gerrit-Comment-Date: Tue, 12 Feb 2019 20:36:33 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-5861: fix RowsRead for zero-slot table scan
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/12332 ) Change subject: IMPALA-5861: fix RowsRead for zero-slot table scan .. Patch Set 7: Verified+1 -- To view, visit http://gerrit.cloudera.org:8080/12332 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I7a927c6a4f0b8055608cb7a5e2b550a1610cef89 Gerrit-Change-Number: 12332 Gerrit-PatchSet: 7 Gerrit-Owner: Tim Armstrong Gerrit-Reviewer: Csaba Ringhofer Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Tim Armstrong Gerrit-Reviewer: Zoltan Borok-Nagy Gerrit-Comment-Date: Tue, 12 Feb 2019 20:37:44 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-7214: [DOCS] More on decoupling impala and DataNodes
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/12457 ) Change subject: IMPALA-7214: [DOCS] More on decoupling impala and DataNodes .. Patch Set 1: Build Started https://jenkins.impala.io/job/gerrit-docs-auto-test/217/ Testing docs change - this change appears to modify docs/ and no code. This is experimental - please report any issues to tarmstr...@cloudera.com or on this JIRA: IMPALA-7317 -- To view, visit http://gerrit.cloudera.org:8080/12457 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I4b6f1c704c1e328af9f0beec73f8b6b61fba992e Gerrit-Change-Number: 12457 Gerrit-PatchSet: 1 Gerrit-Owner: Alex Rodoni Gerrit-Reviewer: Impala Public Jenkins Gerrit-Comment-Date: Tue, 12 Feb 2019 20:42:07 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-7214: [DOCS] More on decoupling impala and DataNodes
Alex Rodoni has uploaded this change for review. ( http://gerrit.cloudera.org:8080/12457 Change subject: IMPALA-7214: [DOCS] More on decoupling impala and DataNodes .. IMPALA-7214: [DOCS] More on decoupling impala and DataNodes Change-Id: I4b6f1c704c1e328af9f0beec73f8b6b61fba992e --- M docs/topics/impala_processes.xml M docs/topics/impala_troubleshooting.xml 2 files changed, 23 insertions(+), 26 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/57/12457/1 -- To view, visit http://gerrit.cloudera.org:8080/12457 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newchange Gerrit-Change-Id: I4b6f1c704c1e328af9f0beec73f8b6b61fba992e Gerrit-Change-Number: 12457 Gerrit-PatchSet: 1 Gerrit-Owner: Alex Rodoni
[Impala-ASF-CR] IMPALA-5861: fix RowsRead for zero-slot table scan
Impala Public Jenkins has submitted this change and it was merged. ( http://gerrit.cloudera.org:8080/12332 ) Change subject: IMPALA-5861: fix RowsRead for zero-slot table scan .. IMPALA-5861: fix RowsRead for zero-slot table scan Testing: Added regression test based on JIRA and a targeted test for all HDFS file formats. Change-Id: I7a927c6a4f0b8055608cb7a5e2b550a1610cef89 Reviewed-on: http://gerrit.cloudera.org:8080/12332 Reviewed-by: Impala Public Jenkins Tested-by: Impala Public Jenkins --- M be/src/exec/parquet/hdfs-parquet-scanner.cc M testdata/workloads/functional-query/queries/QueryTest/mixed-format.test M testdata/workloads/functional-query/queries/QueryTest/scanners.test 3 files changed, 126 insertions(+), 1 deletion(-) Approvals: Impala Public Jenkins: Looks good to me, approved; Verified -- To view, visit http://gerrit.cloudera.org:8080/12332 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: merged Gerrit-Change-Id: I7a927c6a4f0b8055608cb7a5e2b550a1610cef89 Gerrit-Change-Number: 12332 Gerrit-PatchSet: 8 Gerrit-Owner: Tim Armstrong Gerrit-Reviewer: Csaba Ringhofer Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Tim Armstrong Gerrit-Reviewer: Zoltan Borok-Nagy
[Impala-ASF-CR] IMPALA-8186: script to configure docker network
Tim Armstrong has posted comments on this change. ( http://gerrit.cloudera.org:8080/12452 ) Change subject: IMPALA-8186: script to configure docker network .. Patch Set 2: (1 comment) http://gerrit.cloudera.org:8080/#/c/12452/2//COMMIT_MSG Commit Message: http://gerrit.cloudera.org:8080/#/c/12452/2//COMMIT_MSG@15 PS2, Line 15: "start-impala-cluster.py --docker_network=network-name" > Should the network name end up in the env variables too? I think maybe eventually we'd want to be able to toggle docker mode via impala-config.sh; there isn't a good place to put it right now. Currently you have to put it in TEST_START_CLUSTER_ARGS and similar. -- To view, visit http://gerrit.cloudera.org:8080/12452 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Icb4854aa951bcad7087a9653845b22ffd862057d Gerrit-Change-Number: 12452 Gerrit-PatchSet: 2 Gerrit-Owner: Tim Armstrong Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Joe McDonnell Gerrit-Reviewer: Philip Zeyliger Gerrit-Reviewer: Tim Armstrong Gerrit-Comment-Date: Tue, 12 Feb 2019 20:10:34 + Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-8186: script to configure docker network
Hello Philip Zeyliger, Joe McDonnell, Impala Public Jenkins, I'd like you to reexamine a change. Please visit http://gerrit.cloudera.org:8080/12452 to look at the new patch set (#3). Change subject: IMPALA-8186: script to configure docker network .. IMPALA-8186: script to configure docker network This automates the network setup that I did manually in http://gerrit.cloudera.org:8080/12189 After running the script it should be possible to run "./buildall.sh -format -testdata" to load test data with the right hostnames, then "start-impala-cluster.py --docker_network=network-name" to run a dockerised minicluster. Change-Id: Icb4854aa951bcad7087a9653845b22ffd862057d --- A docker/configure_test_network.sh 1 file changed, 53 insertions(+), 0 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/52/12452/3 -- To view, visit http://gerrit.cloudera.org:8080/12452 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: Icb4854aa951bcad7087a9653845b22ffd862057d Gerrit-Change-Number: 12452 Gerrit-PatchSet: 3 Gerrit-Owner: Tim Armstrong Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Joe McDonnell Gerrit-Reviewer: Philip Zeyliger Gerrit-Reviewer: Tim Armstrong
[Impala-ASF-CR] IMPALA-8182: Add single-node plan to PlanCtx
Bharath Vissapragada has posted comments on this change. (
http://gerrit.cloudera.org:8080/12436 )
Change subject: IMPALA-8182: Add single-node plan to PlanCtx
..
Patch Set 1:
(6 comments)
Few minor comments, lgtm otherwise.
http://gerrit.cloudera.org:8080/#/c/12436/1/fe/src/main/java/org/apache/impala/planner/Planner.java
File fe/src/main/java/org/apache/impala/planner/Planner.java:
http://gerrit.cloudera.org:8080/#/c/12436/1/fe/src/main/java/org/apache/impala/planner/Planner.java@66
PS1, Line 66: public static final class QueryPlan {
Comment.
http://gerrit.cloudera.org:8080/#/c/12436/1/fe/src/main/java/org/apache/impala/planner/Planner.java@99
PS1, Line 99: Returns a list of plan fragments for executing an analyzed parse
tree.
:* May return a single-node or distributed executable plan. If
enabled (through a
:* query option), computes runtime filters for dynamic
partition pruning.
Update
http://gerrit.cloudera.org:8080/#/c/12436/1/fe/src/main/java/org/apache/impala/planner/Planner.java@141
PS1, Line 141: plan.rootFragment_ = rootFragment;
Set this after L179 instead of doing it twice.
http://gerrit.cloudera.org:8080/#/c/12436/1/fe/src/main/java/org/apache/impala/planner/Planner.java@244
PS1, Line 244: Return a list of plans
Update.
http://gerrit.cloudera.org:8080/#/c/12436/1/fe/src/main/java/org/apache/impala/planner/Planner.java@258
PS1, Line 258: plan.parallelPlans_ = parallelPlans;
Do it in L253?
http://gerrit.cloudera.org:8080/#/c/12436/1/fe/src/main/java/org/apache/impala/service/Frontend.java
File fe/src/main/java/org/apache/impala/service/Frontend.java:
http://gerrit.cloudera.org:8080/#/c/12436/1/fe/src/main/java/org/apache/impala/service/Frontend.java@212
PS1, Line 212: public List getPlan() { return planRoots_; }
Looks like there are very limited callers for this. Redo them to use the new
way?
--
To view, visit http://gerrit.cloudera.org:8080/12436
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I983473f259df8af17d8dd38e497fe33e839cd0dc
Gerrit-Change-Number: 12436
Gerrit-PatchSet: 1
Gerrit-Owner: Paul Rogers
Gerrit-Reviewer: Bharath Vissapragada
Gerrit-Reviewer: Impala Public Jenkins
Gerrit-Reviewer: Paul Rogers
Gerrit-Comment-Date: Tue, 12 Feb 2019 19:42:10 +
Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-7961: Avoid adding unmodified objects to DDL response
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/12428 ) Change subject: IMPALA-7961: Avoid adding unmodified objects to DDL response .. Patch Set 4: Build Successful https://jenkins.impala.io/job/gerrit-code-review-checks/2087/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests. -- To view, visit http://gerrit.cloudera.org:8080/12428 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: If3e914b70ba796c9b224e9dea559b8c40aa25d83 Gerrit-Change-Number: 12428 Gerrit-PatchSet: 4 Gerrit-Owner: Bharath Vissapragada Gerrit-Reviewer: Bharath Vissapragada Gerrit-Reviewer: Fredy Wijaya Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Paul Rogers Gerrit-Comment-Date: Tue, 12 Feb 2019 19:27:58 + Gerrit-HasComments: No
[Impala-ASF-CR] Turn off shell debug tracing for create-load-data.sh
Joe McDonnell has posted comments on this change. ( http://gerrit.cloudera.org:8080/12398 ) Change subject: Turn off shell debug tracing for create-load-data.sh .. Patch Set 2: Code-Review+2 Rebase, carry +2 -- To view, visit http://gerrit.cloudera.org:8080/12398 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I524ec48d0264f6180a13d6d068832809bcc86596 Gerrit-Change-Number: 12398 Gerrit-PatchSet: 2 Gerrit-Owner: Joe McDonnell Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Joe McDonnell Gerrit-Reviewer: Philip Zeyliger Gerrit-Reviewer: Tim Armstrong Gerrit-Comment-Date: Tue, 12 Feb 2019 19:25:43 + Gerrit-HasComments: No
[Impala-ASF-CR] Turn off shell debug tracing for create-load-data.sh
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/12398 ) Change subject: Turn off shell debug tracing for create-load-data.sh .. Patch Set 2: Build started: https://jenkins.impala.io/job/gerrit-verify-dryrun/3766/ DRY_RUN=false -- To view, visit http://gerrit.cloudera.org:8080/12398 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I524ec48d0264f6180a13d6d068832809bcc86596 Gerrit-Change-Number: 12398 Gerrit-PatchSet: 2 Gerrit-Owner: Joe McDonnell Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Joe McDonnell Gerrit-Reviewer: Philip Zeyliger Gerrit-Reviewer: Tim Armstrong Gerrit-Comment-Date: Tue, 12 Feb 2019 19:26:10 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-7935: Disable /catalog object in local catalog mode.
Bharath Vissapragada has posted comments on this change. (
http://gerrit.cloudera.org:8080/12443 )
Change subject: IMPALA-7935: Disable /catalog_object in local catalog mode.
..
Patch Set 3:
(5 comments)
http://gerrit.cloudera.org:8080/#/c/12443/3/be/src/service/impala-http-handler.cc
File be/src/service/impala-http-handler.cc:
http://gerrit.cloudera.org:8080/#/c/12443/3/be/src/service/impala-http-handler.cc@100
PS3, Line 100: if(!FLAGS_use_local_catalog)
nit: we generally use braces for multiline if blocks.
if (condition) {
.
}
Add a comment why we disable this end-point?
http://gerrit.cloudera.org:8080/#/c/12443/3/be/src/service/impala-http-handler.cc@540
PS3, Line 540: if(!FLAGS_use_local_catalog){
Add a comment that this creates hrefs for /catalog_object which is disabled in
local catalog mode?
http://gerrit.cloudera.org:8080/#/c/12443/3/www/catalog.tmpl
File www/catalog.tmpl:
http://gerrit.cloudera.org:8080/#/c/12443/3/www/catalog.tmpl@42
PS3, Line 42: {{^fqtn}}
Use 2 space formatting for nested blocks to be consistent with rest of the
file. (multiple places)
http://gerrit.cloudera.org:8080/#/c/12443/3/www/catalog.tmpl@45
PS3, Line 45: ?fqtn
This works as expected right? I see the upstream docs has something like
{{#fqtn?}}
{{/fqtn}}
https://mustache.github.io/mustache.5.html
http://gerrit.cloudera.org:8080/#/c/12443/3/www/catalog.tmpl@126
PS3, Line 126:
:
: {{name}}
: {{num_tables}}
table(s)
:
Just stumbled upon this We probably need to remove this too? In which case we
need to pass the "local_catalog" flag. sorry for the confusion. You may have to
redo that part.
--
To view, visit http://gerrit.cloudera.org:8080/12443
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Ia04797b32964c2edaa2e860dcf510d6f9cccd81c
Gerrit-Change-Number: 12443
Gerrit-PatchSet: 3
Gerrit-Owner: Anurag Mantripragada
Gerrit-Reviewer: Anurag Mantripragada
Gerrit-Reviewer: Bharath Vissapragada
Gerrit-Reviewer: Impala Public Jenkins
Gerrit-Comment-Date: Tue, 12 Feb 2019 19:20:02 +
Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-8186: script to configure docker network
Philip Zeyliger has posted comments on this change. (
http://gerrit.cloudera.org:8080/12452 )
Change subject: IMPALA-8186: script to configure docker network
..
Patch Set 2:
(4 comments)
No objections here; just thinking out loud about how some of this stuff works.
Fine by me to make incremental progress.
http://gerrit.cloudera.org:8080/#/c/12452/2//COMMIT_MSG
Commit Message:
http://gerrit.cloudera.org:8080/#/c/12452/2//COMMIT_MSG@15
PS2, Line 15: "start-impala-cluster.py --docker_network=network-name"
Should the network name end up in the env variables too?
http://gerrit.cloudera.org:8080/#/c/12452/2/docker/configure_test_network.sh
File docker/configure_test_network.sh:
http://gerrit.cloudera.org:8080/#/c/12452/2/docker/configure_test_network.sh@23
PS2, Line 23: # cluster configurations need to be regenerated, all minicluster
processes restarted,
nit, extra space.
http://gerrit.cloudera.org:8080/#/c/12452/2/docker/configure_test_network.sh@45
PS2, Line 45: GATEWAY=$(docker network inspect "$NETWORK_NAME" | grep -o
'"Gateway": "[0-9\.]*"' | \
: grep -o "[0-9.]*")
I don't know if the following makes you any happier:
$docker network inspect bridge -f '{{(index .IPAM.Config 0).Gateway}}'
10.250.0.1
http://gerrit.cloudera.org:8080/#/c/12452/2/docker/configure_test_network.sh@50
PS2, Line 50: echo "export INTERNAL_LISTEN_HOST=${GATEWAY}" >>
bin/impala-config-local.sh
This clearly persists across restarts. Does the "docker network create" command
persist across reboots? Is Docker deterministic about the gateway?
--
To view, visit http://gerrit.cloudera.org:8080/12452
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Icb4854aa951bcad7087a9653845b22ffd862057d
Gerrit-Change-Number: 12452
Gerrit-PatchSet: 2
Gerrit-Owner: Tim Armstrong
Gerrit-Reviewer: Impala Public Jenkins
Gerrit-Reviewer: Joe McDonnell
Gerrit-Reviewer: Philip Zeyliger
Gerrit-Reviewer: Tim Armstrong
Gerrit-Comment-Date: Tue, 12 Feb 2019 19:02:19 +
Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-7961: Avoid adding unmodified objects to DDL response
Hello Paul Rogers, Impala Public Jenkins, I'd like you to reexamine a change. Please visit http://gerrit.cloudera.org:8080/12428 to look at the new patch set (#4). Change subject: IMPALA-7961: Avoid adding unmodified objects to DDL response .. IMPALA-7961: Avoid adding unmodified objects to DDL response When a DDL is processed, we typically add the affected (added/removed) objects to the response TCatalogUpdateResult struct. This response is processed on the coordinator and the changes are applied locally. When SYNC_DDL is enabled, the Catalog server also includes a topic version number that should include all the affected objects so that the coordinator can wait for that miniumum topic version to be applied on all other coordinators before returning the control back to the user. This covering topic version is calculated by looking at the topic update log, which contains all the in-flight updates (and to an extent past updates) that are perodically GC'ed. Bug: In certain cases like CREATE TBL IF NOT EXISTS, we could end up adding objects to the DDL response which haven't been modified in a while (> TOPIC_UPDATE_LOG_GC_FREQUENCY) and hence could be potentially GC'ed from the TopicUpdateLog. This means that the Catalog server wouldn't be able to find a covering topic update version and eventually gives up throwing an error as described in the jira. Fix: The idea here is to avoid any additions that include objects that haven't been modified by this DDL. Also added some additional diagnostic logging that could've simplified debugging an issue like this. Testing: Since this is a racy bug, I could only repro it by forcing frequent topic update log GCs along with a specific sequence of actions. Couldn't reproduce it with the patch. Change-Id: If3e914b70ba796c9b224e9dea559b8c40aa25d83 --- M fe/src/main/java/org/apache/impala/catalog/CatalogServiceCatalog.java M fe/src/main/java/org/apache/impala/catalog/FeCatalogUtils.java M fe/src/main/java/org/apache/impala/catalog/TopicUpdateLog.java M fe/src/main/java/org/apache/impala/service/CatalogOpExecutor.java 4 files changed, 48 insertions(+), 31 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/28/12428/4 -- To view, visit http://gerrit.cloudera.org:8080/12428 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: If3e914b70ba796c9b224e9dea559b8c40aa25d83 Gerrit-Change-Number: 12428 Gerrit-PatchSet: 4 Gerrit-Owner: Bharath Vissapragada Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Paul Rogers
[Impala-ASF-CR] IMPALA-7961: Avoid adding unmodified objects to DDL response
Bharath Vissapragada has posted comments on this change. (
http://gerrit.cloudera.org:8080/12428 )
Change subject: IMPALA-7961: Avoid adding unmodified objects to DDL response
..
Patch Set 4:
(5 comments)
http://gerrit.cloudera.org:8080/#/c/12428/3/fe/src/main/java/org/apache/impala/catalog/CatalogServiceCatalog.java
File fe/src/main/java/org/apache/impala/catalog/CatalogServiceCatalog.java:
http://gerrit.cloudera.org:8080/#/c/12428/3/fe/src/main/java/org/apache/impala/catalog/CatalogServiceCatalog.java@2243
PS3, Line 2243: "catalog objects. Updated objects: %s,
deleted objects: %s",
> Nit: There is a trick you can use here. The logger has its own (clunky) for
Probably doesn't matter since this is in "error" severity which is almost
always enabled in most clusters. I surrounded this with if {} block since the
function args seem complex. Otherwise, for simple arguments parameterized
logging seems the right away. (thanks for the pointers)
http://gerrit.cloudera.org:8080/#/c/12428/3/fe/src/main/java/org/apache/impala/catalog/TopicUpdateLog.java
File fe/src/main/java/org/apache/impala/catalog/TopicUpdateLog.java:
http://gerrit.cloudera.org:8080/#/c/12428/3/fe/src/main/java/org/apache/impala/catalog/TopicUpdateLog.java@107
PS3, Line 107: if (numTopicUpdatesToGc_ == 0) {
> Here you can use the formatting trick mentioned previously.
Done
http://gerrit.cloudera.org:8080/#/c/12428/3/fe/src/main/java/org/apache/impala/service/CatalogOpExecutor.java
File fe/src/main/java/org/apache/impala/service/CatalogOpExecutor.java:
http://gerrit.cloudera.org:8080/#/c/12428/3/fe/src/main/java/org/apache/impala/service/CatalogOpExecutor.java@1765
PS3, Line 1765:* exceptions encountered during the create.
> Nit: @return true if ...
Done
http://gerrit.cloudera.org:8080/#/c/12428/3/fe/src/main/java/org/apache/impala/service/CatalogOpExecutor.java@1766
PS3, Line 1766:* @return true if a new table has been created with the
given params, false
> Might be useful to explain the response. I see a comment saying we are not
Tried to clarify in the code comments. Let me know if that is not clear.
http://gerrit.cloudera.org:8080/#/c/12428/3/fe/src/main/java/org/apache/impala/service/CatalogOpExecutor.java@1782
PS3, Line 1782: // Tables that are not modified by this DDL are not added
to the response, since
> Nit: Using the log formatting can avoid the if-statement
Done
--
To view, visit http://gerrit.cloudera.org:8080/12428
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: If3e914b70ba796c9b224e9dea559b8c40aa25d83
Gerrit-Change-Number: 12428
Gerrit-PatchSet: 4
Gerrit-Owner: Bharath Vissapragada
Gerrit-Reviewer: Bharath Vissapragada
Gerrit-Reviewer: Impala Public Jenkins
Gerrit-Reviewer: Paul Rogers
Gerrit-Comment-Date: Tue, 12 Feb 2019 18:45:05 +
Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-7961: Avoid adding unmodified objects to DDL response
Impala Public Jenkins has posted comments on this change. (
http://gerrit.cloudera.org:8080/12428 )
Change subject: IMPALA-7961: Avoid adding unmodified objects to DDL response
..
Patch Set 4:
(1 comment)
http://gerrit.cloudera.org:8080/#/c/12428/4/fe/src/main/java/org/apache/impala/catalog/CatalogServiceCatalog.java
File fe/src/main/java/org/apache/impala/catalog/CatalogServiceCatalog.java:
http://gerrit.cloudera.org:8080/#/c/12428/4/fe/src/main/java/org/apache/impala/catalog/CatalogServiceCatalog.java@2242
PS4, Line 2242: LOG.error(String.format("Couldn't retrieve the
covering topic version for " +
line too long (91 > 90)
--
To view, visit http://gerrit.cloudera.org:8080/12428
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: If3e914b70ba796c9b224e9dea559b8c40aa25d83
Gerrit-Change-Number: 12428
Gerrit-PatchSet: 4
Gerrit-Owner: Bharath Vissapragada
Gerrit-Reviewer: Impala Public Jenkins
Gerrit-Reviewer: Paul Rogers
Gerrit-Comment-Date: Tue, 12 Feb 2019 18:44:17 +
Gerrit-HasComments: Yes
[Impala-ASF-CR] Add support for compiling using OpenSSL 1.1
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/12420 ) Change subject: Add support for compiling using OpenSSL 1.1 .. Patch Set 4: Build started: https://jenkins.impala.io/job/gerrit-verify-dryrun/3765/ DRY_RUN=false -- To view, visit http://gerrit.cloudera.org:8080/12420 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Iaccf1b2dedf0d957a2665df8f9afca4139754264 Gerrit-Change-Number: 12420 Gerrit-PatchSet: 4 Gerrit-Owner: hector.aco...@cloudera.com Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Michael Ho Gerrit-Reviewer: Tim Armstrong Gerrit-Reviewer: hector.aco...@cloudera.com Gerrit-Comment-Date: Tue, 12 Feb 2019 18:40:28 + Gerrit-HasComments: No
[Impala-ASF-CR] Add support for compiling using OpenSSL 1.1
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/12420 ) Change subject: Add support for compiling using OpenSSL 1.1 .. Patch Set 4: Code-Review+2 -- To view, visit http://gerrit.cloudera.org:8080/12420 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Iaccf1b2dedf0d957a2665df8f9afca4139754264 Gerrit-Change-Number: 12420 Gerrit-PatchSet: 4 Gerrit-Owner: hector.aco...@cloudera.com Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Michael Ho Gerrit-Reviewer: Tim Armstrong Gerrit-Reviewer: hector.aco...@cloudera.com Gerrit-Comment-Date: Tue, 12 Feb 2019 18:40:27 + Gerrit-HasComments: No
[Impala-ASF-CR] Add support for compiling using OpenSSL 1.1
Michael Ho has posted comments on this change. ( http://gerrit.cloudera.org:8080/12420 ) Change subject: Add support for compiling using OpenSSL 1.1 .. Patch Set 3: Code-Review+2 (1 comment) http://gerrit.cloudera.org:8080/#/c/12420/1/be/src/util/openssl-util.cc File be/src/util/openssl-util.cc: http://gerrit.cloudera.org:8080/#/c/12420/1/be/src/util/openssl-util.cc@76 PS1, Line 76: #else : // OpenSSL 1.1+ doesn't let us detect the supported TLS version at runtime. Assume : // that the OpenSSL library we're linked against supports only up to TLS1.2 : > I guess OpenSSL is expecting that callers will pass in the version they wan Fair enough. Agree that if we compile against OpenSSL 1.1, we would expect the OpenSSL version which can be linked against Impala at runtime should support TLS 1.2. -- To view, visit http://gerrit.cloudera.org:8080/12420 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Iaccf1b2dedf0d957a2665df8f9afca4139754264 Gerrit-Change-Number: 12420 Gerrit-PatchSet: 3 Gerrit-Owner: hector.aco...@cloudera.com Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Michael Ho Gerrit-Reviewer: Tim Armstrong Gerrit-Reviewer: hector.aco...@cloudera.com Gerrit-Comment-Date: Tue, 12 Feb 2019 18:33:00 + Gerrit-HasComments: Yes
[Impala-ASF-CR] Add support for compiling using OpenSSL 1.1
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/12420 ) Change subject: Add support for compiling using OpenSSL 1.1 .. Patch Set 3: Build Successful https://jenkins.impala.io/job/gerrit-code-review-checks/2086/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests. -- To view, visit http://gerrit.cloudera.org:8080/12420 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Iaccf1b2dedf0d957a2665df8f9afca4139754264 Gerrit-Change-Number: 12420 Gerrit-PatchSet: 3 Gerrit-Owner: hector.aco...@cloudera.com Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Michael Ho Gerrit-Reviewer: Tim Armstrong Gerrit-Reviewer: hector.aco...@cloudera.com Gerrit-Comment-Date: Tue, 12 Feb 2019 17:54:03 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-7087: Read Parquet decimal columns with lower precision/scale than table metadata
Zoltan Borok-Nagy has posted comments on this change. ( http://gerrit.cloudera.org:8080/12163 ) Change subject: IMPALA-7087: Read Parquet decimal columns with lower precision/scale than table metadata .. Patch Set 3: (3 comments) Just noticed the comments about the README, but I plan to take a deeper look soon. http://gerrit.cloudera.org:8080/#/c/12163/3/testdata/data/README File testdata/data/README: http://gerrit.cloudera.org:8080/#/c/12163/3/testdata/data/README@183 PS3, Line 183: decimal_stored_as_int32.parquet: : Parquet file generated by Spark 2.3.1 that contains decimals stored as int32. : Impala needs to be able to read such values (IMPALA-5542) : : decimal_stored_as_int64.parquet: : Parquet file generated by Spark 2.3.1 that contains decimals stored as int64. : Impala needs to be able to read such values (IMPALA-5542) Here. http://gerrit.cloudera.org:8080/#/c/12163/1/tests/query_test/test_scanners.py File tests/query_test/test_scanners.py: http://gerrit.cloudera.org:8080/#/c/12163/1/tests/query_test/test_scanners.py@842 PS1, Line 842: pressed_page_size_summaries: > Thanks for taking care of the other files too! The decimal_stored_as_* files were in the README already. http://gerrit.cloudera.org:8080/#/c/12163/1/tests/query_test/test_scanners.py@848 PS1, Line 848: " limit 10") > I do not see this file in the patch and it is not mentioned in the README. This was already in the README. -- To view, visit http://gerrit.cloudera.org:8080/12163 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Iafc8efd12379a39756e3e70f022a81a636dadb61 Gerrit-Change-Number: 12163 Gerrit-PatchSet: 3 Gerrit-Owner: Sahil Takiar Gerrit-Reviewer: Csaba Ringhofer Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Lars Volker Gerrit-Reviewer: Sahil Takiar Gerrit-Reviewer: Zoltan Borok-Nagy Gerrit-Comment-Date: Tue, 12 Feb 2019 17:51:45 + Gerrit-HasComments: Yes
[Impala-ASF-CR] Add support for compiling using OpenSSL 1.1
Tim Armstrong has posted comments on this change. ( http://gerrit.cloudera.org:8080/12420 ) Change subject: Add support for compiling using OpenSSL 1.1 .. Patch Set 1: (1 comment) http://gerrit.cloudera.org:8080/#/c/12420/1/be/src/util/openssl-util.cc File be/src/util/openssl-util.cc: http://gerrit.cloudera.org:8080/#/c/12420/1/be/src/util/openssl-util.cc@76 PS1, Line 76: #else : return TLS1_2_VERSION; : #endif : } > That results in: I guess OpenSSL is expecting that callers will pass in the version they want and rely on OpenSSL returning an error, rather than asking the library what it supports, i.e. https://www.openssl.org/docs/man1.1.0/man3/SSL_CTX_set_min_proto_version.html We support up to TLS1.2 with the newer OpenSSL, and we only document --ssl_minimum_version as supporting up to that, so that seems fine. Can you leave a comment, something like "OpenSSL 1.1+ doesn't let us detect the supported TLS version at runtime. Assume that the OpenSSL library we're linked against supports only up to TLS1.2. -- To view, visit http://gerrit.cloudera.org:8080/12420 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Iaccf1b2dedf0d957a2665df8f9afca4139754264 Gerrit-Change-Number: 12420 Gerrit-PatchSet: 1 Gerrit-Owner: hector.aco...@cloudera.com Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Michael Ho Gerrit-Reviewer: Tim Armstrong Gerrit-Reviewer: hector.aco...@cloudera.com Gerrit-Comment-Date: Tue, 12 Feb 2019 16:44:23 + Gerrit-HasComments: Yes
[Impala-ASF-CR] Turn off shell debug tracing for create-load-data.sh
Tim Armstrong has posted comments on this change. ( http://gerrit.cloudera.org:8080/12398 ) Change subject: Turn off shell debug tracing for create-load-data.sh .. Patch Set 1: Code-Review+2 Oops, sorry about leaving this in there. -- To view, visit http://gerrit.cloudera.org:8080/12398 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I524ec48d0264f6180a13d6d068832809bcc86596 Gerrit-Change-Number: 12398 Gerrit-PatchSet: 1 Gerrit-Owner: Joe McDonnell Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Philip Zeyliger Gerrit-Reviewer: Tim Armstrong Gerrit-Comment-Date: Tue, 12 Feb 2019 16:49:57 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-8187: UDF samples hide symbols by default
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/12451 ) Change subject: IMPALA-8187: UDF samples hide symbols by default .. Patch Set 3: (1 comment) http://gerrit.cloudera.org:8080/#/c/12451/3/tests/query_test/test_udfs.py File tests/query_test/test_udfs.py: http://gerrit.cloudera.org:8080/#/c/12451/3/tests/query_test/test_udfs.py@442 PS3, Line 442: u flake8: E126 continuation line over-indented for hanging indent -- To view, visit http://gerrit.cloudera.org:8080/12451 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ie17b74b9bce9cc6962393017879b65409ce23b28 Gerrit-Change-Number: 12451 Gerrit-PatchSet: 3 Gerrit-Owner: Tim Armstrong Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Michael Ho Gerrit-Reviewer: Zoram Thanga Gerrit-Comment-Date: Tue, 12 Feb 2019 16:15:28 + Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-5861: fix RowsRead for zero-slot table scan
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/12332 ) Change subject: IMPALA-5861: fix RowsRead for zero-slot table scan .. Patch Set 7: Code-Review+2 -- To view, visit http://gerrit.cloudera.org:8080/12332 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I7a927c6a4f0b8055608cb7a5e2b550a1610cef89 Gerrit-Change-Number: 12332 Gerrit-PatchSet: 7 Gerrit-Owner: Tim Armstrong Gerrit-Reviewer: Csaba Ringhofer Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Tim Armstrong Gerrit-Reviewer: Zoltan Borok-Nagy Gerrit-Comment-Date: Tue, 12 Feb 2019 16:19:02 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-5861: fix RowsRead for zero-slot table scan
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/12332 ) Change subject: IMPALA-5861: fix RowsRead for zero-slot table scan .. Patch Set 7: Build started: https://jenkins.impala.io/job/gerrit-verify-dryrun/3764/ DRY_RUN=false -- To view, visit http://gerrit.cloudera.org:8080/12332 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I7a927c6a4f0b8055608cb7a5e2b550a1610cef89 Gerrit-Change-Number: 12332 Gerrit-PatchSet: 7 Gerrit-Owner: Tim Armstrong Gerrit-Reviewer: Csaba Ringhofer Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Tim Armstrong Gerrit-Reviewer: Zoltan Borok-Nagy Gerrit-Comment-Date: Tue, 12 Feb 2019 16:19:03 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-8187: UDF samples hide symbols by default
Tim Armstrong has uploaded this change for review. ( http://gerrit.cloudera.org:8080/12451 Change subject: IMPALA-8187: UDF samples hide symbols by default .. IMPALA-8187: UDF samples hide symbols by default Testing: Ran UDF tests, confirmed that changing the compile flags without adding IMPALA_UDF_EXPORT caused them to fail because they couldn't find the symbols. Added a test to check that symbols are actually hidden. Change-Id: Ie17b74b9bce9cc6962393017879b65409ce23b28 --- M be/src/testutil/CMakeLists.txt M be/src/testutil/test-udas.cc M be/src/testutil/test-udfs.cc M be/src/udf/udf.h M be/src/udf_samples/CMakeLists.txt M be/src/udf_samples/hyperloglog-uda.cc M be/src/udf_samples/uda-sample.cc M be/src/udf_samples/udf-sample.cc M tests/query_test/test_udfs.py 9 files changed, 168 insertions(+), 1 deletion(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/51/12451/3 -- To view, visit http://gerrit.cloudera.org:8080/12451 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newchange Gerrit-Change-Id: Ie17b74b9bce9cc6962393017879b65409ce23b28 Gerrit-Change-Number: 12451 Gerrit-PatchSet: 3 Gerrit-Owner: Tim Armstrong Gerrit-Reviewer: Impala Public Jenkins
[Impala-ASF-CR] IMPALA-7935: Disable /catalog object in local catalog mode.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/12443 ) Change subject: IMPALA-7935: Disable /catalog_object in local catalog mode. .. Patch Set 3: Build Successful https://jenkins.impala.io/job/gerrit-code-review-checks/2084/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests. -- To view, visit http://gerrit.cloudera.org:8080/12443 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ia04797b32964c2edaa2e860dcf510d6f9cccd81c Gerrit-Change-Number: 12443 Gerrit-PatchSet: 3 Gerrit-Owner: Anurag Mantripragada Gerrit-Reviewer: Anurag Mantripragada Gerrit-Reviewer: Bharath Vissapragada Gerrit-Reviewer: Impala Public Jenkins Gerrit-Comment-Date: Tue, 12 Feb 2019 15:21:34 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-7935: Disable /catalog object in local catalog mode.
Anurag Mantripragada has uploaded a new patch set (#3). ( http://gerrit.cloudera.org:8080/12443 ) Change subject: IMPALA-7935: Disable /catalog_object in local catalog mode. .. IMPALA-7935: Disable /catalog_object in local catalog mode. getTCatalogObject() is not supported in local catalog mode since metadata is partially fetched on demand. Removed hyperlinks to the /catalog_object endpoints when local_catalog_mode is enabled. Testing: Added a test in test_local_catalog to verify /catalog_mode endpoint is disabled. Change-Id: Ia04797b32964c2edaa2e860dcf510d6f9cccd81c --- M be/src/service/impala-http-handler.cc M tests/custom_cluster/test_local_catalog.py M www/catalog.tmpl 3 files changed, 34 insertions(+), 8 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/43/12443/3 -- To view, visit http://gerrit.cloudera.org:8080/12443 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: Ia04797b32964c2edaa2e860dcf510d6f9cccd81c Gerrit-Change-Number: 12443 Gerrit-PatchSet: 3 Gerrit-Owner: Anurag Mantripragada Gerrit-Reviewer: Anurag Mantripragada Gerrit-Reviewer: Bharath Vissapragada Gerrit-Reviewer: Impala Public Jenkins
[Impala-ASF-CR] IMPALA-7935: Disable /catalog object in local catalog mode.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/12443 ) Change subject: IMPALA-7935: Disable /catalog_object in local catalog mode. .. Patch Set 2: Build Failed https://jenkins.impala.io/job/gerrit-code-review-checks/2083/ : Initial code review checks failed. See linked job for details on the failure. -- To view, visit http://gerrit.cloudera.org:8080/12443 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ia04797b32964c2edaa2e860dcf510d6f9cccd81c Gerrit-Change-Number: 12443 Gerrit-PatchSet: 2 Gerrit-Owner: Anurag Mantripragada Gerrit-Reviewer: Anurag Mantripragada Gerrit-Reviewer: Bharath Vissapragada Gerrit-Reviewer: Impala Public Jenkins Gerrit-Comment-Date: Tue, 12 Feb 2019 14:29:55 + Gerrit-HasComments: No
[Impala-ASF-CR](2.x) IMPALA-6035: Add query options to limit thread reservation
Impala Public Jenkins has submitted this change and it was merged. ( http://gerrit.cloudera.org:8080/12429 ) Change subject: IMPALA-6035: Add query options to limit thread reservation .. IMPALA-6035: Add query options to limit thread reservation Adds two options: THREAD_RESERVATION_LIMIT and THREAD_RESERVATION_AGGREGATE_LIMIT, which are both enforced by admission control based on planner resource requirements and the schedule. The mechanism used is the same as the minimum reservation checks. THREAD_RESERVATION_LIMIT limits the total number of reserved threads in fragments scheduled on a single backend. THREAD_RESERVATION_AGGREGATE_LIMIT limits the sum of reserved threads across all fragments. This also slightly improves the minimum reservation error message to include the host name. Testing: Added end-to-end tests that exercise the code paths. Ran core tests. Change-Id: I5b5bbbdad5cd6b24442eb6c99a4d38c2ad710007 Reviewed-on: http://gerrit.cloudera.org:8080/10365 Reviewed-by: Impala Public Jenkins Tested-by: Impala Public Jenkins Reviewed-on: http://gerrit.cloudera.org:8080/12429 Reviewed-by: Quanlong Huang --- M be/src/scheduling/admission-controller.cc M be/src/scheduling/query-schedule.h M be/src/scheduling/scheduler.cc M be/src/service/query-options-test.cc M be/src/service/query-options.cc M be/src/service/query-options.h M common/thrift/ImpalaInternalService.thrift M common/thrift/ImpalaService.thrift M testdata/workloads/functional-query/queries/QueryTest/admission-reject-min-reservation.test M testdata/workloads/functional-query/queries/QueryTest/runtime_row_filters.test A testdata/workloads/functional-query/queries/QueryTest/thread-limits.test A tests/query_test/test_resource_limits.py 12 files changed, 254 insertions(+), 29 deletions(-) Approvals: Quanlong Huang: Looks good to me, approved Impala Public Jenkins: Verified -- To view, visit http://gerrit.cloudera.org:8080/12429 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: 2.x Gerrit-MessageType: merged Gerrit-Change-Id: I5b5bbbdad5cd6b24442eb6c99a4d38c2ad710007 Gerrit-Change-Number: 12429 Gerrit-PatchSet: 4 Gerrit-Owner: Quanlong Huang Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Quanlong Huang Gerrit-Reviewer: Tim Armstrong
[Impala-ASF-CR] IMPALA-5861: fix RowsRead for zero-slot table scan
Zoltan Borok-Nagy has posted comments on this change. ( http://gerrit.cloudera.org:8080/12332 ) Change subject: IMPALA-5861: fix RowsRead for zero-slot table scan .. Patch Set 6: Code-Review+2 -- To view, visit http://gerrit.cloudera.org:8080/12332 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I7a927c6a4f0b8055608cb7a5e2b550a1610cef89 Gerrit-Change-Number: 12332 Gerrit-PatchSet: 6 Gerrit-Owner: Tim Armstrong Gerrit-Reviewer: Csaba Ringhofer Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Tim Armstrong Gerrit-Reviewer: Zoltan Borok-Nagy Gerrit-Comment-Date: Tue, 12 Feb 2019 10:16:13 + Gerrit-HasComments: No
