[Impala-ASF-CR] IMPALA-10554: Block updates when row-filter/column-mask is enabled for the user
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/17230 ) Change subject: IMPALA-10554: Block updates when row-filter/column-mask is enabled for the user .. Patch Set 7: Verified+1 -- To view, visit http://gerrit.cloudera.org:8080/17230 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I1c899f2ec24b895867cbf2cf9ed23bc7b5a77326 Gerrit-Change-Number: 17230 Gerrit-PatchSet: 7 Gerrit-Owner: Quanlong Huang Gerrit-Reviewer: Aman Sinha Gerrit-Reviewer: Csaba Ringhofer Gerrit-Reviewer: Fang-Yu Rao Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Quanlong Huang Gerrit-Comment-Date: Wed, 31 Mar 2021 05:38:49 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-10554: Block updates when row-filter/column-mask is enabled for the user
Impala Public Jenkins has submitted this change and it was merged. ( http://gerrit.cloudera.org:8080/17230 ) Change subject: IMPALA-10554: Block updates when row-filter/column-mask is enabled for the user .. IMPALA-10554: Block updates when row-filter/column-mask is enabled for the user Per RANGER-1087 and RANGER-1100, table updates(e.g. insert, delete, truncate, upsert, alter, etc.) should be blocked when row-filtering or column-masking policy is enabled for the user. This patch adds the check for any row-filtering or column-masking policy on the table and rejects the update operation if any of them exisits. Tests: - Add FE unit tests - Add audit tests - Add e2e tests Change-Id: I1c899f2ec24b895867cbf2cf9ed23bc7b5a77326 Reviewed-on: http://gerrit.cloudera.org:8080/17230 Reviewed-by: Impala Public Jenkins Tested-by: Impala Public Jenkins --- M fe/src/main/java/org/apache/impala/authorization/AuthorizableTable.java M fe/src/main/java/org/apache/impala/authorization/BaseAuthorizationChecker.java M fe/src/main/java/org/apache/impala/authorization/Privilege.java M fe/src/main/java/org/apache/impala/authorization/ranger/RangerAuthorizationChecker.java M fe/src/main/java/org/apache/impala/authorization/ranger/RangerImpalaPlugin.java M fe/src/test/java/org/apache/impala/authorization/AuthorizationStmtTest.java M fe/src/test/java/org/apache/impala/authorization/AuthorizationTestBase.java M fe/src/test/java/org/apache/impala/authorization/ranger/RangerAuditLogTest.java M testdata/workloads/functional-query/queries/QueryTest/ranger_column_masking.test M testdata/workloads/functional-query/queries/QueryTest/ranger_column_masking_and_row_filtering.test M testdata/workloads/functional-query/queries/QueryTest/ranger_row_filtering.test 11 files changed, 602 insertions(+), 116 deletions(-) Approvals: Impala Public Jenkins: Looks good to me, approved; Verified -- To view, visit http://gerrit.cloudera.org:8080/17230 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: merged Gerrit-Change-Id: I1c899f2ec24b895867cbf2cf9ed23bc7b5a77326 Gerrit-Change-Number: 17230 Gerrit-PatchSet: 8 Gerrit-Owner: Quanlong Huang Gerrit-Reviewer: Aman Sinha Gerrit-Reviewer: Csaba Ringhofer Gerrit-Reviewer: Fang-Yu Rao Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Quanlong Huang
[Impala-ASF-CR] Bump up the GBN number to 11920537
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/17243 ) Change subject: Bump up the GBN number to 11920537 .. Patch Set 1: Verified-1 Build failed: https://jenkins.impala.io/job/gerrit-verify-dryrun/7024/ -- To view, visit http://gerrit.cloudera.org:8080/17243 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: If88ceeaffc94e5bedf2c9953122109e20663f743 Gerrit-Change-Number: 17243 Gerrit-PatchSet: 1 Gerrit-Owner: Vihang Karajgaonkar Gerrit-Reviewer: Impala Public Jenkins Gerrit-Comment-Date: Wed, 31 Mar 2021 05:24:19 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-10606 (part 2): Clean up ordering of requirements.txt
Joe McDonnell has submitted this change and it was merged. ( http://gerrit.cloudera.org:8080/17229 ) Change subject: IMPALA-10606 (part 2): Clean up ordering of requirements.txt .. IMPALA-10606 (part 2): Clean up ordering of requirements.txt This is a followup to the original IMPALA-10606 that reorders the requirements.txt alphabetically. No versions changed as part of this. Change-Id: I2f13ec8f8af80c4bac5da30d08a2ea4c56806d27 Reviewed-on: http://gerrit.cloudera.org:8080/17229 Reviewed-by: Joe McDonnell Tested-by: Joe McDonnell --- M infra/python/deps/requirements.txt 1 file changed, 22 insertions(+), 34 deletions(-) Approvals: Joe McDonnell: Looks good to me, approved; Verified -- To view, visit http://gerrit.cloudera.org:8080/17229 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: merged Gerrit-Change-Id: I2f13ec8f8af80c4bac5da30d08a2ea4c56806d27 Gerrit-Change-Number: 17229 Gerrit-PatchSet: 4 Gerrit-Owner: Joe McDonnell Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Joe McDonnell Gerrit-Reviewer: Laszlo Gaal
[Impala-ASF-CR] IMPALA-10608: Update kudu-python version and remove some unused packages
Joe McDonnell has submitted this change and it was merged. ( http://gerrit.cloudera.org:8080/17231 ) Change subject: IMPALA-10608: Update kudu-python version and remove some unused packages .. IMPALA-10608: Update kudu-python version and remove some unused packages This updates kudu-python to version 1.14.0 (from 1.2.0). As part of this, it disables ccache for bootstrap_virtualenv.py. ccache wasn't working anyway, because pip install uses random temporary directories. It also needs to copy a few files to the build directory for the Kudu install. The advantage to upgrading is that the new version no longer has a numpy dependency. Additionally, this modifies a few minor packages: - virtualenv moves to the latest version prior to the rewrite that accompanied version 20 (i.e. 16.10.7). - setuptools moves to the last version that supports python 2.7 (44.1.1) - remove botos3, ipython, and ordereddict These changes speed up installing the virtualenv Before: real3m11.956s user2m49.620s sys 0m14.266s After: real1m38.798s user1m33.591s sys 0m8.112s Testing: - Hand tests, GVO run Change-Id: Ib47770df9e46de448fe2bffef7abe2c3aa942fb9 Reviewed-on: http://gerrit.cloudera.org:8080/17231 Reviewed-by: Joe McDonnell Tested-by: Impala Public Jenkins --- M bin/compare_branches.py D bin/impala-ipython M infra/python/bootstrap_virtualenv.py M infra/python/deps/kudu-requirements.txt M infra/python/deps/requirements.txt M infra/python/deps/setuptools-requirements.txt M tests/comparison/cluster.py 7 files changed, 15 insertions(+), 60 deletions(-) Approvals: Joe McDonnell: Looks good to me, approved Impala Public Jenkins: Verified -- To view, visit http://gerrit.cloudera.org:8080/17231 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: merged Gerrit-Change-Id: Ib47770df9e46de448fe2bffef7abe2c3aa942fb9 Gerrit-Change-Number: 17231 Gerrit-PatchSet: 4 Gerrit-Owner: Joe McDonnell Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Joe McDonnell Gerrit-Reviewer: Laszlo Gaal
[Impala-ASF-CR] IMPALA-10606: Simplify impala-python virtualenv bootstrapping
Joe McDonnell has submitted this change and it was merged. ( http://gerrit.cloudera.org:8080/17226 ) Change subject: IMPALA-10606: Simplify impala-python virtualenv bootstrapping .. IMPALA-10606: Simplify impala-python virtualenv bootstrapping Bootstrapping the impala-python virtualenv requires multiple rounds of pip installs with different sets of requirements. This consolidates the requirements.txt, stage2-requirements.txt, and compiled-requirements.txt into a single requirements.txt. This will make it easier to upgrade python packages. This also splits out setuptools into its own setuptools-requirements.txt. Setuptools is used during the pip install for several of the dependencies. Recent versions of setuptools do not support Python 2, but some of the install tools (like easy_install) don't know how to pick a version of setuptools that works with Python 2. Splitting it out to its own requirements file lets us pin the version. To make review easier, this does not change any of the versions of the dependencies. It also leaves the stage2-requirements.txt and compiled-requirements.txt split out in separate sections of requirements.txt. These will later be turned into a single alphabetical list. Testing: - Tested impala-python locally - Ran GVO Change-Id: I8e920e5a257f1e0613065685078624a50d59bf2e Reviewed-on: http://gerrit.cloudera.org:8080/17226 Reviewed-by: Joe McDonnell Tested-by: Joe McDonnell --- M infra/python/bootstrap_virtualenv.py M infra/python/deps/pip_download.py M infra/python/deps/requirements.txt R infra/python/deps/setuptools-requirements.txt D infra/python/deps/stage2-requirements.txt 5 files changed, 88 insertions(+), 121 deletions(-) Approvals: Joe McDonnell: Looks good to me, approved; Verified -- To view, visit http://gerrit.cloudera.org:8080/17226 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: merged Gerrit-Change-Id: I8e920e5a257f1e0613065685078624a50d59bf2e Gerrit-Change-Number: 17226 Gerrit-PatchSet: 5 Gerrit-Owner: Joe McDonnell Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Joe McDonnell Gerrit-Reviewer: Laszlo Gaal
[Impala-ASF-CR] IMPALA-10606: Simplify impala-python virtualenv bootstrapping
Joe McDonnell has posted comments on this change. ( http://gerrit.cloudera.org:8080/17226 ) Change subject: IMPALA-10606: Simplify impala-python virtualenv bootstrapping .. Patch Set 4: Verified+1 GVO on whole stack succeeded -- To view, visit http://gerrit.cloudera.org:8080/17226 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I8e920e5a257f1e0613065685078624a50d59bf2e Gerrit-Change-Number: 17226 Gerrit-PatchSet: 4 Gerrit-Owner: Joe McDonnell Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Joe McDonnell Gerrit-Reviewer: Laszlo Gaal Gerrit-Comment-Date: Wed, 31 Mar 2021 03:17:20 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-10606 (part 2): Clean up ordering of requirements.txt
Joe McDonnell has posted comments on this change. ( http://gerrit.cloudera.org:8080/17229 ) Change subject: IMPALA-10606 (part 2): Clean up ordering of requirements.txt .. Patch Set 3: Verified+1 GVO on whole stack succeeded. -- To view, visit http://gerrit.cloudera.org:8080/17229 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I2f13ec8f8af80c4bac5da30d08a2ea4c56806d27 Gerrit-Change-Number: 17229 Gerrit-PatchSet: 3 Gerrit-Owner: Joe McDonnell Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Joe McDonnell Gerrit-Reviewer: Laszlo Gaal Gerrit-Comment-Date: Wed, 31 Mar 2021 03:17:09 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-10608: Update kudu-python version and remove some unused packages
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/17231 ) Change subject: IMPALA-10608: Update kudu-python version and remove some unused packages .. Patch Set 3: Verified+1 -- To view, visit http://gerrit.cloudera.org:8080/17231 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ib47770df9e46de448fe2bffef7abe2c3aa942fb9 Gerrit-Change-Number: 17231 Gerrit-PatchSet: 3 Gerrit-Owner: Joe McDonnell Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Joe McDonnell Gerrit-Reviewer: Laszlo Gaal Gerrit-Comment-Date: Wed, 31 Mar 2021 03:07:43 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-10579: Fix usage of RemoteIterator in FileSystemUtil
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/17171 ) Change subject: IMPALA-10579: Fix usage of RemoteIterator in FileSystemUtil .. Patch Set 6: Code-Review+2 -- To view, visit http://gerrit.cloudera.org:8080/17171 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I859bd4f976c51a34eb6a03cefd2ddcdf11656cea Gerrit-Change-Number: 17171 Gerrit-PatchSet: 6 Gerrit-Owner: Quanlong Huang Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Kurt Deschler Gerrit-Reviewer: Quanlong Huang Gerrit-Reviewer: Vihang Karajgaonkar Gerrit-Comment-Date: Wed, 31 Mar 2021 02:55:18 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-10579: Fix usage of RemoteIterator in FileSystemUtil
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/17171 ) Change subject: IMPALA-10579: Fix usage of RemoteIterator in FileSystemUtil .. Patch Set 6: Build started: https://jenkins.impala.io/job/gerrit-verify-dryrun/7026/ DRY_RUN=false -- To view, visit http://gerrit.cloudera.org:8080/17171 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I859bd4f976c51a34eb6a03cefd2ddcdf11656cea Gerrit-Change-Number: 17171 Gerrit-PatchSet: 6 Gerrit-Owner: Quanlong Huang Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Kurt Deschler Gerrit-Reviewer: Quanlong Huang Gerrit-Reviewer: Vihang Karajgaonkar Gerrit-Comment-Date: Wed, 31 Mar 2021 02:55:19 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-10613 : Standup HMS thrift server in Catalog
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/17244 ) Change subject: IMPALA-10613 : Standup HMS thrift server in Catalog .. Patch Set 3: Build Successful https://jenkins.impala.io/job/gerrit-code-review-checks/8472/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests. -- To view, visit http://gerrit.cloudera.org:8080/17244 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I1b306f91d63cb5137c178e8e72b6e8b578a907b5 Gerrit-Change-Number: 17244 Gerrit-PatchSet: 3 Gerrit-Owner: Vihang Karajgaonkar Gerrit-Reviewer: Aman Sinha Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Quanlong Huang Gerrit-Reviewer: Vihang Karajgaonkar Gerrit-Comment-Date: Tue, 30 Mar 2021 23:51:16 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-10554: Block updates when row-filter/column-mask is enabled for the user
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/17230 ) Change subject: IMPALA-10554: Block updates when row-filter/column-mask is enabled for the user .. Patch Set 7: Build started: https://jenkins.impala.io/job/gerrit-verify-dryrun/7025/ DRY_RUN=false -- To view, visit http://gerrit.cloudera.org:8080/17230 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I1c899f2ec24b895867cbf2cf9ed23bc7b5a77326 Gerrit-Change-Number: 17230 Gerrit-PatchSet: 7 Gerrit-Owner: Quanlong Huang Gerrit-Reviewer: Aman Sinha Gerrit-Reviewer: Csaba Ringhofer Gerrit-Reviewer: Fang-Yu Rao Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Quanlong Huang Gerrit-Comment-Date: Tue, 30 Mar 2021 23:48:03 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-10554: Block updates when row-filter/column-mask is enabled for the user
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/17230 ) Change subject: IMPALA-10554: Block updates when row-filter/column-mask is enabled for the user .. Patch Set 7: Code-Review+2 -- To view, visit http://gerrit.cloudera.org:8080/17230 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I1c899f2ec24b895867cbf2cf9ed23bc7b5a77326 Gerrit-Change-Number: 17230 Gerrit-PatchSet: 7 Gerrit-Owner: Quanlong Huang Gerrit-Reviewer: Aman Sinha Gerrit-Reviewer: Csaba Ringhofer Gerrit-Reviewer: Fang-Yu Rao Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Quanlong Huang Gerrit-Comment-Date: Tue, 30 Mar 2021 23:48:02 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-10397: De-flake test single workload
Impala Public Jenkins has submitted this change and it was merged. ( http://gerrit.cloudera.org:8080/17239 ) Change subject: IMPALA-10397: De-flake test_single_workload .. IMPALA-10397: De-flake test_single_workload This patch removes a flaky part of the test that relies on query completion rate. Since we are already verifying that number of healthy executor groups increases, this additional check is not adding much to the test. Change-Id: I6f75afdbe676d9dd6922b6ba8aa1919daa161947 Reviewed-on: http://gerrit.cloudera.org:8080/17239 Reviewed-by: Impala Public Jenkins Tested-by: Impala Public Jenkins --- M tests/custom_cluster/test_auto_scaling.py 1 file changed, 3 insertions(+), 23 deletions(-) Approvals: Impala Public Jenkins: Looks good to me, approved; Verified -- To view, visit http://gerrit.cloudera.org:8080/17239 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: merged Gerrit-Change-Id: I6f75afdbe676d9dd6922b6ba8aa1919daa161947 Gerrit-Change-Number: 17239 Gerrit-PatchSet: 3 Gerrit-Owner: Bikramjeet Vig Gerrit-Reviewer: Andrew Sherman Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Quanlong Huang
[Impala-ASF-CR] IMPALA-10397: De-flake test single workload
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/17239 ) Change subject: IMPALA-10397: De-flake test_single_workload .. Patch Set 2: Verified+1 -- To view, visit http://gerrit.cloudera.org:8080/17239 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I6f75afdbe676d9dd6922b6ba8aa1919daa161947 Gerrit-Change-Number: 17239 Gerrit-PatchSet: 2 Gerrit-Owner: Bikramjeet Vig Gerrit-Reviewer: Andrew Sherman Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Quanlong Huang Gerrit-Comment-Date: Tue, 30 Mar 2021 23:46:47 + Gerrit-HasComments: No
[Impala-ASF-CR] Bump up the GBN number to 11920537
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/17243 ) Change subject: Bump up the GBN number to 11920537 .. Patch Set 1: Build started: https://jenkins.impala.io/job/gerrit-verify-dryrun/7024/ DRY_RUN=false -- To view, visit http://gerrit.cloudera.org:8080/17243 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: If88ceeaffc94e5bedf2c9953122109e20663f743 Gerrit-Change-Number: 17243 Gerrit-PatchSet: 1 Gerrit-Owner: Vihang Karajgaonkar Gerrit-Reviewer: Impala Public Jenkins Gerrit-Comment-Date: Tue, 30 Mar 2021 23:44:18 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-10554: Block updates when row-filter/column-mask is enabled for the user
Aman Sinha has posted comments on this change. ( http://gerrit.cloudera.org:8080/17230 ) Change subject: IMPALA-10554: Block updates when row-filter/column-mask is enabled for the user .. Patch Set 6: Code-Review+2 -- To view, visit http://gerrit.cloudera.org:8080/17230 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I1c899f2ec24b895867cbf2cf9ed23bc7b5a77326 Gerrit-Change-Number: 17230 Gerrit-PatchSet: 6 Gerrit-Owner: Quanlong Huang Gerrit-Reviewer: Aman Sinha Gerrit-Reviewer: Csaba Ringhofer Gerrit-Reviewer: Fang-Yu Rao Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Quanlong Huang Gerrit-Comment-Date: Tue, 30 Mar 2021 23:34:02 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-10579: Fix usage of RemoteIterator in FileSystemUtil
Vihang Karajgaonkar has posted comments on this change. ( http://gerrit.cloudera.org:8080/17171 ) Change subject: IMPALA-10579: Fix usage of RemoteIterator in FileSystemUtil .. Patch Set 5: Code-Review+2 -- To view, visit http://gerrit.cloudera.org:8080/17171 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I859bd4f976c51a34eb6a03cefd2ddcdf11656cea Gerrit-Change-Number: 17171 Gerrit-PatchSet: 5 Gerrit-Owner: Quanlong Huang Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Kurt Deschler Gerrit-Reviewer: Quanlong Huang Gerrit-Reviewer: Vihang Karajgaonkar Gerrit-Comment-Date: Tue, 30 Mar 2021 23:28:54 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-10613 : Standup HMS thrift server in Catalog
Impala Public Jenkins has posted comments on this change. (
http://gerrit.cloudera.org:8080/17244 )
Change subject: IMPALA-10613 : Standup HMS thrift server in Catalog
..
Patch Set 3:
(10 comments)
http://gerrit.cloudera.org:8080/#/c/17244/3/fe/src/main/java/org/apache/impala/catalog/CatalogHMSAPIHelper.java
File fe/src/main/java/org/apache/impala/catalog/CatalogHMSAPIHelper.java:
http://gerrit.cloudera.org:8080/#/c/17244/3/fe/src/main/java/org/apache/impala/catalog/CatalogHMSAPIHelper.java@121
PS3, Line 121: GetPartialCatalogObjectRequestBuilder reqBuilder = new
GetPartialCatalogObjectRequestBuilder()
line too long (98 > 90)
http://gerrit.cloudera.org:8080/#/c/17244/3/fe/src/main/java/org/apache/impala/catalog/CatalogHMSAPIHelper.java@225
PS3, Line 225: GetPartialCatalogObjectRequestBuilder catalogReq = new
GetPartialCatalogObjectRequestBuilder()
line too long (98 > 90)
http://gerrit.cloudera.org:8080/#/c/17244/3/fe/src/main/java/org/apache/impala/catalog/CatalogHMSAPIHelper.java@519
PS3, Line 519: + "fallback path. Time taken: {} msec",
getPartsResult.getPartitionsSize(),
line too long (91 > 90)
http://gerrit.cloudera.org:8080/#/c/17244/3/fe/src/main/java/org/apache/impala/catalog/CatalogHMSAPIHelper.java@523
PS3, Line 523: + "fallback path. Time taken: {} msec",
getPartsResult.getPartitionsSize(),
line too long (91 > 90)
http://gerrit.cloudera.org:8080/#/c/17244/3/fe/src/main/java/org/apache/impala/catalog/metastore/MetastoreServiceHandler.java
File
fe/src/main/java/org/apache/impala/catalog/metastore/MetastoreServiceHandler.java:
http://gerrit.cloudera.org:8080/#/c/17244/3/fe/src/main/java/org/apache/impala/catalog/metastore/MetastoreServiceHandler.java@1237
PS3, Line 1237: public GetPartitionsPsWithAuthResponse
get_partitions_ps_with_auth_req(GetPartitionsPsWithAuthRequest req)
line too long (107 > 90)
http://gerrit.cloudera.org:8080/#/c/17244/3/fe/src/main/java/org/apache/impala/catalog/metastore/MetastoreServiceHandler.java@1632
PS3, Line 1632: throws NoSuchObjectException, InvalidObjectException,
MetaException, InvalidInputException, TException {
line too long (110 > 90)
http://gerrit.cloudera.org:8080/#/c/17244/3/fe/src/main/java/org/apache/impala/catalog/metastore/MetastoreServiceHandler.java@1643
PS3, Line 1643: throws NoSuchObjectException, MetaException,
InvalidObjectException, InvalidInputException, TException {
line too long (110 > 90)
http://gerrit.cloudera.org:8080/#/c/17244/3/fe/src/main/java/org/apache/impala/catalog/metastore/MetastoreServiceHandler.java@1654
PS3, Line 1654: throws NoSuchObjectException, MetaException,
InvalidObjectException, InvalidInputException, TException {
line too long (110 > 90)
http://gerrit.cloudera.org:8080/#/c/17244/3/fe/src/main/java/org/apache/impala/catalog/metastore/MetastoreServiceHandler.java@1664
PS3, Line 1664: throws AlreadyExistsException, InvalidObjectException,
MetaException, NoSuchObjectException, TException {
line too long (111 > 90)
http://gerrit.cloudera.org:8080/#/c/17244/3/fe/src/main/java/org/apache/impala/catalog/metastore/MetastoreServiceHandler.java@2435
PS3, Line 2435: public WMCreateOrDropTriggerToPoolMappingResponse
create_or_drop_wm_trigger_to_pool_mapping(
line too long (94 > 90)
--
To view, visit http://gerrit.cloudera.org:8080/17244
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I1b306f91d63cb5137c178e8e72b6e8b578a907b5
Gerrit-Change-Number: 17244
Gerrit-PatchSet: 3
Gerrit-Owner: Vihang Karajgaonkar
Gerrit-Reviewer: Aman Sinha
Gerrit-Reviewer: Impala Public Jenkins
Gerrit-Reviewer: Quanlong Huang
Gerrit-Reviewer: Vihang Karajgaonkar
Gerrit-Comment-Date: Tue, 30 Mar 2021 23:22:01 +
Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-10613 : Standup HMS thrift server in Catalog
Vihang Karajgaonkar has uploaded a new patch set (#3). ( http://gerrit.cloudera.org:8080/17244 ) Change subject: IMPALA-10613 : Standup HMS thrift server in Catalog .. IMPALA-10613 : Standup HMS thrift server in Catalog This change adds the basic infrastructure to start the HMS server in Catalog. It introduces a new configuration (--start_hms_server) along with a config for the port and starts a HMS thrift server in the CatalogServiceCatalog instance. Currently, all the HMS APIs are "pass-through" to the backing HMS service. Except for the following 3 HMS APIs which can be used to request a table and its partitions. Additionally, there is another flag (--enable_catalogd_hms_cache) which can be used to disable the usage of catalogd for providing the table and partition metadata. This contribution was done by Kishen Das. 1. get_table_req 2. get_partitions_by_expr 3. get_partitions_by_names In case of get_partitions_by_expr we need the hive-exec jar to be present in the classpath since it needs to load the PartitionExpressionProxy to push down the partition predicates to the HMS database. In case of get_table_req if column statistics are requested, we return the table level statistics. Additionally, this patch adds a new configuration fallback_to_hms_on_errors for the catalog which is used to determine if the Catalog falls back to HMS service in case of errors while executing the API. This is useful for testing purposes. In order to expose the file-metadata for the tables and partitions, HMS API changes were made to add the filemetadata fields to table and partitions. In case of transactional tables, the file-metadata which is returned is consistent with the provided ValidWriteIdList in the API call. There are a few TODOs which will be done in follow up tasks: 1. Add support for SASL support. 2. Pin the hive_metastore.thrift in the code so that any changes to HMS APIs in the hive branch doesn't break Catalog's HMS service. Testing: 1. Added a new end-to-end test which starts the HMS service in Catalog and runs some basic HMS APIs against it. 2. Ran a modification of TestRemoteHiveMetastore in the Hive code base and confirmed most tests are working. There were some test failures but they are unrelated since the test assumes an empty warehouse whereas we run against the actual HMS service running in the mini-cluster. Change-Id: I1b306f91d63cb5137c178e8e72b6e8b578a907b5 --- M be/src/catalog/catalog-server.cc M be/src/common/global-flags.cc M be/src/util/backend-gflag-util.cc M common/thrift/BackendGflags.thrift M common/thrift/CatalogService.thrift A fe/src/main/java/org/apache/impala/catalog/CatalogHMSAPIHelper.java M fe/src/main/java/org/apache/impala/catalog/CatalogServiceCatalog.java A fe/src/main/java/org/apache/impala/catalog/GetPartialCatalogObjectRequestBuilder.java M fe/src/main/java/org/apache/impala/catalog/HdfsTable.java M fe/src/main/java/org/apache/impala/catalog/ParallelFileMetadataLoader.java M fe/src/main/java/org/apache/impala/catalog/Table.java A fe/src/main/java/org/apache/impala/catalog/metastore/CatalogHMSClientUtils.java A fe/src/main/java/org/apache/impala/catalog/metastore/CatalogMetastoreServer.java A fe/src/main/java/org/apache/impala/catalog/metastore/CatalogMetastoreServiceHandler.java A fe/src/main/java/org/apache/impala/catalog/metastore/ICatalogMetastoreServer.java A fe/src/main/java/org/apache/impala/catalog/metastore/MetastoreServiceHandler.java A fe/src/main/java/org/apache/impala/catalog/metastore/NoOpCatalogMetastoreServer.java M fe/src/main/java/org/apache/impala/service/BackendConfig.java A fe/src/test/java/org/apache/impala/catalog/metastore/CatalogHMSFileMetadataTest.java A fe/src/test/java/org/apache/impala/catalog/metastore/EnableCatalogdHMSCacheFlagTest.java M fe/src/test/java/org/apache/impala/testutil/CatalogServiceTestCatalog.java M tests/common/impala_test_suite.py A tests/custom_cluster/test_metastore_service.py 23 files changed, 5,542 insertions(+), 22 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/44/17244/3 -- To view, visit http://gerrit.cloudera.org:8080/17244 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: I1b306f91d63cb5137c178e8e72b6e8b578a907b5 Gerrit-Change-Number: 17244 Gerrit-PatchSet: 3 Gerrit-Owner: Vihang Karajgaonkar Gerrit-Reviewer: Aman Sinha Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Quanlong Huang Gerrit-Reviewer: Vihang Karajgaonkar
[Impala-ASF-CR] IMPALA-10613 : Standup HMS thrift server in Catalog
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/17244 ) Change subject: IMPALA-10613 : Standup HMS thrift server in Catalog .. Patch Set 2: Build Successful https://jenkins.impala.io/job/gerrit-code-review-checks/8471/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests. -- To view, visit http://gerrit.cloudera.org:8080/17244 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I1b306f91d63cb5137c178e8e72b6e8b578a907b5 Gerrit-Change-Number: 17244 Gerrit-PatchSet: 2 Gerrit-Owner: Vihang Karajgaonkar Gerrit-Reviewer: Aman Sinha Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Quanlong Huang Gerrit-Reviewer: Vihang Karajgaonkar Gerrit-Comment-Date: Tue, 30 Mar 2021 23:13:29 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-10613 : Standup HMS thrift server in Catalog
Impala Public Jenkins has posted comments on this change. (
http://gerrit.cloudera.org:8080/17244 )
Change subject: IMPALA-10613 : Standup HMS thrift server in Catalog
..
Patch Set 2:
(9 comments)
http://gerrit.cloudera.org:8080/#/c/17244/2/fe/src/main/java/org/apache/impala/catalog/CatalogHMSAPIHelper.java
File fe/src/main/java/org/apache/impala/catalog/CatalogHMSAPIHelper.java:
http://gerrit.cloudera.org:8080/#/c/17244/2/fe/src/main/java/org/apache/impala/catalog/CatalogHMSAPIHelper.java@121
PS2, Line 121: GetPartialCatalogObjectRequestBuilder reqBuilder = new
GetPartialCatalogObjectRequestBuilder()
line too long (98 > 90)
http://gerrit.cloudera.org:8080/#/c/17244/2/fe/src/main/java/org/apache/impala/catalog/CatalogHMSAPIHelper.java@225
PS2, Line 225: GetPartialCatalogObjectRequestBuilder catalogReq = new
GetPartialCatalogObjectRequestBuilder()
line too long (98 > 90)
http://gerrit.cloudera.org:8080/#/c/17244/2/fe/src/main/java/org/apache/impala/catalog/CatalogHMSAPIHelper.java@519
PS2, Line 519: + "fallback path. Time taken: {} msec",
getPartsResult.getPartitionsSize(),
line too long (91 > 90)
http://gerrit.cloudera.org:8080/#/c/17244/2/fe/src/main/java/org/apache/impala/catalog/CatalogHMSAPIHelper.java@523
PS2, Line 523: + "fallback path. Time taken: {} msec",
getPartsResult.getPartitionsSize(),
line too long (91 > 90)
http://gerrit.cloudera.org:8080/#/c/17244/2/fe/src/main/java/org/apache/impala/catalog/metastore/MetastoreServiceHandler.java
File
fe/src/main/java/org/apache/impala/catalog/metastore/MetastoreServiceHandler.java:
http://gerrit.cloudera.org:8080/#/c/17244/2/fe/src/main/java/org/apache/impala/catalog/metastore/MetastoreServiceHandler.java@1600
PS2, Line 1600: throws NoSuchObjectException, InvalidObjectException,
MetaException, InvalidInputException, TException {
line too long (110 > 90)
http://gerrit.cloudera.org:8080/#/c/17244/2/fe/src/main/java/org/apache/impala/catalog/metastore/MetastoreServiceHandler.java@1611
PS2, Line 1611: throws NoSuchObjectException, MetaException,
InvalidObjectException, InvalidInputException, TException {
line too long (110 > 90)
http://gerrit.cloudera.org:8080/#/c/17244/2/fe/src/main/java/org/apache/impala/catalog/metastore/MetastoreServiceHandler.java@1622
PS2, Line 1622: throws NoSuchObjectException, MetaException,
InvalidObjectException, InvalidInputException, TException {
line too long (110 > 90)
http://gerrit.cloudera.org:8080/#/c/17244/2/fe/src/main/java/org/apache/impala/catalog/metastore/MetastoreServiceHandler.java@1632
PS2, Line 1632: throws AlreadyExistsException, InvalidObjectException,
MetaException, NoSuchObjectException, TException {
line too long (111 > 90)
http://gerrit.cloudera.org:8080/#/c/17244/2/fe/src/main/java/org/apache/impala/catalog/metastore/MetastoreServiceHandler.java@2403
PS2, Line 2403: public WMCreateOrDropTriggerToPoolMappingResponse
create_or_drop_wm_trigger_to_pool_mapping(
line too long (94 > 90)
--
To view, visit http://gerrit.cloudera.org:8080/17244
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I1b306f91d63cb5137c178e8e72b6e8b578a907b5
Gerrit-Change-Number: 17244
Gerrit-PatchSet: 2
Gerrit-Owner: Vihang Karajgaonkar
Gerrit-Reviewer: Aman Sinha
Gerrit-Reviewer: Impala Public Jenkins
Gerrit-Reviewer: Quanlong Huang
Gerrit-Reviewer: Vihang Karajgaonkar
Gerrit-Comment-Date: Tue, 30 Mar 2021 23:00:28 +
Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-10613 : Standup HMS thrift server in Catalog
Vihang Karajgaonkar has uploaded a new patch set (#2). ( http://gerrit.cloudera.org:8080/17244 ) Change subject: IMPALA-10613 : Standup HMS thrift server in Catalog .. IMPALA-10613 : Standup HMS thrift server in Catalog This change adds the basic infrastructure to start the HMS server in Catalog. It introduces a new configuration (--start_hms_server) along with a config for the port and starts a HMS thrift server in the CatalogServiceCatalog instance. Currently, all the HMS APIs are "pass-through" to the backing HMS service. Except for the following 3 HMS APIs which can be used to request a table and its partitions. Additionally, there is another flag (--enable_catalogd_hms_cache) which can be used to disable the usage of catalogd for providing the table and partition metadata. This contribution was done by Kishen Das. 1. get_table_req 2. get_partitions_by_expr 3. get_partitions_by_names In case of get_partitions_by_expr we need the hive-exec jar to be present in the classpath since it needs to load the PartitionExpressionProxy to push down the partition predicates to the HMS database. In case of get_table_req if column statistics are requested, we return the table level statistics. Additionally, this patch adds a new configuration fallback_to_hms_on_errors for the catalog which is used to determine if the Catalog falls back to HMS service in case of errors while executing the API. This is useful for testing purposes. In order to expose the file-metadata for the tables and partitions, HMS API changes were made to add the filemetadata fields to table and partitions. In case of transactional tables, the file-metadata which is returned is consistent with the provided ValidWriteIdList in the API call. There are a few TODOs which will be done in follow up tasks: 1. Add support for SASL support. 2. Pin the hive_metastore.thrift in the code so that any changes to HMS APIs in the hive branch doesn't break Catalog's HMS service. Testing: 1. Added a new end-to-end test which starts the HMS service in Catalog and runs some basic HMS APIs against it. 2. Ran a modification of TestRemoteHiveMetastore in the Hive code base and confirmed most tests are working. There were some test failures but they are unrelated since the test assumes an empty warehouse whereas we run against the actual HMS service running in the mini-cluster. Change-Id: I1b306f91d63cb5137c178e8e72b6e8b578a907b5 --- M be/src/catalog/catalog-server.cc M be/src/common/global-flags.cc M be/src/util/backend-gflag-util.cc M common/thrift/BackendGflags.thrift M common/thrift/CatalogService.thrift A fe/src/main/java/org/apache/impala/catalog/CatalogHMSAPIHelper.java M fe/src/main/java/org/apache/impala/catalog/CatalogServiceCatalog.java A fe/src/main/java/org/apache/impala/catalog/GetPartialCatalogObjectRequestBuilder.java M fe/src/main/java/org/apache/impala/catalog/HdfsTable.java M fe/src/main/java/org/apache/impala/catalog/ParallelFileMetadataLoader.java M fe/src/main/java/org/apache/impala/catalog/Table.java A fe/src/main/java/org/apache/impala/catalog/metastore/CatalogHMSClientUtils.java A fe/src/main/java/org/apache/impala/catalog/metastore/CatalogMetastoreServer.java A fe/src/main/java/org/apache/impala/catalog/metastore/CatalogMetastoreServiceHandler.java A fe/src/main/java/org/apache/impala/catalog/metastore/ICatalogMetastoreServer.java A fe/src/main/java/org/apache/impala/catalog/metastore/MetastoreServiceHandler.java A fe/src/main/java/org/apache/impala/catalog/metastore/NoOpCatalogMetastoreServer.java M fe/src/main/java/org/apache/impala/service/BackendConfig.java A fe/src/test/java/org/apache/impala/catalog/metastore/CatalogHMSFileMetadataTest.java A fe/src/test/java/org/apache/impala/catalog/metastore/EnableCatalogdHMSCacheFlagTest.java M fe/src/test/java/org/apache/impala/testutil/CatalogServiceTestCatalog.java M tests/common/impala_test_suite.py A tests/custom_cluster/test_metastore_service.py 23 files changed, 5,510 insertions(+), 22 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/44/17244/2 -- To view, visit http://gerrit.cloudera.org:8080/17244 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: I1b306f91d63cb5137c178e8e72b6e8b578a907b5 Gerrit-Change-Number: 17244 Gerrit-PatchSet: 2 Gerrit-Owner: Vihang Karajgaonkar Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Vihang Karajgaonkar
[Impala-ASF-CR] IMPALA-10613 : Standup HMS thrift server in Catalog
Vihang Karajgaonkar has posted comments on this change. (
http://gerrit.cloudera.org:8080/17244 )
Change subject: IMPALA-10613 : Standup HMS thrift server in Catalog
..
Patch Set 1:
(14 comments)
http://gerrit.cloudera.org:8080/#/c/17244/1/fe/src/main/java/org/apache/impala/catalog/CatalogServiceCatalog.java
File fe/src/main/java/org/apache/impala/catalog/CatalogServiceCatalog.java:
http://gerrit.cloudera.org:8080/#/c/17244/1/fe/src/main/java/org/apache/impala/catalog/CatalogServiceCatalog.java@3397
PS1, Line 3397: TGetPartialCatalogObjectRequest req, String
tableLoadReason) throws CatalogException {
> line too long (92 > 90)
Done
http://gerrit.cloudera.org:8080/#/c/17244/1/fe/src/main/java/org/apache/impala/catalog/HdfsTable.java
File fe/src/main/java/org/apache/impala/catalog/HdfsTable.java:
http://gerrit.cloudera.org:8080/#/c/17244/1/fe/src/main/java/org/apache/impala/catalog/HdfsTable.java@745
PS1, Line 745: new ParallelFileMetadataLoader(getFileSystem(),
partBuilders, validWriteIds_, validTxnList,
> line too long (95 > 90)
Done
http://gerrit.cloudera.org:8080/#/c/17244/1/fe/src/main/java/org/apache/impala/catalog/HdfsTable.java@746
PS1, Line 746: Utils.shouldRecursivelyListPartitions(this),
getHostIndex(), debugActions, logPrefix)
> line too long (93 > 90)
Done
http://gerrit.cloudera.org:8080/#/c/17244/1/fe/src/main/java/org/apache/impala/catalog/metastore/CatalogMetastoreServer.java
File
fe/src/main/java/org/apache/impala/catalog/metastore/CatalogMetastoreServer.java:
http://gerrit.cloudera.org:8080/#/c/17244/1/fe/src/main/java/org/apache/impala/catalog/metastore/CatalogMetastoreServer.java@199
PS1, Line 199: new Class[]{ThriftHiveMetastore.Iface.class,
ICatalogMetastoreServer.class},
> line too long (92 > 90)
Done
http://gerrit.cloudera.org:8080/#/c/17244/1/fe/src/test/java/org/apache/impala/catalog/metastore/EnableCatalogdHMSCacheFlagTest.java
File
fe/src/test/java/org/apache/impala/catalog/metastore/EnableCatalogdHMSCacheFlagTest.java:
http://gerrit.cloudera.org:8080/#/c/17244/1/fe/src/test/java/org/apache/impala/catalog/metastore/EnableCatalogdHMSCacheFlagTest.java@25
PS1, Line 25: import static
org.apache.impala.catalog.metastore.CatalogHMSFileMetadataTest.assertFdsAreSame;
> line too long (94 > 90)
Done
http://gerrit.cloudera.org:8080/#/c/17244/1/tests/common/impala_test_suite.py
File tests/common/impala_test_suite.py:
http://gerrit.cloudera.org:8080/#/c/17244/1/tests/common/impala_test_suite.py@156
PS1, Line 156: @
> flake8: E303 too many blank lines (2)
Done
http://gerrit.cloudera.org:8080/#/c/17244/1/tests/custom_cluster/test_metastore_service.py
File tests/custom_cluster/test_metastore_service.py:
http://gerrit.cloudera.org:8080/#/c/17244/1/tests/custom_cluster/test_metastore_service.py@19
PS1, Line 19: import logging
> flake8: F401 'logging' imported but unused
Done
http://gerrit.cloudera.org:8080/#/c/17244/1/tests/custom_cluster/test_metastore_service.py@25
PS1, Line 25: from hive_metastore.ttypes import GetValidWriteIdsRequest
> flake8: F401 'hive_metastore.ttypes.GetValidWriteIdsRequest' imported but u
Done
http://gerrit.cloudera.org:8080/#/c/17244/1/tests/custom_cluster/test_metastore_service.py@230
PS1, Line 230: @
> flake8: E303 too many blank lines (2)
Done
http://gerrit.cloudera.org:8080/#/c/17244/1/tests/custom_cluster/test_metastore_service.py@294
PS1, Line 294: "
> flake8: E501 line too long (91 > 90 characters)
Done
http://gerrit.cloudera.org:8080/#/c/17244/1/tests/custom_cluster/test_metastore_service.py@405
PS1, Line 405: #
> flake8: E265 block comment should start with '# '
Done
http://gerrit.cloudera.org:8080/#/c/17244/1/tests/custom_cluster/test_metastore_service.py@420
PS1, Line 420: t
> flake8: E501 line too long (92 > 90 characters)
Done
http://gerrit.cloudera.org:8080/#/c/17244/1/tests/custom_cluster/test_metastore_service.py@427
PS1, Line 427: _
> flake8: E501 line too long (96 > 90 characters)
Done
http://gerrit.cloudera.org:8080/#/c/17244/1/tests/custom_cluster/test_metastore_service.py@430
PS1, Line 430: )
> flake8: E501 line too long (92 > 90 characters)
Done
--
To view, visit http://gerrit.cloudera.org:8080/17244
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I1b306f91d63cb5137c178e8e72b6e8b578a907b5
Gerrit-Change-Number: 17244
Gerrit-PatchSet: 1
Gerrit-Owner: Vihang Karajgaonkar
Gerrit-Reviewer: Impala Public Jenkins
Gerrit-Reviewer: Vihang Karajgaonkar
Gerrit-Comment-Date: Tue, 30 Mar 2021 22:59:21 +
Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-10613 : Standup HMS thrift server in Catalog
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/17244 ) Change subject: IMPALA-10613 : Standup HMS thrift server in Catalog .. Patch Set 1: Build Successful https://jenkins.impala.io/job/gerrit-code-review-checks/8470/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests. -- To view, visit http://gerrit.cloudera.org:8080/17244 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I1b306f91d63cb5137c178e8e72b6e8b578a907b5 Gerrit-Change-Number: 17244 Gerrit-PatchSet: 1 Gerrit-Owner: Vihang Karajgaonkar Gerrit-Reviewer: Impala Public Jenkins Gerrit-Comment-Date: Tue, 30 Mar 2021 22:57:28 + Gerrit-HasComments: No
[Impala-ASF-CR] Bump up the GBN number to 11920537
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/17243 ) Change subject: Bump up the GBN number to 11920537 .. Patch Set 1: Build Successful https://jenkins.impala.io/job/gerrit-code-review-checks/8469/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests. -- To view, visit http://gerrit.cloudera.org:8080/17243 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: If88ceeaffc94e5bedf2c9953122109e20663f743 Gerrit-Change-Number: 17243 Gerrit-PatchSet: 1 Gerrit-Owner: Vihang Karajgaonkar Gerrit-Reviewer: Impala Public Jenkins Gerrit-Comment-Date: Tue, 30 Mar 2021 22:57:13 + Gerrit-HasComments: No
[Impala-ASF-CR] Bump up CDP BUILD NUMBER to 11918249
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/17241 ) Change subject: Bump up CDP_BUILD_NUMBER to 11918249 .. Patch Set 1: Verified-1 Build failed: https://jenkins.impala.io/job/gerrit-verify-dryrun/7021/ -- To view, visit http://gerrit.cloudera.org:8080/17241 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I9b84afeb8cec13f9fba4e676077cde3c676e6537 Gerrit-Change-Number: 17241 Gerrit-PatchSet: 1 Gerrit-Owner: Zoltan Borok-Nagy Gerrit-Reviewer: Impala Public Jenkins Gerrit-Comment-Date: Tue, 30 Mar 2021 22:47:34 + Gerrit-HasComments: No
[native-toolchain-CR] IMPALA-10544: Switch from Centos 7.2 to Centos 7.4
Joe McDonnell has submitted this change and it was merged. ( http://gerrit.cloudera.org:8080/17119 ) Change subject: IMPALA-10544: Switch from Centos 7.2 to Centos 7.4 .. IMPALA-10544: Switch from Centos 7.2 to Centos 7.4 jwt-cpp requires OpenSSL 1.0.2. Centos 7.2 is now too old, as it still uses OpenSSL 1.0.1e. This changes the redhat7 docker image to use Centos 7.4, which has OpenSSL 1.0.2. This removes several obsolete fixes required for Centos 7.2 along with the repository descriptions for Centos 6. Testing: - Ran a build of the docker images and then built the native toolchain (including jwt-cpp) with them. Change-Id: Ibeba445ff87e2addef81543b025c398bde08a377 Reviewed-on: http://gerrit.cloudera.org:8080/17119 Reviewed-by: Laszlo Gaal Tested-by: Joe McDonnell --- D docker/redhat/CentOS-6.6-Base.repo R docker/redhat/CentOS-7.4.1708-Base.repo D docker/redhat/Centos6-Vault.repo D docker/redhat/Centos7-Vault.repo M docker/redhat7.df 5 files changed, 10 insertions(+), 66 deletions(-) Approvals: Laszlo Gaal: Looks good to me, approved Joe McDonnell: Verified -- To view, visit http://gerrit.cloudera.org:8080/17119 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: native-toolchain Gerrit-Branch: master Gerrit-MessageType: merged Gerrit-Change-Id: Ibeba445ff87e2addef81543b025c398bde08a377 Gerrit-Change-Number: 17119 Gerrit-PatchSet: 3 Gerrit-Owner: Joe McDonnell Gerrit-Reviewer: Joe McDonnell Gerrit-Reviewer: Laszlo Gaal
[native-toolchain-CR] IMPALA-10544: Switch from Centos 7.2 to Centos 7.4
Joe McDonnell has posted comments on this change. ( http://gerrit.cloudera.org:8080/17119 ) Change subject: IMPALA-10544: Switch from Centos 7.2 to Centos 7.4 .. Patch Set 2: Verified+1 -- To view, visit http://gerrit.cloudera.org:8080/17119 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: native-toolchain Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ibeba445ff87e2addef81543b025c398bde08a377 Gerrit-Change-Number: 17119 Gerrit-PatchSet: 2 Gerrit-Owner: Joe McDonnell Gerrit-Reviewer: Joe McDonnell Gerrit-Reviewer: Laszlo Gaal Gerrit-Comment-Date: Tue, 30 Mar 2021 22:41:44 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-10613 : Standup HMS thrift server in Catalog
Impala Public Jenkins has posted comments on this change. (
http://gerrit.cloudera.org:8080/17244 )
Change subject: IMPALA-10613 : Standup HMS thrift server in Catalog
..
Patch Set 1:
(81 comments)
http://gerrit.cloudera.org:8080/#/c/17244/1/fe/src/main/java/org/apache/impala/catalog/CatalogHMSAPIHelper.java
File fe/src/main/java/org/apache/impala/catalog/CatalogHMSAPIHelper.java:
http://gerrit.cloudera.org:8080/#/c/17244/1/fe/src/main/java/org/apache/impala/catalog/CatalogHMSAPIHelper.java@118
PS1, Line 118: GetPartialCatalogObjectRequestBuilder reqBuilder = new
GetPartialCatalogObjectRequestBuilder()
line too long (98 > 90)
http://gerrit.cloudera.org:8080/#/c/17244/1/fe/src/main/java/org/apache/impala/catalog/CatalogHMSAPIHelper.java@219
PS1, Line 219: GetPartialCatalogObjectRequestBuilder catalogReq = new
GetPartialCatalogObjectRequestBuilder()
line too long (98 > 90)
http://gerrit.cloudera.org:8080/#/c/17244/1/fe/src/main/java/org/apache/impala/catalog/CatalogHMSAPIHelper.java@504
PS1, Line 504: + "fallback path. Time taken: {} msec",
getPartsResult.getPartitionsSize(),
line too long (91 > 90)
http://gerrit.cloudera.org:8080/#/c/17244/1/fe/src/main/java/org/apache/impala/catalog/CatalogHMSAPIHelper.java@508
PS1, Line 508: + "fallback path. Time taken: {} msec",
getPartsResult.getPartitionsSize(),
line too long (91 > 90)
http://gerrit.cloudera.org:8080/#/c/17244/1/fe/src/main/java/org/apache/impala/catalog/CatalogServiceCatalog.java
File fe/src/main/java/org/apache/impala/catalog/CatalogServiceCatalog.java:
http://gerrit.cloudera.org:8080/#/c/17244/1/fe/src/main/java/org/apache/impala/catalog/CatalogServiceCatalog.java@3397
PS1, Line 3397: TGetPartialCatalogObjectRequest req, String
tableLoadReason) throws CatalogException {
line too long (92 > 90)
http://gerrit.cloudera.org:8080/#/c/17244/1/fe/src/main/java/org/apache/impala/catalog/HdfsTable.java
File fe/src/main/java/org/apache/impala/catalog/HdfsTable.java:
http://gerrit.cloudera.org:8080/#/c/17244/1/fe/src/main/java/org/apache/impala/catalog/HdfsTable.java@745
PS1, Line 745: new ParallelFileMetadataLoader(getFileSystem(),
partBuilders, validWriteIds_, validTxnList,
line too long (95 > 90)
http://gerrit.cloudera.org:8080/#/c/17244/1/fe/src/main/java/org/apache/impala/catalog/HdfsTable.java@746
PS1, Line 746: Utils.shouldRecursivelyListPartitions(this),
getHostIndex(), debugActions, logPrefix)
line too long (93 > 90)
http://gerrit.cloudera.org:8080/#/c/17244/1/fe/src/main/java/org/apache/impala/catalog/metastore/CatalogMetastoreServer.java
File
fe/src/main/java/org/apache/impala/catalog/metastore/CatalogMetastoreServer.java:
http://gerrit.cloudera.org:8080/#/c/17244/1/fe/src/main/java/org/apache/impala/catalog/metastore/CatalogMetastoreServer.java@199
PS1, Line 199: new Class[]{ThriftHiveMetastore.Iface.class,
ICatalogMetastoreServer.class},
line too long (92 > 90)
http://gerrit.cloudera.org:8080/#/c/17244/1/fe/src/main/java/org/apache/impala/catalog/metastore/MetastoreServiceHandler.java
File
fe/src/main/java/org/apache/impala/catalog/metastore/MetastoreServiceHandler.java:
http://gerrit.cloudera.org:8080/#/c/17244/1/fe/src/main/java/org/apache/impala/catalog/metastore/MetastoreServiceHandler.java@277
PS1, Line 277: * APIs that should be served from CatalogD must be overridden
in {@link CatalogMetastoreServer}.
line too long (97 > 90)
http://gerrit.cloudera.org:8080/#/c/17244/1/fe/src/main/java/org/apache/impala/catalog/metastore/MetastoreServiceHandler.java@291
PS1, Line 291: protected static final String METAEXCEPTION_MSG_FORMAT =
"Unexpected error occurred while"
line too long (92 > 90)
http://gerrit.cloudera.org:8080/#/c/17244/1/fe/src/main/java/org/apache/impala/catalog/metastore/MetastoreServiceHandler.java@416
PS1, Line 416: return
client.getHiveClient().getThriftClient().get_database_req(getDatabaseRequest);
line too long (91 > 90)
http://gerrit.cloudera.org:8080/#/c/17244/1/fe/src/main/java/org/apache/impala/catalog/metastore/MetastoreServiceHandler.java@521
PS1, Line 521: throws AlreadyExistsException, InvalidObjectException,
MetaException, NoSuchObjectException, TException {
line too long (111 > 90)
http://gerrit.cloudera.org:8080/#/c/17244/1/fe/src/main/java/org/apache/impala/catalog/metastore/MetastoreServiceHandler.java@530
PS1, Line 530: throws AlreadyExistsException, InvalidObjectException,
MetaException, NoSuchObjectException, TException {
line too long (111 > 90)
http://gerrit.cloudera.org:8080/#/c/17244/1/fe/src/main/java/org/apache/impala/catalog/metastore/MetastoreServiceHandler.java@544
PS1, Line 544: throws AlreadyExistsException, InvalidObjectException,
MetaException, NoSuchObjectException, TException {
line too long (111 > 90)
[Impala-ASF-CR] IMPALA-10613 : Standup HMS thrift server in Catalog
Vihang Karajgaonkar has uploaded this change for review. ( http://gerrit.cloudera.org:8080/17244 Change subject: IMPALA-10613 : Standup HMS thrift server in Catalog .. IMPALA-10613 : Standup HMS thrift server in Catalog This change adds the basic infrastructure to start the HMS server in Catalog. It introduces a new configuration (--start_hms_server) along with a config for the port and starts a HMS thrift server in the CatalogServiceCatalog instance. Currently, all the HMS APIs are "pass-through" to the backing HMS service. Except for the following 3 HMS APIs which can be used to request a table and its partitions. Additionally, there is another flag (--enable_catalogd_hms_cache) which can be used to disable the usage of catalogd for providing the table and partition metadata. This contribution was done by Kishen Das. 1. get_table_req 2. get_partitions_by_expr 3. get_partitions_by_names In case of get_partitions_by_expr we need the hive-exec jar to be present in the classpath since it needs to load the PartitionExpressionProxy to push down the partition predicates to the HMS database. In case of get_table_req if column statistics are requested, we return the table level statistics. Additionally, this patch adds a new configuration fallback_to_hms_on_errors for the catalog which is used to determine if the Catalog falls back to HMS service in case of errors while executing the API. This is useful for testing purposes. In order to expose the file-metadata for the tables and partitions, HMS API changes were made to add the filemetadata fields to table and partitions. In case of transactional tables, the file-metadata which is returned is consistent with the provided ValidWriteIdList in the API call. There are a few TODOs which will be done in follow up tasks: 1. Add support for SASL support. 2. Pin the hive_metastore.thrift in the code so that any changes to HMS APIs in the hive branch doesn't break Catalog's HMS service. Testing: 1. Added a new end-to-end test which starts the HMS service in Catalog and runs some basic HMS APIs against it. 2. Ran a modification of TestRemoteHiveMetastore in the Hive code base and confirmed most tests are working. There were some test failures but they are unrelated since the test assumes an empty warehouse whereas we run against the actual HMS service running in the mini-cluster. Change-Id: I1b306f91d63cb5137c178e8e72b6e8b578a907b5 --- M be/src/catalog/catalog-server.cc M be/src/common/global-flags.cc M be/src/util/backend-gflag-util.cc M common/thrift/BackendGflags.thrift M common/thrift/CatalogService.thrift A fe/src/main/java/org/apache/impala/catalog/CatalogHMSAPIHelper.java M fe/src/main/java/org/apache/impala/catalog/CatalogServiceCatalog.java A fe/src/main/java/org/apache/impala/catalog/GetPartialCatalogObjectRequestBuilder.java M fe/src/main/java/org/apache/impala/catalog/HdfsTable.java M fe/src/main/java/org/apache/impala/catalog/ParallelFileMetadataLoader.java M fe/src/main/java/org/apache/impala/catalog/Table.java A fe/src/main/java/org/apache/impala/catalog/metastore/CatalogHMSClientUtils.java A fe/src/main/java/org/apache/impala/catalog/metastore/CatalogMetastoreServer.java A fe/src/main/java/org/apache/impala/catalog/metastore/CatalogMetastoreServiceHandler.java A fe/src/main/java/org/apache/impala/catalog/metastore/ICatalogMetastoreServer.java A fe/src/main/java/org/apache/impala/catalog/metastore/MetastoreServiceHandler.java A fe/src/main/java/org/apache/impala/catalog/metastore/NoOpCatalogMetastoreServer.java M fe/src/main/java/org/apache/impala/service/BackendConfig.java A fe/src/test/java/org/apache/impala/catalog/metastore/CatalogHMSFileMetadataTest.java A fe/src/test/java/org/apache/impala/catalog/metastore/EnableCatalogdHMSCacheFlagTest.java M fe/src/test/java/org/apache/impala/testutil/CatalogServiceTestCatalog.java M tests/common/impala_test_suite.py A tests/custom_cluster/test_metastore_service.py 23 files changed, 5,444 insertions(+), 22 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/44/17244/1 -- To view, visit http://gerrit.cloudera.org:8080/17244 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newchange Gerrit-Change-Id: I1b306f91d63cb5137c178e8e72b6e8b578a907b5 Gerrit-Change-Number: 17244 Gerrit-PatchSet: 1 Gerrit-Owner: Vihang Karajgaonkar
[Impala-ASF-CR] Bump up the GBN number to 11920537
Vihang Karajgaonkar has uploaded this change for review. ( http://gerrit.cloudera.org:8080/17243 Change subject: Bump up the GBN number to 11920537 .. Bump up the GBN number to 11920537 This change bumps up the GBN to 11920537 which includes several changes to Hive needed to support Catalogd's HMS endpoint for supporting external frontends. Additionally, it excludes some dependencies from the pom.xml which are not uploaded by default to the toolchain. Change-Id: If88ceeaffc94e5bedf2c9953122109e20663f743 --- M bin/impala-config.sh M fe/pom.xml M java/shaded-deps/hive-exec/pom.xml M java/test-hive-udfs/pom.xml 4 files changed, 28 insertions(+), 12 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/43/17243/1 -- To view, visit http://gerrit.cloudera.org:8080/17243 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newchange Gerrit-Change-Id: If88ceeaffc94e5bedf2c9953122109e20663f743 Gerrit-Change-Number: 17243 Gerrit-PatchSet: 1 Gerrit-Owner: Vihang Karajgaonkar
[Impala-ASF-CR] IMPALA-10606 (part 2): Clean up ordering of requirements.txt
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/17229 ) Change subject: IMPALA-10606 (part 2): Clean up ordering of requirements.txt .. Patch Set 3: Build Successful https://jenkins.impala.io/job/gerrit-code-review-checks/8468/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests. -- To view, visit http://gerrit.cloudera.org:8080/17229 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I2f13ec8f8af80c4bac5da30d08a2ea4c56806d27 Gerrit-Change-Number: 17229 Gerrit-PatchSet: 3 Gerrit-Owner: Joe McDonnell Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Joe McDonnell Gerrit-Reviewer: Laszlo Gaal Gerrit-Comment-Date: Tue, 30 Mar 2021 21:40:33 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-10606: Simplify impala-python virtualenv bootstrapping
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/17226 ) Change subject: IMPALA-10606: Simplify impala-python virtualenv bootstrapping .. Patch Set 4: Build Successful https://jenkins.impala.io/job/gerrit-code-review-checks/8467/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests. -- To view, visit http://gerrit.cloudera.org:8080/17226 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I8e920e5a257f1e0613065685078624a50d59bf2e Gerrit-Change-Number: 17226 Gerrit-PatchSet: 4 Gerrit-Owner: Joe McDonnell Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Joe McDonnell Gerrit-Reviewer: Laszlo Gaal Gerrit-Comment-Date: Tue, 30 Mar 2021 21:39:45 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-10608: Update kudu-python version and remove some unused packages
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/17231 ) Change subject: IMPALA-10608: Update kudu-python version and remove some unused packages .. Patch Set 3: Build started: https://jenkins.impala.io/job/gerrit-verify-dryrun/7023/ DRY_RUN=true -- To view, visit http://gerrit.cloudera.org:8080/17231 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ib47770df9e46de448fe2bffef7abe2c3aa942fb9 Gerrit-Change-Number: 17231 Gerrit-PatchSet: 3 Gerrit-Owner: Joe McDonnell Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Joe McDonnell Gerrit-Reviewer: Laszlo Gaal Gerrit-Comment-Date: Tue, 30 Mar 2021 21:21:32 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-10608: Update kudu-python version and remove some unused packages
Joe McDonnell has posted comments on this change. ( http://gerrit.cloudera.org:8080/17231 ) Change subject: IMPALA-10608: Update kudu-python version and remove some unused packages .. Patch Set 3: Code-Review+2 Only change is to the requests version that got updated, carrying +2 -- To view, visit http://gerrit.cloudera.org:8080/17231 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ib47770df9e46de448fe2bffef7abe2c3aa942fb9 Gerrit-Change-Number: 17231 Gerrit-PatchSet: 3 Gerrit-Owner: Joe McDonnell Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Joe McDonnell Gerrit-Reviewer: Laszlo Gaal Gerrit-Comment-Date: Tue, 30 Mar 2021 21:21:04 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-10606 (part 2): Clean up ordering of requirements.txt
Joe McDonnell has posted comments on this change. ( http://gerrit.cloudera.org:8080/17229 ) Change subject: IMPALA-10606 (part 2): Clean up ordering of requirements.txt .. Patch Set 3: Code-Review+2 Only change is to the requests version that got updated, carrying +2 -- To view, visit http://gerrit.cloudera.org:8080/17229 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I2f13ec8f8af80c4bac5da30d08a2ea4c56806d27 Gerrit-Change-Number: 17229 Gerrit-PatchSet: 3 Gerrit-Owner: Joe McDonnell Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Joe McDonnell Gerrit-Reviewer: Laszlo Gaal Gerrit-Comment-Date: Tue, 30 Mar 2021 21:21:00 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-10606: Simplify impala-python virtualenv bootstrapping
Joe McDonnell has posted comments on this change. ( http://gerrit.cloudera.org:8080/17226 ) Change subject: IMPALA-10606: Simplify impala-python virtualenv bootstrapping .. Patch Set 4: Code-Review+2 Only change is to the requests version that got updated, carrying +2 -- To view, visit http://gerrit.cloudera.org:8080/17226 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I8e920e5a257f1e0613065685078624a50d59bf2e Gerrit-Change-Number: 17226 Gerrit-PatchSet: 4 Gerrit-Owner: Joe McDonnell Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Joe McDonnell Gerrit-Reviewer: Laszlo Gaal Gerrit-Comment-Date: Tue, 30 Mar 2021 21:20:39 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-10606: Simplify impala-python virtualenv bootstrapping
Hello Laszlo Gaal, Impala Public Jenkins, I'd like you to reexamine a change. Please visit http://gerrit.cloudera.org:8080/17226 to look at the new patch set (#4). Change subject: IMPALA-10606: Simplify impala-python virtualenv bootstrapping .. IMPALA-10606: Simplify impala-python virtualenv bootstrapping Bootstrapping the impala-python virtualenv requires multiple rounds of pip installs with different sets of requirements. This consolidates the requirements.txt, stage2-requirements.txt, and compiled-requirements.txt into a single requirements.txt. This will make it easier to upgrade python packages. This also splits out setuptools into its own setuptools-requirements.txt. Setuptools is used during the pip install for several of the dependencies. Recent versions of setuptools do not support Python 2, but some of the install tools (like easy_install) don't know how to pick a version of setuptools that works with Python 2. Splitting it out to its own requirements file lets us pin the version. To make review easier, this does not change any of the versions of the dependencies. It also leaves the stage2-requirements.txt and compiled-requirements.txt split out in separate sections of requirements.txt. These will later be turned into a single alphabetical list. Testing: - Tested impala-python locally - Ran GVO Change-Id: I8e920e5a257f1e0613065685078624a50d59bf2e --- M infra/python/bootstrap_virtualenv.py M infra/python/deps/pip_download.py M infra/python/deps/requirements.txt R infra/python/deps/setuptools-requirements.txt D infra/python/deps/stage2-requirements.txt 5 files changed, 88 insertions(+), 121 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/26/17226/4 -- To view, visit http://gerrit.cloudera.org:8080/17226 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: I8e920e5a257f1e0613065685078624a50d59bf2e Gerrit-Change-Number: 17226 Gerrit-PatchSet: 4 Gerrit-Owner: Joe McDonnell Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Laszlo Gaal
[Impala-ASF-CR] IMPALA-10606 (part 2): Clean up ordering of requirements.txt
Hello Laszlo Gaal, Impala Public Jenkins, I'd like you to reexamine a change. Please visit http://gerrit.cloudera.org:8080/17229 to look at the new patch set (#3). Change subject: IMPALA-10606 (part 2): Clean up ordering of requirements.txt .. IMPALA-10606 (part 2): Clean up ordering of requirements.txt This is a followup to the original IMPALA-10606 that reorders the requirements.txt alphabetically. No versions changed as part of this. Change-Id: I2f13ec8f8af80c4bac5da30d08a2ea4c56806d27 --- M infra/python/deps/requirements.txt 1 file changed, 22 insertions(+), 34 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/29/17229/3 -- To view, visit http://gerrit.cloudera.org:8080/17229 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: I2f13ec8f8af80c4bac5da30d08a2ea4c56806d27 Gerrit-Change-Number: 17229 Gerrit-PatchSet: 3 Gerrit-Owner: Joe McDonnell Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Laszlo Gaal
[Impala-ASF-CR] IMPALA-9331: Add symptom for dataload failing on schema mismatch
Andrew Sherman has posted comments on this change. ( http://gerrit.cloudera.org:8080/17242 ) Change subject: IMPALA-9331: Add symptom for dataload failing on schema mismatch .. Patch Set 2: Code-Review+2 LGTM, plus also useful -- To view, visit http://gerrit.cloudera.org:8080/17242 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I2fa193740a2764fdda799d6a9cc64f89cab64aba Gerrit-Change-Number: 17242 Gerrit-PatchSet: 2 Gerrit-Owner: Laszlo Gaal Gerrit-Reviewer: Andrew Sherman Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Joe McDonnell Gerrit-Reviewer: Laszlo Gaal Gerrit-Comment-Date: Tue, 30 Mar 2021 21:05:48 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-9331: Add symptom for dataload failing on schema mismatch
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/17242 ) Change subject: IMPALA-9331: Add symptom for dataload failing on schema mismatch .. Patch Set 1: Build Successful https://jenkins.impala.io/job/gerrit-code-review-checks/8466/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests. -- To view, visit http://gerrit.cloudera.org:8080/17242 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I2fa193740a2764fdda799d6a9cc64f89cab64aba Gerrit-Change-Number: 17242 Gerrit-PatchSet: 1 Gerrit-Owner: Laszlo Gaal Gerrit-Reviewer: Andrew Sherman Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Joe McDonnell Gerrit-Reviewer: Laszlo Gaal Gerrit-Comment-Date: Tue, 30 Mar 2021 21:05:20 + Gerrit-HasComments: No
[native-toolchain-CR] IMPALA-10544: Switch from Centos 7.2 to Centos 7.4
Laszlo Gaal has posted comments on this change. ( http://gerrit.cloudera.org:8080/17119 ) Change subject: IMPALA-10544: Switch from Centos 7.2 to Centos 7.4 .. Patch Set 2: Code-Review+2 -- To view, visit http://gerrit.cloudera.org:8080/17119 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: native-toolchain Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ibeba445ff87e2addef81543b025c398bde08a377 Gerrit-Change-Number: 17119 Gerrit-PatchSet: 2 Gerrit-Owner: Joe McDonnell Gerrit-Reviewer: Laszlo Gaal Gerrit-Comment-Date: Tue, 30 Mar 2021 20:50:31 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-9331: Add symptom for dataload failing on schema mismatch
Hello Impala Public Jenkins, I'd like you to reexamine a change. Please visit http://gerrit.cloudera.org:8080/17242 to look at the new patch set (#2). Change subject: IMPALA-9331: Add symptom for dataload failing on schema mismatch .. IMPALA-9331: Add symptom for dataload failing on schema mismatch Streamline triaging a bit. When this fails, it does so in a specific location, and until now you had to scan the build log to find the problem. This JUnitXML symptom should make this failure mode obvious. Tested by running an S3 build on private infrastructure with a knowingly mismatched data snapshot. Change-Id: I2fa193740a2764fdda799d6a9cc64f89cab64aba --- M testdata/bin/create-load-data.sh 1 file changed, 7 insertions(+), 0 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/42/17242/2 -- To view, visit http://gerrit.cloudera.org:8080/17242 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: I2fa193740a2764fdda799d6a9cc64f89cab64aba Gerrit-Change-Number: 17242 Gerrit-PatchSet: 2 Gerrit-Owner: Laszlo Gaal Gerrit-Reviewer: Impala Public Jenkins
[Impala-ASF-CR] IMPALA-9331: Add symptom for dataload failing on schema mismatch
Laszlo Gaal has uploaded this change for review. ( http://gerrit.cloudera.org:8080/17242 Change subject: IMPALA-9331: Add symptom for dataload failing on schema mismatch .. IMPALA-9331: Add symptom for dataload failing on schema mismatch Streamline triaging a bit. When this fails, it does so in a specific location, and until now you had to scan the build log to find the problem. This JUnitXML symptom should make this failure mode obvious. Change-Id: I2fa193740a2764fdda799d6a9cc64f89cab64aba --- M testdata/bin/create-load-data.sh 1 file changed, 7 insertions(+), 0 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/42/17242/1 -- To view, visit http://gerrit.cloudera.org:8080/17242 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newchange Gerrit-Change-Id: I2fa193740a2764fdda799d6a9cc64f89cab64aba Gerrit-Change-Number: 17242 Gerrit-PatchSet: 1 Gerrit-Owner: Laszlo Gaal
[Impala-ASF-CR] IMPALA-10606: Simplify impala-python virtualenv bootstrapping
Laszlo Gaal has posted comments on this change. ( http://gerrit.cloudera.org:8080/17226 ) Change subject: IMPALA-10606: Simplify impala-python virtualenv bootstrapping .. Patch Set 3: Code-Review+2 -- To view, visit http://gerrit.cloudera.org:8080/17226 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I8e920e5a257f1e0613065685078624a50d59bf2e Gerrit-Change-Number: 17226 Gerrit-PatchSet: 3 Gerrit-Owner: Joe McDonnell Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Laszlo Gaal Gerrit-Comment-Date: Tue, 30 Mar 2021 20:27:30 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-10606 (part 2): Clean up ordering of requirements.txt
Laszlo Gaal has posted comments on this change. ( http://gerrit.cloudera.org:8080/17229 ) Change subject: IMPALA-10606 (part 2): Clean up ordering of requirements.txt .. Patch Set 2: Code-Review+2 -- To view, visit http://gerrit.cloudera.org:8080/17229 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I2f13ec8f8af80c4bac5da30d08a2ea4c56806d27 Gerrit-Change-Number: 17229 Gerrit-PatchSet: 2 Gerrit-Owner: Joe McDonnell Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Laszlo Gaal Gerrit-Comment-Date: Tue, 30 Mar 2021 20:27:22 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-10608: Update kudu-python version and remove some unused packages
Laszlo Gaal has posted comments on this change. ( http://gerrit.cloudera.org:8080/17231 ) Change subject: IMPALA-10608: Update kudu-python version and remove some unused packages .. Patch Set 2: Code-Review+2 (1 comment) http://gerrit.cloudera.org:8080/#/c/17231/2//COMMIT_MSG Commit Message: http://gerrit.cloudera.org:8080/#/c/17231/2//COMMIT_MSG@14 PS2, Line 14: no longer has a numpy dependency Yay! -- To view, visit http://gerrit.cloudera.org:8080/17231 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ib47770df9e46de448fe2bffef7abe2c3aa942fb9 Gerrit-Change-Number: 17231 Gerrit-PatchSet: 2 Gerrit-Owner: Joe McDonnell Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Laszlo Gaal Gerrit-Comment-Date: Tue, 30 Mar 2021 20:27:13 + Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-10397: De-flake test single workload
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/17239 ) Change subject: IMPALA-10397: De-flake test_single_workload .. Patch Set 2: Build started: https://jenkins.impala.io/job/gerrit-verify-dryrun/7022/ DRY_RUN=false -- To view, visit http://gerrit.cloudera.org:8080/17239 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I6f75afdbe676d9dd6922b6ba8aa1919daa161947 Gerrit-Change-Number: 17239 Gerrit-PatchSet: 2 Gerrit-Owner: Bikramjeet Vig Gerrit-Reviewer: Andrew Sherman Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Quanlong Huang Gerrit-Comment-Date: Tue, 30 Mar 2021 17:52:30 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-10397: De-flake test single workload
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/17239 ) Change subject: IMPALA-10397: De-flake test_single_workload .. Patch Set 2: Code-Review+2 -- To view, visit http://gerrit.cloudera.org:8080/17239 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I6f75afdbe676d9dd6922b6ba8aa1919daa161947 Gerrit-Change-Number: 17239 Gerrit-PatchSet: 2 Gerrit-Owner: Bikramjeet Vig Gerrit-Reviewer: Andrew Sherman Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Quanlong Huang Gerrit-Comment-Date: Tue, 30 Mar 2021 17:52:29 + Gerrit-HasComments: No
[Impala-ASF-CR] Bump up CDP BUILD NUMBER to 11918249
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/17241 ) Change subject: Bump up CDP_BUILD_NUMBER to 11918249 .. Patch Set 1: Build Successful https://jenkins.impala.io/job/gerrit-code-review-checks/8465/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests. -- To view, visit http://gerrit.cloudera.org:8080/17241 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I9b84afeb8cec13f9fba4e676077cde3c676e6537 Gerrit-Change-Number: 17241 Gerrit-PatchSet: 1 Gerrit-Owner: Zoltan Borok-Nagy Gerrit-Reviewer: Impala Public Jenkins Gerrit-Comment-Date: Tue, 30 Mar 2021 17:29:15 + Gerrit-HasComments: No
[Impala-ASF-CR] Bump up CDP BUILD NUMBER to 11918249
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/17241 ) Change subject: Bump up CDP_BUILD_NUMBER to 11918249 .. Patch Set 1: Build started: https://jenkins.impala.io/job/gerrit-verify-dryrun/7021/ DRY_RUN=true -- To view, visit http://gerrit.cloudera.org:8080/17241 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I9b84afeb8cec13f9fba4e676077cde3c676e6537 Gerrit-Change-Number: 17241 Gerrit-PatchSet: 1 Gerrit-Owner: Zoltan Borok-Nagy Gerrit-Reviewer: Impala Public Jenkins Gerrit-Comment-Date: Tue, 30 Mar 2021 17:07:20 + Gerrit-HasComments: No
[Impala-ASF-CR] Bump up CDP BUILD NUMBER to 11918249
Zoltan Borok-Nagy has uploaded this change for review. ( http://gerrit.cloudera.org:8080/17241 Change subject: Bump up CDP_BUILD_NUMBER to 11918249 .. Bump up CDP_BUILD_NUMBER to 11918249 This change bumps up the CDP_BUILD_NUMBER to 11918249. Among other things, the new CDP build has important features and fixes related to Iceberg. Change-Id: I9b84afeb8cec13f9fba4e676077cde3c676e6537 --- M bin/impala-config.sh 1 file changed, 12 insertions(+), 12 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/41/17241/1 -- To view, visit http://gerrit.cloudera.org:8080/17241 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newchange Gerrit-Change-Id: I9b84afeb8cec13f9fba4e676077cde3c676e6537 Gerrit-Change-Number: 17241 Gerrit-PatchSet: 1 Gerrit-Owner: Zoltan Borok-Nagy
[Impala-ASF-CR] IMPALA-10618: Update bootstrap system for Ubuntu 20.04
Jim Apple has posted comments on this change. ( http://gerrit.cloudera.org:8080/17240 ) Change subject: IMPALA-10618: Update bootstrap_system for Ubuntu 20.04 .. Patch Set 1: Code-Review+2 Should we have a "the minicluster starts" test in Jenkins that runs on 20.04? -- To view, visit http://gerrit.cloudera.org:8080/17240 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I212f6df3657cf9d621a0669573e1e511eae13662 Gerrit-Change-Number: 17240 Gerrit-PatchSet: 1 Gerrit-Owner: Laszlo Gaal Gerrit-Reviewer: Grant Henke Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Jim Apple Gerrit-Reviewer: Joe McDonnell Gerrit-Reviewer: Laszlo Gaal Gerrit-Reviewer: Qifan Chen Gerrit-Comment-Date: Tue, 30 Mar 2021 13:50:04 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-9470: Use Parquet Bloom filters - Part 1
Daniel Becker has posted comments on this change. ( http://gerrit.cloudera.org:8080/17026 ) Change subject: IMPALA-9470: Use Parquet Bloom filters - Part 1 .. Patch Set 20: (4 comments) Thanks for the reviews. There are a few TODOs about which I'm not quite sure, could you please help me with them? http://gerrit.cloudera.org:8080/#/c/17026/20/be/src/exec/parquet/hdfs-parquet-scanner.cc File be/src/exec/parquet/hdfs-parquet-scanner.cc: http://gerrit.cloudera.org:8080/#/c/17026/20/be/src/exec/parquet/hdfs-parquet-scanner.cc@1482 PS20, Line 1482: DCHECK(false); // TODO: Is it possible? Do you think this is a possibility? http://gerrit.cloudera.org:8080/#/c/17026/20/be/src/exec/parquet/hdfs-parquet-scanner.cc@1533 PS20, Line 1533: // TODO: That's what we do for stats filtering. Is it the correct way to handle What do you think of this? http://gerrit.cloudera.org:8080/#/c/17026/18/be/src/exec/parquet/parquet-bloom-filter-util.cc File be/src/exec/parquet/parquet-bloom-filter-util.cc: http://gerrit.cloudera.org:8080/#/c/17026/18/be/src/exec/parquet/parquet-bloom-filter-util.cc@170 PS18, Line 170: *ptr = storage->data(); : *len = storage->size(); > Resizing could be moved inside ParquetBloomFilterEncodeValue to make the ca Done http://gerrit.cloudera.org:8080/#/c/17026/20/be/src/exprs/expr-value.h File be/src/exprs/expr-value.h: http://gerrit.cloudera.org:8080/#/c/17026/20/be/src/exprs/expr-value.h@236 PS20, Line 236: // TODO: Is it OK for all these string types? What do you think of this, especially for TYPE_FIXED_UDA_INTERMEDIATE? -- To view, visit http://gerrit.cloudera.org:8080/17026 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I7119c7161fa3658e561fc1265430cb90079d8287 Gerrit-Change-Number: 17026 Gerrit-PatchSet: 20 Gerrit-Owner: Daniel Becker Gerrit-Reviewer: Csaba Ringhofer Gerrit-Reviewer: Daniel Becker Gerrit-Reviewer: Impala Public Jenkins Gerrit-Comment-Date: Tue, 30 Mar 2021 13:34:19 + Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-10619: Minor refactoring of standardize method for analytic functions
Quanlong Huang has posted comments on this change. ( http://gerrit.cloudera.org:8080/17237 ) Change subject: IMPALA-10619: Minor refactoring of standardize method for analytic functions .. Patch Set 1: (1 comment) http://gerrit.cloudera.org:8080/#/c/17237/1/fe/src/main/java/org/apache/impala/analysis/AnalyticExpr.java File fe/src/main/java/org/apache/impala/analysis/AnalyticExpr.java: http://gerrit.cloudera.org:8080/#/c/17237/1/fe/src/main/java/org/apache/impala/analysis/AnalyticExpr.java@833 PS1, Line 833: protected FunctionCallExpr createRewrittenFunction(FunctionName funcName, This seems a static method for FunctionCallExpr. Nothing special for analytic. Are we putting it here just for subclasses to override it? Is it possible to move it to FunctionCallExpr? -- To view, visit http://gerrit.cloudera.org:8080/17237 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I39e4268c0c5500f09acf98357a80763c28f615c2 Gerrit-Change-Number: 17237 Gerrit-PatchSet: 1 Gerrit-Owner: Aman Sinha Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Quanlong Huang Gerrit-Comment-Date: Tue, 30 Mar 2021 12:51:38 + Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-10618: Update bootstrap system for Ubuntu 20.04
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/17240 ) Change subject: IMPALA-10618: Update bootstrap_system for Ubuntu 20.04 .. Patch Set 1: Build Successful https://jenkins.impala.io/job/gerrit-code-review-checks/8464/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests. -- To view, visit http://gerrit.cloudera.org:8080/17240 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I212f6df3657cf9d621a0669573e1e511eae13662 Gerrit-Change-Number: 17240 Gerrit-PatchSet: 1 Gerrit-Owner: Laszlo Gaal Gerrit-Reviewer: Grant Henke Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Jim Apple Gerrit-Reviewer: Joe McDonnell Gerrit-Reviewer: Laszlo Gaal Gerrit-Reviewer: Qifan Chen Gerrit-Comment-Date: Tue, 30 Mar 2021 11:57:45 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-10618: Update bootstrap system for Ubuntu 20.04
Laszlo Gaal has uploaded this change for review. ( http://gerrit.cloudera.org:8080/17240 Change subject: IMPALA-10618: Update bootstrap_system for Ubuntu 20.04 .. IMPALA-10618: Update bootstrap_system for Ubuntu 20.04 Impala started adding Ubuntu 20.04 support in various places. This patch extends bootstrap_config.sh for Ubuntu 20.04 coverage: 1. The runtime version check error message is updated to claim support for Ubuntu 20.04. 2. Kudu needs libtinfo.5.so on Ubuntu 20.04 for the minicluster binaries. bin/bootstrap_system.sh now installs it when running on Ubuntu 20.04. 3. The OpenJDK default version reset to JDK 8 is extended to Ubuntu 20.04. Tested by running the code using docker/test-with-docker.py using --base-image=ubuntu:20.04 and observing that Kudu was able to start in the minicluster. The test runs completed, but there were test failures, for which separate tickets will be filed. Change-Id: I212f6df3657cf9d621a0669573e1e511eae13662 --- M bin/bootstrap_system.sh 1 file changed, 6 insertions(+), 4 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/40/17240/1 -- To view, visit http://gerrit.cloudera.org:8080/17240 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newchange Gerrit-Change-Id: I212f6df3657cf9d621a0669573e1e511eae13662 Gerrit-Change-Number: 17240 Gerrit-PatchSet: 1 Gerrit-Owner: Laszlo Gaal
[Impala-ASF-CR] IMPALA-10554: Block updates when row-filter/column-mask is enabled for the user
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/17230 ) Change subject: IMPALA-10554: Block updates when row-filter/column-mask is enabled for the user .. Patch Set 6: Verified+1 -- To view, visit http://gerrit.cloudera.org:8080/17230 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I1c899f2ec24b895867cbf2cf9ed23bc7b5a77326 Gerrit-Change-Number: 17230 Gerrit-PatchSet: 6 Gerrit-Owner: Quanlong Huang Gerrit-Reviewer: Aman Sinha Gerrit-Reviewer: Csaba Ringhofer Gerrit-Reviewer: Fang-Yu Rao Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Quanlong Huang Gerrit-Comment-Date: Tue, 30 Mar 2021 10:11:46 + Gerrit-HasComments: No
