[Impala-ASF-CR] IMPALA-8587: show grant does not produce correct privileges
Austin Nobis has posted comments on this change. ( http://gerrit.cloudera.org:8080/14356 ) Change subject: IMPALA-8587: show grant does not produce correct privileges .. Patch Set 4: Csaba - In regards to the Impala + Sentry implementation I don't believe it shows inherited privileges when you do SHOW GRANT on a resource, however, it is possible to just do SHOW GRANT with Impala + Sentry which will show all of the user's privileges. When Fredy and I were working on the initial Impala + Ranger integration we investigated if it would be possible to build the SHOW GRANT , but it seems that the functionality isn't supported by the current Ranger API. In regards to the Hive + Ranger implementation, it is similar to the change that Fang-Yu is currently proposing. Note that this may have changed in the months that have passed since I last worked on Ranger + Impala integration. Hive + Ranger will show that you have inherited privileges but it will not provide the "parent" that granted the privilege. I consider the change I initially proposed as an improvement compared to the Hive + Ranger implementation. > Patch Set 4: > > (5 comments) > > About the high level design: I prefer https://gerrit.cloudera.org/#/c/13673/ > , as it gives back the exact privileges that the user/group has. I can > imagine the scenario when you want to revoke someone's privilege to access a > given object, so you call SHOW GRANT, and then revoke the privileges you see > there. This will be more tricky if you cannot distinguish between > server/db/table/column level privileges in SHOW GRANT's output. > > Your change is simpler, but the whole class is just as complex in my opinion. > So I would prefer to take over Austin's change and some comments to make it > clearer. > > It would be also good to know how Hive handles inherited privileges + how > this works with Sentry in Impala. -- To view, visit http://gerrit.cloudera.org:8080/14356 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I8511656fe386a37a66d20e07ce1b875190bc4b65 Gerrit-Change-Number: 14356 Gerrit-PatchSet: 4 Gerrit-Owner: Fang-Yu Rao Gerrit-Reviewer: Austin Nobis Gerrit-Reviewer: Csaba Ringhofer Gerrit-Reviewer: Fang-Yu Rao Gerrit-Reviewer: Fredy Wijaya Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Quanlong Huang Gerrit-Comment-Date: Mon, 27 Jan 2020 15:34:05 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-8587: show grant does not produce correct privileges
Austin Nobis has posted comments on this change. ( http://gerrit.cloudera.org:8080/14356 ) Change subject: IMPALA-8587: show grant does not produce correct privileges .. Patch Set 1: Can you please add/update tests demonstrating the new functionality? I'm not sure this solves the issue outlined in IMPALA-8587. -- To view, visit http://gerrit.cloudera.org:8080/14356 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I8511656fe386a37a66d20e07ce1b875190bc4b65 Gerrit-Change-Number: 14356 Gerrit-PatchSet: 1 Gerrit-Owner: Fang-Yu Rao Gerrit-Reviewer: Austin Nobis Gerrit-Reviewer: Fang-Yu Rao Gerrit-Reviewer: Fredy Wijaya Gerrit-Reviewer: Impala Public Jenkins Gerrit-Comment-Date: Thu, 03 Oct 2019 00:14:56 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-8493 IMPALA-8494: [DOCS] GRANT/REVOKE User and Group
Austin Nobis has posted comments on this change. ( http://gerrit.cloudera.org:8080/13861 ) Change subject: IMPALA-8493 IMPALA-8494: [DOCS] GRANT/REVOKE User and Group .. Patch Set 1: Code-Review+1 -- To view, visit http://gerrit.cloudera.org:8080/13861 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I99165bcffa76b8da983fdbf905df4035f4afafe9 Gerrit-Change-Number: 13861 Gerrit-PatchSet: 1 Gerrit-Owner: Alex Rodoni Gerrit-Reviewer: Alex Rodoni Gerrit-Reviewer: Austin Nobis Gerrit-Reviewer: Impala Public Jenkins Gerrit-Comment-Date: Thu, 18 Jul 2019 14:34:49 + Gerrit-HasComments: No
[Impala-ASF-CR] [IMPALA-8587] Show inherited privileges with Ranger show grant
Austin Nobis has uploaded a new patch set (#2). ( http://gerrit.cloudera.org:8080/13673 ) Change subject: [IMPALA-8587] Show inherited privileges with Ranger show grant .. [IMPALA-8587] Show inherited privileges with Ranger show grant Previously when executing a show grant statement on a resource with Ranger authorization enabled, Impala would not show inherited privileges. For example, if a user had database level privileges such as: GRANT SELECT ON DATABASE db TO USER user; If a user then requested table level privileges such as: SHOW GRANT USER user ON TABLE db.table; They would see no results. After this change, the user will see database level privileges when executing the previous statement. If a user has SELECT privilege on DATABASE and on TABLE and issues a show grant on TABLE, they will only see the SELECT privilege for TABLE. Users will not see multiple instances of SELECT or any other privilege type in a SHOW GRANT statemenet. Testing - Ran all FE tests - Ran all authorization E2E tests - Added E2E tests in test_ranger verifying functionality Change-Id: I5c4c9327acb12abc12d130ef5c1ace6a08ed193c --- M fe/src/main/java/org/apache/impala/authorization/ranger/RangerImpaladAuthorizationManager.java M tests/authorization/test_ranger.py 2 files changed, 144 insertions(+), 24 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/73/13673/2 -- To view, visit http://gerrit.cloudera.org:8080/13673 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: I5c4c9327acb12abc12d130ef5c1ace6a08ed193c Gerrit-Change-Number: 13673 Gerrit-PatchSet: 2 Gerrit-Owner: Austin Nobis
[Impala-ASF-CR] [IMPALA-8587] Show inherited privileges with Ranger show grant
Hello Impala Public Jenkins, I'd like you to reexamine a change. Please visit http://gerrit.cloudera.org:8080/13673 to look at the new patch set (#3). Change subject: [IMPALA-8587] Show inherited privileges with Ranger show grant .. [IMPALA-8587] Show inherited privileges with Ranger show grant Previously when executing a show grant statement on a resource with Ranger authorization enabled, Impala would not show inherited privileges. For example, if a user had database level privileges such as: GRANT SELECT ON DATABASE db TO USER user; If a user then requested table level privileges such as: SHOW GRANT USER user ON TABLE db.table; They would see no results. After this change, the user will see database level privileges when executing the previous statement. If a user has SELECT privilege on DATABASE and on TABLE and issues a show grant on TABLE, they will only see the SELECT privilege for TABLE. Users will not see multiple instances of SELECT or any other privilege type in a SHOW GRANT statemenet. Testing - Ran all FE tests - Ran all authorization E2E tests - Added E2E tests in test_ranger verifying functionality Change-Id: I5c4c9327acb12abc12d130ef5c1ace6a08ed193c --- M fe/src/main/java/org/apache/impala/authorization/ranger/RangerImpaladAuthorizationManager.java M tests/authorization/test_ranger.py 2 files changed, 147 insertions(+), 24 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/73/13673/3 -- To view, visit http://gerrit.cloudera.org:8080/13673 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: I5c4c9327acb12abc12d130ef5c1ace6a08ed193c Gerrit-Change-Number: 13673 Gerrit-PatchSet: 3 Gerrit-Owner: Austin Nobis Gerrit-Reviewer: Impala Public Jenkins
[Impala-ASF-CR] [IMPALA-8587] Show inherited privileges with Ranger show grant
Austin Nobis has uploaded this change for review. ( http://gerrit.cloudera.org:8080/13673 Change subject: [IMPALA-8587] Show inherited privileges with Ranger show grant .. [IMPALA-8587] Show inherited privileges with Ranger show grant Previously when executing a show grant statement on a resource with Ranger authorization enabled, Impala would not show inherited privileges. For example, if a user had database level privileges such as: GRANT SELECT ON DATABASE db TO USER user; If a user then requested table level privileges such as: SHOW GRANT USER user ON TABLE db.table; They would see no results. After this change, the user will see database level privileges when executing the previous statement. If a user has SELECT privilege on DATABASE and on TABLE and issues a show grant on TABLE, they will only see the SELECT privilege for TABLE. Users will not see multiple instances of SELECT or any other privilege type in a SHOW GRANT statemenet. Testing - Ran all FE tests - Ran all authorization E2E tests - Added E2E tests in test_ranger verifying functionality Change-Id: I5c4c9327acb12abc12d130ef5c1ace6a08ed193c --- M fe/src/main/java/org/apache/impala/authorization/ranger/RangerImpaladAuthorizationManager.java M tests/authorization/test_ranger.py 2 files changed, 147 insertions(+), 27 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/73/13673/1 -- To view, visit http://gerrit.cloudera.org:8080/13673 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newchange Gerrit-Change-Id: I5c4c9327acb12abc12d130ef5c1ace6a08ed193c Gerrit-Change-Number: 13673 Gerrit-PatchSet: 1 Gerrit-Owner: Austin Nobis
[Impala-ASF-CR] IMPALA-8649: Fix confusing SHOW GRANT error messages
Austin Nobis has posted comments on this change. ( http://gerrit.cloudera.org:8080/13587 ) Change subject: IMPALA-8649: Fix confusing SHOW GRANT error messages .. Patch Set 2: Code-Review+1 -- To view, visit http://gerrit.cloudera.org:8080/13587 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ibb88bdc19cd1223902b44e3634f756d086332266 Gerrit-Change-Number: 13587 Gerrit-PatchSet: 2 Gerrit-Owner: Fredy Wijaya Gerrit-Reviewer: Austin Nobis Gerrit-Reviewer: Fredy Wijaya Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Todd Lipcon Gerrit-Comment-Date: Tue, 11 Jun 2019 22:09:32 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-8588: Fix revoke grant option with Ranger
Austin Nobis has posted comments on this change. ( http://gerrit.cloudera.org:8080/13450 ) Change subject: IMPALA-8588: Fix revoke grant option with Ranger .. Patch Set 2: (1 comment) > Patch Set 2: > > (1 comment) http://gerrit.cloudera.org:8080/#/c/13450/2//COMMIT_MSG Commit Message: http://gerrit.cloudera.org:8080/#/c/13450/2//COMMIT_MSG@20 PS2, Line 20: REVOKE GRANT OPTION FOR SELECT ON DATABASE FROM USER : : This will revoke the grant option for all privileges on this database : resource. It will not revoke the SELECT privilege on the resource. > i have some concern with this, especially since Ranger behaves differently I think Option 1 makes more sense as it offers better functionality to the user. That being said, I don't think it is required for this CR. I created a follow up JIRA, IMPALA-8651, for this work. -- To view, visit http://gerrit.cloudera.org:8080/13450 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Iddfccb442c3be3c266dbc2d8ae85c5674c534d7c Gerrit-Change-Number: 13450 Gerrit-PatchSet: 2 Gerrit-Owner: Austin Nobis Gerrit-Reviewer: Austin Nobis Gerrit-Reviewer: Fredy Wijaya Gerrit-Reviewer: Impala Public Jenkins Gerrit-Comment-Date: Tue, 11 Jun 2019 18:56:00 + Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-8604: Improve authorization test coverage for update/upsert/delete statements
Austin Nobis has posted comments on this change. ( http://gerrit.cloudera.org:8080/13480 ) Change subject: IMPALA-8604: Improve authorization test coverage for update/upsert/delete statements .. Patch Set 1: Code-Review+1 -- To view, visit http://gerrit.cloudera.org:8080/13480 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ic4095476945ff413fc59ec99dc3b9dfd71d95e96 Gerrit-Change-Number: 13480 Gerrit-PatchSet: 1 Gerrit-Owner: Fredy Wijaya Gerrit-Reviewer: Austin Nobis Gerrit-Reviewer: Fredy Wijaya Gerrit-Reviewer: Todd Lipcon Gerrit-Comment-Date: Thu, 30 May 2019 20:22:57 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-8588: Fix revoke grant option with Ranger
Hello Fredy Wijaya, Impala Public Jenkins, I'd like you to reexamine a change. Please visit http://gerrit.cloudera.org:8080/13450 to look at the new patch set (#2). Change subject: IMPALA-8588: Fix revoke grant option with Ranger .. IMPALA-8588: Fix revoke grant option with Ranger Previously, the REVOKE GRANT OPTION type statements would not only revoke the grant option, but also the privilege as well. The behavior has been updated to only revoke the grant option. In Ranger UI, this is seen as the delegate admin option. Examples: REVOKE SELECT ON DATABASE FROM USER This will revoke the SELECT privilege on the database resource, but if there are other privileges on that resource the grant option will remain for those privileges. REVOKE GRANT OPTION FOR SELECT ON DATABASE FROM USER This will revoke the grant option for all privileges on this database resource. It will not revoke the SELECT privilege on the resource. Testing: - Ran all FE tests - Ran all E2E tests - Updated test_ranger to test behavior for when REVOKE GRANT OPTION statements are submitted. Change-Id: Iddfccb442c3be3c266dbc2d8ae85c5674c534d7c --- M fe/src/main/java/org/apache/impala/authorization/ranger/RangerCatalogdAuthorizationManager.java M fe/src/test/java/org/apache/impala/authorization/AuthorizationTestBase.java M tests/authorization/test_ranger.py 3 files changed, 47 insertions(+), 26 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/50/13450/2 -- To view, visit http://gerrit.cloudera.org:8080/13450 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: Iddfccb442c3be3c266dbc2d8ae85c5674c534d7c Gerrit-Change-Number: 13450 Gerrit-PatchSet: 2 Gerrit-Owner: Austin Nobis Gerrit-Reviewer: Austin Nobis Gerrit-Reviewer: Fredy Wijaya Gerrit-Reviewer: Impala Public Jenkins
[Impala-ASF-CR] IMPALA-8588: Fix revoke grant option with Ranger
Austin Nobis has posted comments on this change. ( http://gerrit.cloudera.org:8080/13450 ) Change subject: IMPALA-8588: Fix revoke grant option with Ranger .. Patch Set 2: (1 comment) > Patch Set 1: > > (1 comment) http://gerrit.cloudera.org:8080/#/c/13450/1//COMMIT_MSG Commit Message: http://gerrit.cloudera.org:8080/#/c/13450/1//COMMIT_MSG@12 PS1, Line 12: seen as the delegate admin option. Examples: > It will be clearer to provide some examples here: Done -- To view, visit http://gerrit.cloudera.org:8080/13450 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Iddfccb442c3be3c266dbc2d8ae85c5674c534d7c Gerrit-Change-Number: 13450 Gerrit-PatchSet: 2 Gerrit-Owner: Austin Nobis Gerrit-Reviewer: Austin Nobis Gerrit-Reviewer: Fredy Wijaya Gerrit-Reviewer: Impala Public Jenkins Gerrit-Comment-Date: Thu, 30 May 2019 14:29:44 + Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-8588: Fix revoke grant option with Ranger
Austin Nobis has uploaded this change for review. ( http://gerrit.cloudera.org:8080/13450 Change subject: IMPALA-8588: Fix revoke grant option with Ranger .. IMPALA-8588: Fix revoke grant option with Ranger Previously, the REVOKE GRANT OPTION type statements would not only revoke the grant option, but also the privilege as well. The behavior has been updated to only revoke the grant option. In Ranger UI, this is seen as the delegate admin option. Testing: - Ran all FE tests - Ran all E2E tests - Updated test_ranger to test behavior for when REVOKE GRANT OPTION statements are submitted. Change-Id: Iddfccb442c3be3c266dbc2d8ae85c5674c534d7c --- M fe/src/main/java/org/apache/impala/authorization/ranger/RangerCatalogdAuthorizationManager.java M fe/src/test/java/org/apache/impala/authorization/AuthorizationTestBase.java M tests/authorization/test_ranger.py 3 files changed, 47 insertions(+), 26 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/50/13450/1 -- To view, visit http://gerrit.cloudera.org:8080/13450 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newchange Gerrit-Change-Id: Iddfccb442c3be3c266dbc2d8ae85c5674c534d7c Gerrit-Change-Number: 13450 Gerrit-PatchSet: 1 Gerrit-Owner: Austin Nobis Gerrit-Reviewer: Austin Nobis Gerrit-Reviewer: Fredy Wijaya
[Impala-ASF-CR] IMPALA-8248: Improve Ranger test coverage
Hello Fredy Wijaya, Impala Public Jenkins, I'd like you to reexamine a change. Please visit http://gerrit.cloudera.org:8080/13413 to look at the new patch set (#4). Change subject: IMPALA-8248: Improve Ranger test coverage .. IMPALA-8248: Improve Ranger test coverage This patch adds increased coverage for Apache Ranger integration. Specifically, tests were added that interact directly with Apache Ranger via the REST API and then assertions were made against Impala to test proper behavior. Testing: - Ran all E2E authorization tests - Added a test that adds hive privileges to Ranger and verifies they do not show in Impala. - Added a test that grants privileges to Ranger via the UI and runs "refresh authorization" to verify they exist in Impala. Change-Id: I15ce57ea96fbf6bff9bcabf7300fbadea7c55b09 --- M tests/authorization/test_ranger.py 1 file changed, 175 insertions(+), 0 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/13/13413/4 -- To view, visit http://gerrit.cloudera.org:8080/13413 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: I15ce57ea96fbf6bff9bcabf7300fbadea7c55b09 Gerrit-Change-Number: 13413 Gerrit-PatchSet: 4 Gerrit-Owner: Austin Nobis Gerrit-Reviewer: Austin Nobis Gerrit-Reviewer: Fredy Wijaya Gerrit-Reviewer: Impala Public Jenkins
[Impala-ASF-CR] [IMPALA-8248] Improve Ranger test coverage
Austin Nobis has uploaded a new patch set (#3). ( http://gerrit.cloudera.org:8080/13413 ) Change subject: [IMPALA-8248] Improve Ranger test coverage .. [IMPALA-8248] Improve Ranger test coverage This patch adds increased coverage for Apache Ranger integration. Specifically, tests were added that interact directly with Apache Ranger via the REST API and then assertions were made against Impala to test proper behavior. Testing: - Ran all E2E authorization tests - Added a test that adds hive privileges to Ranger and verifies they do not show in Impala. - Added a test that grants privileges to Ranger via the UI and runs "refresh authorization" to verify they exist in Impala. Change-Id: I15ce57ea96fbf6bff9bcabf7300fbadea7c55b09 --- M tests/authorization/test_ranger.py 1 file changed, 175 insertions(+), 0 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/13/13413/3 -- To view, visit http://gerrit.cloudera.org:8080/13413 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: I15ce57ea96fbf6bff9bcabf7300fbadea7c55b09 Gerrit-Change-Number: 13413 Gerrit-PatchSet: 3 Gerrit-Owner: Austin Nobis Gerrit-Reviewer: Austin Nobis Gerrit-Reviewer: Fredy Wijaya Gerrit-Reviewer: Impala Public Jenkins
[Impala-ASF-CR] [IMPALA-8248] Improve Ranger test coverage
Austin Nobis has posted comments on this change. (
http://gerrit.cloudera.org:8080/13413 )
Change subject: [IMPALA-8248] Improve Ranger test coverage
..
Patch Set 3:
(4 comments)
http://gerrit.cloudera.org:8080/#/c/13413/2/tests/authorization/test_ranger.py
File tests/authorization/test_ranger.py:
http://gerrit.cloudera.org:8080/#/c/13413/2/tests/authorization/test_ranger.py@352
PS2, Line 352: # Privileges should be stale before a refresh
: result = self.client.execute("show grant user {0} on
database {1}"
:.format(user, unique_db))
: TestRanger._check_privileges(result, [])
:
: # Refresh and check updated privileges
: admin_client.execute("refresh authorization")
: result = self.client.execute("show grant user {0} on
database {1}"
:.format(user, unique_db))
:
: TestRanger._check_privileges(result, [
: ["USER", user, unique_db, "*", "*", "", "", "create",
"false"],
: ["USER", user, unique_db, "*", "*", "", "", "select",
"false"]
: ])
:
> we should add some asserts before the "refresh authorization" that the newl
Done
http://gerrit.cloudera.org:8080/#/c/13413/2/tests/authorization/test_ranger.py@369
PS2, Line 369:
> nit: remove extra new line
Done
http://gerrit.cloudera.org:8080/#/c/13413/2/tests/authorization/test_ranger.py@371
PS2, Line 371: result = self.client.execute("show grant user {0} on
database {1}"
:.format(user, uniqu
> why is this commented out?
Done
http://gerrit.cloudera.org:8080/#/c/13413/2/tests/authorization/test_ranger.py@474
PS2, Line 474: }
:
: headers = {"Content-Type": "application/json", "Accept":
"application/json"}
: r =
requests.post("{0}/service/plugins/services/revoke/test_impala?pluginId=impala"
: .format(RANGER_HOST),
: auth=RANGER_AUTH, jso
> nit; let's not mix '' with "" for strings. For consistency, let's use "".
Done
--
To view, visit http://gerrit.cloudera.org:8080/13413
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I15ce57ea96fbf6bff9bcabf7300fbadea7c55b09
Gerrit-Change-Number: 13413
Gerrit-PatchSet: 3
Gerrit-Owner: Austin Nobis
Gerrit-Reviewer: Austin Nobis
Gerrit-Reviewer: Fredy Wijaya
Gerrit-Reviewer: Impala Public Jenkins
Gerrit-Comment-Date: Fri, 24 May 2019 15:04:37 +
Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-8049: [DOCS] Ranger authz support in impala
Austin Nobis has posted comments on this change. ( http://gerrit.cloudera.org:8080/13368 ) Change subject: IMPALA-8049: [DOCS] Ranger authz support in impala .. Patch Set 3: Code-Review+1 -- To view, visit http://gerrit.cloudera.org:8080/13368 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I4858bc49c1ed6d5e65ddbaebc96e56427446bad6 Gerrit-Change-Number: 13368 Gerrit-PatchSet: 3 Gerrit-Owner: Alex Rodoni Gerrit-Reviewer: Alex Rodoni Gerrit-Reviewer: Austin Nobis Gerrit-Reviewer: Fredy Wijaya Gerrit-Reviewer: Impala Public Jenkins Gerrit-Comment-Date: Thu, 23 May 2019 23:19:33 + Gerrit-HasComments: No
[Impala-ASF-CR] [IMPALA-8248] Improve Ranger test coverage
Austin Nobis has uploaded this change for review. ( http://gerrit.cloudera.org:8080/13413 Change subject: [IMPALA-8248] Improve Ranger test coverage .. [IMPALA-8248] Improve Ranger test coverage This patch adds increased coverage for Apache Ranger integration. Specifically, tests were added that interact directly with Apache Ranger via the REST API and then assertions were made against Impala to test proper behavior. Testing: - Ran all E2E authorization tests - Added a test that adds hive privileges to Ranger and verifies they do not show in Impala. - Added a test that grants privileges to Ranger via the UI and runs "refresh authorization" to verify they exist in Impala. Change-Id: I15ce57ea96fbf6bff9bcabf7300fbadea7c55b09 --- M tests/authorization/test_ranger.py 1 file changed, 161 insertions(+), 0 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/13/13413/2 -- To view, visit http://gerrit.cloudera.org:8080/13413 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newchange Gerrit-Change-Id: I15ce57ea96fbf6bff9bcabf7300fbadea7c55b09 Gerrit-Change-Number: 13413 Gerrit-PatchSet: 2 Gerrit-Owner: Austin Nobis
[Impala-ASF-CR] IMPALA-8049: [DOCS] Ranger authz support in impala
Austin Nobis has posted comments on this change. ( http://gerrit.cloudera.org:8080/13368 ) Change subject: IMPALA-8049: [DOCS] Ranger authz support in impala .. Patch Set 2: (1 comment) http://gerrit.cloudera.org:8080/#/c/13368/2/docs/topics/impala_authorization.xml File docs/topics/impala_authorization.xml: http://gerrit.cloudera.org:8080/#/c/13368/2/docs/topics/impala_authorization.xml@226 PS2, Line 226: or Ranger There is also a polling frequency for refreshing Ranger policies but I don't believe it is exposed as an Impala flag. Instead it depends on the Ranger config XML that needs to be on the classpath when starting Impala w/ Ranger. -- To view, visit http://gerrit.cloudera.org:8080/13368 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I4858bc49c1ed6d5e65ddbaebc96e56427446bad6 Gerrit-Change-Number: 13368 Gerrit-PatchSet: 2 Gerrit-Owner: Alex Rodoni Gerrit-Reviewer: Austin Nobis Gerrit-Reviewer: Fredy Wijaya Gerrit-Reviewer: Impala Public Jenkins Gerrit-Comment-Date: Thu, 23 May 2019 14:53:56 + Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-8049: [DOCS] Ranger authz support in impala
Austin Nobis has posted comments on this change. ( http://gerrit.cloudera.org:8080/13368 ) Change subject: IMPALA-8049: [DOCS] Ranger authz support in impala .. Patch Set 2: (1 comment) http://gerrit.cloudera.org:8080/#/c/13368/2/docs/topics/impala_authorization.xml File docs/topics/impala_authorization.xml: http://gerrit.cloudera.org:8080/#/c/13368/2/docs/topics/impala_authorization.xml@239 PS2, Line 239: INVALIDATE METADATA For Ranger I think you only need REFRESH AUTHORIZATION and not INVALIDATE METADATA at least for "modifying privileges". I'm not sure how it interacts with "adding a user" and "removing a user". -- To view, visit http://gerrit.cloudera.org:8080/13368 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I4858bc49c1ed6d5e65ddbaebc96e56427446bad6 Gerrit-Change-Number: 13368 Gerrit-PatchSet: 2 Gerrit-Owner: Alex Rodoni Gerrit-Reviewer: Austin Nobis Gerrit-Reviewer: Fredy Wijaya Gerrit-Reviewer: Impala Public Jenkins Gerrit-Comment-Date: Thu, 23 May 2019 14:51:24 + Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-8400: Implement Ranger audit event handler
Austin Nobis has posted comments on this change. ( http://gerrit.cloudera.org:8080/13309 ) Change subject: IMPALA-8400: Implement Ranger audit event handler .. Patch Set 7: Code-Review+1 -- To view, visit http://gerrit.cloudera.org:8080/13309 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ife93c3e708875ef5fc0117153ad8ee225a88518b Gerrit-Change-Number: 13309 Gerrit-PatchSet: 7 Gerrit-Owner: Fredy Wijaya Gerrit-Reviewer: Austin Nobis Gerrit-Reviewer: Fredy Wijaya Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Todd Lipcon Gerrit-Comment-Date: Tue, 21 May 2019 02:45:40 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-8473: publish lineage info via hook
Austin Nobis has posted comments on this change. (
http://gerrit.cloudera.org:8080/13352 )
Change subject: IMPALA-8473: publish lineage info via hook
..
Patch Set 15:
(3 comments)
http://gerrit.cloudera.org:8080/#/c/13352/15/be/src/service/impala-server.cc
File be/src/service/impala-server.cc:
http://gerrit.cloudera.org:8080/#/c/13352/15/be/src/service/impala-server.cc@505
PS15, Line 505: if (!status.ok()) {
I don't see a test case for this.
Looking at the code, the Hook function execution is sent into a List>
which won't be resolved because they are asynchronous.
However, the return of this status is synchronous and exceptions thrown by the
Future probably won't be thrown before this status is returned.
I definitely think we need tests for this feature.
http://gerrit.cloudera.org:8080/#/c/13352/15/fe/src/main/java/org/apache/impala/hooks/QueryExecHook.java
File fe/src/main/java/org/apache/impala/hooks/QueryExecHook.java:
http://gerrit.cloudera.org:8080/#/c/13352/15/fe/src/main/java/org/apache/impala/hooks/QueryExecHook.java@35
PS15, Line 35:* Any {@link Exception} thrown from this method will
effectively fail
Is this the proper behavior? Do we really want to prevent impala from starting
up if a hook doesn't properly start?
Maybe a better name for this function would be `onStartup` or `onImpalaStartup`
since the function executes when startup occurs.
http://gerrit.cloudera.org:8080/#/c/13352/15/fe/src/main/java/org/apache/impala/hooks/QueryExecHook.java@52
PS15, Line 52:* Any {@link Exception} thrown from this method will only be
caught
Aren't the exceptions from this function re-thrown in the Manager?
--
To view, visit http://gerrit.cloudera.org:8080/13352
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I23a896537a98bfef07fb27c70e9a87c105cd77a1
Gerrit-Change-Number: 13352
Gerrit-PatchSet: 15
Gerrit-Owner: radford nguyen
Gerrit-Reviewer: Andrew Sherman
Gerrit-Reviewer: Anonymous Coward (498)
Gerrit-Reviewer: Austin Nobis
Gerrit-Reviewer: Fredy Wijaya
Gerrit-Reviewer: Impala Public Jenkins
Gerrit-Reviewer: radford nguyen
Gerrit-Comment-Date: Mon, 20 May 2019 19:08:54 +
Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-8473: publish lineage info via hook
Austin Nobis has posted comments on this change. ( http://gerrit.cloudera.org:8080/13352 ) Change subject: IMPALA-8473: publish lineage info via hook .. Patch Set 15: (1 comment) http://gerrit.cloudera.org:8080/#/c/13352/15/fe/src/main/java/org/apache/impala/hooks/QueryExecHookManager.java File fe/src/main/java/org/apache/impala/hooks/QueryExecHookManager.java: http://gerrit.cloudera.org:8080/#/c/13352/15/fe/src/main/java/org/apache/impala/hooks/QueryExecHookManager.java@161 PS15, Line 161: hookExecutor_.shutdown(); I think it is possible to lose logs during a shutdown if we don't `awaitTermination` -- To view, visit http://gerrit.cloudera.org:8080/13352 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I23a896537a98bfef07fb27c70e9a87c105cd77a1 Gerrit-Change-Number: 13352 Gerrit-PatchSet: 15 Gerrit-Owner: radford nguyen Gerrit-Reviewer: Andrew Sherman Gerrit-Reviewer: Anonymous Coward (498) Gerrit-Reviewer: Austin Nobis Gerrit-Reviewer: Fredy Wijaya Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: radford nguyen Gerrit-Comment-Date: Mon, 20 May 2019 19:18:51 + Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-8400: Implement Ranger audit event handler
Austin Nobis has posted comments on this change. (
http://gerrit.cloudera.org:8080/13309 )
Change subject: IMPALA-8400: Implement Ranger audit event handler
..
Patch Set 6:
(2 comments)
http://gerrit.cloudera.org:8080/#/c/13309/4/fe/src/main/java/org/apache/impala/analysis/AnalysisContext.java
File fe/src/main/java/org/apache/impala/analysis/AnalysisContext.java:
http://gerrit.cloudera.org:8080/#/c/13309/4/fe/src/main/java/org/apache/impala/analysis/AnalysisContext.java@435
PS4, Line 435: authzChecker.authorize(analysisResult_, catalog_,
authzCtxConsumer);
: } catch (AuthorizationException e) {
: authException = e;
: }
:
: // AuthorizationExceptions take precedence over
AnalysisExceptions so as not
: // to reveal the existence/absence of objects the user is
not authorized to see.
: if (authException != null) throw authException;
: if (analysisException != null) throw analysisException;
: r
> Done. Rename the overloaded authorize to authorizeResource().
I think I prefer the implementation in Patch Set 4. The
preAuthorize/postAuthorize weren't actually removed in Patch Set 6 and adding
the Consumer<> makes everything even more confusing.
I believe the implementation is correct in patch set 4 so I'll +1 on the revert.
http://gerrit.cloudera.org:8080/#/c/13309/6/fe/src/main/java/org/apache/impala/authorization/BaseAuthorizationChecker.java
File
fe/src/main/java/org/apache/impala/authorization/BaseAuthorizationChecker.java:
http://gerrit.cloudera.org:8080/#/c/13309/6/fe/src/main/java/org/apache/impala/authorization/BaseAuthorizationChecker.java@64
PS6, Line 64: is used "show databases"
nit: is used by
--
To view, visit http://gerrit.cloudera.org:8080/13309
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Ife93c3e708875ef5fc0117153ad8ee225a88518b
Gerrit-Change-Number: 13309
Gerrit-PatchSet: 6
Gerrit-Owner: Fredy Wijaya
Gerrit-Reviewer: Austin Nobis
Gerrit-Reviewer: Fredy Wijaya
Gerrit-Reviewer: Impala Public Jenkins
Gerrit-Reviewer: Todd Lipcon
Gerrit-Comment-Date: Mon, 20 May 2019 18:14:19 +
Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-8443: Record time spent in authorization in the runtime profile
Austin Nobis has posted comments on this change. ( http://gerrit.cloudera.org:8080/13353 ) Change subject: IMPALA-8443: Record time spent in authorization in the runtime profile .. Patch Set 2: (1 comment) http://gerrit.cloudera.org:8080/#/c/13353/2/fe/src/main/java/org/apache/impala/analysis/AnalysisContext.java File fe/src/main/java/org/apache/impala/analysis/AnalysisContext.java: http://gerrit.cloudera.org:8080/#/c/13353/2/fe/src/main/java/org/apache/impala/analysis/AnalysisContext.java@426 PS2, Line 426: (Ranger|Sentry) > Adding the current provider sounds good to me, it would be one less thing t I think going with that approach will cause a lot of conflicts with Fredy's open CR https://gerrit.cloudera.org/c/13309/. Would the approach be to move all of the Authorization related code from AnalysisContext into Frontend? I'm not opposed to moving the code out of the AnalysisContext but maybe we should create a new class as opposed to putting it all in the Frontend. Thoughts? Would also be helpful if Fredy commented because he may have other plans for this code. -- To view, visit http://gerrit.cloudera.org:8080/13353 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I5bb85e57fcc75d41f3eb2911e6d375e0da6f82ae Gerrit-Change-Number: 13353 Gerrit-PatchSet: 2 Gerrit-Owner: Tamas Mate Gerrit-Reviewer: Austin Nobis Gerrit-Reviewer: Fredy Wijaya Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Tamas Mate Gerrit-Comment-Date: Mon, 20 May 2019 18:29:04 + Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-8443: Record time spent in authorization in the runtime profile
Austin Nobis has posted comments on this change. ( http://gerrit.cloudera.org:8080/13353 ) Change subject: IMPALA-8443: Record time spent in authorization in the runtime profile .. Patch Set 2: (1 comment) http://gerrit.cloudera.org:8080/#/c/13353/2/fe/src/main/java/org/apache/impala/analysis/AnalysisContext.java File fe/src/main/java/org/apache/impala/analysis/AnalysisContext.java: http://gerrit.cloudera.org:8080/#/c/13353/2/fe/src/main/java/org/apache/impala/analysis/AnalysisContext.java@426 PS2, Line 426: (Ranger|Sentry) If we expose access to the AuthorizationConfig stored in the AuthorizationChecker, it would be possible to determine which of Ranger/Sentry/NOOP authorization is being used. Would this be useful for profiling? -- To view, visit http://gerrit.cloudera.org:8080/13353 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I5bb85e57fcc75d41f3eb2911e6d375e0da6f82ae Gerrit-Change-Number: 13353 Gerrit-PatchSet: 2 Gerrit-Owner: Tamas Mate Gerrit-Reviewer: Austin Nobis Gerrit-Reviewer: Fredy Wijaya Gerrit-Reviewer: Impala Public Jenkins Gerrit-Comment-Date: Mon, 20 May 2019 14:32:48 + Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-8400: Implement Ranger audit event handler
Austin Nobis has posted comments on this change. (
http://gerrit.cloudera.org:8080/13309 )
Change subject: IMPALA-8400: Implement Ranger audit event handler
..
Patch Set 4:
(1 comment)
http://gerrit.cloudera.org:8080/#/c/13309/4/fe/src/main/java/org/apache/impala/analysis/AnalysisContext.java
File fe/src/main/java/org/apache/impala/analysis/AnalysisContext.java:
http://gerrit.cloudera.org:8080/#/c/13309/4/fe/src/main/java/org/apache/impala/analysis/AnalysisContext.java@435
PS4, Line 435: try {
: authzCtx = authzChecker.preAuthorize(analysisResult_,
catalog_);
: authzChecker.authorize(authzCtx, analysisResult_,
catalog_);
: } catch (AuthorizationException e) {
: authException = e;
: } finally {
: if (authzCtx != null) {
: authzChecker.postAuthorize(authzCtx, analysisResult_,
catalog_);
: }
: }
Had a discussion with Fredy because I didn't understand the purpose of having
`preAuthorize`, `authorize`, and then `postAuthorize` called sequentially.
The `BaseAuthorizationChecker` class has 2 `authorize` methods with different
signatures. The `abstract` one is the one that is being `override`'d. This is
why these 3 methods can't be merged into a single `authorize`.
Fredy and I came to an agreement that we should rename the `abstract authorize`
method to `authorizeResource` as it is an authorization that occurs per
resources, whereas the `authorize` being called here is the authorization for
the entire SQL statement.
The logic flow is as follows:
RangerAuthorizationChecker#preAuthorize
BaseAuthorizationChecker#authorize
foreach resource: RangerAuthorizationChecker#authorize (authorizeResource)
RangerAuthorizationChecker#postAuthorize
I still think the `preAuthorize`, `authorize`, `postAuthorize` sequence is a
little strange as the methods execute sequentially.
--
To view, visit http://gerrit.cloudera.org:8080/13309
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Ife93c3e708875ef5fc0117153ad8ee225a88518b
Gerrit-Change-Number: 13309
Gerrit-PatchSet: 4
Gerrit-Owner: Fredy Wijaya
Gerrit-Reviewer: Austin Nobis
Gerrit-Reviewer: Fredy Wijaya
Gerrit-Reviewer: Impala Public Jenkins
Gerrit-Reviewer: Todd Lipcon
Gerrit-Comment-Date: Tue, 14 May 2019 22:44:14 +
Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-8497: dealing with query ends with '\n'
Austin Nobis has posted comments on this change. ( http://gerrit.cloudera.org:8080/13293 ) Change subject: IMPALA-8497: dealing with query ends with '\n' .. Patch Set 4: (3 comments) http://gerrit.cloudera.org:8080/#/c/13293/4//COMMIT_MSG Commit Message: http://gerrit.cloudera.org:8080/#/c/13293/4//COMMIT_MSG@10 PS4, Line 10: Syntax nit: syntax* http://gerrit.cloudera.org:8080/#/c/13293/4//COMMIT_MSG@11 PS4, Line 11: sumbit nit: submit* http://gerrit.cloudera.org:8080/#/c/13293/4//COMMIT_MSG@15 PS4, Line 15: Tests: : * Add test cases in ParserTest.java Did you run the FE tests via maven? If so, add it here. If not, please run the FE tests. -- To view, visit http://gerrit.cloudera.org:8080/13293 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I3f034b351d0468a3f6482e27ddef818b34d8 Gerrit-Change-Number: 13293 Gerrit-PatchSet: 4 Gerrit-Owner: wangsheng Gerrit-Reviewer: Austin Nobis Gerrit-Reviewer: Fredy Wijaya Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Quanlong Huang Gerrit-Comment-Date: Mon, 13 May 2019 16:57:41 + Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-8528: Refactor authorization check in AnalysisContext
Austin Nobis has posted comments on this change. ( http://gerrit.cloudera.org:8080/13285 ) Change subject: IMPALA-8528: Refactor authorization check in AnalysisContext .. Patch Set 6: Code-Review+1 -- To view, visit http://gerrit.cloudera.org:8080/13285 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I3bc3a11220dae0f49ef3e73d9ff27a90e9d4a71c Gerrit-Change-Number: 13285 Gerrit-PatchSet: 6 Gerrit-Owner: Fredy Wijaya Gerrit-Reviewer: Austin Nobis Gerrit-Reviewer: Fredy Wijaya Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Tim Armstrong Gerrit-Reviewer: Todd Lipcon Gerrit-Comment-Date: Mon, 13 May 2019 03:30:43 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-8528: Refactor authorization check in AnalysisContext
Austin Nobis has posted comments on this change. (
http://gerrit.cloudera.org:8080/13285 )
Change subject: IMPALA-8528: Refactor authorization check in AnalysisContext
..
Patch Set 5:
(4 comments)
http://gerrit.cloudera.org:8080/#/c/13285/5/fe/src/main/java/org/apache/impala/analysis/AnalysisContext.java
File fe/src/main/java/org/apache/impala/analysis/AnalysisContext.java:
http://gerrit.cloudera.org:8080/#/c/13285/5/fe/src/main/java/org/apache/impala/analysis/AnalysisContext.java@421
PS5, Line 421: AuthorizationContext authzCtx = null;
: try {
: authzCtx = authzChecker.preAuthorize(analysisResult_,
catalog_);
: authzChecker.authorize(authzCtx, analysisResult_,
catalog_);
: } catch (AuthorizationException e) {
: authException = e;
: } finally {
: if (authzCtx != null) {
: authzChecker.postAuthorize(authzCtx, analysisResult_,
catalog_);
: }
: }
:
: // AuthorizationExceptions take precedence over
AnalysisExceptions so as not
: // to reveal the existence/absence of objects the user is
not authorized to see.
: if (authException != null) throw authException;
Can't this just be a try { } finally { } and you can remove the authException
local variable?
Also it seems like preAuthorize, authorize, postAuthorize are all called
sequentially. I'm not sure of the benefit of having them be separate methods in
the interface.
http://gerrit.cloudera.org:8080/#/c/13285/5/fe/src/main/java/org/apache/impala/authorization/AuthorizationChecker.java
File fe/src/main/java/org/apache/impala/authorization/AuthorizationChecker.java:
http://gerrit.cloudera.org:8080/#/c/13285/5/fe/src/main/java/org/apache/impala/authorization/AuthorizationChecker.java@29
PS5, Line 29: public interface AuthorizationChecker {
Is there a reason that we need preAuth, auth, postAuth as opposed to just
having an auth method and leaving those details up the class that implements
this interface?
http://gerrit.cloudera.org:8080/#/c/13285/5/fe/src/main/java/org/apache/impala/authorization/AuthorizationChecker.java@38
PS5, Line 38: Executes some code before the authorization check.
nit: this documentation seems a bit casual.
maybe: "Function to be executed before an authorization check occurs."
http://gerrit.cloudera.org:8080/#/c/13285/5/fe/src/main/java/org/apache/impala/authorization/AuthorizationChecker.java@53
PS5, Line 53:* Executes some code after the authorization check.
nit: same as above
--
To view, visit http://gerrit.cloudera.org:8080/13285
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I3bc3a11220dae0f49ef3e73d9ff27a90e9d4a71c
Gerrit-Change-Number: 13285
Gerrit-PatchSet: 5
Gerrit-Owner: Fredy Wijaya
Gerrit-Reviewer: Austin Nobis
Gerrit-Reviewer: Fredy Wijaya
Gerrit-Reviewer: Impala Public Jenkins
Gerrit-Reviewer: Tim Armstrong
Gerrit-Reviewer: Todd Lipcon
Gerrit-Comment-Date: Fri, 10 May 2019 19:47:50 +
Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-8281: Misc Sentry decoupling clean up
Austin Nobis has uploaded a new patch set (#11). ( http://gerrit.cloudera.org:8080/13284 ) Change subject: IMPALA-8281: Misc Sentry decoupling clean up .. IMPALA-8281: Misc Sentry decoupling clean up This patch moves Sentry specific code to the Sentry specific plugin implementation. Testing: - Ran all FE tests - Ran E2E authorization tests - Added new E2E tests in test_sentry Change-Id: Id24a00dd395e30e4c392f085893e9561da2ee539 --- M common/thrift/Frontend.thrift M fe/src/main/java/org/apache/impala/analysis/CreateDropRoleStmt.java M fe/src/main/java/org/apache/impala/analysis/GrantRevokePrivStmt.java M fe/src/main/java/org/apache/impala/analysis/GrantRevokeRoleStmt.java M fe/src/main/java/org/apache/impala/analysis/ShowGrantPrincipalStmt.java M fe/src/main/java/org/apache/impala/analysis/ShowRolesStmt.java M fe/src/main/java/org/apache/impala/authorization/sentry/SentryCatalogdAuthorizationManager.java M fe/src/main/java/org/apache/impala/authorization/sentry/SentryImpaladAuthorizationManager.java M fe/src/main/java/org/apache/impala/service/Frontend.java M fe/src/test/java/org/apache/impala/analysis/AnalyzeAuthStmtsTest.java M tests/authorization/test_sentry.py 11 files changed, 140 insertions(+), 104 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/84/13284/11 -- To view, visit http://gerrit.cloudera.org:8080/13284 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: Id24a00dd395e30e4c392f085893e9561da2ee539 Gerrit-Change-Number: 13284 Gerrit-PatchSet: 11 Gerrit-Owner: Austin Nobis Gerrit-Reviewer: Austin Nobis Gerrit-Reviewer: Fredy Wijaya Gerrit-Reviewer: Impala Public Jenkins
[Impala-ASF-CR] IMPALA-8281: Misc Sentry decoupling clean up
Austin Nobis has uploaded a new patch set (#10). ( http://gerrit.cloudera.org:8080/13284 ) Change subject: IMPALA-8281: Misc Sentry decoupling clean up .. IMPALA-8281: Misc Sentry decoupling clean up This patch moves Sentry specific code to the Sentry specific plugin implementation. Testing: - Ran all FE tests - Ran E2E authorization tests - Added new E2E tests in test_sentry Change-Id: Id24a00dd395e30e4c392f085893e9561da2ee539 --- M common/thrift/Frontend.thrift M fe/src/main/java/org/apache/impala/analysis/CreateDropRoleStmt.java M fe/src/main/java/org/apache/impala/analysis/GrantRevokePrivStmt.java M fe/src/main/java/org/apache/impala/analysis/GrantRevokeRoleStmt.java M fe/src/main/java/org/apache/impala/analysis/ShowGrantPrincipalStmt.java M fe/src/main/java/org/apache/impala/analysis/ShowRolesStmt.java M fe/src/main/java/org/apache/impala/authorization/sentry/SentryCatalogdAuthorizationManager.java M fe/src/main/java/org/apache/impala/authorization/sentry/SentryImpaladAuthorizationManager.java M fe/src/main/java/org/apache/impala/service/Frontend.java M fe/src/test/java/org/apache/impala/analysis/AnalyzeAuthStmtsTest.java M tests/authorization/test_sentry.py 11 files changed, 140 insertions(+), 104 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/84/13284/10 -- To view, visit http://gerrit.cloudera.org:8080/13284 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: Id24a00dd395e30e4c392f085893e9561da2ee539 Gerrit-Change-Number: 13284 Gerrit-PatchSet: 10 Gerrit-Owner: Austin Nobis Gerrit-Reviewer: Austin Nobis Gerrit-Reviewer: Fredy Wijaya Gerrit-Reviewer: Impala Public Jenkins
[Impala-ASF-CR] IMPALA-8281: Misc Sentry decoupling clean up
Austin Nobis has posted comments on this change. (
http://gerrit.cloudera.org:8080/13284 )
Change subject: IMPALA-8281: Misc Sentry decoupling clean up
..
Patch Set 9:
(1 comment)
http://gerrit.cloudera.org:8080/#/c/13284/7/tests/authorization/test_sentry.py
File tests/authorization/test_sentry.py:
http://gerrit.cloudera.org:8080/#/c/13284/7/tests/authorization/test_sentry.py@93
PS7, Line 93: finally:
: self.client.execute("drop database {0}".format(unique_name))
: self.client.execute(
> I did a quick test from master, I don't think this behavior in this CR is c
Done
--
To view, visit http://gerrit.cloudera.org:8080/13284
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Id24a00dd395e30e4c392f085893e9561da2ee539
Gerrit-Change-Number: 13284
Gerrit-PatchSet: 9
Gerrit-Owner: Austin Nobis
Gerrit-Reviewer: Austin Nobis
Gerrit-Reviewer: Fredy Wijaya
Gerrit-Reviewer: Impala Public Jenkins
Gerrit-Comment-Date: Fri, 10 May 2019 16:59:38 +
Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-8281: Misc Sentry decoupling clean up
Hello Fredy Wijaya, Impala Public Jenkins, I'd like you to reexamine a change. Please visit http://gerrit.cloudera.org:8080/13284 to look at the new patch set (#9). Change subject: IMPALA-8281: Misc Sentry decoupling clean up .. IMPALA-8281: Misc Sentry decoupling clean up This patch moves Sentry specific code to the Sentry specific plugin implementation. Testing: - Ran all FE tests - Ran E2E authorization tests - Added new E2E tests in test_sentry Change-Id: Id24a00dd395e30e4c392f085893e9561da2ee539 --- M common/thrift/Frontend.thrift M fe/src/main/java/org/apache/impala/analysis/CreateDropRoleStmt.java M fe/src/main/java/org/apache/impala/analysis/GrantRevokePrivStmt.java M fe/src/main/java/org/apache/impala/analysis/GrantRevokeRoleStmt.java M fe/src/main/java/org/apache/impala/analysis/ShowGrantPrincipalStmt.java M fe/src/main/java/org/apache/impala/analysis/ShowRolesStmt.java M fe/src/main/java/org/apache/impala/authorization/sentry/SentryCatalogdAuthorizationManager.java M fe/src/main/java/org/apache/impala/authorization/sentry/SentryImpaladAuthorizationManager.java M fe/src/main/java/org/apache/impala/service/Frontend.java M fe/src/test/java/org/apache/impala/analysis/AnalyzeAuthStmtsTest.java M tests/authorization/test_sentry.py 11 files changed, 128 insertions(+), 106 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/84/13284/9 -- To view, visit http://gerrit.cloudera.org:8080/13284 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: Id24a00dd395e30e4c392f085893e9561da2ee539 Gerrit-Change-Number: 13284 Gerrit-PatchSet: 9 Gerrit-Owner: Austin Nobis Gerrit-Reviewer: Austin Nobis Gerrit-Reviewer: Fredy Wijaya Gerrit-Reviewer: Impala Public Jenkins
[Impala-ASF-CR] IMPALA-8281: Misc Sentry decoupling clean up
Hello Fredy Wijaya, Impala Public Jenkins, I'd like you to reexamine a change. Please visit http://gerrit.cloudera.org:8080/13284 to look at the new patch set (#8). Change subject: IMPALA-8281: Misc Sentry decoupling clean up .. IMPALA-8281: Misc Sentry decoupling clean up This patch moves Sentry specific code to the Sentry specific plugin implementation. Testing: - Ran all FE tests - Ran E2E authorization tests - Added new E2E tests in test_sentry Change-Id: Id24a00dd395e30e4c392f085893e9561da2ee539 --- M common/thrift/Frontend.thrift M fe/src/main/java/org/apache/impala/analysis/GrantRevokePrivStmt.java M fe/src/main/java/org/apache/impala/analysis/GrantRevokeRoleStmt.java M fe/src/main/java/org/apache/impala/analysis/ShowGrantPrincipalStmt.java M fe/src/main/java/org/apache/impala/analysis/ShowRolesStmt.java M fe/src/main/java/org/apache/impala/authorization/sentry/SentryCatalogdAuthorizationManager.java M fe/src/main/java/org/apache/impala/authorization/sentry/SentryImpaladAuthorizationManager.java M fe/src/main/java/org/apache/impala/service/Frontend.java M fe/src/test/java/org/apache/impala/analysis/AnalyzeAuthStmtsTest.java M tests/authorization/test_sentry.py 10 files changed, 119 insertions(+), 92 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/84/13284/8 -- To view, visit http://gerrit.cloudera.org:8080/13284 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: Id24a00dd395e30e4c392f085893e9561da2ee539 Gerrit-Change-Number: 13284 Gerrit-PatchSet: 8 Gerrit-Owner: Austin Nobis Gerrit-Reviewer: Austin Nobis Gerrit-Reviewer: Fredy Wijaya Gerrit-Reviewer: Impala Public Jenkins
[Impala-ASF-CR] IMPALA-8281: Misc Sentry decoupling clean up
Austin Nobis has posted comments on this change. (
http://gerrit.cloudera.org:8080/13284 )
Change subject: IMPALA-8281: Misc Sentry decoupling clean up
..
Patch Set 8:
(1 comment)
http://gerrit.cloudera.org:8080/#/c/13284/7/tests/authorization/test_sentry.py
File tests/authorization/test_sentry.py:
http://gerrit.cloudera.org:8080/#/c/13284/7/tests/authorization/test_sentry.py@93
PS7, Line 93: finally:
: self.client.execute("drop database {0}".format(unique_name))
: self.client.execute(
> this is an odd behavior, we should document this in as comment for backward
My mistake. This test case is invalid. Calling `drop role` on a non-existent
role does not result in an exception.
--
To view, visit http://gerrit.cloudera.org:8080/13284
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Id24a00dd395e30e4c392f085893e9561da2ee539
Gerrit-Change-Number: 13284
Gerrit-PatchSet: 8
Gerrit-Owner: Austin Nobis
Gerrit-Reviewer: Austin Nobis
Gerrit-Reviewer: Fredy Wijaya
Gerrit-Reviewer: Impala Public Jenkins
Gerrit-Comment-Date: Fri, 10 May 2019 15:39:54 +
Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-8281: Misc Sentry decoupling clean up
Hello Fredy Wijaya, Impala Public Jenkins, I'd like you to reexamine a change. Please visit http://gerrit.cloudera.org:8080/13284 to look at the new patch set (#7). Change subject: IMPALA-8281: Misc Sentry decoupling clean up .. IMPALA-8281: Misc Sentry decoupling clean up This patch moves Sentry specific code to the Sentry specific plugin implementation. Testing: - Ran all FE tests - Ran E2E authorization tests - Added new E2E tests in test_sentry Change-Id: Id24a00dd395e30e4c392f085893e9561da2ee539 --- M common/thrift/Frontend.thrift M fe/src/main/java/org/apache/impala/analysis/GrantRevokePrivStmt.java M fe/src/main/java/org/apache/impala/analysis/GrantRevokeRoleStmt.java M fe/src/main/java/org/apache/impala/analysis/ShowGrantPrincipalStmt.java M fe/src/main/java/org/apache/impala/analysis/ShowRolesStmt.java M fe/src/main/java/org/apache/impala/authorization/sentry/SentryCatalogdAuthorizationManager.java M fe/src/main/java/org/apache/impala/authorization/sentry/SentryImpaladAuthorizationManager.java M fe/src/main/java/org/apache/impala/service/Frontend.java M fe/src/test/java/org/apache/impala/analysis/AnalyzeAuthStmtsTest.java M tests/authorization/test_sentry.py 10 files changed, 122 insertions(+), 92 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/84/13284/7 -- To view, visit http://gerrit.cloudera.org:8080/13284 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: Id24a00dd395e30e4c392f085893e9561da2ee539 Gerrit-Change-Number: 13284 Gerrit-PatchSet: 7 Gerrit-Owner: Austin Nobis Gerrit-Reviewer: Austin Nobis Gerrit-Reviewer: Fredy Wijaya Gerrit-Reviewer: Impala Public Jenkins
[Impala-ASF-CR] IMPALA-8281: Misc Sentry decoupling clean up
Hello Fredy Wijaya, Impala Public Jenkins, I'd like you to reexamine a change. Please visit http://gerrit.cloudera.org:8080/13284 to look at the new patch set (#6). Change subject: IMPALA-8281: Misc Sentry decoupling clean up .. IMPALA-8281: Misc Sentry decoupling clean up This patch moves Sentry specific code to the Sentry specific plugin implementation. Testing: - Ran all FE tests - Ran E2E authorization tests - Added new E2E tests in test_sentry Change-Id: Id24a00dd395e30e4c392f085893e9561da2ee539 --- M common/thrift/Frontend.thrift M fe/src/main/java/org/apache/impala/analysis/GrantRevokePrivStmt.java M fe/src/main/java/org/apache/impala/analysis/GrantRevokeRoleStmt.java M fe/src/main/java/org/apache/impala/analysis/ShowGrantPrincipalStmt.java M fe/src/main/java/org/apache/impala/analysis/ShowRolesStmt.java M fe/src/main/java/org/apache/impala/authorization/sentry/SentryCatalogdAuthorizationManager.java M fe/src/main/java/org/apache/impala/authorization/sentry/SentryImpaladAuthorizationManager.java M fe/src/main/java/org/apache/impala/service/Frontend.java M fe/src/test/java/org/apache/impala/analysis/AnalyzeAuthStmtsTest.java M tests/authorization/test_sentry.py 10 files changed, 122 insertions(+), 92 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/84/13284/6 -- To view, visit http://gerrit.cloudera.org:8080/13284 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: Id24a00dd395e30e4c392f085893e9561da2ee539 Gerrit-Change-Number: 13284 Gerrit-PatchSet: 6 Gerrit-Owner: Austin Nobis Gerrit-Reviewer: Austin Nobis Gerrit-Reviewer: Fredy Wijaya Gerrit-Reviewer: Impala Public Jenkins
[Impala-ASF-CR] IMPALA-8281 Sentry frontend decoupling
Austin Nobis has uploaded a new patch set (#5). ( http://gerrit.cloudera.org:8080/13284 ) Change subject: IMPALA-8281 Sentry frontend decoupling .. IMPALA-8281 Sentry frontend decoupling This patch moves Sentry specific code from the Frontend into the SentryImpaladAuthorizationManager. Testing: - Ran all FE tests - Ran E2E authorization tests Change-Id: Id24a00dd395e30e4c392f085893e9561da2ee539 --- M common/thrift/Frontend.thrift M fe/src/main/java/org/apache/impala/analysis/GrantRevokePrivStmt.java M fe/src/main/java/org/apache/impala/analysis/GrantRevokeRoleStmt.java M fe/src/main/java/org/apache/impala/analysis/ShowGrantPrincipalStmt.java M fe/src/main/java/org/apache/impala/analysis/ShowRolesStmt.java M fe/src/main/java/org/apache/impala/authorization/sentry/SentryCatalogdAuthorizationManager.java M fe/src/main/java/org/apache/impala/authorization/sentry/SentryImpaladAuthorizationManager.java M fe/src/main/java/org/apache/impala/service/Frontend.java M fe/src/test/java/org/apache/impala/analysis/AnalyzeAuthStmtsTest.java M tests/authorization/test_grant_revoke.py 10 files changed, 113 insertions(+), 92 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/84/13284/5 -- To view, visit http://gerrit.cloudera.org:8080/13284 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: Id24a00dd395e30e4c392f085893e9561da2ee539 Gerrit-Change-Number: 13284 Gerrit-PatchSet: 5 Gerrit-Owner: Austin Nobis Gerrit-Reviewer: Austin Nobis Gerrit-Reviewer: Fredy Wijaya Gerrit-Reviewer: Impala Public Jenkins
[Impala-ASF-CR] [IMPALA-8281] Sentry frontend decoupling
Hello Fredy Wijaya, Impala Public Jenkins, I'd like you to reexamine a change. Please visit http://gerrit.cloudera.org:8080/13284 to look at the new patch set (#3). Change subject: [IMPALA-8281] Sentry frontend decoupling .. [IMPALA-8281] Sentry frontend decoupling This patch moves Sentry specific code from the Frontend into the SentryImpaladAuthorizationManager. Testing: - Ran all FE tests - Ran E2E authorization tests Change-Id: Id24a00dd395e30e4c392f085893e9561da2ee539 --- M common/thrift/Frontend.thrift M fe/src/main/java/org/apache/impala/analysis/GrantRevokePrivStmt.java M fe/src/main/java/org/apache/impala/analysis/ShowGrantPrincipalStmt.java M fe/src/main/java/org/apache/impala/analysis/ShowRolesStmt.java M fe/src/main/java/org/apache/impala/authorization/sentry/SentryImpaladAuthorizationManager.java M fe/src/main/java/org/apache/impala/service/Frontend.java M fe/src/test/java/org/apache/impala/analysis/AnalyzeAuthStmtsTest.java 7 files changed, 49 insertions(+), 65 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/84/13284/3 -- To view, visit http://gerrit.cloudera.org:8080/13284 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: Id24a00dd395e30e4c392f085893e9561da2ee539 Gerrit-Change-Number: 13284 Gerrit-PatchSet: 3 Gerrit-Owner: Austin Nobis Gerrit-Reviewer: Austin Nobis Gerrit-Reviewer: Fredy Wijaya Gerrit-Reviewer: Impala Public Jenkins
[Impala-ASF-CR] [IMPALA-8281] Sentry frontend decoupling
Austin Nobis has uploaded this change for review. ( http://gerrit.cloudera.org:8080/13284 Change subject: [IMPALA-8281] Sentry frontend decoupling .. [IMPALA-8281] Sentry frontend decoupling This patch moves Sentry specific code from the Frontend into the SentryImpaladAuthorizationManager. Testing: - Ran all FE tests - Ran E2E authorization tests Change-Id: Id24a00dd395e30e4c392f085893e9561da2ee539 --- M common/thrift/Frontend.thrift M fe/src/main/java/org/apache/impala/analysis/GrantRevokePrivStmt.java M fe/src/main/java/org/apache/impala/analysis/ShowGrantPrincipalStmt.java M fe/src/main/java/org/apache/impala/analysis/ShowRolesStmt.java M fe/src/main/java/org/apache/impala/authorization/AuthorizationPolicy.java M fe/src/main/java/org/apache/impala/authorization/sentry/SentryImpaladAuthorizationManager.java M fe/src/main/java/org/apache/impala/service/Frontend.java M fe/src/test/java/org/apache/impala/analysis/AnalyzeAuthStmtsTest.java 8 files changed, 50 insertions(+), 65 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/84/13284/2 -- To view, visit http://gerrit.cloudera.org:8080/13284 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newchange Gerrit-Change-Id: Id24a00dd395e30e4c392f085893e9561da2ee539 Gerrit-Change-Number: 13284 Gerrit-PatchSet: 2 Gerrit-Owner: Austin Nobis Gerrit-Reviewer: Austin Nobis Gerrit-Reviewer: Fredy Wijaya
[Impala-ASF-CR] IMPALA-8488: Fix hardcoded path in Ranger E2E test on S3
Hello Laszlo Gaal, Fredy Wijaya, Impala Public Jenkins, I'd like you to reexamine a change. Please visit http://gerrit.cloudera.org:8080/13234 to look at the new patch set (#2). Change subject: IMPALA-8488: Fix hardcoded path in Ranger E2E test on S3 .. IMPALA-8488: Fix hardcoded path in Ranger E2E test on S3 A hardcoded path in test_ranger.py for URI testing was updated to support S3, local, and HDFS as opposed to just HDFS. Testing: - Ran authorization E2E tests - Ran all FE tests - Ran test_ranger.py with S3 Change-Id: Ie2c021ce212f483a644fdab4e77ab95031066b14 --- M tests/authorization/test_ranger.py 1 file changed, 2 insertions(+), 1 deletion(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/34/13234/2 -- To view, visit http://gerrit.cloudera.org:8080/13234 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: Ie2c021ce212f483a644fdab4e77ab95031066b14 Gerrit-Change-Number: 13234 Gerrit-PatchSet: 2 Gerrit-Owner: Austin Nobis Gerrit-Reviewer: Austin Nobis Gerrit-Reviewer: Fredy Wijaya Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Laszlo Gaal
[Impala-ASF-CR] IMPALA-8488: Fix hardcoded path in Ranger E2E test on S3
Austin Nobis has posted comments on this change. (
http://gerrit.cloudera.org:8080/13234 )
Change subject: IMPALA-8488: Fix hardcoded path in Ranger E2E test on S3
..
Patch Set 2:
(2 comments)
http://gerrit.cloudera.org:8080/#/c/13234/1//COMMIT_MSG
Commit Message:
http://gerrit.cloudera.org:8080/#/c/13234/1//COMMIT_MSG@7
PS1, Line 7: Fix hardcoded path in Ranger E2E test on S3
> nit: usually we try to say something like "Fix hardcoded path in Ranger E2E
Done
http://gerrit.cloudera.org:8080/#/c/13234/1/tests/authorization/test_ranger.py
File tests/authorization/test_ranger.py:
http://gerrit.cloudera.org:8080/#/c/13234/1/tests/authorization/test_ranger.py@262
PS1, Line 262: "{0}{1}".forma
> nit: "{0}{1}".format(NAMENODE, uri)
Done
--
To view, visit http://gerrit.cloudera.org:8080/13234
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Ie2c021ce212f483a644fdab4e77ab95031066b14
Gerrit-Change-Number: 13234
Gerrit-PatchSet: 2
Gerrit-Owner: Austin Nobis
Gerrit-Reviewer: Austin Nobis
Gerrit-Reviewer: Fredy Wijaya
Gerrit-Reviewer: Impala Public Jenkins
Gerrit-Reviewer: Laszlo Gaal
Gerrit-Comment-Date: Mon, 06 May 2019 16:33:45 +
Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-8488: Hardcoded path in Ranger E2E test breaks for S3
Austin Nobis has uploaded this change for review. ( http://gerrit.cloudera.org:8080/13234 Change subject: IMPALA-8488: Hardcoded path in Ranger E2E test breaks for S3 .. IMPALA-8488: Hardcoded path in Ranger E2E test breaks for S3 A hardcoded path in test_ranger.py for URI testing was updated to support S3, local, and HDFS as opposed to just HDFS. Testing: - Ran authorization E2E tests - Ran all FE tests - Ran test_ranger.py with S3 Change-Id: Ie2c021ce212f483a644fdab4e77ab95031066b14 --- M tests/authorization/test_ranger.py 1 file changed, 2 insertions(+), 1 deletion(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/34/13234/1 -- To view, visit http://gerrit.cloudera.org:8080/13234 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newchange Gerrit-Change-Id: Ie2c021ce212f483a644fdab4e77ab95031066b14 Gerrit-Change-Number: 13234 Gerrit-PatchSet: 1 Gerrit-Owner: Austin Nobis Gerrit-Reviewer: Austin Nobis Gerrit-Reviewer: Fredy Wijaya Gerrit-Reviewer: Laszlo Gaal
[Impala-ASF-CR] IMPALA-8472: Fix the refresh privilege workaround in Ranger
Austin Nobis has posted comments on this change. ( http://gerrit.cloudera.org:8080/13229 ) Change subject: IMPALA-8472: Fix the refresh privilege workaround in Ranger .. Patch Set 2: Code-Review+1 -- To view, visit http://gerrit.cloudera.org:8080/13229 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ica8216353b10ad7366a2b5b7b7d86a4e0af844f8 Gerrit-Change-Number: 13229 Gerrit-PatchSet: 2 Gerrit-Owner: Fredy Wijaya Gerrit-Reviewer: Austin Nobis Gerrit-Reviewer: Fredy Wijaya Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Todd Lipcon Gerrit-Comment-Date: Fri, 03 May 2019 19:08:02 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-8477: [DOCS] SHOW GRANT GROUP for Ranger authorization
Austin Nobis has posted comments on this change. ( http://gerrit.cloudera.org:8080/13220 ) Change subject: IMPALA-8477: [DOCS] SHOW GRANT GROUP for Ranger authorization .. Patch Set 1: Code-Review+1 -- To view, visit http://gerrit.cloudera.org:8080/13220 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Iadf0d5c8b43809880f194e0bc810df06bfab2075 Gerrit-Change-Number: 13220 Gerrit-PatchSet: 1 Gerrit-Owner: Alex Rodoni Gerrit-Reviewer: Austin Nobis Gerrit-Reviewer: Fredy Wijaya Gerrit-Reviewer: Impala Public Jenkins Gerrit-Comment-Date: Fri, 03 May 2019 15:21:53 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-8485: Authorization policy file clean up
Austin Nobis has uploaded this change for review. ( http://gerrit.cloudera.org:8080/13222 Change subject: IMPALA-8485: Authorization policy file clean up .. IMPALA-8485: Authorization policy file clean up This patch cleans up references to the deprecated authorization_policy_file flag. The authz-policy.ini file is no longer created during the test config creation. The reference is also removed from the gitignore. Testing: - All FE tests were run - All authorization E2E tests were run - test_authorization.py E2E test was updated to no longer have references to the authz-policy.ini file. Change-Id: Ib1e90973cb3d5b243844d379e5cdcb2add4eec75 --- M bin/create-test-configuration.sh M fe/.gitignore M tests/authorization/test_authorization.py 3 files changed, 2 insertions(+), 8 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/22/13222/1 -- To view, visit http://gerrit.cloudera.org:8080/13222 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newchange Gerrit-Change-Id: Ib1e90973cb3d5b243844d379e5cdcb2add4eec75 Gerrit-Change-Number: 13222 Gerrit-PatchSet: 1 Gerrit-Owner: Austin Nobis
[Impala-ASF-CR] IMPALA-8269: Refactor authorization test package structure
Austin Nobis has posted comments on this change. ( http://gerrit.cloudera.org:8080/13208 ) Change subject: IMPALA-8269: Refactor authorization test package structure .. Patch Set 2: Code-Review+1 -- To view, visit http://gerrit.cloudera.org:8080/13208 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I525ff71f63d7c306d82b4c111f98ff327e4a07b3 Gerrit-Change-Number: 13208 Gerrit-PatchSet: 2 Gerrit-Owner: Fredy Wijaya Gerrit-Reviewer: Austin Nobis Gerrit-Reviewer: Fredy Wijaya Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Joe McDonnell Gerrit-Comment-Date: Thu, 02 May 2019 15:08:59 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-8293 (Part 2): Add support for Ranger cache invalidation
Austin Nobis has posted comments on this change. ( http://gerrit.cloudera.org:8080/13134 ) Change subject: IMPALA-8293 (Part 2): Add support for Ranger cache invalidation .. Patch Set 8: Code-Review+1 -- To view, visit http://gerrit.cloudera.org:8080/13134 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ia7160c082298e0b8cc2742dd3facbd4978581288 Gerrit-Change-Number: 13134 Gerrit-PatchSet: 8 Gerrit-Owner: Fredy Wijaya Gerrit-Reviewer: Austin Nobis Gerrit-Reviewer: Bharath Vissapragada Gerrit-Reviewer: Fredy Wijaya Gerrit-Reviewer: Impala Public Jenkins Gerrit-Comment-Date: Tue, 30 Apr 2019 21:52:39 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-8293 (Part 2): Add support for Ranger cache invalidation
Austin Nobis has posted comments on this change. (
http://gerrit.cloudera.org:8080/13134 )
Change subject: IMPALA-8293 (Part 2): Add support for Ranger cache invalidation
..
Patch Set 8:
(1 comment)
http://gerrit.cloudera.org:8080/#/c/13134/8/fe/src/main/java/org/apache/impala/authorization/NoopAuthorizationFactory.java
File
fe/src/main/java/org/apache/impala/authorization/NoopAuthorizationFactory.java:
http://gerrit.cloudera.org:8080/#/c/13134/8/fe/src/main/java/org/apache/impala/authorization/NoopAuthorizationFactory.java@204
PS8, Line 204: public void invalidateAuthorizationCache() {
nit: one line this method
--
To view, visit http://gerrit.cloudera.org:8080/13134
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Ia7160c082298e0b8cc2742dd3facbd4978581288
Gerrit-Change-Number: 13134
Gerrit-PatchSet: 8
Gerrit-Owner: Fredy Wijaya
Gerrit-Reviewer: Austin Nobis
Gerrit-Reviewer: Bharath Vissapragada
Gerrit-Reviewer: Fredy Wijaya
Gerrit-Reviewer: Impala Public Jenkins
Gerrit-Comment-Date: Tue, 30 Apr 2019 20:23:18 +
Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-8280, IMPALA-8281: Add support for show grant user/group with Ranger
Austin Nobis has posted comments on this change. ( http://gerrit.cloudera.org:8080/13074 ) Change subject: IMPALA-8280, IMPALA-8281: Add support for show grant user/group with Ranger .. Patch Set 13: Fixed compilation error that was caused by not rebasing. Commit has been rebased and tested. -- To view, visit http://gerrit.cloudera.org:8080/13074 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ic46fb9fc36c9e11ec78d5840d22eb0668150c2a4 Gerrit-Change-Number: 13074 Gerrit-PatchSet: 13 Gerrit-Owner: Austin Nobis Gerrit-Reviewer: Austin Nobis Gerrit-Reviewer: Fredy Wijaya Gerrit-Reviewer: Impala Public Jenkins Gerrit-Comment-Date: Tue, 30 Apr 2019 17:52:51 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-8280, IMPALA-8281: Add support for show grant user/group with Ranger
Austin Nobis has uploaded a new patch set (#13). ( http://gerrit.cloudera.org:8080/13074 ) Change subject: IMPALA-8280, IMPALA-8281: Add support for show grant user/group with Ranger .. IMPALA-8280, IMPALA-8281: Add support for show grant user/group with Ranger Add support for SHOW GRANT statements for Apache Ranger. This patch also adds the RangerImpaladAuthorizationManager as the show grant statement is called from impalad. The new supported syntax is: SHOW GRANT USER/GROUP ON server; SHOW GRANT USER/GROUP ON database ; SHOW GRANT USER/GROUP ON uri ; SHOW GRANT USER/GROUP ON table .; SHOW GRANT USER/GROUP ON column ..; The following syntax is valid SQL, but is not supported currently by the Apache Ranger integration with Impala: SHOW GRANT USER/GROUP Testing: - Ran all FE unit tests - Ran authorization E2E tests - Updated test_ranger to use show grant statement for verification of granted privileges Change-Id: Ic46fb9fc36c9e11ec78d5840d22eb0668150c2a4 --- M fe/src/main/cup/sql-parser.cup M fe/src/main/java/org/apache/impala/analysis/ShowGrantPrincipalStmt.java M fe/src/main/java/org/apache/impala/authorization/ranger/RangerAuthorizationFactory.java M fe/src/main/java/org/apache/impala/authorization/ranger/RangerCatalogdAuthorizationManager.java A fe/src/main/java/org/apache/impala/authorization/ranger/RangerImpaladAuthorizationManager.java A fe/src/main/java/org/apache/impala/authorization/ranger/RangerUtil.java M fe/src/main/java/org/apache/impala/catalog/Principal.java M fe/src/test/java/org/apache/impala/analysis/AnalyzeAuthStmtsTest.java M fe/src/test/java/org/apache/impala/analysis/ParserTest.java M tests/authorization/test_ranger.py 10 files changed, 804 insertions(+), 123 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/74/13074/13 -- To view, visit http://gerrit.cloudera.org:8080/13074 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: Ic46fb9fc36c9e11ec78d5840d22eb0668150c2a4 Gerrit-Change-Number: 13074 Gerrit-PatchSet: 13 Gerrit-Owner: Austin Nobis Gerrit-Reviewer: Austin Nobis Gerrit-Reviewer: Fredy Wijaya Gerrit-Reviewer: Impala Public Jenkins
[Impala-ASF-CR] IMPALA-8280, IMPALA-8281: Add support for show grant user/group with Ranger
Austin Nobis has posted comments on this change. (
http://gerrit.cloudera.org:8080/13074 )
Change subject: IMPALA-8280, IMPALA-8281: Add support for show grant user/group
with Ranger
..
Patch Set 11:
(4 comments)
http://gerrit.cloudera.org:8080/#/c/13074/9/fe/src/main/java/org/apache/impala/analysis/ShowGrantPrincipalStmt.java
File fe/src/main/java/org/apache/impala/analysis/ShowGrantPrincipalStmt.java:
http://gerrit.cloudera.org:8080/#/c/13074/9/fe/src/main/java/org/apache/impala/analysis/ShowGrantPrincipalStmt.java@65
PS9, Line 65: if (principal_ == null) {
: throw new AnalysisException(String.format("%s '%s' " +
> since this spans more than one line, can you use {}
Done
http://gerrit.cloudera.org:8080/#/c/13074/9/fe/src/main/java/org/apache/impala/authorization/ranger/RangerUtil.java
File fe/src/main/java/org/apache/impala/authorization/ranger/RangerUtil.java:
http://gerrit.cloudera.org:8080/#/c/13074/9/fe/src/main/java/org/apache/impala/authorization/ranger/RangerUtil.java@26
PS9, Line 26: * Collection of static functions to support Apache Ranger
implementation
> nit: remove an extra empty new line
Done
http://gerrit.cloudera.org:8080/#/c/13074/9/fe/src/main/java/org/apache/impala/authorization/ranger/RangerUtil.java@28
PS9, Line 28: public class RangerUtil {
> nit: remove an extra empty new line
Done
http://gerrit.cloudera.org:8080/#/c/13074/9/fe/src/main/java/org/apache/impala/authorization/ranger/RangerUtil.java@30
PS9, Line 30:
: /**
:* Creates a column resource for Ranger. Column resources also
include
:* database and table information.
:*/
: public static Map
createColumnResource(TPrivilege privilege) {
: Map resource = new HashMap<>();
:
: resource.put(RangerImpalaResourceBuilder.DATABASE,
getOrAll(privilege.getDb_name()));
: resource.put(RangerImpalaResourceBuilder.TABLE,
getOrAll(privilege.getTable_name()));
: resource.put(RangerImpalaResourceBuilder.COLUMN,
: getOrAll(privilege.getColumn_name()));
:
: return resource;
: }
:
: /**
:* Creates a URI resource for Ranger. In Ranger a URI is known
as a URL.
:*/
: public static Map createUriResource(TPrivilege
privilege) {
: Map resource = new HashMap<>();
: String uri = privilege.getUri();
: resource.put(RangerImpalaResourceBuilder.URL, uri == null ?
"*" : uri);
:
: return resource;
: }
:
: /**
:* Creates a function resource for Ranger. Function resources
also include
:* database information.
:
> add javadoc for public methods
Done
--
To view, visit http://gerrit.cloudera.org:8080/13074
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Ic46fb9fc36c9e11ec78d5840d22eb0668150c2a4
Gerrit-Change-Number: 13074
Gerrit-PatchSet: 11
Gerrit-Owner: Austin Nobis
Gerrit-Reviewer: Austin Nobis
Gerrit-Reviewer: Fredy Wijaya
Gerrit-Reviewer: Impala Public Jenkins
Gerrit-Comment-Date: Tue, 30 Apr 2019 15:30:07 +
Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-8280, IMPALA-8281: Add support for show grant user/group with Ranger
Austin Nobis has uploaded a new patch set (#11). ( http://gerrit.cloudera.org:8080/13074 ) Change subject: IMPALA-8280, IMPALA-8281: Add support for show grant user/group with Ranger .. IMPALA-8280, IMPALA-8281: Add support for show grant user/group with Ranger Add support for SHOW GRANT statements for Apache Ranger. This patch also adds the RangerImpaladAuthorizationManager as the show grant statement is called from impalad. The new supported syntax is: SHOW GRANT USER/GROUP ON server; SHOW GRANT USER/GROUP ON database ; SHOW GRANT USER/GROUP ON uri ; SHOW GRANT USER/GROUP ON table .; SHOW GRANT USER/GROUP ON column ..; The following syntax is valid SQL, but is not supported currently by the Apache Ranger integration with Impala: SHOW GRANT USER/GROUP Testing: - Ran all FE unit tests - Ran authorization E2E tests - Updated test_ranger to use show grant statement for verification of granted privileges Change-Id: Ic46fb9fc36c9e11ec78d5840d22eb0668150c2a4 --- M fe/src/main/cup/sql-parser.cup M fe/src/main/java/org/apache/impala/analysis/ShowGrantPrincipalStmt.java M fe/src/main/java/org/apache/impala/authorization/ranger/RangerAuthorizationFactory.java M fe/src/main/java/org/apache/impala/authorization/ranger/RangerCatalogdAuthorizationManager.java A fe/src/main/java/org/apache/impala/authorization/ranger/RangerImpaladAuthorizationManager.java A fe/src/main/java/org/apache/impala/authorization/ranger/RangerUtil.java M fe/src/main/java/org/apache/impala/catalog/Principal.java M fe/src/test/java/org/apache/impala/analysis/AnalyzeAuthStmtsTest.java M fe/src/test/java/org/apache/impala/analysis/ParserTest.java M tests/authorization/test_ranger.py 10 files changed, 807 insertions(+), 124 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/74/13074/11 -- To view, visit http://gerrit.cloudera.org:8080/13074 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: Ic46fb9fc36c9e11ec78d5840d22eb0668150c2a4 Gerrit-Change-Number: 13074 Gerrit-PatchSet: 11 Gerrit-Owner: Austin Nobis Gerrit-Reviewer: Austin Nobis Gerrit-Reviewer: Fredy Wijaya Gerrit-Reviewer: Impala Public Jenkins
[Impala-ASF-CR] IMPALA-8280, IMPALA-8281: Add support for show grant user/group with Ranger
Austin Nobis has posted comments on this change. (
http://gerrit.cloudera.org:8080/13074 )
Change subject: IMPALA-8280, IMPALA-8281: Add support for show grant user/group
with Ranger
..
Patch Set 9:
(2 comments)
http://gerrit.cloudera.org:8080/#/c/13074/8//COMMIT_MSG
Commit Message:
http://gerrit.cloudera.org:8080/#/c/13074/8//COMMIT_MSG@13
PS8, Line 13: SHOW GRANT USER/GROUP ON server;
> Instead of can we list down what the valid resources?
Done
http://gerrit.cloudera.org:8080/#/c/13074/8/fe/src/main/java/org/apache/impala/authorization/ranger/RangerImpaladAuthorizationManager.java
File
fe/src/main/java/org/apache/impala/authorization/ranger/RangerImpaladAuthorizationManager.java:
http://gerrit.cloudera.org:8080/#/c/13074/8/fe/src/main/java/org/apache/impala/authorization/ranger/RangerImpaladAuthorizationManager.java@267
PS8, Line 267: } else if (privilege.getUri() != null) {
: resources.add(RangerUtil.createUriResource(privilege));
: } else if (privilege.getDb_name() != null) {
: // DB is used by column and function resources.
: resources.add(RangerUtil.createColumnResource(privilege));
:
resources.add(RangerUtil.createFunctionResource(privilege));
: } else {
: // Server is used by column, function, and URI resources.
: resources.add(RangerUtil.createColumnResource(privilege));
: resources.add(RangerUtil.createUriResource(privilege));
:
resources.add(RangerUtil.createFunctionResource(privilege));
: }
:
: L
> I feel like we shouldn't be calling functions from RangerCatalogdAuthorizat
Done
--
To view, visit http://gerrit.cloudera.org:8080/13074
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Ic46fb9fc36c9e11ec78d5840d22eb0668150c2a4
Gerrit-Change-Number: 13074
Gerrit-PatchSet: 9
Gerrit-Owner: Austin Nobis
Gerrit-Reviewer: Austin Nobis
Gerrit-Reviewer: Fredy Wijaya
Gerrit-Reviewer: Impala Public Jenkins
Gerrit-Comment-Date: Tue, 30 Apr 2019 15:14:49 +
Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-8280, IMPALA-8281: Add support for show grant user/group with Ranger
Austin Nobis has uploaded a new patch set (#9). ( http://gerrit.cloudera.org:8080/13074 ) Change subject: IMPALA-8280, IMPALA-8281: Add support for show grant user/group with Ranger .. IMPALA-8280, IMPALA-8281: Add support for show grant user/group with Ranger Add support for SHOW GRANT statements for Apache Ranger. This patch also adds the RangerImpaladAuthorizationManager as the show grant statement is called from impalad. The new supported syntax is: SHOW GRANT USER/GROUP ON server; SHOW GRANT USER/GROUP ON database ; SHOW GRANT USER/GROUP ON uri ; SHOW GRANT USER/GROUP ON table .; SHOW GRANT USER/GROUP ON column ..; The following syntax is valid SQL, but is not supported currently by the Apache Ranger integration with Impala: SHOW GRANT USER/GROUP Testing: - Ran all FE unit tests - Ran authorization E2E tests - Updated test_ranger to use show grant statement for verification of granted privileges Change-Id: Ic46fb9fc36c9e11ec78d5840d22eb0668150c2a4 --- M fe/src/main/cup/sql-parser.cup M fe/src/main/java/org/apache/impala/analysis/ShowGrantPrincipalStmt.java M fe/src/main/java/org/apache/impala/authorization/ranger/RangerAuthorizationFactory.java M fe/src/main/java/org/apache/impala/authorization/ranger/RangerCatalogdAuthorizationManager.java A fe/src/main/java/org/apache/impala/authorization/ranger/RangerImpaladAuthorizationManager.java A fe/src/main/java/org/apache/impala/authorization/ranger/RangerUtil.java M fe/src/main/java/org/apache/impala/catalog/Principal.java M fe/src/test/java/org/apache/impala/analysis/AnalyzeAuthStmtsTest.java M fe/src/test/java/org/apache/impala/analysis/ParserTest.java M tests/authorization/test_ranger.py 10 files changed, 793 insertions(+), 124 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/74/13074/9 -- To view, visit http://gerrit.cloudera.org:8080/13074 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: Ic46fb9fc36c9e11ec78d5840d22eb0668150c2a4 Gerrit-Change-Number: 13074 Gerrit-PatchSet: 9 Gerrit-Owner: Austin Nobis Gerrit-Reviewer: Austin Nobis Gerrit-Reviewer: Fredy Wijaya Gerrit-Reviewer: Impala Public Jenkins
[Impala-ASF-CR] IMPALA-8280, IMPALA-8281: Add support for show grant user/group with Ranger
Austin Nobis has posted comments on this change. (
http://gerrit.cloudera.org:8080/13074 )
Change subject: IMPALA-8280, IMPALA-8281: Add support for show grant user/group
with Ranger
..
Patch Set 8:
(20 comments)
http://gerrit.cloudera.org:8080/#/c/13074/7//COMMIT_MSG
Commit Message:
http://gerrit.cloudera.org:8080/#/c/13074/7//COMMIT_MSG@9
PS7, Line 9: Add support for SHOW GRANT statements for Apache Ranger. This
patch also
: adds the RangerImpaladAuthorizationManager as the show grant
statement
: is called from impalad.
> mention the list of new syntax and also mention that "show grant" without "
Done
http://gerrit.cloudera.org:8080/#/c/13074/7/fe/src/main/java/org/apache/impala/authorization/AuthorizationPolicy.java
File fe/src/main/java/org/apache/impala/authorization/AuthorizationPolicy.java:
http://gerrit.cloudera.org:8080/#/c/13074/7/fe/src/main/java/org/apache/impala/authorization/AuthorizationPolicy.java@526
PS7, Line 526: Li
> does this need to be public?
Done
http://gerrit.cloudera.org:8080/#/c/13074/7/fe/src/main/java/org/apache/impala/authorization/AuthorizationPolicy.java@529
PS7, Line 529: oThrift();
> This is an unordered set, will this be a problem like the output of the row
Done
http://gerrit.cloudera.org:8080/#/c/13074/7/fe/src/main/java/org/apache/impala/authorization/ranger/RangerCatalogdAuthorizationManager.java
File
fe/src/main/java/org/apache/impala/authorization/ranger/RangerCatalogdAuthorizationManager.java:
http://gerrit.cloudera.org:8080/#/c/13074/7/fe/src/main/java/org/apache/impala/authorization/ranger/RangerCatalogdAuthorizationManager.java@260
PS7, Line 260: resource.put(RangerImpalaResourceBuilder.DATABASE,
getOrAll(privilege.getDb_name()));
: resource.put(RangerImpalaResourceBuilder.TABLE,
getOrAll(privilege.getTable_name()));
: resource.put(RangerImpalaResourceBuilder.COLUMN,
: getOrAll(privilege.getColumn_name()));
:
: return resource;
: }
:
: public static Map createUriResource(TPrivilege
privilege) {
: Map resource = new HashMap<>();
: String uri = privilege.getUri();
: resource.put(RangerImpalaResourceBuilder.URL, uri == null ?
"*" : uri);
:
: return resource;
: }
:
: public static Map
createFunctionResource(TPrivilege privilege) {
: Map resource = new HashMap<>();
:
: resource.put(RangerImpalaResourceBuilder.DATABASE,
getOrAll(privilege.getDb_name()));
: resource.put(RangerImpalaResourceBuilder.UDF, "*");
:
: return resource;
: }
:
: private static String getOrAll(String resource) {
:
> can we make these private now?
They are used in the RangerImpaladAuthorizationManager as well.
http://gerrit.cloudera.org:8080/#/c/13074/7/fe/src/main/java/org/apache/impala/authorization/ranger/RangerImpaladAuthorizationManager.java
File
fe/src/main/java/org/apache/impala/authorization/ranger/RangerImpaladAuthorizationManager.java:
http://gerrit.cloudera.org:8080/#/c/13074/7/fe/src/main/java/org/apache/impala/authorization/ranger/RangerImpaladAuthorizationManager.java@162
PS7, Line 162: upInformation ugi = UserGroupInforma
> let's try to use the one from JDK instead, i.e. new HashSet<>(ugi.getGroupN
The JDK HashSet constructor doesn't take a String[]. The code would have to be:
`return new HashSet<>(Arrays.asList(ugi.getGroupNames());`
http://gerrit.cloudera.org:8080/#/c/13074/7/fe/src/main/java/org/apache/impala/authorization/ranger/RangerImpaladAuthorizationManager.java@236
PS7, Line 236:
: switch (privilege.getScope()) {
: case COLUMN:
: if (!column.isPresent() || column.get().equals("*"))
return null;
: case TABLE:
: if (!table.isPresent() || table.get().equals("*"))
return null;
: case DATABASE:
: if (!database.isPresent() || database.get().equals("*"))
return null;
: break;
: case URI:
: if (!uri.isPresent() || uri.get().equals("*")) return
null;
:
> add default case to to make the compiler happy
Done
http://gerrit.cloudera.org:8080/#/c/13074/7/fe/src/main/java/org/apache/impala/authorization/ranger/RangerImpaladAuthorizationManager.java@259
PS7, Line 259: vate
> since L257 is sa throw. We can make it just an if instead of else if.
Done
http://gerrit.cloudera.org:8080/#/c/13074/7/fe/src/main/java/org/apache/impala/authorization/ranger/RangerImpaladAuthorizationManager.java@276
PS7, Line 276: // Server is used by column, function, and URI
[Impala-ASF-CR] IMPALA-8280, IMPALA-8281: Add support for show grant user/group with Ranger
Austin Nobis has uploaded a new patch set (#8). ( http://gerrit.cloudera.org:8080/13074 ) Change subject: IMPALA-8280, IMPALA-8281: Add support for show grant user/group with Ranger .. IMPALA-8280, IMPALA-8281: Add support for show grant user/group with Ranger Add support for SHOW GRANT statements for Apache Ranger. This patch also adds the RangerImpaladAuthorizationManager as the show grant statement is called from impalad. The new supported syntax is: SHOW GRANT USER/GROUP ON The following syntax is valid SQL, but is not supported currently by the Apache Ranger integration with Impala: SHOW GRANT USER/GROUP Testing: - Ran all FE unit tests - Ran authorization E2E tests - Updated test_ranger to use show grant statement for verification of granted privileges Change-Id: Ic46fb9fc36c9e11ec78d5840d22eb0668150c2a4 --- M fe/src/main/cup/sql-parser.cup M fe/src/main/java/org/apache/impala/analysis/ShowGrantPrincipalStmt.java M fe/src/main/java/org/apache/impala/authorization/ranger/RangerAuthorizationFactory.java M fe/src/main/java/org/apache/impala/authorization/ranger/RangerCatalogdAuthorizationManager.java A fe/src/main/java/org/apache/impala/authorization/ranger/RangerImpaladAuthorizationManager.java M fe/src/main/java/org/apache/impala/catalog/Principal.java M fe/src/test/java/org/apache/impala/analysis/AnalyzeAuthStmtsTest.java M fe/src/test/java/org/apache/impala/analysis/ParserTest.java M tests/authorization/test_ranger.py 9 files changed, 730 insertions(+), 88 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/74/13074/8 -- To view, visit http://gerrit.cloudera.org:8080/13074 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: Ic46fb9fc36c9e11ec78d5840d22eb0668150c2a4 Gerrit-Change-Number: 13074 Gerrit-PatchSet: 8 Gerrit-Owner: Austin Nobis Gerrit-Reviewer: Austin Nobis Gerrit-Reviewer: Fredy Wijaya Gerrit-Reviewer: Impala Public Jenkins
[Impala-ASF-CR] IMPALA-8280, IMPALA-8281: Add support for show grant user/group with Ranger
Austin Nobis has uploaded a new patch set (#7). ( http://gerrit.cloudera.org:8080/13074 ) Change subject: IMPALA-8280, IMPALA-8281: Add support for show grant user/group with Ranger .. IMPALA-8280, IMPALA-8281: Add support for show grant user/group with Ranger Add support for SHOW GRANT statements for Apache Ranger. This patch also adds the RangerImpaladAuthorizationManager as the show grant statement is called from impalad. Testing: - Ran all FE unit tests - Ran authorization E2E tests - Updated test_ranger to use show grant statement for verification of granted privileges Change-Id: Ic46fb9fc36c9e11ec78d5840d22eb0668150c2a4 --- M fe/src/main/cup/sql-parser.cup M fe/src/main/java/org/apache/impala/analysis/ShowGrantPrincipalStmt.java M fe/src/main/java/org/apache/impala/authorization/AuthorizationPolicy.java M fe/src/main/java/org/apache/impala/authorization/ranger/RangerAuthorizationFactory.java M fe/src/main/java/org/apache/impala/authorization/ranger/RangerCatalogdAuthorizationManager.java A fe/src/main/java/org/apache/impala/authorization/ranger/RangerImpaladAuthorizationManager.java M fe/src/main/java/org/apache/impala/catalog/Principal.java M fe/src/test/java/org/apache/impala/analysis/AnalyzeAuthStmtsTest.java M fe/src/test/java/org/apache/impala/analysis/ParserTest.java M tests/authorization/test_ranger.py 10 files changed, 709 insertions(+), 97 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/74/13074/7 -- To view, visit http://gerrit.cloudera.org:8080/13074 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: Ic46fb9fc36c9e11ec78d5840d22eb0668150c2a4 Gerrit-Change-Number: 13074 Gerrit-PatchSet: 7 Gerrit-Owner: Austin Nobis Gerrit-Reviewer: Austin Nobis Gerrit-Reviewer: Fredy Wijaya Gerrit-Reviewer: Impala Public Jenkins
[Impala-ASF-CR] IMPALA-8309: add user authorization provider flag
Austin Nobis has uploaded a new patch set (#18) to the change originally created by radford nguyen. ( http://gerrit.cloudera.org:8080/12901 ) Change subject: IMPALA-8309: add user authorization_provider flag .. IMPALA-8309: add user authorization_provider flag This commit adds a `authorization_provider` user-facing flag in order to provide a more human-readable alternative to the `authorization_factory_class` for internally-provided authorization strategies. The `authorization_factory_class` flag is retained, but no longer takes a default value if not specified. The default for `authorization_provider` is "sentry" in order to retain backwards-compatibility. If specified, `authorization_factory_class` will take precedence. Testing: - Manually started minicluster with each of following flags and verified correct authorization strategy chosen: - provider='' factory='' => sentry - provider=sentry factory='' => sentry - provider=ranger factory='' => ranger - provider='' factory=sentry => sentry - provider='' factory=ranger => ranger - provider=sentry factory=sentry => sentry - provider=ranger factory=sentry => sentry - provider=sentry factory=ranger => ranger - provider=ranger factory=ranger => ranger - Wrote unit tests to capture above assertions - Ran fe unit and e2e tests - Wrote e2e test to verify new flag behavior Change-Id: I001c20505ba4f0562b60fdef73d15308e8500c19 --- M be/src/service/frontend.cc M be/src/util/backend-gflag-util.cc M common/thrift/BackendGflags.thrift M fe/src/main/java/org/apache/impala/authorization/AuthorizationConfig.java M fe/src/main/java/org/apache/impala/authorization/AuthorizationProvider.java M fe/src/main/java/org/apache/impala/authorization/NoopAuthorizationFactory.java M fe/src/main/java/org/apache/impala/authorization/ranger/RangerAuthorizationConfig.java M fe/src/main/java/org/apache/impala/authorization/sentry/SentryAuthorizationConfig.java M fe/src/main/java/org/apache/impala/service/BackendConfig.java M fe/src/main/java/org/apache/impala/service/JniCatalog.java M fe/src/main/java/org/apache/impala/service/JniFrontend.java A fe/src/main/java/org/apache/impala/util/AuthorizationUtil.java M fe/src/test/java/org/apache/impala/common/FrontendTestBase.java A fe/src/test/java/org/apache/impala/util/AuthorizationUtilTest.java A tests/authorization/test_provider.py M tests/authorization/test_ranger.py 16 files changed, 338 insertions(+), 62 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/01/12901/18 -- To view, visit http://gerrit.cloudera.org:8080/12901 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: I001c20505ba4f0562b60fdef73d15308e8500c19 Gerrit-Change-Number: 12901 Gerrit-PatchSet: 18 Gerrit-Owner: radford nguyen Gerrit-Reviewer: Austin Nobis Gerrit-Reviewer: Fredy Wijaya Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Todd Lipcon Gerrit-Reviewer: radford nguyen
[Impala-ASF-CR] IMPALA-8309: add user authorization provider flag
Austin Nobis has uploaded a new patch set (#16) to the change originally created by radford nguyen. ( http://gerrit.cloudera.org:8080/12901 ) Change subject: IMPALA-8309: add user authorization_provider flag .. IMPALA-8309: add user authorization_provider flag This commit adds a `authorization_provider` user-facing flag in order to provide a more human-readable alternative to the `authorization_factory_class` for internally-provided authorization strategies. The `authorization_factory_class` flag is retained, but no longer takes a default value if not specified. The default for `authorization_provider` is "sentry" in order to retain backwards-compatibility. If specified, `authorization_factory_class` will take precedence. Testing: - Manually started minicluster with each of following flags and verified correct authorization strategy chosen: - provider='' factory='' => sentry - provider=sentry factory='' => sentry - provider=ranger factory='' => ranger - provider='' factory=sentry => sentry - provider='' factory=ranger => ranger - provider=sentry factory=sentry => sentry - provider=ranger factory=sentry => sentry - provider=sentry factory=ranger => ranger - provider=ranger factory=ranger => ranger - Wrote unit tests to capture above assertions - Ran fe unit and e2e tests - Wrote e2e test to verify new flag behavior Change-Id: I001c20505ba4f0562b60fdef73d15308e8500c19 --- M be/src/service/frontend.cc M be/src/util/backend-gflag-util.cc M common/thrift/BackendGflags.thrift M fe/src/main/java/org/apache/impala/authorization/AuthorizationConfig.java M fe/src/main/java/org/apache/impala/authorization/AuthorizationProvider.java M fe/src/main/java/org/apache/impala/authorization/NoopAuthorizationFactory.java M fe/src/main/java/org/apache/impala/authorization/ranger/RangerAuthorizationConfig.java M fe/src/main/java/org/apache/impala/authorization/sentry/SentryAuthorizationConfig.java M fe/src/main/java/org/apache/impala/service/BackendConfig.java M fe/src/main/java/org/apache/impala/service/JniCatalog.java M fe/src/main/java/org/apache/impala/service/JniFrontend.java A fe/src/main/java/org/apache/impala/util/AuthorizationUtil.java M fe/src/test/java/org/apache/impala/common/FrontendTestBase.java A fe/src/test/java/org/apache/impala/util/AuthorizationUtilTest.java A tests/authorization/test_provider.py M tests/authorization/test_ranger.py 16 files changed, 335 insertions(+), 62 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/01/12901/16 -- To view, visit http://gerrit.cloudera.org:8080/12901 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: I001c20505ba4f0562b60fdef73d15308e8500c19 Gerrit-Change-Number: 12901 Gerrit-PatchSet: 16 Gerrit-Owner: radford nguyen Gerrit-Reviewer: Austin Nobis Gerrit-Reviewer: Fredy Wijaya Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Todd Lipcon Gerrit-Reviewer: radford nguyen
[Impala-ASF-CR] IMPALA-8309: add user authorization provider flag
Austin Nobis has uploaded a new patch set (#15) to the change originally created by radford nguyen. ( http://gerrit.cloudera.org:8080/12901 ) Change subject: IMPALA-8309: add user authorization_provider flag .. IMPALA-8309: add user authorization_provider flag This commit adds a `authorization_provider` user-facing flag in order to provide a more human-readable alternative to the `authorization_factory_class` for internally-provided authorization strategies. The `authorization_factory_class` flag is retained, but no longer takes a default value if not specified. The default for `authorization_provider` is "sentry" in order to retain backwards-compatibility. If specified, `authorization_factory_class` will take precedence. Testing: - Manually started minicluster with each of following flags and verified correct authorization strategy chosen: - provider='' factory='' => sentry - provider=sentry factory='' => sentry - provider=ranger factory='' => ranger - provider='' factory=sentry => sentry - provider='' factory=ranger => ranger - provider=sentry factory=sentry => sentry - provider=ranger factory=sentry => sentry - provider=sentry factory=ranger => ranger - provider=ranger factory=ranger => ranger - Wrote unit tests to capture above assertions - Ran fe unit and e2e tests - Wrote e2e test to verify new flag behavior Change-Id: I001c20505ba4f0562b60fdef73d15308e8500c19 --- M be/src/service/frontend.cc M be/src/util/backend-gflag-util.cc M common/thrift/BackendGflags.thrift M fe/src/main/java/org/apache/impala/authorization/AuthorizationConfig.java M fe/src/main/java/org/apache/impala/authorization/AuthorizationProvider.java M fe/src/main/java/org/apache/impala/authorization/NoopAuthorizationFactory.java M fe/src/main/java/org/apache/impala/authorization/ranger/RangerAuthorizationConfig.java M fe/src/main/java/org/apache/impala/authorization/sentry/SentryAuthorizationConfig.java M fe/src/main/java/org/apache/impala/service/BackendConfig.java M fe/src/main/java/org/apache/impala/service/JniCatalog.java M fe/src/main/java/org/apache/impala/service/JniFrontend.java A fe/src/main/java/org/apache/impala/util/AuthorizationUtil.java M fe/src/test/java/org/apache/impala/common/FrontendTestBase.java A fe/src/test/java/org/apache/impala/util/AuthorizationUtilTest.java A tests/authorization/test_provider.py M tests/authorization/test_ranger.py 16 files changed, 318 insertions(+), 61 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/01/12901/15 -- To view, visit http://gerrit.cloudera.org:8080/12901 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: I001c20505ba4f0562b60fdef73d15308e8500c19 Gerrit-Change-Number: 12901 Gerrit-PatchSet: 15 Gerrit-Owner: radford nguyen Gerrit-Reviewer: Austin Nobis Gerrit-Reviewer: Fredy Wijaya Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Todd Lipcon Gerrit-Reviewer: radford nguyen
[Impala-ASF-CR] IMPALA-8309: add user authorization provider flag
Austin Nobis has uploaded a new patch set (#14) to the change originally created by radford nguyen. ( http://gerrit.cloudera.org:8080/12901 ) Change subject: IMPALA-8309: add user authorization_provider flag .. IMPALA-8309: add user authorization_provider flag This commit adds a `authorization_provider` user-facing flag in order to provide a more human-readable alternative to the `authorization_factory_class` for internally-provided authorization strategies. The `authorization_factory_class` flag is retained, but no longer takes a default value if not specified. The default for `authorization_provider` is "sentry" in order to retain backwards-compatibility. If specified, `authorization_factory_class` will take precedence. Testing: - Manually started minicluster with each of following flags and verified correct authorization strategy chosen: - provider='' factory='' => sentry - provider=sentry factory='' => sentry - provider=ranger factory='' => ranger - provider='' factory=sentry => sentry - provider='' factory=ranger => ranger - provider=sentry factory=sentry => sentry - provider=ranger factory=sentry => sentry - provider=sentry factory=ranger => ranger - provider=ranger factory=ranger => ranger - Wrote unit tests to capture above assertions - Ran fe unit and e2e tests - Wrote e2e test to verify new flag behavior Change-Id: I001c20505ba4f0562b60fdef73d15308e8500c19 --- M be/src/service/frontend.cc M be/src/util/backend-gflag-util.cc M common/thrift/BackendGflags.thrift M fe/src/main/java/org/apache/impala/authorization/AuthorizationConfig.java M fe/src/main/java/org/apache/impala/authorization/AuthorizationProvider.java M fe/src/main/java/org/apache/impala/authorization/NoopAuthorizationFactory.java M fe/src/main/java/org/apache/impala/authorization/ranger/RangerAuthorizationConfig.java M fe/src/main/java/org/apache/impala/authorization/sentry/SentryAuthorizationConfig.java M fe/src/main/java/org/apache/impala/service/BackendConfig.java M fe/src/main/java/org/apache/impala/service/JniCatalog.java M fe/src/main/java/org/apache/impala/service/JniFrontend.java A fe/src/main/java/org/apache/impala/util/AuthorizationUtil.java M fe/src/test/java/org/apache/impala/common/FrontendTestBase.java M fe/src/test/java/org/apache/impala/service/JniFrontendTest.java A fe/src/test/java/org/apache/impala/util/AuthorizationUtilTest.java A tests/authorization/test_provider.py M tests/authorization/test_ranger.py 17 files changed, 337 insertions(+), 61 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/01/12901/14 -- To view, visit http://gerrit.cloudera.org:8080/12901 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: I001c20505ba4f0562b60fdef73d15308e8500c19 Gerrit-Change-Number: 12901 Gerrit-PatchSet: 14 Gerrit-Owner: radford nguyen Gerrit-Reviewer: Austin Nobis Gerrit-Reviewer: Fredy Wijaya Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Todd Lipcon Gerrit-Reviewer: radford nguyen
[Impala-ASF-CR] IMPALA-8309: add user authorization provider flag
Austin Nobis has uploaded a new patch set (#13) to the change originally created by radford nguyen. ( http://gerrit.cloudera.org:8080/12901 ) Change subject: IMPALA-8309: add user authorization_provider flag .. IMPALA-8309: add user authorization_provider flag This commit adds a `authorization_provider` user-facing flag in order to provide a more human-readable alternative to the `authorization_factory_class` for internally-provided authorization strategies. The `authorization_factory_class` flag is retained, but no longer takes a default value if not specified. The default for `authorization_provider` is "sentry" in order to retain backwards-compatibility. If specified, `authorization_factory_class` will take precedence. Testing: - Manually started minicluster with each of following flags and verified correct authorization strategy chosen: - provider='' factory='' => sentry - provider=sentry factory='' => sentry - provider=ranger factory='' => ranger - provider='' factory=sentry => sentry - provider='' factory=ranger => ranger - provider=sentry factory=sentry => sentry - provider=ranger factory=sentry => sentry - provider=sentry factory=ranger => ranger - provider=ranger factory=ranger => ranger - Wrote unit tests to capture above assertions - Ran fe unit and e2e tests - Wrote e2e test to verify new flag behavior Change-Id: I001c20505ba4f0562b60fdef73d15308e8500c19 --- M be/src/service/frontend.cc M be/src/util/backend-gflag-util.cc M common/thrift/BackendGflags.thrift M fe/src/main/java/org/apache/impala/authorization/AuthorizationConfig.java M fe/src/main/java/org/apache/impala/authorization/AuthorizationProvider.java M fe/src/main/java/org/apache/impala/authorization/NoopAuthorizationFactory.java M fe/src/main/java/org/apache/impala/authorization/ranger/RangerAuthorizationConfig.java M fe/src/main/java/org/apache/impala/authorization/sentry/SentryAuthorizationConfig.java M fe/src/main/java/org/apache/impala/service/BackendConfig.java M fe/src/main/java/org/apache/impala/service/JniCatalog.java M fe/src/main/java/org/apache/impala/service/JniFrontend.java A fe/src/main/java/org/apache/impala/util/AuthorizationUtil.java M fe/src/test/java/org/apache/impala/common/FrontendTestBase.java M fe/src/test/java/org/apache/impala/service/JniFrontendTest.java A fe/src/test/java/org/apache/impala/util/AuthorizationUtilTest.java A tests/authorization/test_provider.py M tests/authorization/test_ranger.py 17 files changed, 335 insertions(+), 61 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/01/12901/13 -- To view, visit http://gerrit.cloudera.org:8080/12901 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: I001c20505ba4f0562b60fdef73d15308e8500c19 Gerrit-Change-Number: 12901 Gerrit-PatchSet: 13 Gerrit-Owner: radford nguyen Gerrit-Reviewer: Austin Nobis Gerrit-Reviewer: Fredy Wijaya Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Todd Lipcon Gerrit-Reviewer: radford nguyen
[Impala-ASF-CR] [WIP] IMPALA-8281: Add support for show grant user/group with Ranger
Austin Nobis has uploaded a new patch set (#6). ( http://gerrit.cloudera.org:8080/13074 ) Change subject: [WIP] IMPALA-8281: Add support for show grant user/group with Ranger .. [WIP] IMPALA-8281: Add support for show grant user/group with Ranger Add support for SHOW GRANT statements for Apache Ranger. This patch also adds the RangerImpaladAuthorizationManager as the show grant statement is called from impalad. Testing: - Ran all FE unit tests - Ran authorization E2E tests - Updated test_ranger to use show grant statement for verification of granted privileges Change-Id: Ic46fb9fc36c9e11ec78d5840d22eb0668150c2a4 --- M fe/src/main/cup/sql-parser.cup M fe/src/main/java/org/apache/impala/analysis/ShowGrantPrincipalStmt.java M fe/src/main/java/org/apache/impala/authorization/AuthorizationPolicy.java M fe/src/main/java/org/apache/impala/authorization/ranger/RangerAuthorizationFactory.java M fe/src/main/java/org/apache/impala/authorization/ranger/RangerCatalogdAuthorizationManager.java A fe/src/main/java/org/apache/impala/authorization/ranger/RangerImpaladAuthorizationManager.java M fe/src/main/java/org/apache/impala/catalog/Principal.java M fe/src/test/java/org/apache/impala/analysis/AnalyzeAuthStmtsTest.java M fe/src/test/java/org/apache/impala/analysis/ParserTest.java M tests/authorization/test_ranger.py 10 files changed, 489 insertions(+), 73 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/74/13074/6 -- To view, visit http://gerrit.cloudera.org:8080/13074 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: Ic46fb9fc36c9e11ec78d5840d22eb0668150c2a4 Gerrit-Change-Number: 13074 Gerrit-PatchSet: 6 Gerrit-Owner: Austin Nobis Gerrit-Reviewer: Austin Nobis Gerrit-Reviewer: Fredy Wijaya Gerrit-Reviewer: Impala Public Jenkins
[Impala-ASF-CR] [WIP] IMPALA-8281: Add support for show grant user/group with Ranger
Austin Nobis has uploaded this change for review. ( http://gerrit.cloudera.org:8080/13074 Change subject: [WIP] IMPALA-8281: Add support for show grant user/group with Ranger .. [WIP] IMPALA-8281: Add support for show grant user/group with Ranger Add support for SHOW GRANT statements for Apache Ranger. This patch also adds the RangerImpaladAuthorizationManager as the show grant statement is called from impalad. Testing: - Ran all FE unit tests - Ran authorization E2E tests - Updated test_ranger to use show grant statement for verification of granted privileges Change-Id: Ic46fb9fc36c9e11ec78d5840d22eb0668150c2a4 --- M fe/src/main/cup/sql-parser.cup M fe/src/main/java/org/apache/impala/analysis/ShowGrantPrincipalStmt.java M fe/src/main/java/org/apache/impala/authorization/AuthorizationPolicy.java M fe/src/main/java/org/apache/impala/authorization/ranger/RangerAuthorizationFactory.java M fe/src/main/java/org/apache/impala/authorization/ranger/RangerCatalogdAuthorizationManager.java A fe/src/main/java/org/apache/impala/authorization/ranger/RangerImpaladAuthorizationManager.java M fe/src/main/java/org/apache/impala/catalog/Principal.java M fe/src/test/java/org/apache/impala/analysis/AnalyzeAuthStmtsTest.java M fe/src/test/java/org/apache/impala/analysis/ParserTest.java M tests/authorization/test_ranger.py 10 files changed, 489 insertions(+), 73 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/74/13074/5 -- To view, visit http://gerrit.cloudera.org:8080/13074 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newchange Gerrit-Change-Id: Ic46fb9fc36c9e11ec78d5840d22eb0668150c2a4 Gerrit-Change-Number: 13074 Gerrit-PatchSet: 5 Gerrit-Owner: Austin Nobis
[Impala-ASF-CR] IMPALA-8309: add user authorization provider flag
Austin Nobis has posted comments on this change. ( http://gerrit.cloudera.org:8080/12901 ) Change subject: IMPALA-8309: add user authorization_provider flag .. Patch Set 10: Code-Review+1 -- To view, visit http://gerrit.cloudera.org:8080/12901 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I001c20505ba4f0562b60fdef73d15308e8500c19 Gerrit-Change-Number: 12901 Gerrit-PatchSet: 10 Gerrit-Owner: radford nguyen Gerrit-Reviewer: Austin Nobis Gerrit-Reviewer: Fredy Wijaya Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Todd Lipcon Gerrit-Reviewer: radford nguyen Gerrit-Comment-Date: Mon, 22 Apr 2019 21:33:09 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-8309: add user authorization provider flag
Austin Nobis has posted comments on this change. ( http://gerrit.cloudera.org:8080/12901 ) Change subject: IMPALA-8309: add user authorization_provider flag .. Patch Set 9: (2 comments) http://gerrit.cloudera.org:8080/#/c/12901/9/fe/src/main/java/org/apache/impala/service/BackendConfig.java File fe/src/main/java/org/apache/impala/service/BackendConfig.java: http://gerrit.cloudera.org:8080/#/c/12901/9/fe/src/main/java/org/apache/impala/service/BackendConfig.java@141 PS9, Line 141: getAuthorizationFactoryClassOrNull nit: remove the `orNull` in the function name http://gerrit.cloudera.org:8080/#/c/12901/9/fe/src/main/java/org/apache/impala/service/JniCatalog.java File fe/src/main/java/org/apache/impala/service/JniCatalog.java: http://gerrit.cloudera.org:8080/#/c/12901/9/fe/src/main/java/org/apache/impala/service/JniCatalog.java@114 PS9, Line 114: //<<< 67f77d41d40523074385b8dbccfa6ef6ef81dd57 Why is this commented out? -- To view, visit http://gerrit.cloudera.org:8080/12901 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I001c20505ba4f0562b60fdef73d15308e8500c19 Gerrit-Change-Number: 12901 Gerrit-PatchSet: 9 Gerrit-Owner: radford nguyen Gerrit-Reviewer: Austin Nobis Gerrit-Reviewer: Fredy Wijaya Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Todd Lipcon Gerrit-Reviewer: radford nguyen Gerrit-Comment-Date: Mon, 22 Apr 2019 19:58:37 + Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-8309: add user authorization provider flag
Austin Nobis has posted comments on this change. ( http://gerrit.cloudera.org:8080/12901 ) Change subject: IMPALA-8309: add user authorization_provider flag .. Patch Set 4: (2 comments) http://gerrit.cloudera.org:8080/#/c/12901/2/be/src/service/frontend.cc File be/src/service/frontend.cc: http://gerrit.cloudera.org:8080/#/c/12901/2/be/src/service/frontend.cc@41 PS2, Line 41: "sentry", > Honest question: why? It's a nit so it is not something that needs to be fixed for the change to be merged. I just noticed all the other DEFINE_string in this file had the default value on the same line so I pointed it out. http://gerrit.cloudera.org:8080/#/c/12901/4/tests/authorization/test_ranger.py File tests/authorization/test_ranger.py: http://gerrit.cloudera.org:8080/#/c/12901/4/tests/authorization/test_ranger.py@38 PS4, Line 38: "--authorization_provider=ranger") > I changed these to use the new flag as I imagine this is the preferred way Would prefer if there was an e2e test with an invalid as well as valid `--authorization_factory_class`. These tests could be put in a new, separate e2e test file. -- To view, visit http://gerrit.cloudera.org:8080/12901 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I001c20505ba4f0562b60fdef73d15308e8500c19 Gerrit-Change-Number: 12901 Gerrit-PatchSet: 4 Gerrit-Owner: radford nguyen Gerrit-Reviewer: Austin Nobis Gerrit-Reviewer: Fredy Wijaya Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Todd Lipcon Gerrit-Reviewer: radford nguyen Gerrit-Comment-Date: Thu, 11 Apr 2019 05:33:25 + Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-8227: Add support for WITH GRANT OPTION with Ranger
Austin Nobis has uploaded a new patch set (#6). ( http://gerrit.cloudera.org:8080/12962 ) Change subject: IMPALA-8227: Add support for WITH GRANT OPTION with Ranger .. IMPALA-8227: Add support for WITH GRANT OPTION with Ranger This patch adds support for the WITH GRANT OPTION SQL syntax when granting privileges to users and groups in Ranger. This allows users who have been granted a privilege to then grant that privilege to other users/groups. Testing: - Ran all FE tests - Ran authorization E2E tests - Added an E2E authorization test in test_ranger to verify the functionality Change-Id: I9c2384f0a9fe30bea1eaceac5b27b1c432383aa8 --- M fe/src/main/java/org/apache/impala/authorization/ranger/RangerCatalogdAuthorizationManager.java M tests/authorization/test_ranger.py 2 files changed, 97 insertions(+), 6 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/62/12962/6 -- To view, visit http://gerrit.cloudera.org:8080/12962 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: I9c2384f0a9fe30bea1eaceac5b27b1c432383aa8 Gerrit-Change-Number: 12962 Gerrit-PatchSet: 6 Gerrit-Owner: Austin Nobis Gerrit-Reviewer: Austin Nobis Gerrit-Reviewer: Fredy Wijaya Gerrit-Reviewer: Impala Public Jenkins
[Impala-ASF-CR] IMPALA-8227: Add support for WITH GRANT OPTION with Ranger
Austin Nobis has uploaded a new patch set (#5). ( http://gerrit.cloudera.org:8080/12962 ) Change subject: IMPALA-8227: Add support for WITH GRANT OPTION with Ranger .. IMPALA-8227: Add support for WITH GRANT OPTION with Ranger This patch adds support for the WITH GRANT OPTION SQL syntax when granting privileges to users and groups in Ranger. This allows users who have been granted a privilege to then grant that privilege to other users/groups. Testing: - Ran all FE tests - Ran authorization E2E tests - Added an E2E authorization test in test_ranger to verify the functionality Change-Id: I9c2384f0a9fe30bea1eaceac5b27b1c432383aa8 --- M fe/src/main/java/org/apache/impala/authorization/ranger/RangerCatalogdAuthorizationManager.java M tests/authorization/test_ranger.py 2 files changed, 90 insertions(+), 6 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/62/12962/5 -- To view, visit http://gerrit.cloudera.org:8080/12962 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: I9c2384f0a9fe30bea1eaceac5b27b1c432383aa8 Gerrit-Change-Number: 12962 Gerrit-PatchSet: 5 Gerrit-Owner: Austin Nobis Gerrit-Reviewer: Austin Nobis Gerrit-Reviewer: Fredy Wijaya Gerrit-Reviewer: Impala Public Jenkins
[Impala-ASF-CR] IMPALA-8227: Add support for WITH GRANT OPTION with Ranger
Austin Nobis has posted comments on this change. (
http://gerrit.cloudera.org:8080/12962 )
Change subject: IMPALA-8227: Add support for WITH GRANT OPTION with Ranger
..
Patch Set 4:
(5 comments)
http://gerrit.cloudera.org:8080/#/c/12962/3/tests/authorization/test_ranger.py
File tests/authorization/test_ranger.py:
http://gerrit.cloudera.org:8080/#/c/12962/3/tests/authorization/test_ranger.py@138
PS3, Line 138:
> flake8: W291 trailing whitespace
Done
http://gerrit.cloudera.org:8080/#/c/12962/3/tests/authorization/test_ranger.py@138
PS3, Line 138: self.execute_query_expect_success(admin_client, "revoke
grant option for select "
> line has trailing whitespace
Done
http://gerrit.cloudera.org:8080/#/c/12962/3/tests/authorization/test_ranger.py@159
PS3, Line 159:
> flake8: W291 trailing whitespace
Done
http://gerrit.cloudera.org:8080/#/c/12962/3/tests/authorization/test_ranger.py@159
PS3, Line 159: r =
requests.post("{0}/service/xusers/secure/users".format(RANGER_HOST),
> line has trailing whitespace
Done
http://gerrit.cloudera.org:8080/#/c/12962/3/tests/authorization/test_ranger.py@168
PS3, Line 168:
> flake8: W391 blank line at end of file
Done
--
To view, visit http://gerrit.cloudera.org:8080/12962
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I9c2384f0a9fe30bea1eaceac5b27b1c432383aa8
Gerrit-Change-Number: 12962
Gerrit-PatchSet: 4
Gerrit-Owner: Austin Nobis
Gerrit-Reviewer: Austin Nobis
Gerrit-Reviewer: Fredy Wijaya
Gerrit-Reviewer: Impala Public Jenkins
Gerrit-Comment-Date: Tue, 09 Apr 2019 19:32:09 +
Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-8227: Add support for WITH GRANT OPTION with Ranger
Austin Nobis has uploaded a new patch set (#4). ( http://gerrit.cloudera.org:8080/12962 ) Change subject: IMPALA-8227: Add support for WITH GRANT OPTION with Ranger .. IMPALA-8227: Add support for WITH GRANT OPTION with Ranger This patch adds support for the WITH GRANT OPTION SQL syntax when granting privileges to users and groups in Ranger. This allows users who have been granted a privilege to then grant that privilege to other users/groups. Testing: - Ran all FE tests - Ran authorization E2E tests - Added an E2E authorization test in test_ranger to verify the functionality Change-Id: I9c2384f0a9fe30bea1eaceac5b27b1c432383aa8 --- M fe/src/main/java/org/apache/impala/authorization/ranger/RangerCatalogdAuthorizationManager.java M tests/authorization/test_ranger.py 2 files changed, 91 insertions(+), 6 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/62/12962/4 -- To view, visit http://gerrit.cloudera.org:8080/12962 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: I9c2384f0a9fe30bea1eaceac5b27b1c432383aa8 Gerrit-Change-Number: 12962 Gerrit-PatchSet: 4 Gerrit-Owner: Austin Nobis Gerrit-Reviewer: Austin Nobis Gerrit-Reviewer: Fredy Wijaya Gerrit-Reviewer: Impala Public Jenkins
[Impala-ASF-CR] IMPALA-8227: Add support for WITH GRANT OPTION with Ranger
Austin Nobis has uploaded a new patch set (#3). ( http://gerrit.cloudera.org:8080/12962 ) Change subject: IMPALA-8227: Add support for WITH GRANT OPTION with Ranger .. IMPALA-8227: Add support for WITH GRANT OPTION with Ranger This patch adds support for the WITH GRANT OPTION SQL syntax when granting privileges to users and groups in Ranger. This allows users who have been granted a privilege to then grant that privilege to other users/groups. Testing: - Ran all FE tests - Ran authorization E2E tests - Added an E2E authorization test in test_ranger to verify the functionality Change-Id: I9c2384f0a9fe30bea1eaceac5b27b1c432383aa8 --- M fe/src/main/java/org/apache/impala/authorization/ranger/RangerCatalogdAuthorizationManager.java M tests/authorization/test_ranger.py 2 files changed, 91 insertions(+), 6 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/62/12962/3 -- To view, visit http://gerrit.cloudera.org:8080/12962 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: I9c2384f0a9fe30bea1eaceac5b27b1c432383aa8 Gerrit-Change-Number: 12962 Gerrit-PatchSet: 3 Gerrit-Owner: Austin Nobis Gerrit-Reviewer: Austin Nobis Gerrit-Reviewer: Fredy Wijaya Gerrit-Reviewer: Impala Public Jenkins
[Impala-ASF-CR] [IMPALA-8227] Add support for WITH GRANT OPTION with Ranger
Austin Nobis has uploaded this change for review. ( http://gerrit.cloudera.org:8080/12962 Change subject: [IMPALA-8227] Add support for WITH GRANT OPTION with Ranger .. [IMPALA-8227] Add support for WITH GRANT OPTION with Ranger This patch adds support for the WITH GRANT OPTION SQL syntax when granting privileges to users and groups in Ranger. This allows users who have been granted a privilege to then grant that privilege to other users/groups. Testing: - Ran all FE tests - Ran authorization E2E tests - Added an E2E authorization test in test_ranger to verify the functionality Change-Id: I9c2384f0a9fe30bea1eaceac5b27b1c432383aa8 --- M fe/src/main/java/org/apache/impala/authorization/ranger/RangerCatalogdAuthorizationManager.java M tests/authorization/test_ranger.py 2 files changed, 77 insertions(+), 6 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/62/12962/2 -- To view, visit http://gerrit.cloudera.org:8080/12962 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newchange Gerrit-Change-Id: I9c2384f0a9fe30bea1eaceac5b27b1c432383aa8 Gerrit-Change-Number: 12962 Gerrit-PatchSet: 2 Gerrit-Owner: Austin Nobis
[Impala-ASF-CR] IMPALA-8385: Refactor Sentry admin user check
Austin Nobis has posted comments on this change. ( http://gerrit.cloudera.org:8080/12963 ) Change subject: IMPALA-8385: Refactor Sentry admin user check .. Patch Set 5: (1 comment) http://gerrit.cloudera.org:8080/#/c/12963/5/tests/authorization/test_sentry.py File tests/authorization/test_sentry.py: http://gerrit.cloudera.org:8080/#/c/12963/5/tests/authorization/test_sentry.py@39 PS5, Line 39: non_admin = "foobar" Consider using unique_name instead of "foobar" -- To view, visit http://gerrit.cloudera.org:8080/12963 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I911228b09af7eed5d5dc002b20591ef64dc625d3 Gerrit-Change-Number: 12963 Gerrit-PatchSet: 5 Gerrit-Owner: Fredy Wijaya Gerrit-Reviewer: Austin Nobis Gerrit-Reviewer: Bharath Vissapragada Gerrit-Reviewer: Fredy Wijaya Gerrit-Reviewer: Impala Public Jenkins Gerrit-Comment-Date: Tue, 09 Apr 2019 15:36:19 + Gerrit-HasComments: Yes
[Impala-ASF-CR] Add impala group.json to .gitignore
Austin Nobis has posted comments on this change. ( http://gerrit.cloudera.org:8080/12964 ) Change subject: Add impala_group.json to .gitignore .. Patch Set 1: Code-Review+1 -- To view, visit http://gerrit.cloudera.org:8080/12964 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I20ab5080fe931ac37af2403d7b85adb78916660a Gerrit-Change-Number: 12964 Gerrit-PatchSet: 1 Gerrit-Owner: Fredy Wijaya Gerrit-Reviewer: Austin Nobis Gerrit-Reviewer: Bharath Vissapragada Gerrit-Reviewer: Fredy Wijaya Gerrit-Reviewer: Impala Public Jenkins Gerrit-Comment-Date: Tue, 09 Apr 2019 05:13:52 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-8393: Skip ranger setup for unsupported environments
Austin Nobis has uploaded a new patch set (#6). ( http://gerrit.cloudera.org:8080/12957 ) Change subject: IMPALA-8393: Skip ranger setup for unsupported environments .. IMPALA-8393: Skip ranger setup for unsupported environments Previously, the setup-ranger step in create-load-data.sh was hard coded with localhost as the host for Ranger. This patch makes it possible to skip the setup for Ranger by using the flag -skip_ranger. The script was also updated to set the SKIP_RANGER variable when the REMOTE_LOAD environment variable is set. Testing: - Testing was performed by calling the script with and without the setup-ranger flag set as well as calling the script with and without the REMOTE_LOAD environment variable set. Change-Id: Ie81dda992cf29792468580b182e570132d5ce0a1 --- M testdata/bin/create-load-data.sh 1 file changed, 12 insertions(+), 1 deletion(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/57/12957/6 -- To view, visit http://gerrit.cloudera.org:8080/12957 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: Ie81dda992cf29792468580b182e570132d5ce0a1 Gerrit-Change-Number: 12957 Gerrit-PatchSet: 6 Gerrit-Owner: Austin Nobis Gerrit-Reviewer: Austin Nobis Gerrit-Reviewer: David Knupp Gerrit-Reviewer: Fredy Wijaya Gerrit-Reviewer: Impala Public Jenkins
[Impala-ASF-CR] IMPALA-8393: Skip ranger setup for unsupported environments
Austin Nobis has posted comments on this change. ( http://gerrit.cloudera.org:8080/12957 ) Change subject: IMPALA-8393: Skip ranger setup for unsupported environments .. Patch Set 5: (2 comments) http://gerrit.cloudera.org:8080/#/c/12957/2//COMMIT_MSG Commit Message: http://gerrit.cloudera.org:8080/#/c/12957/2//COMMIT_MSG@7 PS2, Line 7: Skip ranger setup for unsupported environments > Agreed -- could be something like: Done http://gerrit.cloudera.org:8080/#/c/12957/2/testdata/bin/create-load-data.sh File testdata/bin/create-load-data.sh: http://gerrit.cloudera.org:8080/#/c/12957/2/testdata/bin/create-load-data.sh@105 PS2, Line 105: > use [[ -n $REMOTE_LOAD ]] instead? Some of the code in this file uses -z: h Done -- To view, visit http://gerrit.cloudera.org:8080/12957 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ie81dda992cf29792468580b182e570132d5ce0a1 Gerrit-Change-Number: 12957 Gerrit-PatchSet: 5 Gerrit-Owner: Austin Nobis Gerrit-Reviewer: Austin Nobis Gerrit-Reviewer: David Knupp Gerrit-Reviewer: Fredy Wijaya Gerrit-Reviewer: Impala Public Jenkins Gerrit-Comment-Date: Mon, 08 Apr 2019 20:05:28 + Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-8393: Skip ranger setup for unsupported environments
Austin Nobis has uploaded a new patch set (#5). ( http://gerrit.cloudera.org:8080/12957 ) Change subject: IMPALA-8393: Skip ranger setup for unsupported environments .. IMPALA-8393: Skip ranger setup for unsupported environments Previously, the setup-ranger step in create-load-data.sh was hard coded with localhost as the host for Ranger. This patch makes it possible to skip the setup for Ranger by using the flag -skip_ranger. The script was also updated to set the SKIP_RANGER variable when the REMOTE_LOAD environment variable is set. Testing: - Testing was performed by calling the script with and without the setup-ranger flag set as well as calling the script with and without the REMOTE_LOAD environment variable set. Change-Id: Ie81dda992cf29792468580b182e570132d5ce0a1 --- M testdata/bin/create-load-data.sh 1 file changed, 13 insertions(+), 1 deletion(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/57/12957/5 -- To view, visit http://gerrit.cloudera.org:8080/12957 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: Ie81dda992cf29792468580b182e570132d5ce0a1 Gerrit-Change-Number: 12957 Gerrit-PatchSet: 5 Gerrit-Owner: Austin Nobis Gerrit-Reviewer: Austin Nobis Gerrit-Reviewer: David Knupp Gerrit-Reviewer: Fredy Wijaya Gerrit-Reviewer: Impala Public Jenkins
[Impala-ASF-CR] IMPALA-8393: Skip ranger setup for unsupported environments
Austin Nobis has uploaded a new patch set (#4). ( http://gerrit.cloudera.org:8080/12957 ) Change subject: IMPALA-8393: Skip ranger setup for unsupported environments .. IMPALA-8393: Skip ranger setup for unsupported environments Previously, the setup-ranger step in create-load-data.sh was hard coded with localhost as the host for Ranger. This patch makes it possible to skip the setup for Ranger by using the flag -skip_ranger. The script was also updated to set the SKIP_RANGER variable when the REMOTE_LOAD environment variable is set. Testing: - Testing was performed by calling the script with and without the setup-ranger flag set as well as calling the script with and without the REMOTE_LOAD environment variable set. Change-Id: Ie81dda992cf29792468580b182e570132d5ce0a1 --- M testdata/bin/create-load-data.sh 1 file changed, 14 insertions(+), 1 deletion(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/57/12957/4 -- To view, visit http://gerrit.cloudera.org:8080/12957 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: Ie81dda992cf29792468580b182e570132d5ce0a1 Gerrit-Change-Number: 12957 Gerrit-PatchSet: 4 Gerrit-Owner: Austin Nobis Gerrit-Reviewer: Austin Nobis Gerrit-Reviewer: David Knupp Gerrit-Reviewer: Fredy Wijaya Gerrit-Reviewer: Impala Public Jenkins
[Impala-ASF-CR] IMPALA-8393: setup-ranger in create-load-data.sh breaks for clusters
Austin Nobis has uploaded a new patch set (#2). ( http://gerrit.cloudera.org:8080/12957 ) Change subject: IMPALA-8393: setup-ranger in create-load-data.sh breaks for clusters .. IMPALA-8393: setup-ranger in create-load-data.sh breaks for clusters Previously, the setup-ranger step in create-load-data.sh was hard coded with localhost as the host for Ranger. This patch makes it possible to skip the setup for Ranger by using the flag -skip_ranger. The script was also updated to set the SKIP_RANGER variable when the REMOTE_LOAD environment variable is set. Testing: - Testing was performed by calling the script with and without the setup-ranger flag set as well as calling the script with and without the REMOTE_LOAD environment variable set. Change-Id: Ie81dda992cf29792468580b182e570132d5ce0a1 --- M testdata/bin/create-load-data.sh 1 file changed, 13 insertions(+), 1 deletion(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/57/12957/2 -- To view, visit http://gerrit.cloudera.org:8080/12957 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: Ie81dda992cf29792468580b182e570132d5ce0a1 Gerrit-Change-Number: 12957 Gerrit-PatchSet: 2 Gerrit-Owner: Austin Nobis Gerrit-Reviewer: Austin Nobis Gerrit-Reviewer: David Knupp Gerrit-Reviewer: Fredy Wijaya
[Impala-ASF-CR] IMPALA-8393: setup-ranger in create-load-data.sh breaks for clusters
Austin Nobis has uploaded this change for review. ( http://gerrit.cloudera.org:8080/12957 Change subject: IMPALA-8393: setup-ranger in create-load-data.sh breaks for clusters .. IMPALA-8393: setup-ranger in create-load-data.sh breaks for clusters Previously, the setup-ranger step in create-load-data.sh was hard coded with localhost as the host for Ranger. This patch makes it possible to skip the setup for Ranger by using the flag -skip_ranger. The script was also updated to set the SKIP_RANGER variable in when the REMOTE_LOAD environment variable is set. Testing: - Testing was performed by calling the script with and without the setup-ranger flag set as well as calling the script with and without the REMOTE_LOAD environment variable set. Change-Id: Ie81dda992cf29792468580b182e570132d5ce0a1 --- M testdata/bin/create-load-data.sh 1 file changed, 13 insertions(+), 1 deletion(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/57/12957/1 -- To view, visit http://gerrit.cloudera.org:8080/12957 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newchange Gerrit-Change-Id: Ie81dda992cf29792468580b182e570132d5ce0a1 Gerrit-Change-Number: 12957 Gerrit-PatchSet: 1 Gerrit-Owner: Austin Nobis Gerrit-Reviewer: Austin Nobis Gerrit-Reviewer: David Knupp Gerrit-Reviewer: Fredy Wijaya
[Impala-ASF-CR] IMPALA-8363: Deny access when column masking or row filtering is enabled in Ranger
Austin Nobis has posted comments on this change. (
http://gerrit.cloudera.org:8080/12927 )
Change subject: IMPALA-8363: Deny access when column masking or row filtering
is enabled in Ranger
..
Patch Set 2:
(4 comments)
http://gerrit.cloudera.org:8080/#/c/12927/2/fe/src/main/java/org/apache/impala/authorization/ranger/RangerAuthorizationChecker.java
File
fe/src/main/java/org/apache/impala/authorization/ranger/RangerAuthorizationChecker.java:
http://gerrit.cloudera.org:8080/#/c/12927/2/fe/src/main/java/org/apache/impala/authorization/ranger/RangerAuthorizationChecker.java@70
PS2, Line 70: private static class Table {
Wouldn't this represent a column and not a table?
http://gerrit.cloudera.org:8080/#/c/12927/2/fe/src/main/java/org/apache/impala/authorization/ranger/RangerAuthorizationChecker.java@246
PS2, Line 246: for (Table table: tables) {
So you're iterating over the `Table` which is actually more like a `Column` but
ignoring the columnName_?
Won't this cause a lot of duplicate entries in the `rowFilterTables`?
http://gerrit.cloudera.org:8080/#/c/12927/2/fe/src/main/java/org/apache/impala/authorization/ranger/RangerAuthorizationChecker.java@269
PS2, Line 269: private static List getTables(List exprs) {
Is this getting the tables or the columns?
http://gerrit.cloudera.org:8080/#/c/12927/2/fe/src/main/java/org/apache/impala/authorization/ranger/RangerAuthorizationChecker.java@279
PS2, Line 279: // Column can be null when it's complex type.
If it is a complex type do you need to parse the complex type for more
potential columns?
--
To view, visit http://gerrit.cloudera.org:8080/12927
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: If46b4bf24d916e4a4ea8a36ff4acfd95d5f45c8e
Gerrit-Change-Number: 12927
Gerrit-PatchSet: 2
Gerrit-Owner: Fredy Wijaya
Gerrit-Reviewer: Austin Nobis
Gerrit-Reviewer: Bharath Vissapragada
Gerrit-Reviewer: Fredy Wijaya
Gerrit-Reviewer: Impala Public Jenkins
Gerrit-Reviewer: Todd Lipcon
Gerrit-Comment-Date: Thu, 04 Apr 2019 19:26:03 +
Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-8226: Add grant/revoke to/from group for Ranger
Austin Nobis has uploaded a new patch set (#10). ( http://gerrit.cloudera.org:8080/12914 ) Change subject: IMPALA-8226: Add grant/revoke to/from group for Ranger .. IMPALA-8226: Add grant/revoke to/from group for Ranger This patch adds fupport for GRANT privilege statements to GROUP and REVOKE privilege statements from GROUP. The grammar has been updated to support FROM GROUP and TO GROUP for GRANT/REVOKE statements, i.e: GRANT ON TO GROUP REVOKE ON FROM GROUP Currently, only Ranger's authorization implementation supports GROUP based privileges. Sentry will throw an UnsupportedOperationException if it is the enabled authorization provider and this new grammar is used. Testing: - AuthorizationStmtTest was updated to also test for GROUP authorization. - ToSqlTest was updated to test for GROUP changes to the grammar. - A GROUP based E2E test was added to test_ranger.py - ParserTest was updated to test combinations for GrantRevokePrivilege - AnalyzeAuthStmtsTest was updated to test for USER and GROUP identities - Ran all FE tests - Ran authorization E2E tests Change-Id: I28b7b3e4c776ad1bb5bdc184c7d733d0b5ef5e96 --- M common/thrift/CatalogObjects.thrift M fe/src/main/cup/sql-parser.cup M fe/src/main/java/org/apache/impala/authorization/AuthorizationManager.java M fe/src/main/java/org/apache/impala/authorization/NoneAuthorizationFactory.java M fe/src/main/java/org/apache/impala/authorization/ranger/RangerCatalogdAuthorizationManager.java M fe/src/main/java/org/apache/impala/authorization/sentry/SentryCatalogdAuthorizationManager.java M fe/src/main/java/org/apache/impala/authorization/sentry/SentryImpaladAuthorizationManager.java M fe/src/main/java/org/apache/impala/service/CatalogOpExecutor.java M fe/src/test/java/org/apache/impala/analysis/AnalyzeAuthStmtsTest.java M fe/src/test/java/org/apache/impala/analysis/AuthorizationStmtTest.java M fe/src/test/java/org/apache/impala/analysis/ParserTest.java M fe/src/test/java/org/apache/impala/analysis/ToSqlTest.java M fe/src/test/resources/ranger-hive-security.xml M testdata/bin/create-load-data.sh A testdata/cluster/ranger/setup/impala_group.json.template M testdata/cluster/ranger/setup/impala_user.json.template M tests/authorization/test_ranger.py 17 files changed, 461 insertions(+), 323 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/14/12914/10 -- To view, visit http://gerrit.cloudera.org:8080/12914 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: I28b7b3e4c776ad1bb5bdc184c7d733d0b5ef5e96 Gerrit-Change-Number: 12914 Gerrit-PatchSet: 10 Gerrit-Owner: Austin Nobis Gerrit-Reviewer: Austin Nobis Gerrit-Reviewer: Fredy Wijaya Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: radford nguyen
[Impala-ASF-CR] IMPALA-8226: Add grant/revoke to/from group for Ranger
Austin Nobis has posted comments on this change. ( http://gerrit.cloudera.org:8080/12914 ) Change subject: IMPALA-8226: Add grant/revoke to/from group for Ranger .. Patch Set 9: Fixed an issue caused by the group configuration on the Jenkins host that caused the merge to fail. Tested here: https://master-02.jenkins.cloudera.com/view/Impala/view/Private/job/impala-private-parameterized/4690/ -- To view, visit http://gerrit.cloudera.org:8080/12914 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I28b7b3e4c776ad1bb5bdc184c7d733d0b5ef5e96 Gerrit-Change-Number: 12914 Gerrit-PatchSet: 9 Gerrit-Owner: Austin Nobis Gerrit-Reviewer: Austin Nobis Gerrit-Reviewer: Fredy Wijaya Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: radford nguyen Gerrit-Comment-Date: Thu, 04 Apr 2019 18:56:42 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-8226: Add grant/revoke to/from group for Ranger
Austin Nobis has uploaded a new patch set (#9). ( http://gerrit.cloudera.org:8080/12914 ) Change subject: IMPALA-8226: Add grant/revoke to/from group for Ranger .. IMPALA-8226: Add grant/revoke to/from group for Ranger This patch adds fupport for GRANT privilege statements to GROUP and REVOKE privilege statements from GROUP. The grammar has been updated to support FROM GROUP and TO GROUP for GRANT/REVOKE statements, i.e: GRANT ON TO GROUP REVOKE ON FROM GROUP Currently, only Ranger's authorization implementation supports GROUP based privileges. Sentry will throw an UnsupportedOperationException if it is the enabled authorization provider and this new grammar is used. Testing: - AuthorizationStmtTest was updated to also test for GROUP authorization. - ToSqlTest was updated to test for GROUP changes to the grammar. - A GROUP based E2E test was added to test_ranger.py - ParserTest was updated to test combinations for GrantRevokePrivilege - AnalyzeAuthStmtsTest was updated to test for USER and GROUP identities - Ran all FE tests - Ran authorization E2E tests Change-Id: I28b7b3e4c776ad1bb5bdc184c7d733d0b5ef5e96 --- M common/thrift/CatalogObjects.thrift M fe/src/main/cup/sql-parser.cup M fe/src/main/java/org/apache/impala/authorization/AuthorizationManager.java M fe/src/main/java/org/apache/impala/authorization/NoneAuthorizationFactory.java M fe/src/main/java/org/apache/impala/authorization/ranger/RangerCatalogdAuthorizationManager.java M fe/src/main/java/org/apache/impala/authorization/sentry/SentryCatalogdAuthorizationManager.java M fe/src/main/java/org/apache/impala/authorization/sentry/SentryImpaladAuthorizationManager.java M fe/src/main/java/org/apache/impala/service/CatalogOpExecutor.java M fe/src/test/java/org/apache/impala/analysis/AnalyzeAuthStmtsTest.java M fe/src/test/java/org/apache/impala/analysis/AuthorizationStmtTest.java M fe/src/test/java/org/apache/impala/analysis/ParserTest.java M fe/src/test/java/org/apache/impala/analysis/ToSqlTest.java M fe/src/test/resources/ranger-hive-security.xml M testdata/bin/create-load-data.sh A testdata/cluster/ranger/setup/impala_group.json.template M testdata/cluster/ranger/setup/impala_user.json.template M tests/authorization/test_ranger.py 17 files changed, 463 insertions(+), 323 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/14/12914/9 -- To view, visit http://gerrit.cloudera.org:8080/12914 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: I28b7b3e4c776ad1bb5bdc184c7d733d0b5ef5e96 Gerrit-Change-Number: 12914 Gerrit-PatchSet: 9 Gerrit-Owner: Austin Nobis Gerrit-Reviewer: Austin Nobis Gerrit-Reviewer: Fredy Wijaya Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: radford nguyen
[Impala-ASF-CR] IMPALA-8226: Add grant/revoke to/from group for Ranger
Austin Nobis has uploaded a new patch set (#8). ( http://gerrit.cloudera.org:8080/12914 ) Change subject: IMPALA-8226: Add grant/revoke to/from group for Ranger .. IMPALA-8226: Add grant/revoke to/from group for Ranger This patch adds fupport for GRANT privilege statements to GROUP and REVOKE privilege statements from GROUP. The grammar has been updated to support FROM GROUP and TO GROUP for GRANT/REVOKE statements, i.e: GRANT ON TO GROUP REVOKE ON FROM GROUP Currently, only Ranger's authorization implementation supports GROUP based privileges. Sentry will throw an UnsupportedOperationException if it is the enabled authorization provider and this new grammar is used. Testing: - AuthorizationStmtTest was updated to also test for GROUP authorization. - ToSqlTest was updated to test for GROUP changes to the grammar. - A GROUP based E2E test was added to test_ranger.py - ParserTest was updated to test combinations for GrantRevokePrivilege - AnalyzeAuthStmtsTest was updated to test for USER and GROUP identities - Ran all FE tests - Ran authorization E2E tests Change-Id: I28b7b3e4c776ad1bb5bdc184c7d733d0b5ef5e96 --- M common/thrift/CatalogObjects.thrift M fe/src/main/cup/sql-parser.cup M fe/src/main/java/org/apache/impala/authorization/AuthorizationManager.java M fe/src/main/java/org/apache/impala/authorization/NoneAuthorizationFactory.java M fe/src/main/java/org/apache/impala/authorization/ranger/RangerCatalogdAuthorizationManager.java M fe/src/main/java/org/apache/impala/authorization/sentry/SentryCatalogdAuthorizationManager.java M fe/src/main/java/org/apache/impala/authorization/sentry/SentryImpaladAuthorizationManager.java M fe/src/main/java/org/apache/impala/service/CatalogOpExecutor.java M fe/src/test/java/org/apache/impala/analysis/AnalyzeAuthStmtsTest.java M fe/src/test/java/org/apache/impala/analysis/AuthorizationStmtTest.java M fe/src/test/java/org/apache/impala/analysis/ParserTest.java M fe/src/test/java/org/apache/impala/analysis/ToSqlTest.java M fe/src/test/resources/ranger-hive-security.xml M testdata/bin/create-load-data.sh A testdata/cluster/ranger/setup/impala_group.json.template M testdata/cluster/ranger/setup/impala_user.json.template M tests/authorization/test_ranger.py 17 files changed, 470 insertions(+), 323 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/14/12914/8 -- To view, visit http://gerrit.cloudera.org:8080/12914 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: I28b7b3e4c776ad1bb5bdc184c7d733d0b5ef5e96 Gerrit-Change-Number: 12914 Gerrit-PatchSet: 8 Gerrit-Owner: Austin Nobis Gerrit-Reviewer: Austin Nobis Gerrit-Reviewer: Fredy Wijaya Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: radford nguyen
[Impala-ASF-CR] IMPALA-8226: Add grant/revoke to/from group for Ranger
Austin Nobis has uploaded a new patch set (#6). ( http://gerrit.cloudera.org:8080/12914 ) Change subject: IMPALA-8226: Add grant/revoke to/from group for Ranger .. IMPALA-8226: Add grant/revoke to/from group for Ranger This patch adds fupport for GRANT privilege statements to GROUP and REVOKE privilege statements from GROUP. The grammar has been updated to support FROM GROUP and TO GROUP for GRANT/REVOKE statements, i.e: GRANT ON TO GROUP REVOKE ON FROM GROUP Currently, only Ranger's authorization implementation supports GROUP based privileges. Sentry will throw an UnsupportedOperationException if it is the enabled authorization provider and this new grammar is used. Testing: - AuthorizationStmtTest was updated to also test for GROUP authorization. - ToSqlTest was updated to test for GROUP changes to the grammar. - A GROUP based E2E test was added to test_ranger.py - ParserTest was updated to test combinations for GrantRevokePrivilege - AnalyzeAuthStmtsTest was updated to test for USER and GROUP identities - Ran all FE tests - Ran authorization E2E tests Change-Id: I28b7b3e4c776ad1bb5bdc184c7d733d0b5ef5e96 --- M common/thrift/CatalogObjects.thrift M fe/src/main/cup/sql-parser.cup M fe/src/main/java/org/apache/impala/authorization/AuthorizationManager.java M fe/src/main/java/org/apache/impala/authorization/NoneAuthorizationFactory.java M fe/src/main/java/org/apache/impala/authorization/ranger/RangerCatalogdAuthorizationManager.java M fe/src/main/java/org/apache/impala/authorization/sentry/SentryCatalogdAuthorizationManager.java M fe/src/main/java/org/apache/impala/authorization/sentry/SentryImpaladAuthorizationManager.java M fe/src/main/java/org/apache/impala/service/CatalogOpExecutor.java M fe/src/test/java/org/apache/impala/analysis/AnalyzeAuthStmtsTest.java M fe/src/test/java/org/apache/impala/analysis/AuthorizationStmtTest.java M fe/src/test/java/org/apache/impala/analysis/ParserTest.java M fe/src/test/java/org/apache/impala/analysis/ToSqlTest.java M fe/src/test/resources/ranger-hive-security.xml M testdata/bin/create-load-data.sh A testdata/cluster/ranger/setup/impala_group.json.template M testdata/cluster/ranger/setup/impala_user.json.template M tests/authorization/test_ranger.py 17 files changed, 468 insertions(+), 323 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/14/12914/6 -- To view, visit http://gerrit.cloudera.org:8080/12914 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: I28b7b3e4c776ad1bb5bdc184c7d733d0b5ef5e96 Gerrit-Change-Number: 12914 Gerrit-PatchSet: 6 Gerrit-Owner: Austin Nobis Gerrit-Reviewer: Austin Nobis Gerrit-Reviewer: Fredy Wijaya Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: radford nguyen
[Impala-ASF-CR] IMPALA-8226: Add grant/revoke to/from group for Ranger
Austin Nobis has uploaded a new patch set (#5). ( http://gerrit.cloudera.org:8080/12914 ) Change subject: IMPALA-8226: Add grant/revoke to/from group for Ranger .. IMPALA-8226: Add grant/revoke to/from group for Ranger This patch adds fupport for GRANT privilege statements to GROUP and REVOKE privilege statements from GROUP. The grammar has been updated to support FROM GROUP and TO GROUP for GRANT/REVOKE statements, i.e: GRANT ON TO GROUP REVOKE ON FROM GROUP Currently, only Ranger's authorization implementation supports GROUP based privileges. Sentry will throw an UnsupportedOperationException if it is the enabled authorization provider and this new grammar is used. Testing: - AuthorizationStmtTest was updated to also test for GROUP authorization. - ToSqlTest was updated to test for GROUP changes to the grammar. - A GROUP based E2E test was added to test_ranger.py - ParserTest was updated to test combinations for GrantRevokePrivilege - AnalyzeAuthStmtsTest was updated to test for USER and GROUP identities - Ran all FE tests - Ran authorization E2E tests Change-Id: I28b7b3e4c776ad1bb5bdc184c7d733d0b5ef5e96 --- M common/thrift/CatalogObjects.thrift M fe/src/main/cup/sql-parser.cup M fe/src/main/java/org/apache/impala/authorization/AuthorizationManager.java M fe/src/main/java/org/apache/impala/authorization/NoneAuthorizationFactory.java M fe/src/main/java/org/apache/impala/authorization/ranger/RangerCatalogdAuthorizationManager.java M fe/src/main/java/org/apache/impala/authorization/sentry/SentryCatalogdAuthorizationManager.java M fe/src/main/java/org/apache/impala/authorization/sentry/SentryImpaladAuthorizationManager.java M fe/src/main/java/org/apache/impala/service/CatalogOpExecutor.java M fe/src/test/java/org/apache/impala/analysis/AnalyzeAuthStmtsTest.java M fe/src/test/java/org/apache/impala/analysis/AuthorizationStmtTest.java M fe/src/test/java/org/apache/impala/analysis/ParserTest.java M fe/src/test/java/org/apache/impala/analysis/ToSqlTest.java M fe/src/test/resources/ranger-hive-security.xml M testdata/bin/create-load-data.sh A testdata/cluster/ranger/setup/impala_group.json.template M testdata/cluster/ranger/setup/impala_user.json.template M tests/authorization/test_ranger.py 17 files changed, 467 insertions(+), 321 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/14/12914/5 -- To view, visit http://gerrit.cloudera.org:8080/12914 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: I28b7b3e4c776ad1bb5bdc184c7d733d0b5ef5e96 Gerrit-Change-Number: 12914 Gerrit-PatchSet: 5 Gerrit-Owner: Austin Nobis Gerrit-Reviewer: Austin Nobis Gerrit-Reviewer: Fredy Wijaya Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: radford nguyen
[Impala-ASF-CR] IMPALA-8226: Add grant/revoke to/from group for Ranger
Austin Nobis has uploaded a new patch set (#4). ( http://gerrit.cloudera.org:8080/12914 ) Change subject: IMPALA-8226: Add grant/revoke to/from group for Ranger .. IMPALA-8226: Add grant/revoke to/from group for Ranger This patch adds fupport for GRANT privilege statements to GROUP and REVOKE privilege statements from GROUP. The grammar has been updated to support FROM GROUP and TO GROUP for GRANT/REVOKE statements, i.e: GRANT ON TO GROUP REVOKE ON FROM GROUP Currently, only Ranger's authorization implementation supports GROUP based privileges. Sentry will throw an UnsupportedOperationException if it is the enabled authorization provider and this new grammar is used. Testing: - AuthorizationStmtTest was updated to also test for GROUP authorization. - ToSqlTest was updated to test for GROUP changes to the grammar. - A GROUP based E2E test was added to test_ranger.py - ParserTest was updated to test combinations for GrantRevokePrivilege - AnalyzeAuthStmtsTest was updated to test for USER and GROUP identities - Ran all FE tests - Ran authorization E2E tests Change-Id: I28b7b3e4c776ad1bb5bdc184c7d733d0b5ef5e96 --- M common/thrift/CatalogObjects.thrift M fe/src/main/cup/sql-parser.cup M fe/src/main/java/org/apache/impala/authorization/AuthorizationManager.java M fe/src/main/java/org/apache/impala/authorization/NoneAuthorizationFactory.java M fe/src/main/java/org/apache/impala/authorization/ranger/RangerCatalogdAuthorizationManager.java M fe/src/main/java/org/apache/impala/authorization/sentry/SentryCatalogdAuthorizationManager.java M fe/src/main/java/org/apache/impala/authorization/sentry/SentryImpaladAuthorizationManager.java M fe/src/main/java/org/apache/impala/service/CatalogOpExecutor.java M fe/src/test/java/org/apache/impala/analysis/AnalyzeAuthStmtsTest.java M fe/src/test/java/org/apache/impala/analysis/AuthorizationStmtTest.java M fe/src/test/java/org/apache/impala/analysis/ParserTest.java M fe/src/test/java/org/apache/impala/analysis/ToSqlTest.java M fe/src/test/resources/ranger-hive-security.xml M testdata/bin/create-load-data.sh A testdata/cluster/ranger/setup/impala_group.json.template M testdata/cluster/ranger/setup/impala_user.json.template M tests/authorization/test_ranger.py 17 files changed, 470 insertions(+), 325 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/14/12914/4 -- To view, visit http://gerrit.cloudera.org:8080/12914 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: I28b7b3e4c776ad1bb5bdc184c7d733d0b5ef5e96 Gerrit-Change-Number: 12914 Gerrit-PatchSet: 4 Gerrit-Owner: Austin Nobis Gerrit-Reviewer: Austin Nobis Gerrit-Reviewer: Fredy Wijaya Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: radford nguyen
[Impala-ASF-CR] IMPALA-8226: Add grant/revoke to/from group for Ranger
Austin Nobis has uploaded a new patch set (#3). ( http://gerrit.cloudera.org:8080/12914 ) Change subject: IMPALA-8226: Add grant/revoke to/from group for Ranger .. IMPALA-8226: Add grant/revoke to/from group for Ranger This patch adds fupport for GRANT privilege statements to GROUP and REVOKE privilege statements from GROUP. The grammar has been updated to support FROM GROUP and TO GROUP for GRANT/REVOKE statements, i.e: GRANT ON TO GROUP REVOKE ON FROM GROUP Currently, only Ranger's authorization implementation supports GROUP based privileges. Sentry will throw an UnsupportedOperationException if it is the enabled authorization provider and this new grammar is used. Testing: - AuthorizationStmtTest was updated to also test for GROUP authorization. - ToSqlTest was updated to test for GROUP changes to the grammar. - A GROUP based E2E test was added to test_ranger.py - Ran all FE tests - Ran authorization E2E tests Change-Id: I28b7b3e4c776ad1bb5bdc184c7d733d0b5ef5e96 --- M common/thrift/CatalogObjects.thrift M fe/src/main/cup/sql-parser.cup M fe/src/main/java/org/apache/impala/authorization/AuthorizationManager.java M fe/src/main/java/org/apache/impala/authorization/NoneAuthorizationFactory.java M fe/src/main/java/org/apache/impala/authorization/ranger/RangerCatalogdAuthorizationManager.java M fe/src/main/java/org/apache/impala/authorization/sentry/SentryCatalogdAuthorizationManager.java M fe/src/main/java/org/apache/impala/authorization/sentry/SentryImpaladAuthorizationManager.java M fe/src/main/java/org/apache/impala/service/CatalogOpExecutor.java M fe/src/test/java/org/apache/impala/analysis/AuthorizationStmtTest.java M fe/src/test/java/org/apache/impala/analysis/ToSqlTest.java M fe/src/test/resources/ranger-hive-security.xml M testdata/bin/create-load-data.sh A testdata/cluster/ranger/setup/impala_group.json.template M testdata/cluster/ranger/setup/impala_user.json.template M tests/authorization/test_ranger.py 15 files changed, 212 insertions(+), 83 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/14/12914/3 -- To view, visit http://gerrit.cloudera.org:8080/12914 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: I28b7b3e4c776ad1bb5bdc184c7d733d0b5ef5e96 Gerrit-Change-Number: 12914 Gerrit-PatchSet: 3 Gerrit-Owner: Austin Nobis Gerrit-Reviewer: Austin Nobis Gerrit-Reviewer: Fredy Wijaya Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: radford nguyen
[Impala-ASF-CR] IMPALA-8226: Add grant/revoke to/from group for Ranger
Austin Nobis has uploaded this change for review. ( http://gerrit.cloudera.org:8080/12914 Change subject: IMPALA-8226: Add grant/revoke to/from group for Ranger .. IMPALA-8226: Add grant/revoke to/from group for Ranger This patch adds fupport for GRANT privilege statements to GROUP and REVOKE privilege statements from GROUP. The grammar has been updated to support FROM GROUP and TO GROUP for GRANT/REVOKE statements, i.e: GRANT ON TO GROUP REVOKE ON FROM GROUP Currently, only Ranger's authorization implementation supports GROUP based privileges. Sentry will throw an UnsupportedOperationException if it is the enabled authorization provider and this new grammar is used. Testing: - AuthorizationStmtTest was updated to also test for GROUP authorization. - ToSqlTest was updated to test for GROUP changes to the grammar. - A GROUP based E2E test was added to test_ranger.py - Ran all FE tests - Ran authorization E2E tests Change-Id: I28b7b3e4c776ad1bb5bdc184c7d733d0b5ef5e96 --- M common/thrift/CatalogObjects.thrift M fe/src/main/cup/sql-parser.cup M fe/src/main/java/org/apache/impala/authorization/AuthorizationManager.java M fe/src/main/java/org/apache/impala/authorization/NoneAuthorizationFactory.java M fe/src/main/java/org/apache/impala/authorization/ranger/RangerCatalogdAuthorizationManager.java M fe/src/main/java/org/apache/impala/authorization/sentry/SentryCatalogdAuthorizationManager.java M fe/src/main/java/org/apache/impala/authorization/sentry/SentryImpaladAuthorizationManager.java M fe/src/main/java/org/apache/impala/service/CatalogOpExecutor.java M fe/src/test/java/org/apache/impala/analysis/AuthorizationStmtTest.java M fe/src/test/java/org/apache/impala/analysis/ToSqlTest.java M fe/src/test/resources/ranger-hive-security.xml M testdata/bin/create-load-data.sh A testdata/cluster/ranger/setup/impala_group.json.template M testdata/cluster/ranger/setup/impala_user.json.template M tests/authorization/test_ranger.py 15 files changed, 212 insertions(+), 83 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/14/12914/2 -- To view, visit http://gerrit.cloudera.org:8080/12914 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newchange Gerrit-Change-Id: I28b7b3e4c776ad1bb5bdc184c7d733d0b5ef5e96 Gerrit-Change-Number: 12914 Gerrit-PatchSet: 2 Gerrit-Owner: Austin Nobis Gerrit-Reviewer: Austin Nobis Gerrit-Reviewer: Fredy Wijaya Gerrit-Reviewer: radford nguyen
[Impala-ASF-CR] IMPALA-8309: add user authorization provider flag
Austin Nobis has posted comments on this change. (
http://gerrit.cloudera.org:8080/12901 )
Change subject: IMPALA-8309: add user authorization_provider flag
..
Patch Set 2:
(13 comments)
You also need to add E2E tests in the python testing.
http://gerrit.cloudera.org:8080/#/c/12901/2/be/src/service/frontend.cc
File be/src/service/frontend.cc:
http://gerrit.cloudera.org:8080/#/c/12901/2/be/src/service/frontend.cc@41
PS2, Line 41: "sentry",
nit: move default to line above like other DEFINE
http://gerrit.cloudera.org:8080/#/c/12901/2/be/src/service/frontend.cc@45
PS2, Line 45: "",
nit: move default to line above like other DEFINE.
http://gerrit.cloudera.org:8080/#/c/12901/2/be/src/service/frontend.cc@46
PS2, Line 46: "Specifies the class name that implements the authorization
provider. "
: "This will override the authorization_provider flag if both
are specified."
nit: Wrap at the character limit not at the end of a sentence.
http://gerrit.cloudera.org:8080/#/c/12901/2/fe/src/main/java/org/apache/impala/authorization/AuthorizationProvider.java
File
fe/src/main/java/org/apache/impala/authorization/AuthorizationProvider.java:
http://gerrit.cloudera.org:8080/#/c/12901/2/fe/src/main/java/org/apache/impala/authorization/AuthorizationProvider.java@31
PS2, Line 31: factoryClass
Should this be `factoryClassName`?
http://gerrit.cloudera.org:8080/#/c/12901/2/fe/src/main/java/org/apache/impala/authorization/NoneAuthorizationFactory.java
File
fe/src/main/java/org/apache/impala/authorization/NoneAuthorizationFactory.java:
http://gerrit.cloudera.org:8080/#/c/12901/2/fe/src/main/java/org/apache/impala/authorization/NoneAuthorizationFactory.java@67
PS2, Line 67: public String getProviderName() { return "none"; }
Can we make "none" a public static final String?
http://gerrit.cloudera.org:8080/#/c/12901/2/fe/src/main/java/org/apache/impala/authorization/ranger/RangerAuthorizationConfig.java
File
fe/src/main/java/org/apache/impala/authorization/ranger/RangerAuthorizationConfig.java:
http://gerrit.cloudera.org:8080/#/c/12901/2/fe/src/main/java/org/apache/impala/authorization/ranger/RangerAuthorizationConfig.java@45
PS2, Line 45: public String getProviderName() { return "ranger"; }
Can we make "ranger" a public static final String?
http://gerrit.cloudera.org:8080/#/c/12901/2/fe/src/main/java/org/apache/impala/authorization/sentry/SentryAuthorizationConfig.java
File
fe/src/main/java/org/apache/impala/authorization/sentry/SentryAuthorizationConfig.java:
http://gerrit.cloudera.org:8080/#/c/12901/2/fe/src/main/java/org/apache/impala/authorization/sentry/SentryAuthorizationConfig.java@130
PS2, Line 130: public String getProviderName() { return "sentry"; }
Can we make "sentry" a public static final String?
http://gerrit.cloudera.org:8080/#/c/12901/2/fe/src/main/java/org/apache/impala/service/BackendConfig.java
File fe/src/main/java/org/apache/impala/service/BackendConfig.java:
http://gerrit.cloudera.org:8080/#/c/12901/2/fe/src/main/java/org/apache/impala/service/BackendConfig.java@141
PS2, Line 141: public @Nullable String getAuthorizationFactoryClassOrNull() {
> Not sure if there's a standard for handling optional flags... at least I di
nit: I'm not opposed to adding the @Nullable annotation and adding `orNull` in
the class name but it is inconsistent with the rest of the code base.
http://gerrit.cloudera.org:8080/#/c/12901/2/fe/src/main/java/org/apache/impala/service/JniFrontend.java
File fe/src/main/java/org/apache/impala/service/JniFrontend.java:
http://gerrit.cloudera.org:8080/#/c/12901/2/fe/src/main/java/org/apache/impala/service/JniFrontend.java@773
PS2, Line 773: * @param beCfg
:* @return
:* @throws InternalException
Finish the documentation
http://gerrit.cloudera.org:8080/#/c/12901/2/fe/src/main/java/org/apache/impala/service/JniFrontend.java@778
PS2, Line 778: throws InternalException {
nit: indent 4
http://gerrit.cloudera.org:8080/#/c/12901/2/fe/src/main/java/org/apache/impala/service/JniFrontend.java@794
PS2, Line 794: +authzProvider
nit: add space around the `+`
http://gerrit.cloudera.org:8080/#/c/12901/2/fe/src/main/java/org/apache/impala/service/JniFrontend.java@812
PS2, Line 812: throws InternalException {
nit: indent 4
http://gerrit.cloudera.org:8080/#/c/12901/2/fe/src/main/java/org/apache/impala/service/JniFrontend.java@822
PS2, Line 822: +authzFactoryClassName
nit: add space around the `+`
--
To view, visit http://gerrit.cloudera.org:8080/12901
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I001c20505ba4f0562b60fdef73d15308e8500c19
Gerrit-Change-Number: 12901
Gerrit-PatchSet: 2
Gerrit-Owner: radford nguyen
Gerrit-Reviewer: Austin Nobis
Gerrit-Reviewer: Fredy Wijaya
Gerrit-Reviewer: Impala Public Jenkins
[Impala-ASF-CR] IMPALA-8368: Fix exception thrown by create database/table with Ranger enabled
Austin Nobis has uploaded a new patch set (#3). ( http://gerrit.cloudera.org:8080/12878 ) Change subject: IMPALA-8368: Fix exception thrown by create database/table with Ranger enabled .. IMPALA-8368: Fix exception thrown by create database/table with Ranger enabled Previously, an exception would be thrown when a create database or create table command was run and Ranger was enabled. This patch removes the code that would throw the invalid exception from RangerCatalogdAuthorizationManager. Testing: - The python E2E test was updated to create and drop a database to cover the missing test case. - All E2E tests were run - All FE tests were run Change-Id: I430b2e67e97dc0fc99f655fcfd7db2072b13 --- M fe/src/main/java/org/apache/impala/authorization/ranger/RangerCatalogdAuthorizationManager.java M tests/authorization/test_ranger.py 2 files changed, 23 insertions(+), 14 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/78/12878/3 -- To view, visit http://gerrit.cloudera.org:8080/12878 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: I430b2e67e97dc0fc99f655fcfd7db2072b13 Gerrit-Change-Number: 12878 Gerrit-PatchSet: 3 Gerrit-Owner: Austin Nobis Gerrit-Reviewer: Austin Nobis Gerrit-Reviewer: Fredy Wijaya Gerrit-Reviewer: Impala Public Jenkins
[Impala-ASF-CR] IMPALA-8293: Support for Ranger cache invalidation
Austin Nobis has posted comments on this change. (
http://gerrit.cloudera.org:8080/12748 )
Change subject: IMPALA-8293: Support for Ranger cache invalidation
..
Patch Set 6:
(9 comments)
http://gerrit.cloudera.org:8080/#/c/12748/6/fe/src/main/java/org/apache/impala/authorization/AuthorizationChecker.java
File fe/src/main/java/org/apache/impala/authorization/AuthorizationChecker.java:
http://gerrit.cloudera.org:8080/#/c/12748/6/fe/src/main/java/org/apache/impala/authorization/AuthorizationChecker.java@121
PS6, Line 121:* Performs a cache refresh.
Can you add more information? How is this cache related to the
AuthorizationChecker interface.?
http://gerrit.cloudera.org:8080/#/c/12748/6/fe/src/main/java/org/apache/impala/authorization/AuthorizationDelta.java
File fe/src/main/java/org/apache/impala/authorization/AuthorizationDelta.java:
http://gerrit.cloudera.org:8080/#/c/12748/6/fe/src/main/java/org/apache/impala/authorization/AuthorizationDelta.java@44
PS6, Line 44: public List getAdded() { return added_; }
: public List getRemoved() { return removed_; }
nit: Consider wrapping these in Collections.unmodifiableList().
http://gerrit.cloudera.org:8080/#/c/12748/6/fe/src/main/java/org/apache/impala/authorization/AuthorizationFactory.java
File fe/src/main/java/org/apache/impala/authorization/AuthorizationFactory.java:
http://gerrit.cloudera.org:8080/#/c/12748/6/fe/src/main/java/org/apache/impala/authorization/AuthorizationFactory.java@79
PS6, Line 79: of {@link AuthorizationRefresher}
nit: of *an* {@link AuthorizationRefresher}
http://gerrit.cloudera.org:8080/#/c/12748/6/fe/src/main/java/org/apache/impala/authorization/AuthorizationRefresher.java
File
fe/src/main/java/org/apache/impala/authorization/AuthorizationRefresher.java:
http://gerrit.cloudera.org:8080/#/c/12748/6/fe/src/main/java/org/apache/impala/authorization/AuthorizationRefresher.java@30
PS6, Line 30: rest
typo: reset
http://gerrit.cloudera.org:8080/#/c/12748/6/fe/src/main/java/org/apache/impala/authorization/ranger/RangerAuthorizationChecker.java
File
fe/src/main/java/org/apache/impala/authorization/ranger/RangerAuthorizationChecker.java:
http://gerrit.cloudera.org:8080/#/c/12748/6/fe/src/main/java/org/apache/impala/authorization/ranger/RangerAuthorizationChecker.java@64
PS6, Line 64: synchronized
Can we limit the scope of the synchronization to just the `authorize` calls?
I'm guessing we need to synchronize this in the event that the `refresh` is
called and we want to wait for it to complete?
http://gerrit.cloudera.org:8080/#/c/12748/6/fe/src/main/java/org/apache/impala/authorization/ranger/RangerAuthorizationRefresher.java
File
fe/src/main/java/org/apache/impala/authorization/ranger/RangerAuthorizationRefresher.java:
http://gerrit.cloudera.org:8080/#/c/12748/6/fe/src/main/java/org/apache/impala/authorization/ranger/RangerAuthorizationRefresher.java@45
PS6, Line 45: authzDelta.getAdded().add(authzCache.toTCatalogObject());
nit: Add some method to `AuthorizationDelta` for adding elements to the
`private final List<>` as opposed to exposing the list and mutating its
elements.
http://gerrit.cloudera.org:8080/#/c/12748/6/fe/src/main/java/org/apache/impala/catalog/CatalogServiceCatalog.java
File fe/src/main/java/org/apache/impala/catalog/CatalogServiceCatalog.java:
http://gerrit.cloudera.org:8080/#/c/12748/6/fe/src/main/java/org/apache/impala/catalog/CatalogServiceCatalog.java@2293
PS6, Line 2293: ;
nit: unnecessary semicolon
http://gerrit.cloudera.org:8080/#/c/12748/6/fe/src/main/java/org/apache/impala/catalog/CatalogServiceCatalog.java@2297
PS6, Line 2297: addAuthzCache
I think this method name is slightly misleading. It only adds a new cache if
the current authzCache is null. It also ends up looking weird from the
functions that call this method.
i.e.
https://gerrit.cloudera.org/c/12748/6/fe/src/main/java/org/apache/impala/authorization/ranger/RangerAuthorizationRefresher.java#44
In this case, the RangerAuthorizationRefresher looks like it is adding a new
authzCache everytime refresh is called and returning the created object. In
reality it is just an increment to the catalog version while returning the
mutated authzCache that already existed.
http://gerrit.cloudera.org:8080/#/c/12748/6/fe/src/main/java/org/apache/impala/service/JniCatalog.java
File fe/src/main/java/org/apache/impala/service/JniCatalog.java:
http://gerrit.cloudera.org:8080/#/c/12748/6/fe/src/main/java/org/apache/impala/service/JniCatalog.java@140
PS6, Line 140: User kerberosPrincipal = Strings.isNullOrEmpty(cfg.principal) ?
: new User(System.getProperty("user.name")) : new
User(cfg.principal);
Is it safe to use the System.getProperty('user.name") here if the principal is
not passed from the backend? I'm not sure if something malicious is possible
here by potentially passing -Duser.name=* during start-up. It seems like the
[Impala-ASF-CR] IMPALA-8368: Fix exception thrown by create database or create table with Ranger enabled
Austin Nobis has uploaded a new patch set (#2). ( http://gerrit.cloudera.org:8080/12878 ) Change subject: IMPALA-8368: Fix exception thrown by create database or create table with Ranger enabled .. IMPALA-8368: Fix exception thrown by create database or create table with Ranger enabled Previously, an exception would be thrown when a create database or create table command was run and Ranger was enabled. This patch removes the code that would throw the invalid exception from RangerCatalogdAuthorizationManager. Testing: - The python E2E test was updated to create and drop a database to cover the missing test case. - All E2E tests were run - All FE tests were run Change-Id: I430b2e67e97dc0fc99f655fcfd7db2072b13 --- M fe/src/main/java/org/apache/impala/authorization/ranger/RangerCatalogdAuthorizationManager.java M tests/authorization/test_ranger.py 2 files changed, 23 insertions(+), 14 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/78/12878/2 -- To view, visit http://gerrit.cloudera.org:8080/12878 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: I430b2e67e97dc0fc99f655fcfd7db2072b13 Gerrit-Change-Number: 12878 Gerrit-PatchSet: 2 Gerrit-Owner: Austin Nobis Gerrit-Reviewer: Austin Nobis Gerrit-Reviewer: Fredy Wijaya Gerrit-Reviewer: Impala Public Jenkins
[Impala-ASF-CR] IMPALA-8368: Fix exception thrown by create database with Ranger enabled
Austin Nobis has uploaded this change for review. ( http://gerrit.cloudera.org:8080/12878 Change subject: IMPALA-8368: Fix exception thrown by create database with Ranger enabled .. IMPALA-8368: Fix exception thrown by create database with Ranger enabled Previously, an exception would be thrown when a create database or create table command was run and Ranger was enabled. This patch removes the code that would throw the invalid exception from RangerCatalogdAuthorizationManager. Testing: - The python E2E test was updated to create and drop a database to cover the missing test case. - All E2E tests were run - All FE tests were run Change-Id: I430b2e67e97dc0fc99f655fcfd7db2072b13 --- M fe/src/main/java/org/apache/impala/authorization/ranger/RangerCatalogdAuthorizationManager.java M tests/authorization/test_ranger.py 2 files changed, 23 insertions(+), 16 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/78/12878/1 -- To view, visit http://gerrit.cloudera.org:8080/12878 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newchange Gerrit-Change-Id: I430b2e67e97dc0fc99f655fcfd7db2072b13 Gerrit-Change-Number: 12878 Gerrit-PatchSet: 1 Gerrit-Owner: Austin Nobis Gerrit-Reviewer: Austin Nobis Gerrit-Reviewer: Fredy Wijaya
[Impala-ASF-CR] IMPALA-8225: Add Ranger support for grant/revoke privilege to/from user
Austin Nobis has uploaded a new patch set (#10). ( http://gerrit.cloudera.org:8080/12769 ) Change subject: IMPALA-8225: Add Ranger support for grant/revoke privilege to/from user .. IMPALA-8225: Add Ranger support for grant/revoke privilege to/from user This patch adds support for GRANT privilege statements to USER and REVOKE privilege statements from USER. The RangerAuthorizationManager class has been created and will throw UnsupportedOperationException when an unimplemented method is called. The grammar has been updated to support FROM USER and TO USER for GRANT/REVOKE statements. Previously, privileges could be granted to a ROLE via GRANT/REVOKE statements even when the ROLE keyword was omitted, i.e: GRANT ON TO This is still the case for ROLE based authorization to preserve backward compatibility, but Ranger will throw an exception when a GRANT/REVOKE statement excludes the USER keyword. The syntax for the new statement is: GRANT ON TO USER REVOKE ON FROM USER Sentry does not support grant/revoke to/from user. Testing: - An additional end to end test, test_ranger.py, was added. A single test was added that grants and revokes for a user and asserts permissions on a table. The test uses sleep statements to work with Ranger's polling interval for policy changes. More end to end tests will be added in the future when the refresh authorization statement works properly with Ranger. - AuthorizationStmtTest has been refactored to use the new RangerCatalogdAuthorizationManager grant/revoke methods for better test coverage. - Ran all FE tests - Ran all E2E authorization tests Change-Id: I6ee97bf41546d63385026c0e2b19545565402462 --- M common/thrift/JniCatalog.thrift M fe/src/main/cup/sql-parser.cup M fe/src/main/java/org/apache/impala/analysis/GrantRevokePrivStmt.java M fe/src/main/java/org/apache/impala/authorization/AuthorizationManager.java M fe/src/main/java/org/apache/impala/authorization/NoneAuthorizationFactory.java M fe/src/main/java/org/apache/impala/authorization/ranger/RangerAuthorizationFactory.java A fe/src/main/java/org/apache/impala/authorization/ranger/RangerCatalogdAuthorizationManager.java M fe/src/main/java/org/apache/impala/authorization/sentry/SentryCatalogdAuthorizationManager.java M fe/src/main/java/org/apache/impala/authorization/sentry/SentryImpaladAuthorizationManager.java M fe/src/main/java/org/apache/impala/service/CatalogOpExecutor.java M fe/src/test/java/org/apache/impala/analysis/AuthorizationStmtTest.java M fe/src/test/java/org/apache/impala/analysis/ToSqlTest.java A tests/authorization/test_ranger.py 13 files changed, 605 insertions(+), 224 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/69/12769/10 -- To view, visit http://gerrit.cloudera.org:8080/12769 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: I6ee97bf41546d63385026c0e2b19545565402462 Gerrit-Change-Number: 12769 Gerrit-PatchSet: 10 Gerrit-Owner: Austin Nobis Gerrit-Reviewer: Austin Nobis Gerrit-Reviewer: Fredy Wijaya Gerrit-Reviewer: Impala Public Jenkins
[Impala-ASF-CR] IMPALA-8225: Add Ranger support for grant/revoke privilege to/from user
Austin Nobis has uploaded a new patch set (#8). ( http://gerrit.cloudera.org:8080/12769 ) Change subject: IMPALA-8225: Add Ranger support for grant/revoke privilege to/from user .. IMPALA-8225: Add Ranger support for grant/revoke privilege to/from user This patch adds support for GRANT privilege statements to USER and REVOKE privilege statements from USER. The RangerAuthorizationManager class has been created and will throw UnsupportedOperationException when an unimplemented method is called. The grammar has been updated to support FROM USER and TO USER for GRANT/REVOKE statements. Previously, privileges could be granted to a ROLE via GRANT/REVOKE statements even when the ROLE keyword was omitted, i.e: GRANT ON TO This is still the case for ROLE based authorization to preserve backward compatibility, but Ranger will throw an exception when a GRANT/REVOKE statement excludes the USER keyword. The syntax for the new statement is: GRANT ON TO USER REVOKE ON FROM USER Sentry does not support grant/revoke to/from user. Testing: - An additional end to end test, test_ranger.py, was added. A single test was added that grants and revokes for a user and asserts permissions on a table. The test uses sleep statements to work with Ranger's polling interval for policy changes. More end to end tests will be added in the future when the refresh authorization statement works properly with Ranger. - AuthorizationStmtTest has been refactored to use the new RangerCatalogdAuthorizationManager grant/revoke methods for better test coverage. - Ran all FE tests - Ran all E2E authorization tests Change-Id: I6ee97bf41546d63385026c0e2b19545565402462 --- M common/thrift/JniCatalog.thrift M fe/src/main/cup/sql-parser.cup M fe/src/main/java/org/apache/impala/analysis/GrantRevokePrivStmt.java M fe/src/main/java/org/apache/impala/authorization/AuthorizationManager.java M fe/src/main/java/org/apache/impala/authorization/NoneAuthorizationFactory.java M fe/src/main/java/org/apache/impala/authorization/ranger/RangerAuthorizationFactory.java A fe/src/main/java/org/apache/impala/authorization/ranger/RangerCatalogdAuthorizationManager.java M fe/src/main/java/org/apache/impala/authorization/sentry/SentryCatalogdAuthorizationManager.java M fe/src/main/java/org/apache/impala/authorization/sentry/SentryImpaladAuthorizationManager.java M fe/src/main/java/org/apache/impala/service/CatalogOpExecutor.java M fe/src/test/java/org/apache/impala/analysis/AuthorizationStmtTest.java M fe/src/test/java/org/apache/impala/analysis/ToSqlTest.java A tests/authorization/test_ranger.py 13 files changed, 598 insertions(+), 224 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/69/12769/8 -- To view, visit http://gerrit.cloudera.org:8080/12769 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: I6ee97bf41546d63385026c0e2b19545565402462 Gerrit-Change-Number: 12769 Gerrit-PatchSet: 8 Gerrit-Owner: Austin Nobis Gerrit-Reviewer: Austin Nobis Gerrit-Reviewer: Fredy Wijaya Gerrit-Reviewer: Impala Public Jenkins
[Impala-ASF-CR] IMPALA-8225: Add Ranger support for grant/revoke privilege to/from user
Austin Nobis has uploaded a new patch set (#7). ( http://gerrit.cloudera.org:8080/12769 ) Change subject: IMPALA-8225: Add Ranger support for grant/revoke privilege to/from user .. IMPALA-8225: Add Ranger support for grant/revoke privilege to/from user This patch adds support for GRANT privilege statements to USER and REVOKE privilege statements from USER. The RangerAuthorizationManager class has been created and will throw UnsupportedOperationException when an unimplemented method is called. The grammar has been updated to support FROM USER and TO USER for GRANT/REVOKE statements. Previously, privileges could be granted to a ROLE via GRANT/REVOKE statements even when the ROLE keyword was omitted, i.e: GRANT ON TO This is still the case for ROLE based authorization to preserve backward compatibility, but Ranger will throw an exception when a GRANT/REVOKE statement excludes the USER keyword. The syntax for the new statement is: GRANT ON TO USER REVOKE ON FROM USER Sentry does not support grant/revoke to/from user. Testing: - An additional end to end test, test_ranger.py, was added. A single test was added that grants and revokes for a user and asserts permissions on a table. The test uses sleep statements to work with Ranger's polling interval for policy changes. More end to end tests will be added in the future when the refresh authorization statement works properly with Ranger. - AuthorizationStmtTest has been refactored to use the new RangerCatalogdAuthorizationManager grant/revoke methods for better test coverage. - Ran all FE tests - Ran all E2E authorization tests Change-Id: I6ee97bf41546d63385026c0e2b19545565402462 --- M common/thrift/JniCatalog.thrift M fe/src/main/cup/sql-parser.cup M fe/src/main/java/org/apache/impala/analysis/GrantRevokePrivStmt.java M fe/src/main/java/org/apache/impala/authorization/AuthorizationManager.java M fe/src/main/java/org/apache/impala/authorization/NoneAuthorizationFactory.java M fe/src/main/java/org/apache/impala/authorization/ranger/RangerAuthorizationFactory.java A fe/src/main/java/org/apache/impala/authorization/ranger/RangerCatalogdAuthorizationManager.java M fe/src/main/java/org/apache/impala/authorization/sentry/SentryCatalogdAuthorizationManager.java M fe/src/main/java/org/apache/impala/authorization/sentry/SentryImpaladAuthorizationManager.java M fe/src/main/java/org/apache/impala/service/CatalogOpExecutor.java M fe/src/test/java/org/apache/impala/analysis/AuthorizationStmtTest.java M fe/src/test/java/org/apache/impala/analysis/ToSqlTest.java A tests/authorization/test_ranger.py 13 files changed, 598 insertions(+), 224 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/69/12769/7 -- To view, visit http://gerrit.cloudera.org:8080/12769 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: I6ee97bf41546d63385026c0e2b19545565402462 Gerrit-Change-Number: 12769 Gerrit-PatchSet: 7 Gerrit-Owner: Austin Nobis Gerrit-Reviewer: Austin Nobis Gerrit-Reviewer: Fredy Wijaya Gerrit-Reviewer: Impala Public Jenkins
