[Impala-ASF-CR] IMPALA-8363: Deny access when column masking or row filtering is enabled in Ranger
Impala Public Jenkins has submitted this change and it was merged. ( http://gerrit.cloudera.org:8080/12927 ) Change subject: IMPALA-8363: Deny access when column masking or row filtering is enabled in Ranger .. IMPALA-8363: Deny access when column masking or row filtering is enabled in Ranger This patch updates the Ranger authorization checker code to deny access when column masking and row filtering is enabled in Ranger for queries that that have columns/tables specified in column mask and row filter policies. This is to prevent data leak, such that the data that is masked/filtered in Hive should not be visible at all in Impala until Impala has full support for column masking and row filtering. Testing: - Added tests in AuthorizationStmtTest to test queries with column masking and row filtering enabled. - Ran all FE tests - Ran all E2E tests Change-Id: If46b4bf24d916e4a4ea8a36ff4acfd95d5f45c8e Reviewed-on: http://gerrit.cloudera.org:8080/12927 Reviewed-by: Fredy Wijaya Tested-by: Impala Public Jenkins --- M fe/src/main/java/org/apache/impala/analysis/AnalysisContext.java M fe/src/main/java/org/apache/impala/authorization/AuthorizationChecker.java M fe/src/main/java/org/apache/impala/authorization/AuthorizationFactory.java M fe/src/main/java/org/apache/impala/authorization/NoneAuthorizationFactory.java M fe/src/main/java/org/apache/impala/authorization/ranger/RangerAuthorizationChecker.java M fe/src/main/java/org/apache/impala/authorization/ranger/RangerAuthorizationFactory.java M fe/src/main/java/org/apache/impala/authorization/sentry/SentryAuthorizationChecker.java M fe/src/main/java/org/apache/impala/authorization/sentry/SentryAuthorizationFactory.java M fe/src/test/java/org/apache/impala/analysis/AuthorizationStmtTest.java M fe/src/test/java/org/apache/impala/catalog/events/MetastoreEventsProcessorTest.java M fe/src/test/java/org/apache/impala/common/FrontendTestBase.java 11 files changed, 407 insertions(+), 49 deletions(-) Approvals: Fredy Wijaya: Looks good to me, approved Impala Public Jenkins: Verified -- To view, visit http://gerrit.cloudera.org:8080/12927 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: merged Gerrit-Change-Id: If46b4bf24d916e4a4ea8a36ff4acfd95d5f45c8e Gerrit-Change-Number: 12927 Gerrit-PatchSet: 9 Gerrit-Owner: Fredy Wijaya Gerrit-Reviewer: Austin Nobis Gerrit-Reviewer: Bharath Vissapragada Gerrit-Reviewer: Fredy Wijaya Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Todd Lipcon
[Impala-ASF-CR] IMPALA-8363: Deny access when column masking or row filtering is enabled in Ranger
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/12927 ) Change subject: IMPALA-8363: Deny access when column masking or row filtering is enabled in Ranger .. Patch Set 8: Verified+1 -- To view, visit http://gerrit.cloudera.org:8080/12927 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: If46b4bf24d916e4a4ea8a36ff4acfd95d5f45c8e Gerrit-Change-Number: 12927 Gerrit-PatchSet: 8 Gerrit-Owner: Fredy Wijaya Gerrit-Reviewer: Austin Nobis Gerrit-Reviewer: Bharath Vissapragada Gerrit-Reviewer: Fredy Wijaya Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Todd Lipcon Gerrit-Comment-Date: Tue, 16 Apr 2019 09:40:18 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-8363: Deny access when column masking or row filtering is enabled in Ranger
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/12927 ) Change subject: IMPALA-8363: Deny access when column masking or row filtering is enabled in Ranger .. Patch Set 8: Build Successful https://jenkins.impala.io/job/gerrit-code-review-checks/2793/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests. -- To view, visit http://gerrit.cloudera.org:8080/12927 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: If46b4bf24d916e4a4ea8a36ff4acfd95d5f45c8e Gerrit-Change-Number: 12927 Gerrit-PatchSet: 8 Gerrit-Owner: Fredy Wijaya Gerrit-Reviewer: Austin Nobis Gerrit-Reviewer: Bharath Vissapragada Gerrit-Reviewer: Fredy Wijaya Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Todd Lipcon Gerrit-Comment-Date: Tue, 16 Apr 2019 05:00:59 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-8363: Deny access when column masking or row filtering is enabled in Ranger
Fredy Wijaya has uploaded a new patch set (#8). ( http://gerrit.cloudera.org:8080/12927 ) Change subject: IMPALA-8363: Deny access when column masking or row filtering is enabled in Ranger .. IMPALA-8363: Deny access when column masking or row filtering is enabled in Ranger This patch updates the Ranger authorization checker code to deny access when column masking and row filtering is enabled in Ranger for queries that that have columns/tables specified in column mask and row filter policies. This is to prevent data leak, such that the data that is masked/filtered in Hive should not be visible at all in Impala until Impala has full support for column masking and row filtering. Testing: - Added tests in AuthorizationStmtTest to test queries with column masking and row filtering enabled. - Ran all FE tests - Ran all E2E tests Change-Id: If46b4bf24d916e4a4ea8a36ff4acfd95d5f45c8e --- M fe/src/main/java/org/apache/impala/analysis/AnalysisContext.java M fe/src/main/java/org/apache/impala/authorization/AuthorizationChecker.java M fe/src/main/java/org/apache/impala/authorization/AuthorizationFactory.java M fe/src/main/java/org/apache/impala/authorization/NoneAuthorizationFactory.java M fe/src/main/java/org/apache/impala/authorization/ranger/RangerAuthorizationChecker.java M fe/src/main/java/org/apache/impala/authorization/ranger/RangerAuthorizationFactory.java M fe/src/main/java/org/apache/impala/authorization/sentry/SentryAuthorizationChecker.java M fe/src/main/java/org/apache/impala/authorization/sentry/SentryAuthorizationFactory.java M fe/src/test/java/org/apache/impala/analysis/AuthorizationStmtTest.java M fe/src/test/java/org/apache/impala/catalog/events/MetastoreEventsProcessorTest.java M fe/src/test/java/org/apache/impala/common/FrontendTestBase.java 11 files changed, 407 insertions(+), 49 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/27/12927/8 -- To view, visit http://gerrit.cloudera.org:8080/12927 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: If46b4bf24d916e4a4ea8a36ff4acfd95d5f45c8e Gerrit-Change-Number: 12927 Gerrit-PatchSet: 8 Gerrit-Owner: Fredy Wijaya Gerrit-Reviewer: Austin Nobis Gerrit-Reviewer: Bharath Vissapragada Gerrit-Reviewer: Fredy Wijaya Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Todd Lipcon
[Impala-ASF-CR] IMPALA-8363: Deny access when column masking or row filtering is enabled in Ranger
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/12927 ) Change subject: IMPALA-8363: Deny access when column masking or row filtering is enabled in Ranger .. Patch Set 8: Build started: https://jenkins.impala.io/job/gerrit-verify-dryrun/4025/ DRY_RUN=false -- To view, visit http://gerrit.cloudera.org:8080/12927 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: If46b4bf24d916e4a4ea8a36ff4acfd95d5f45c8e Gerrit-Change-Number: 12927 Gerrit-PatchSet: 8 Gerrit-Owner: Fredy Wijaya Gerrit-Reviewer: Austin Nobis Gerrit-Reviewer: Bharath Vissapragada Gerrit-Reviewer: Fredy Wijaya Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Todd Lipcon Gerrit-Comment-Date: Tue, 16 Apr 2019 04:16:50 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-8363: Deny access when column masking or row filtering is enabled in Ranger
Fredy Wijaya has posted comments on this change. ( http://gerrit.cloudera.org:8080/12927 ) Change subject: IMPALA-8363: Deny access when column masking or row filtering is enabled in Ranger .. Patch Set 8: Code-Review+2 Rebased. Carry +2. -- To view, visit http://gerrit.cloudera.org:8080/12927 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: If46b4bf24d916e4a4ea8a36ff4acfd95d5f45c8e Gerrit-Change-Number: 12927 Gerrit-PatchSet: 8 Gerrit-Owner: Fredy Wijaya Gerrit-Reviewer: Austin Nobis Gerrit-Reviewer: Bharath Vissapragada Gerrit-Reviewer: Fredy Wijaya Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Todd Lipcon Gerrit-Comment-Date: Tue, 16 Apr 2019 04:16:33 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-8363: Deny access when column masking or row filtering is enabled in Ranger
Bharath Vissapragada has posted comments on this change. ( http://gerrit.cloudera.org:8080/12927 ) Change subject: IMPALA-8363: Deny access when column masking or row filtering is enabled in Ranger .. Patch Set 7: Code-Review+2 -- To view, visit http://gerrit.cloudera.org:8080/12927 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: If46b4bf24d916e4a4ea8a36ff4acfd95d5f45c8e Gerrit-Change-Number: 12927 Gerrit-PatchSet: 7 Gerrit-Owner: Fredy Wijaya Gerrit-Reviewer: Austin Nobis Gerrit-Reviewer: Bharath Vissapragada Gerrit-Reviewer: Fredy Wijaya Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Todd Lipcon Gerrit-Comment-Date: Mon, 15 Apr 2019 22:05:33 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-8363: Deny access when column masking or row filtering is enabled in Ranger
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/12927 ) Change subject: IMPALA-8363: Deny access when column masking or row filtering is enabled in Ranger .. Patch Set 7: Build Successful https://jenkins.impala.io/job/gerrit-code-review-checks/2781/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests. -- To view, visit http://gerrit.cloudera.org:8080/12927 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: If46b4bf24d916e4a4ea8a36ff4acfd95d5f45c8e Gerrit-Change-Number: 12927 Gerrit-PatchSet: 7 Gerrit-Owner: Fredy Wijaya Gerrit-Reviewer: Austin Nobis Gerrit-Reviewer: Bharath Vissapragada Gerrit-Reviewer: Fredy Wijaya Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Todd Lipcon Gerrit-Comment-Date: Mon, 15 Apr 2019 18:49:31 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-8363: Deny access when column masking or row filtering is enabled in Ranger
Fredy Wijaya has posted comments on this change. ( http://gerrit.cloudera.org:8080/12927 ) Change subject: IMPALA-8363: Deny access when column masking or row filtering is enabled in Ranger .. Patch Set 7: (4 comments) http://gerrit.cloudera.org:8080/#/c/12927/6/fe/src/main/java/org/apache/impala/authorization/AuthorizationChecker.java File fe/src/main/java/org/apache/impala/authorization/AuthorizationChecker.java: http://gerrit.cloudera.org:8080/#/c/12927/6/fe/src/main/java/org/apache/impala/authorization/AuthorizationChecker.java@129 PS6, Line 129: > Make it abstract? Done http://gerrit.cloudera.org:8080/#/c/12927/6/fe/src/main/java/org/apache/impala/authorization/ranger/RangerAuthorizationChecker.java File fe/src/main/java/org/apache/impala/authorization/ranger/RangerAuthorizationChecker.java: http://gerrit.cloudera.org:8080/#/c/12927/6/fe/src/main/java/org/apache/impala/authorization/ranger/RangerAuthorizationChecker.java@53 PS6, Line 53: // These are Ranger access types (privileges). : public static final String UPDATE_ACCESS_TYPE = "update"; : public static final String REFRESH_ACCESS_TYPE = "read"; > Document these? Done http://gerrit.cloudera.org:8080/#/c/12927/6/fe/src/main/java/org/apache/impala/authorization/ranger/RangerAuthorizationChecker.java@165 PS6, Line 165: getAuthorizable().getT > Move this throw inside authorizeColumnMask() (same below)? (Unless you are Done http://gerrit.cloudera.org:8080/#/c/12927/6/fe/src/main/java/org/apache/impala/authorization/ranger/RangerAuthorizationChecker.java@166 PS6, Line 166: request.getAuthorizable().getColumnName()); > Mention that Impala doesn't support column masking / row filtering yet? (sa Done -- To view, visit http://gerrit.cloudera.org:8080/12927 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: If46b4bf24d916e4a4ea8a36ff4acfd95d5f45c8e Gerrit-Change-Number: 12927 Gerrit-PatchSet: 7 Gerrit-Owner: Fredy Wijaya Gerrit-Reviewer: Austin Nobis Gerrit-Reviewer: Bharath Vissapragada Gerrit-Reviewer: Fredy Wijaya Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Todd Lipcon Gerrit-Comment-Date: Mon, 15 Apr 2019 18:13:34 + Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-8363: Deny access when column masking or row filtering is enabled in Ranger
Fredy Wijaya has uploaded a new patch set (#7). ( http://gerrit.cloudera.org:8080/12927 ) Change subject: IMPALA-8363: Deny access when column masking or row filtering is enabled in Ranger .. IMPALA-8363: Deny access when column masking or row filtering is enabled in Ranger This patch updates the Ranger authorization checker code to deny access when column masking and row filtering is enabled in Ranger for queries that that have columns/tables specified in column mask and row filter policies. This is to prevent data leak, such that the data that is masked/filtered in Hive should not be visible at all in Impala until Impala has full support for column masking and row filtering. Testing: - Added tests in AuthorizationStmtTest to test queries with column masking and row filtering enabled. - Ran all FE tests - Ran all E2E tests Change-Id: If46b4bf24d916e4a4ea8a36ff4acfd95d5f45c8e --- M fe/src/main/java/org/apache/impala/analysis/AnalysisContext.java M fe/src/main/java/org/apache/impala/authorization/AuthorizationChecker.java M fe/src/main/java/org/apache/impala/authorization/AuthorizationFactory.java M fe/src/main/java/org/apache/impala/authorization/NoneAuthorizationFactory.java M fe/src/main/java/org/apache/impala/authorization/ranger/RangerAuthorizationChecker.java M fe/src/main/java/org/apache/impala/authorization/ranger/RangerAuthorizationFactory.java M fe/src/main/java/org/apache/impala/authorization/sentry/SentryAuthorizationChecker.java M fe/src/main/java/org/apache/impala/authorization/sentry/SentryAuthorizationFactory.java M fe/src/test/java/org/apache/impala/analysis/AuthorizationStmtTest.java M fe/src/test/java/org/apache/impala/catalog/events/MetastoreEventsProcessorTest.java M fe/src/test/java/org/apache/impala/common/FrontendTestBase.java 11 files changed, 407 insertions(+), 49 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/27/12927/7 -- To view, visit http://gerrit.cloudera.org:8080/12927 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: If46b4bf24d916e4a4ea8a36ff4acfd95d5f45c8e Gerrit-Change-Number: 12927 Gerrit-PatchSet: 7 Gerrit-Owner: Fredy Wijaya Gerrit-Reviewer: Austin Nobis Gerrit-Reviewer: Bharath Vissapragada Gerrit-Reviewer: Fredy Wijaya Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Todd Lipcon
[Impala-ASF-CR] IMPALA-8363: Deny access when column masking or row filtering is enabled in Ranger
Bharath Vissapragada has posted comments on this change. (
http://gerrit.cloudera.org:8080/12927 )
Change subject: IMPALA-8363: Deny access when column masking or row filtering
is enabled in Ranger
..
Patch Set 6:
(4 comments)
nice, looks much simpler on top of the view column authz patch. lgtm modulo
nits!
http://gerrit.cloudera.org:8080/#/c/12927/6/fe/src/main/java/org/apache/impala/authorization/AuthorizationChecker.java
File fe/src/main/java/org/apache/impala/authorization/AuthorizationChecker.java:
http://gerrit.cloudera.org:8080/#/c/12927/6/fe/src/main/java/org/apache/impala/authorization/AuthorizationChecker.java@129
PS6, Line 129: {}
Make it abstract?
http://gerrit.cloudera.org:8080/#/c/12927/6/fe/src/main/java/org/apache/impala/authorization/ranger/RangerAuthorizationChecker.java
File
fe/src/main/java/org/apache/impala/authorization/ranger/RangerAuthorizationChecker.java:
http://gerrit.cloudera.org:8080/#/c/12927/6/fe/src/main/java/org/apache/impala/authorization/ranger/RangerAuthorizationChecker.java@53
PS6, Line 53: public static final String UPDATE_ACCESS_TYPE = "update";
: public static final String REFRESH_ACCESS_TYPE = "read";
: public static final String SELECT_ACCESS_TYPE = "select";
Document these?
http://gerrit.cloudera.org:8080/#/c/12927/6/fe/src/main/java/org/apache/impala/authorization/ranger/RangerAuthorizationChecker.java@165
PS6, Line 165: AuthorizationException
Move this throw inside authorizeColumnMask() (same below)? (Unless you are
using the helper elsewhere?
http://gerrit.cloudera.org:8080/#/c/12927/6/fe/src/main/java/org/apache/impala/authorization/ranger/RangerAuthorizationChecker.java@166
PS6, Line 166: "Column masking is enabled on column: %s.%s.%s",
Mention that Impala doesn't support column masking / row filtering yet? (same
below too).
--
To view, visit http://gerrit.cloudera.org:8080/12927
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: If46b4bf24d916e4a4ea8a36ff4acfd95d5f45c8e
Gerrit-Change-Number: 12927
Gerrit-PatchSet: 6
Gerrit-Owner: Fredy Wijaya
Gerrit-Reviewer: Austin Nobis
Gerrit-Reviewer: Bharath Vissapragada
Gerrit-Reviewer: Fredy Wijaya
Gerrit-Reviewer: Impala Public Jenkins
Gerrit-Reviewer: Todd Lipcon
Gerrit-Comment-Date: Mon, 15 Apr 2019 05:06:22 +
Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-8363: Deny access when column masking or row filtering is enabled in Ranger
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/12927 ) Change subject: IMPALA-8363: Deny access when column masking or row filtering is enabled in Ranger .. Patch Set 6: Build Successful https://jenkins.impala.io/job/gerrit-code-review-checks/2702/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests. -- To view, visit http://gerrit.cloudera.org:8080/12927 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: If46b4bf24d916e4a4ea8a36ff4acfd95d5f45c8e Gerrit-Change-Number: 12927 Gerrit-PatchSet: 6 Gerrit-Owner: Fredy Wijaya Gerrit-Reviewer: Austin Nobis Gerrit-Reviewer: Bharath Vissapragada Gerrit-Reviewer: Fredy Wijaya Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Todd Lipcon Gerrit-Comment-Date: Tue, 09 Apr 2019 16:58:24 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-8363: Deny access when column masking or row filtering is enabled in Ranger
Fredy Wijaya has posted comments on this change. (
http://gerrit.cloudera.org:8080/12927 )
Change subject: IMPALA-8363: Deny access when column masking or row filtering
is enabled in Ranger
..
Patch Set 6:
(7 comments)
The new implementation based on the work in this CR:
https://gerrit.cloudera.org/c/12959/
http://gerrit.cloudera.org:8080/#/c/12927/4/fe/src/main/java/org/apache/impala/authorization/AuthorizationChecker.java
File fe/src/main/java/org/apache/impala/authorization/AuthorizationChecker.java:
http://gerrit.cloudera.org:8080/#/c/12927/4/fe/src/main/java/org/apache/impala/authorization/AuthorizationChecker.java@106
PS4, Line 106:
> consider adding @Nullable for this parameter
Done. StatementBase is no longer needed.
http://gerrit.cloudera.org:8080/#/c/12927/4/fe/src/main/java/org/apache/impala/authorization/AuthorizationChecker.java@120
PS4, Line 120:
> can you add to the javadoc indicating what 'statement' is used for
Same as above.
http://gerrit.cloudera.org:8080/#/c/12927/4/fe/src/main/java/org/apache/impala/authorization/AuthorizationChecker.java@123
PS4, Line 123: y do, thr
> and this one
Sane as above.
http://gerrit.cloudera.org:8080/#/c/12927/4/fe/src/main/java/org/apache/impala/authorization/NoopAuthorizationFactory.java
File
fe/src/main/java/org/apache/impala/authorization/NoopAuthorizationFactory.java:
http://gerrit.cloudera.org:8080/#/c/12927/4/fe/src/main/java/org/apache/impala/authorization/NoopAuthorizationFactory.java@47
PS4, Line 47:
> if it's not too much of a pain, is it possible to separate out these unrela
Done
http://gerrit.cloudera.org:8080/#/c/12927/4/fe/src/main/java/org/apache/impala/authorization/ranger/RangerAuthorizationChecker.java
File
fe/src/main/java/org/apache/impala/authorization/ranger/RangerAuthorizationChecker.java:
http://gerrit.cloudera.org:8080/#/c/12927/4/fe/src/main/java/org/apache/impala/authorization/ranger/RangerAuthorizationChecker.java@160
PS4, Line 160: if (request.getAuthorizable().getType() == Type.COLUMN) {
> I think part of the problem here is column-level permissions on views are b
I changed the implementation based on https://gerrit.cloudera.org/c/12959/.
Please take a look at it again.
http://gerrit.cloudera.org:8080/#/c/12927/4/fe/src/main/java/org/apache/impala/authorization/ranger/RangerAuthorizationChecker.java@206
PS4, Line 206:* enabled but not in Impala.
> Curious whether it would be feasible to give a different error message indi
Done
http://gerrit.cloudera.org:8080/#/c/12927/4/fe/src/main/java/org/apache/impala/authorization/ranger/RangerAuthorizationChecker.java@250
PS4, Line 250: accessType = RangerPolicyEngine.ANY_ACCESS;
> I'm worried that this pattern will be very hard to miss when adding a new t
No longer needed.
--
To view, visit http://gerrit.cloudera.org:8080/12927
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: If46b4bf24d916e4a4ea8a36ff4acfd95d5f45c8e
Gerrit-Change-Number: 12927
Gerrit-PatchSet: 6
Gerrit-Owner: Fredy Wijaya
Gerrit-Reviewer: Austin Nobis
Gerrit-Reviewer: Bharath Vissapragada
Gerrit-Reviewer: Fredy Wijaya
Gerrit-Reviewer: Impala Public Jenkins
Gerrit-Reviewer: Todd Lipcon
Gerrit-Comment-Date: Tue, 09 Apr 2019 16:33:13 +
Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-8363: Deny access when column masking or row filtering is enabled in Ranger
Fredy Wijaya has uploaded a new patch set (#6). ( http://gerrit.cloudera.org:8080/12927 ) Change subject: IMPALA-8363: Deny access when column masking or row filtering is enabled in Ranger .. IMPALA-8363: Deny access when column masking or row filtering is enabled in Ranger This patch updates the Ranger authorization checker code to deny access when column masking and row filtering is enabled in Ranger for queries that that have columns/tables specified in column mask and row filter policies. This is to prevent data leak, such that the data that is masked/filtered in Hive should not be visible at all in Impala until Impala has full support for column masking and row filtering. Testing: - Added tests in AuthorizationStmtTest to test queries with column masking and row filtering enabled. - Ran all FE tests - Ran all E2E tests Change-Id: If46b4bf24d916e4a4ea8a36ff4acfd95d5f45c8e --- M fe/src/main/java/org/apache/impala/analysis/AnalysisContext.java M fe/src/main/java/org/apache/impala/authorization/AuthorizationChecker.java M fe/src/main/java/org/apache/impala/authorization/AuthorizationFactory.java M fe/src/main/java/org/apache/impala/authorization/NoneAuthorizationFactory.java M fe/src/main/java/org/apache/impala/authorization/ranger/RangerAuthorizationChecker.java M fe/src/main/java/org/apache/impala/authorization/ranger/RangerAuthorizationFactory.java M fe/src/main/java/org/apache/impala/authorization/sentry/SentryAuthorizationChecker.java M fe/src/main/java/org/apache/impala/authorization/sentry/SentryAuthorizationFactory.java M fe/src/test/java/org/apache/impala/analysis/AuthorizationStmtTest.java M fe/src/test/java/org/apache/impala/catalog/events/MetastoreEventsProcessorTest.java M fe/src/test/java/org/apache/impala/common/FrontendTestBase.java 11 files changed, 379 insertions(+), 49 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/27/12927/6 -- To view, visit http://gerrit.cloudera.org:8080/12927 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: If46b4bf24d916e4a4ea8a36ff4acfd95d5f45c8e Gerrit-Change-Number: 12927 Gerrit-PatchSet: 6 Gerrit-Owner: Fredy Wijaya Gerrit-Reviewer: Austin Nobis Gerrit-Reviewer: Bharath Vissapragada Gerrit-Reviewer: Fredy Wijaya Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Todd Lipcon
[Impala-ASF-CR] IMPALA-8363: Deny access when column masking or row filtering is enabled in Ranger
Fredy Wijaya has posted comments on this change. ( http://gerrit.cloudera.org:8080/12927 ) Change subject: IMPALA-8363: Deny access when column masking or row filtering is enabled in Ranger .. Patch Set 4: (1 comment) http://gerrit.cloudera.org:8080/#/c/12927/4/fe/src/main/java/org/apache/impala/authorization/ranger/RangerAuthorizationChecker.java File fe/src/main/java/org/apache/impala/authorization/ranger/RangerAuthorizationChecker.java: http://gerrit.cloudera.org:8080/#/c/12927/4/fe/src/main/java/org/apache/impala/authorization/ranger/RangerAuthorizationChecker.java@160 PS4, Line 160: case COLUMN: > Ranger column masking in views apply on either the view or base table permi I think part of the problem here is column-level permissions on views are broken in Impala. So I'm fixing this one first: https://issues.apache.org/jira/browse/IMPALA-6718 and will come back to this and hopefully it will make things much simpler. -- To view, visit http://gerrit.cloudera.org:8080/12927 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: If46b4bf24d916e4a4ea8a36ff4acfd95d5f45c8e Gerrit-Change-Number: 12927 Gerrit-PatchSet: 4 Gerrit-Owner: Fredy Wijaya Gerrit-Reviewer: Austin Nobis Gerrit-Reviewer: Bharath Vissapragada Gerrit-Reviewer: Fredy Wijaya Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Todd Lipcon Gerrit-Comment-Date: Mon, 08 Apr 2019 20:29:25 + Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-8363: Deny access when column masking or row filtering is enabled in Ranger
Fredy Wijaya has posted comments on this change. ( http://gerrit.cloudera.org:8080/12927 ) Change subject: IMPALA-8363: Deny access when column masking or row filtering is enabled in Ranger .. Patch Set 4: (1 comment) http://gerrit.cloudera.org:8080/#/c/12927/4/fe/src/main/java/org/apache/impala/authorization/ranger/RangerAuthorizationChecker.java File fe/src/main/java/org/apache/impala/authorization/ranger/RangerAuthorizationChecker.java: http://gerrit.cloudera.org:8080/#/c/12927/4/fe/src/main/java/org/apache/impala/authorization/ranger/RangerAuthorizationChecker.java@160 PS4, Line 160: case COLUMN: > >Yea I didn't think about masking in views -- does that apply based on the Ranger column masking in views apply on either the view or base table permissions. Impala doesn't support column-level permissions on view: https://github.com/apache/impala/blob/80b82b5b2c60ff934eabe42dd145c9f84ca3a60b/fe/src/main/java/org/apache/impala/analysis/PrivilegeSpec.java#L248-L251. This means the privilege requests generated for views won't contain any view column privilege requests. The implementation in this CR is actually incorrect for views. This is what I'm planning to do, let me know what you think about it. 1. We will be very strict (this is just a band-aid anyway) that means we will reject anything that has "select" on column/table when row filtering/column masking is enabled regardless whether it's select SQL statement or non-select SQL statement. 2. For tables, we will use the privilege requests since the privilege requests will contain all base table columns. 3. For views, we will use the privilege requests as well as getting the view column names from the catalog to create the missing view column privilege requests. This is because Ranger supports column masking on the view columns directly or base columns. -- To view, visit http://gerrit.cloudera.org:8080/12927 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: If46b4bf24d916e4a4ea8a36ff4acfd95d5f45c8e Gerrit-Change-Number: 12927 Gerrit-PatchSet: 4 Gerrit-Owner: Fredy Wijaya Gerrit-Reviewer: Austin Nobis Gerrit-Reviewer: Bharath Vissapragada Gerrit-Reviewer: Fredy Wijaya Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Todd Lipcon Gerrit-Comment-Date: Fri, 05 Apr 2019 21:58:22 + Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-8363: Deny access when column masking or row filtering is enabled in Ranger
Bharath Vissapragada has posted comments on this change. ( http://gerrit.cloudera.org:8080/12927 ) Change subject: IMPALA-8363: Deny access when column masking or row filtering is enabled in Ranger .. Patch Set 4: (1 comment) http://gerrit.cloudera.org:8080/#/c/12927/4/fe/src/main/java/org/apache/impala/authorization/ranger/RangerAuthorizationChecker.java File fe/src/main/java/org/apache/impala/authorization/ranger/RangerAuthorizationChecker.java: http://gerrit.cloudera.org:8080/#/c/12927/4/fe/src/main/java/org/apache/impala/authorization/ranger/RangerAuthorizationChecker.java@160 PS4, Line 160: case COLUMN: > Yea I didn't think about masking in views -- does that apply based on the b >Yea I didn't think about masking in views -- does that apply based on the base >table masking permissions instead of the view permissions? I have the same question. I'm not sure how hive works in this regard. Fredy probably knows. -- To view, visit http://gerrit.cloudera.org:8080/12927 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: If46b4bf24d916e4a4ea8a36ff4acfd95d5f45c8e Gerrit-Change-Number: 12927 Gerrit-PatchSet: 4 Gerrit-Owner: Fredy Wijaya Gerrit-Reviewer: Austin Nobis Gerrit-Reviewer: Bharath Vissapragada Gerrit-Reviewer: Fredy Wijaya Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Todd Lipcon Gerrit-Comment-Date: Fri, 05 Apr 2019 21:07:08 + Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-8363: Deny access when column masking or row filtering is enabled in Ranger
Todd Lipcon has posted comments on this change. ( http://gerrit.cloudera.org:8080/12927 ) Change subject: IMPALA-8363: Deny access when column masking or row filtering is enabled in Ranger .. Patch Set 4: (1 comment) http://gerrit.cloudera.org:8080/#/c/12927/4/fe/src/main/java/org/apache/impala/authorization/ranger/RangerAuthorizationChecker.java File fe/src/main/java/org/apache/impala/authorization/ranger/RangerAuthorizationChecker.java: http://gerrit.cloudera.org:8080/#/c/12927/4/fe/src/main/java/org/apache/impala/authorization/ranger/RangerAuthorizationChecker.java@160 PS4, Line 160: case COLUMN: > I had somewhat similar concerns and I was chatting with Fredy offline on ho Yea I didn't think about masking in views -- does that apply based on the base table masking permissions instead of the view permissions? Either way I think we could probably extend the PrivilegeRequest structure to have some more info, like whether the requested column/table is directly accessed or via a resolved view, and be more accurate about the requested permission being 'view_metadata' vs 'select' so that we can treat them differently here. -- To view, visit http://gerrit.cloudera.org:8080/12927 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: If46b4bf24d916e4a4ea8a36ff4acfd95d5f45c8e Gerrit-Change-Number: 12927 Gerrit-PatchSet: 4 Gerrit-Owner: Fredy Wijaya Gerrit-Reviewer: Austin Nobis Gerrit-Reviewer: Bharath Vissapragada Gerrit-Reviewer: Fredy Wijaya Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Todd Lipcon Gerrit-Comment-Date: Fri, 05 Apr 2019 20:50:40 + Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-8363: Deny access when column masking or row filtering is enabled in Ranger
Bharath Vissapragada has posted comments on this change. ( http://gerrit.cloudera.org:8080/12927 ) Change subject: IMPALA-8363: Deny access when column masking or row filtering is enabled in Ranger .. Patch Set 4: (1 comment) http://gerrit.cloudera.org:8080/#/c/12927/4/fe/src/main/java/org/apache/impala/authorization/ranger/RangerAuthorizationChecker.java File fe/src/main/java/org/apache/impala/authorization/ranger/RangerAuthorizationChecker.java: http://gerrit.cloudera.org:8080/#/c/12927/4/fe/src/main/java/org/apache/impala/authorization/ranger/RangerAuthorizationChecker.java@160 PS4, Line 160: case COLUMN: > I'm a bit confused by the implementation here... I had somewhat similar concerns and I was chatting with Fredy offline on how we could reuse the privilege reqs populated in the original Analyzer pass. It looks like there are a couple of issues based on our discussion: - We use "SELECT" privilege pretty loosely. For example, something like describe table foo requires a "SELECT" privilege on column foo.a. In such a case if there is masking on foo.a, should describe fail (too restrictive) or should we intelligently catch and ignore it, since masking shouldn't affect a describe output. - There are some subtleties around how we populate column privilege requests for base table columns in views (the whole logic around maskedPrivilegeReqs). We should probably consider those too. Overall, I prefer if we could avoid duplicating the traversal path again here. -- To view, visit http://gerrit.cloudera.org:8080/12927 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: If46b4bf24d916e4a4ea8a36ff4acfd95d5f45c8e Gerrit-Change-Number: 12927 Gerrit-PatchSet: 4 Gerrit-Owner: Fredy Wijaya Gerrit-Reviewer: Austin Nobis Gerrit-Reviewer: Bharath Vissapragada Gerrit-Reviewer: Fredy Wijaya Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Todd Lipcon Gerrit-Comment-Date: Fri, 05 Apr 2019 20:39:43 + Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-8363: Deny access when column masking or row filtering is enabled in Ranger
Todd Lipcon has posted comments on this change. ( http://gerrit.cloudera.org:8080/12927 ) Change subject: IMPALA-8363: Deny access when column masking or row filtering is enabled in Ranger .. Patch Set 4: (7 comments) http://gerrit.cloudera.org:8080/#/c/12927/4/fe/src/main/java/org/apache/impala/authorization/AuthorizationChecker.java File fe/src/main/java/org/apache/impala/authorization/AuthorizationChecker.java: http://gerrit.cloudera.org:8080/#/c/12927/4/fe/src/main/java/org/apache/impala/authorization/AuthorizationChecker.java@106 PS4, Line 106: StatementBase consider adding @Nullable for this parameter http://gerrit.cloudera.org:8080/#/c/12927/4/fe/src/main/java/org/apache/impala/authorization/AuthorizationChecker.java@120 PS4, Line 120:* Performs an authorization for a given user. can you add to the javadoc indicating what 'statement' is used for http://gerrit.cloudera.org:8080/#/c/12927/4/fe/src/main/java/org/apache/impala/authorization/AuthorizationChecker.java@123 PS4, Line 123: statement and this one http://gerrit.cloudera.org:8080/#/c/12927/4/fe/src/main/java/org/apache/impala/authorization/NoopAuthorizationFactory.java File fe/src/main/java/org/apache/impala/authorization/NoopAuthorizationFactory.java: http://gerrit.cloudera.org:8080/#/c/12927/4/fe/src/main/java/org/apache/impala/authorization/NoopAuthorizationFactory.java@47 PS4, Line 47: NoopAuthorizationFactory if it's not too much of a pain, is it possible to separate out these unrelated changes into a different commit? makes it hard to review when there is refactoring/renaming going on at the same time as a functional change http://gerrit.cloudera.org:8080/#/c/12927/4/fe/src/main/java/org/apache/impala/authorization/ranger/RangerAuthorizationChecker.java File fe/src/main/java/org/apache/impala/authorization/ranger/RangerAuthorizationChecker.java: http://gerrit.cloudera.org:8080/#/c/12927/4/fe/src/main/java/org/apache/impala/authorization/ranger/RangerAuthorizationChecker.java@160 PS4, Line 160: case COLUMN: I'm a bit confused by the implementation here... It seems like in AnalysisContext.authorize() we're already doing the work to collect the tables and columns that need access, and calling down to here once per table/column. Now, for each of those calls, you're duplicating the work of collecting the accessed tables/columns and checking them. In some cases this will end up doing a bunch of repeated work, and it also duplicates a bunch of code. The one barrier to reusing the existing code in AnalysisContext looks like it sees SELECT access on the table, and at that point short circuits the column-level checks by setting 'hasTableSelectPriv = true'. In the case that one column is masked, I guess we could either have this return 'false' for table-level select access and then 'true' for all of the non-masked columns, or we could make the code return an enum like 'HAS_MASKED_COLUMNS' for the table-level check, so that it then falls through to checking each referenced column? http://gerrit.cloudera.org:8080/#/c/12927/4/fe/src/main/java/org/apache/impala/authorization/ranger/RangerAuthorizationChecker.java@206 PS4, Line 206: return false; Curious whether it would be feasible to give a different error message indicating that the access is denied due to lack of filtering/masking support. That way, a user hitting this "access denied" can realize that they could use Hive to do their work if they need access to this table. Maybe add a TODO and come back to it since it probably requires a lot of plumbing so that this method would return an enum instead of a boolean. http://gerrit.cloudera.org:8080/#/c/12927/4/fe/src/main/java/org/apache/impala/authorization/ranger/RangerAuthorizationChecker.java@250 PS4, Line 250: } I'm worried that this pattern will be very hard to miss when adding a new type of statement, and we'll end up not doing appropriate authorization Perhaps we can add a 'visitExprs' method to StatementBase or something and ensure that all statement subclasses implement it appropriately? (this may be unnecessary if we rely on the existing analysis code to collect referenced columns) -- To view, visit http://gerrit.cloudera.org:8080/12927 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: If46b4bf24d916e4a4ea8a36ff4acfd95d5f45c8e Gerrit-Change-Number: 12927 Gerrit-PatchSet: 4 Gerrit-Owner: Fredy Wijaya Gerrit-Reviewer: Austin Nobis Gerrit-Reviewer: Bharath Vissapragada Gerrit-Reviewer: Fredy Wijaya Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Todd Lipcon Gerrit-Comment-Date: Fri, 05 Apr 2019 17:08:24 + Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-8363: Deny access when column masking or row filtering is enabled in Ranger
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/12927 ) Change subject: IMPALA-8363: Deny access when column masking or row filtering is enabled in Ranger .. Patch Set 4: Build Successful https://jenkins.impala.io/job/gerrit-code-review-checks/2655/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests. -- To view, visit http://gerrit.cloudera.org:8080/12927 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: If46b4bf24d916e4a4ea8a36ff4acfd95d5f45c8e Gerrit-Change-Number: 12927 Gerrit-PatchSet: 4 Gerrit-Owner: Fredy Wijaya Gerrit-Reviewer: Austin Nobis Gerrit-Reviewer: Bharath Vissapragada Gerrit-Reviewer: Fredy Wijaya Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Todd Lipcon Gerrit-Comment-Date: Thu, 04 Apr 2019 21:33:08 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-8363: Deny access when column masking or row filtering is enabled in Ranger
Fredy Wijaya has posted comments on this change. ( http://gerrit.cloudera.org:8080/12927 ) Change subject: IMPALA-8363: Deny access when column masking or row filtering is enabled in Ranger .. Patch Set 4: Bharath, this is ready for review now. -- To view, visit http://gerrit.cloudera.org:8080/12927 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: If46b4bf24d916e4a4ea8a36ff4acfd95d5f45c8e Gerrit-Change-Number: 12927 Gerrit-PatchSet: 4 Gerrit-Owner: Fredy Wijaya Gerrit-Reviewer: Austin Nobis Gerrit-Reviewer: Bharath Vissapragada Gerrit-Reviewer: Fredy Wijaya Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Todd Lipcon Gerrit-Comment-Date: Thu, 04 Apr 2019 20:59:52 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-8363: Deny access when column masking or row filtering is enabled in Ranger
Fredy Wijaya has uploaded a new patch set (#4). ( http://gerrit.cloudera.org:8080/12927 ) Change subject: IMPALA-8363: Deny access when column masking or row filtering is enabled in Ranger .. IMPALA-8363: Deny access when column masking or row filtering is enabled in Ranger This patch updates the Ranger authorization checker code to deny access when column masking and row filtering is enabled in Ranger for queries that that have columns/tables specified in column mask and row filter policies. This is to prevent data leak, such that the data that is masked/filtered in Hive should not be visible at all in Impala until Impala has full support for column masking and row filtering. Testing: - Added tests in AuthorizationStmtTest to test queries with column masking and row filtering enabled. - Ran all FE tests - Ran all E2E tests Change-Id: If46b4bf24d916e4a4ea8a36ff4acfd95d5f45c8e --- M fe/src/main/java/org/apache/impala/analysis/AnalysisContext.java M fe/src/main/java/org/apache/impala/analysis/CreateOrAlterViewStmtBase.java M fe/src/main/java/org/apache/impala/analysis/ModifyStmt.java M fe/src/main/java/org/apache/impala/authorization/AuthorizationChecker.java M fe/src/main/java/org/apache/impala/authorization/AuthorizationFactory.java M fe/src/main/java/org/apache/impala/authorization/AuthorizationProvider.java R fe/src/main/java/org/apache/impala/authorization/NoopAuthorizationFactory.java M fe/src/main/java/org/apache/impala/authorization/ranger/RangerAuthorizationChecker.java M fe/src/main/java/org/apache/impala/authorization/ranger/RangerAuthorizationFactory.java M fe/src/main/java/org/apache/impala/authorization/sentry/SentryAuthorizationChecker.java M fe/src/main/java/org/apache/impala/authorization/sentry/SentryAuthorizationFactory.java M fe/src/main/java/org/apache/impala/service/JniCatalog.java M fe/src/main/java/org/apache/impala/service/JniFrontend.java M fe/src/test/java/org/apache/impala/analysis/AnalyzeAuthStmtsTest.java M fe/src/test/java/org/apache/impala/analysis/AuthorizationStmtTest.java M fe/src/test/java/org/apache/impala/analysis/StmtMetadataLoaderTest.java M fe/src/test/java/org/apache/impala/catalog/AlterDatabaseTest.java M fe/src/test/java/org/apache/impala/catalog/events/MetastoreEventsProcessorTest.java M fe/src/test/java/org/apache/impala/common/FrontendFixture.java M fe/src/test/java/org/apache/impala/common/FrontendTestBase.java M fe/src/test/java/org/apache/impala/common/QueryFixture.java M fe/src/test/java/org/apache/impala/testutil/ImpaladTestCatalog.java M fe/src/test/java/org/apache/impala/testutil/PlannerTestCaseLoader.java 23 files changed, 486 insertions(+), 81 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/27/12927/4 -- To view, visit http://gerrit.cloudera.org:8080/12927 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: If46b4bf24d916e4a4ea8a36ff4acfd95d5f45c8e Gerrit-Change-Number: 12927 Gerrit-PatchSet: 4 Gerrit-Owner: Fredy Wijaya Gerrit-Reviewer: Austin Nobis Gerrit-Reviewer: Bharath Vissapragada Gerrit-Reviewer: Fredy Wijaya Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Todd Lipcon
[Impala-ASF-CR] IMPALA-8363: Deny access when column masking or row filtering is enabled in Ranger
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/12927 ) Change subject: IMPALA-8363: Deny access when column masking or row filtering is enabled in Ranger .. Patch Set 3: Build Successful https://jenkins.impala.io/job/gerrit-code-review-checks/2653/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests. -- To view, visit http://gerrit.cloudera.org:8080/12927 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: If46b4bf24d916e4a4ea8a36ff4acfd95d5f45c8e Gerrit-Change-Number: 12927 Gerrit-PatchSet: 3 Gerrit-Owner: Fredy Wijaya Gerrit-Reviewer: Austin Nobis Gerrit-Reviewer: Bharath Vissapragada Gerrit-Reviewer: Fredy Wijaya Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Todd Lipcon Gerrit-Comment-Date: Thu, 04 Apr 2019 20:40:00 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-8363: Deny access when column masking or row filtering is enabled in Ranger
Fredy Wijaya has posted comments on this change. (
http://gerrit.cloudera.org:8080/12927 )
Change subject: IMPALA-8363: Deny access when column masking or row filtering
is enabled in Ranger
..
Patch Set 3:
(4 comments)
http://gerrit.cloudera.org:8080/#/c/12927/2/fe/src/main/java/org/apache/impala/authorization/ranger/RangerAuthorizationChecker.java
File
fe/src/main/java/org/apache/impala/authorization/ranger/RangerAuthorizationChecker.java:
http://gerrit.cloudera.org:8080/#/c/12927/2/fe/src/main/java/org/apache/impala/authorization/ranger/RangerAuthorizationChecker.java@70
PS2, Line 70:
> Wouldn't this represent a column and not a table?
Done
http://gerrit.cloudera.org:8080/#/c/12927/2/fe/src/main/java/org/apache/impala/authorization/ranger/RangerAuthorizationChecker.java@246
PS2, Line 246: List exprs = Collections.singletonList(((DeleteStmt)
stmt).getWhereClause());
> So you're iterating over the `Table` which is actually more like a `Column`
For row filter, we don't need to care for columns. My next PS handles the
duplicate entries.
http://gerrit.cloudera.org:8080/#/c/12927/2/fe/src/main/java/org/apache/impala/authorization/ranger/RangerAuthorizationChecker.java@269
PS2, Line 269: for (RangerAccessResourceImpl resource: colMaskTables) {
> Is this getting the tables or the columns?
Updated in the next PS.
http://gerrit.cloudera.org:8080/#/c/12927/2/fe/src/main/java/org/apache/impala/authorization/ranger/RangerAuthorizationChecker.java@279
PS2, Line 279: /**
> If it is a complex type do you need to parse the complex type for more pote
I think Ranger doesn't support column masking with complex types.
--
To view, visit http://gerrit.cloudera.org:8080/12927
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: If46b4bf24d916e4a4ea8a36ff4acfd95d5f45c8e
Gerrit-Change-Number: 12927
Gerrit-PatchSet: 3
Gerrit-Owner: Fredy Wijaya
Gerrit-Reviewer: Austin Nobis
Gerrit-Reviewer: Bharath Vissapragada
Gerrit-Reviewer: Fredy Wijaya
Gerrit-Reviewer: Impala Public Jenkins
Gerrit-Reviewer: Todd Lipcon
Gerrit-Comment-Date: Thu, 04 Apr 2019 19:59:02 +
Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-8363: Deny access when column masking or row filtering is enabled in Ranger
Fredy Wijaya has uploaded a new patch set (#3). ( http://gerrit.cloudera.org:8080/12927 ) Change subject: IMPALA-8363: Deny access when column masking or row filtering is enabled in Ranger .. IMPALA-8363: Deny access when column masking or row filtering is enabled in Ranger This patch updates the Ranger authorization checker code to deny access when column masking and row filtering is enabled in Ranger for queries that that have columns/tables specified in column mask and row filter policies. This is to prevent data leak, such that the data that is masked/filtered in Hive should not be visible at all in Impala until Impala has full support for column masking and row filtering. Testing: - Added tests in AuthorizationStmtTest to test queries with column masking and row filtering enabled. - Ran all FE tests - Ran all E2E tests Change-Id: If46b4bf24d916e4a4ea8a36ff4acfd95d5f45c8e --- M fe/src/main/java/org/apache/impala/analysis/AnalysisContext.java M fe/src/main/java/org/apache/impala/analysis/CreateOrAlterViewStmtBase.java M fe/src/main/java/org/apache/impala/analysis/ModifyStmt.java M fe/src/main/java/org/apache/impala/authorization/AuthorizationChecker.java M fe/src/main/java/org/apache/impala/authorization/AuthorizationFactory.java M fe/src/main/java/org/apache/impala/authorization/AuthorizationProvider.java R fe/src/main/java/org/apache/impala/authorization/NoopAuthorizationFactory.java M fe/src/main/java/org/apache/impala/authorization/ranger/RangerAuthorizationChecker.java M fe/src/main/java/org/apache/impala/authorization/ranger/RangerAuthorizationFactory.java M fe/src/main/java/org/apache/impala/authorization/sentry/SentryAuthorizationChecker.java M fe/src/main/java/org/apache/impala/authorization/sentry/SentryAuthorizationFactory.java M fe/src/main/java/org/apache/impala/service/JniCatalog.java M fe/src/main/java/org/apache/impala/service/JniFrontend.java M fe/src/test/java/org/apache/impala/analysis/AnalyzeAuthStmtsTest.java M fe/src/test/java/org/apache/impala/analysis/AuthorizationStmtTest.java M fe/src/test/java/org/apache/impala/analysis/StmtMetadataLoaderTest.java M fe/src/test/java/org/apache/impala/catalog/AlterDatabaseTest.java M fe/src/test/java/org/apache/impala/catalog/events/MetastoreEventsProcessorTest.java M fe/src/test/java/org/apache/impala/common/FrontendFixture.java M fe/src/test/java/org/apache/impala/common/FrontendTestBase.java M fe/src/test/java/org/apache/impala/common/QueryFixture.java M fe/src/test/java/org/apache/impala/testutil/ImpaladTestCatalog.java M fe/src/test/java/org/apache/impala/testutil/PlannerTestCaseLoader.java 23 files changed, 486 insertions(+), 81 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/27/12927/3 -- To view, visit http://gerrit.cloudera.org:8080/12927 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: If46b4bf24d916e4a4ea8a36ff4acfd95d5f45c8e Gerrit-Change-Number: 12927 Gerrit-PatchSet: 3 Gerrit-Owner: Fredy Wijaya Gerrit-Reviewer: Austin Nobis Gerrit-Reviewer: Bharath Vissapragada Gerrit-Reviewer: Fredy Wijaya Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Todd Lipcon
[Impala-ASF-CR] IMPALA-8363: Deny access when column masking or row filtering is enabled in Ranger
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/12927 ) Change subject: IMPALA-8363: Deny access when column masking or row filtering is enabled in Ranger .. Patch Set 2: Build Successful https://jenkins.impala.io/job/gerrit-code-review-checks/2646/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests. -- To view, visit http://gerrit.cloudera.org:8080/12927 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: If46b4bf24d916e4a4ea8a36ff4acfd95d5f45c8e Gerrit-Change-Number: 12927 Gerrit-PatchSet: 2 Gerrit-Owner: Fredy Wijaya Gerrit-Reviewer: Austin Nobis Gerrit-Reviewer: Bharath Vissapragada Gerrit-Reviewer: Fredy Wijaya Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Todd Lipcon Gerrit-Comment-Date: Thu, 04 Apr 2019 19:27:04 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-8363: Deny access when column masking or row filtering is enabled in Ranger
Austin Nobis has posted comments on this change. (
http://gerrit.cloudera.org:8080/12927 )
Change subject: IMPALA-8363: Deny access when column masking or row filtering
is enabled in Ranger
..
Patch Set 2:
(4 comments)
http://gerrit.cloudera.org:8080/#/c/12927/2/fe/src/main/java/org/apache/impala/authorization/ranger/RangerAuthorizationChecker.java
File
fe/src/main/java/org/apache/impala/authorization/ranger/RangerAuthorizationChecker.java:
http://gerrit.cloudera.org:8080/#/c/12927/2/fe/src/main/java/org/apache/impala/authorization/ranger/RangerAuthorizationChecker.java@70
PS2, Line 70: private static class Table {
Wouldn't this represent a column and not a table?
http://gerrit.cloudera.org:8080/#/c/12927/2/fe/src/main/java/org/apache/impala/authorization/ranger/RangerAuthorizationChecker.java@246
PS2, Line 246: for (Table table: tables) {
So you're iterating over the `Table` which is actually more like a `Column` but
ignoring the columnName_?
Won't this cause a lot of duplicate entries in the `rowFilterTables`?
http://gerrit.cloudera.org:8080/#/c/12927/2/fe/src/main/java/org/apache/impala/authorization/ranger/RangerAuthorizationChecker.java@269
PS2, Line 269: private static List getTables(List exprs) {
Is this getting the tables or the columns?
http://gerrit.cloudera.org:8080/#/c/12927/2/fe/src/main/java/org/apache/impala/authorization/ranger/RangerAuthorizationChecker.java@279
PS2, Line 279: // Column can be null when it's complex type.
If it is a complex type do you need to parse the complex type for more
potential columns?
--
To view, visit http://gerrit.cloudera.org:8080/12927
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: If46b4bf24d916e4a4ea8a36ff4acfd95d5f45c8e
Gerrit-Change-Number: 12927
Gerrit-PatchSet: 2
Gerrit-Owner: Fredy Wijaya
Gerrit-Reviewer: Austin Nobis
Gerrit-Reviewer: Bharath Vissapragada
Gerrit-Reviewer: Fredy Wijaya
Gerrit-Reviewer: Impala Public Jenkins
Gerrit-Reviewer: Todd Lipcon
Gerrit-Comment-Date: Thu, 04 Apr 2019 19:26:03 +
Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-8363: Deny access when column masking or row filtering is enabled in Ranger
Fredy Wijaya has uploaded a new patch set (#2). ( http://gerrit.cloudera.org:8080/12927 ) Change subject: IMPALA-8363: Deny access when column masking or row filtering is enabled in Ranger .. IMPALA-8363: Deny access when column masking or row filtering is enabled in Ranger This patch updates the Ranger authorization checker code to deny access when column masking and row filtering is enabled in Ranger for queries that that have columns/tables specified in column mask and row filter policies. This is to prevent data leak, such that the data that is masked/filtered in Hive should not be visible at all in Impala until Impala has full support for column masking and row filtering. Testing: - Added tests in AuthorizationStmtTest to test queries with column masking and row filtering enabled. - Ran all FE tests - Ran all E2E tests Change-Id: If46b4bf24d916e4a4ea8a36ff4acfd95d5f45c8e --- M fe/src/main/java/org/apache/impala/analysis/AnalysisContext.java M fe/src/main/java/org/apache/impala/analysis/CreateOrAlterViewStmtBase.java M fe/src/main/java/org/apache/impala/analysis/ModifyStmt.java M fe/src/main/java/org/apache/impala/authorization/AuthorizationChecker.java M fe/src/main/java/org/apache/impala/authorization/AuthorizationFactory.java M fe/src/main/java/org/apache/impala/authorization/AuthorizationProvider.java R fe/src/main/java/org/apache/impala/authorization/NoopAuthorizationFactory.java M fe/src/main/java/org/apache/impala/authorization/ranger/RangerAuthorizationChecker.java M fe/src/main/java/org/apache/impala/authorization/ranger/RangerAuthorizationFactory.java M fe/src/main/java/org/apache/impala/authorization/sentry/SentryAuthorizationChecker.java M fe/src/main/java/org/apache/impala/authorization/sentry/SentryAuthorizationFactory.java M fe/src/main/java/org/apache/impala/service/JniCatalog.java M fe/src/main/java/org/apache/impala/service/JniFrontend.java M fe/src/test/java/org/apache/impala/analysis/AnalyzeAuthStmtsTest.java M fe/src/test/java/org/apache/impala/analysis/AuthorizationStmtTest.java M fe/src/test/java/org/apache/impala/analysis/StmtMetadataLoaderTest.java M fe/src/test/java/org/apache/impala/catalog/AlterDatabaseTest.java M fe/src/test/java/org/apache/impala/catalog/events/MetastoreEventsProcessorTest.java M fe/src/test/java/org/apache/impala/common/FrontendFixture.java M fe/src/test/java/org/apache/impala/common/FrontendTestBase.java M fe/src/test/java/org/apache/impala/common/QueryFixture.java M fe/src/test/java/org/apache/impala/testutil/ImpaladTestCatalog.java M fe/src/test/java/org/apache/impala/testutil/PlannerTestCaseLoader.java 23 files changed, 444 insertions(+), 81 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/27/12927/2 -- To view, visit http://gerrit.cloudera.org:8080/12927 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: If46b4bf24d916e4a4ea8a36ff4acfd95d5f45c8e Gerrit-Change-Number: 12927 Gerrit-PatchSet: 2 Gerrit-Owner: Fredy Wijaya Gerrit-Reviewer: Austin Nobis Gerrit-Reviewer: Bharath Vissapragada Gerrit-Reviewer: Fredy Wijaya Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Todd Lipcon
[Impala-ASF-CR] IMPALA-8363: Deny access when column masking or row filtering is enabled in Ranger
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/12927 ) Change subject: IMPALA-8363: Deny access when column masking or row filtering is enabled in Ranger .. Patch Set 1: Build Successful https://jenkins.impala.io/job/gerrit-code-review-checks/2631/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests. -- To view, visit http://gerrit.cloudera.org:8080/12927 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: If46b4bf24d916e4a4ea8a36ff4acfd95d5f45c8e Gerrit-Change-Number: 12927 Gerrit-PatchSet: 1 Gerrit-Owner: Fredy Wijaya Gerrit-Reviewer: Austin Nobis Gerrit-Reviewer: Bharath Vissapragada Gerrit-Reviewer: Fredy Wijaya Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Todd Lipcon Gerrit-Comment-Date: Wed, 03 Apr 2019 23:48:42 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-8363: Deny access when column masking or row filtering is enabled in Ranger
Fredy Wijaya has uploaded this change for review. ( http://gerrit.cloudera.org:8080/12927 Change subject: IMPALA-8363: Deny access when column masking or row filtering is enabled in Ranger .. IMPALA-8363: Deny access when column masking or row filtering is enabled in Ranger This patch updates the Ranger authorization checker code to deny access when column masking and row filtering is enabled in Ranger for queries that that have columns/tables specified in column mask and row filter policies. This is to prevent data leak, such that the data that is masked/filtered in Hive should not be visible at all in Impala until Impala has full support for column masking and row filtering. Testing: - Added tests in AuthorizationStmtTest to test queries with column masking and row filtering enabled. - Ran all FE tests - Ran all E2E tests Change-Id: If46b4bf24d916e4a4ea8a36ff4acfd95d5f45c8e --- M fe/src/main/java/org/apache/impala/analysis/AnalysisContext.java M fe/src/main/java/org/apache/impala/analysis/CreateOrAlterViewStmtBase.java M fe/src/main/java/org/apache/impala/analysis/ModifyStmt.java M fe/src/main/java/org/apache/impala/authorization/AuthorizationChecker.java M fe/src/main/java/org/apache/impala/authorization/AuthorizationFactory.java M fe/src/main/java/org/apache/impala/authorization/AuthorizationProvider.java R fe/src/main/java/org/apache/impala/authorization/NoopAuthorizationFactory.java M fe/src/main/java/org/apache/impala/authorization/ranger/RangerAuthorizationChecker.java M fe/src/main/java/org/apache/impala/authorization/ranger/RangerAuthorizationFactory.java M fe/src/main/java/org/apache/impala/authorization/sentry/SentryAuthorizationChecker.java M fe/src/main/java/org/apache/impala/authorization/sentry/SentryAuthorizationFactory.java M fe/src/main/java/org/apache/impala/service/JniCatalog.java M fe/src/main/java/org/apache/impala/service/JniFrontend.java M fe/src/test/java/org/apache/impala/analysis/AnalyzeAuthStmtsTest.java M fe/src/test/java/org/apache/impala/analysis/AuthorizationStmtTest.java M fe/src/test/java/org/apache/impala/analysis/StmtMetadataLoaderTest.java M fe/src/test/java/org/apache/impala/catalog/AlterDatabaseTest.java M fe/src/test/java/org/apache/impala/catalog/events/MetastoreEventsProcessorTest.java M fe/src/test/java/org/apache/impala/common/FrontendFixture.java M fe/src/test/java/org/apache/impala/common/FrontendTestBase.java M fe/src/test/java/org/apache/impala/common/QueryFixture.java M fe/src/test/java/org/apache/impala/testutil/ImpaladTestCatalog.java M fe/src/test/java/org/apache/impala/testutil/PlannerTestCaseLoader.java 23 files changed, 436 insertions(+), 81 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/27/12927/1 -- To view, visit http://gerrit.cloudera.org:8080/12927 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newchange Gerrit-Change-Id: If46b4bf24d916e4a4ea8a36ff4acfd95d5f45c8e Gerrit-Change-Number: 12927 Gerrit-PatchSet: 1 Gerrit-Owner: Fredy Wijaya
