[Impala-ASF-CR] IMPALA-9348: Add flag to disable column masking
Impala Public Jenkins has submitted this change and it was merged. ( http://gerrit.cloudera.org:8080/15140 ) Change subject: IMPALA-9348: Add flag to disable column masking .. IMPALA-9348: Add flag to disable column masking Add flag --enable_column_masking which defaults to false to disable column masking feature until this feature becomes mature. This patch recovers some legacy codes before IMPALA-9009 (mainly inside RangerAuthorizationChecker). When the flag is set to false, the legacy code paths are used: access of columns containing column masking policies will be denied. Tests: - Recover legacy FE tests for verifying error messages. Change-Id: Ife7c98042cfb831ba00d9d2179367cba257b77a0 Reviewed-on: http://gerrit.cloudera.org:8080/15140 Reviewed-by: Impala Public Jenkins Tested-by: Impala Public Jenkins --- M be/src/common/global-flags.cc M be/src/util/backend-gflag-util.cc M common/thrift/BackendGflags.thrift M fe/src/main/java/org/apache/impala/authorization/BaseAuthorizationChecker.java M fe/src/main/java/org/apache/impala/authorization/ranger/RangerAuthorizationChecker.java M fe/src/main/java/org/apache/impala/authorization/ranger/RangerAuthorizationFactory.java M fe/src/main/java/org/apache/impala/service/BackendConfig.java M fe/src/test/java/org/apache/impala/authorization/AuthorizationStmtTest.java M fe/src/test/java/org/apache/impala/authorization/AuthorizationTestBase.java M tests/authorization/test_ranger.py 10 files changed, 166 insertions(+), 6 deletions(-) Approvals: Impala Public Jenkins: Looks good to me, approved; Verified -- To view, visit http://gerrit.cloudera.org:8080/15140 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: merged Gerrit-Change-Id: Ife7c98042cfb831ba00d9d2179367cba257b77a0 Gerrit-Change-Number: 15140 Gerrit-PatchSet: 7 Gerrit-Owner: Quanlong Huang Gerrit-Reviewer: Aman Sinha Gerrit-Reviewer: Csaba Ringhofer Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Quanlong Huang
[Impala-ASF-CR] IMPALA-9348: Add flag to disable column masking
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/15140 ) Change subject: IMPALA-9348: Add flag to disable column masking .. Patch Set 6: Verified+1 -- To view, visit http://gerrit.cloudera.org:8080/15140 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ife7c98042cfb831ba00d9d2179367cba257b77a0 Gerrit-Change-Number: 15140 Gerrit-PatchSet: 6 Gerrit-Owner: Quanlong Huang Gerrit-Reviewer: Aman Sinha Gerrit-Reviewer: Csaba Ringhofer Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Quanlong Huang Gerrit-Comment-Date: Fri, 31 Jan 2020 15:09:57 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-9348: Add flag to disable column masking
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/15140 ) Change subject: IMPALA-9348: Add flag to disable column masking .. Patch Set 5: Build Successful https://jenkins.impala.io/job/gerrit-code-review-checks/5567/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests. -- To view, visit http://gerrit.cloudera.org:8080/15140 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ife7c98042cfb831ba00d9d2179367cba257b77a0 Gerrit-Change-Number: 15140 Gerrit-PatchSet: 5 Gerrit-Owner: Quanlong Huang Gerrit-Reviewer: Aman Sinha Gerrit-Reviewer: Csaba Ringhofer Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Quanlong Huang Gerrit-Comment-Date: Fri, 31 Jan 2020 10:57:34 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-9348: Add flag to disable column masking
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/15140 ) Change subject: IMPALA-9348: Add flag to disable column masking .. Patch Set 6: Build started: https://jenkins.impala.io/job/gerrit-verify-dryrun/5483/ DRY_RUN=false -- To view, visit http://gerrit.cloudera.org:8080/15140 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ife7c98042cfb831ba00d9d2179367cba257b77a0 Gerrit-Change-Number: 15140 Gerrit-PatchSet: 6 Gerrit-Owner: Quanlong Huang Gerrit-Reviewer: Aman Sinha Gerrit-Reviewer: Csaba Ringhofer Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Quanlong Huang Gerrit-Comment-Date: Fri, 31 Jan 2020 10:21:36 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-9348: Add flag to disable column masking
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/15140 ) Change subject: IMPALA-9348: Add flag to disable column masking .. Patch Set 6: Code-Review+2 -- To view, visit http://gerrit.cloudera.org:8080/15140 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ife7c98042cfb831ba00d9d2179367cba257b77a0 Gerrit-Change-Number: 15140 Gerrit-PatchSet: 6 Gerrit-Owner: Quanlong Huang Gerrit-Reviewer: Aman Sinha Gerrit-Reviewer: Csaba Ringhofer Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Quanlong Huang Gerrit-Comment-Date: Fri, 31 Jan 2020 10:21:35 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-9348: Add flag to disable column masking
Csaba Ringhofer has posted comments on this change. ( http://gerrit.cloudera.org:8080/15140 ) Change subject: IMPALA-9348: Add flag to disable column masking .. Patch Set 5: Code-Review+2 -- To view, visit http://gerrit.cloudera.org:8080/15140 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ife7c98042cfb831ba00d9d2179367cba257b77a0 Gerrit-Change-Number: 15140 Gerrit-PatchSet: 5 Gerrit-Owner: Quanlong Huang Gerrit-Reviewer: Aman Sinha Gerrit-Reviewer: Csaba Ringhofer Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Quanlong Huang Gerrit-Comment-Date: Fri, 31 Jan 2020 10:20:14 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-9348: Add flag to disable column masking
Quanlong Huang has posted comments on this change. ( http://gerrit.cloudera.org:8080/15140 ) Change subject: IMPALA-9348: Add flag to disable column masking .. Patch Set 5: (2 comments) Thanks Aman and Csaba for looking into this so quickly! http://gerrit.cloudera.org:8080/#/c/15140/4/fe/src/main/java/org/apache/impala/authorization/BaseAuthorizationChecker.java File fe/src/main/java/org/apache/impala/authorization/BaseAuthorizationChecker.java: http://gerrit.cloudera.org:8080/#/c/15140/4/fe/src/main/java/org/apache/impala/authorization/BaseAuthorizationChecker.java@228 PS4, Line 228: Also deny access of columns : // containing column masking policies when column masking feature is disabled. > I would prefer to mention this in the commit message. Done http://gerrit.cloudera.org:8080/#/c/15140/4/fe/src/main/java/org/apache/impala/authorization/ranger/RangerAuthorizationChecker.java File fe/src/main/java/org/apache/impala/authorization/ranger/RangerAuthorizationChecker.java: http://gerrit.cloudera.org:8080/#/c/15140/4/fe/src/main/java/org/apache/impala/authorization/ranger/RangerAuthorizationChecker.java@200 PS4, Line 200: authorizeRowFilter(user, > The same condition is already checked in the previous line. Oops, done. -- To view, visit http://gerrit.cloudera.org:8080/15140 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ife7c98042cfb831ba00d9d2179367cba257b77a0 Gerrit-Change-Number: 15140 Gerrit-PatchSet: 5 Gerrit-Owner: Quanlong Huang Gerrit-Reviewer: Aman Sinha Gerrit-Reviewer: Csaba Ringhofer Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Quanlong Huang Gerrit-Comment-Date: Fri, 31 Jan 2020 10:14:10 + Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-9348: Add flag to disable column masking
Hello Aman Sinha, Csaba Ringhofer, Impala Public Jenkins, I'd like you to reexamine a change. Please visit http://gerrit.cloudera.org:8080/15140 to look at the new patch set (#5). Change subject: IMPALA-9348: Add flag to disable column masking .. IMPALA-9348: Add flag to disable column masking Add flag --enable_column_masking which defaults to false to disable column masking feature until this feature becomes mature. This patch recovers some legacy codes before IMPALA-9009 (mainly inside RangerAuthorizationChecker). When the flag is set to false, the legacy code paths are used: access of columns containing column masking policies will be denied. Tests: - Recover legacy FE tests for verifying error messages. Change-Id: Ife7c98042cfb831ba00d9d2179367cba257b77a0 --- M be/src/common/global-flags.cc M be/src/util/backend-gflag-util.cc M common/thrift/BackendGflags.thrift M fe/src/main/java/org/apache/impala/authorization/BaseAuthorizationChecker.java M fe/src/main/java/org/apache/impala/authorization/ranger/RangerAuthorizationChecker.java M fe/src/main/java/org/apache/impala/authorization/ranger/RangerAuthorizationFactory.java M fe/src/main/java/org/apache/impala/service/BackendConfig.java M fe/src/test/java/org/apache/impala/authorization/AuthorizationStmtTest.java M fe/src/test/java/org/apache/impala/authorization/AuthorizationTestBase.java M tests/authorization/test_ranger.py 10 files changed, 166 insertions(+), 6 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/40/15140/5 -- To view, visit http://gerrit.cloudera.org:8080/15140 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: Ife7c98042cfb831ba00d9d2179367cba257b77a0 Gerrit-Change-Number: 15140 Gerrit-PatchSet: 5 Gerrit-Owner: Quanlong Huang Gerrit-Reviewer: Aman Sinha Gerrit-Reviewer: Csaba Ringhofer Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Quanlong Huang
[Impala-ASF-CR] IMPALA-9348: Add flag to disable column masking
Csaba Ringhofer has posted comments on this change. ( http://gerrit.cloudera.org:8080/15140 ) Change subject: IMPALA-9348: Add flag to disable column masking .. Patch Set 4: (2 comments) http://gerrit.cloudera.org:8080/#/c/15140/4/fe/src/main/java/org/apache/impala/authorization/BaseAuthorizationChecker.java File fe/src/main/java/org/apache/impala/authorization/BaseAuthorizationChecker.java: http://gerrit.cloudera.org:8080/#/c/15140/4/fe/src/main/java/org/apache/impala/authorization/BaseAuthorizationChecker.java@228 PS4, Line 228: Also deny access of columns : // containing column masking policies when column masking feature is disabled. I would prefer to mention this in the commit message. http://gerrit.cloudera.org:8080/#/c/15140/4/fe/src/main/java/org/apache/impala/authorization/ranger/RangerAuthorizationChecker.java File fe/src/main/java/org/apache/impala/authorization/ranger/RangerAuthorizationChecker.java: http://gerrit.cloudera.org:8080/#/c/15140/4/fe/src/main/java/org/apache/impala/authorization/ranger/RangerAuthorizationChecker.java@200 PS4, Line 200: if (request.getAuthorizable().getType() == Type.TABLE) The same condition is already checked in the previous line. -- To view, visit http://gerrit.cloudera.org:8080/15140 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ife7c98042cfb831ba00d9d2179367cba257b77a0 Gerrit-Change-Number: 15140 Gerrit-PatchSet: 4 Gerrit-Owner: Quanlong Huang Gerrit-Reviewer: Aman Sinha Gerrit-Reviewer: Csaba Ringhofer Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Quanlong Huang Gerrit-Comment-Date: Fri, 31 Jan 2020 10:07:05 + Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-9348: Add flag to disable column masking
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/15140 ) Change subject: IMPALA-9348: Add flag to disable column masking .. Patch Set 4: Build Successful https://jenkins.impala.io/job/gerrit-code-review-checks/5566/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests. -- To view, visit http://gerrit.cloudera.org:8080/15140 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ife7c98042cfb831ba00d9d2179367cba257b77a0 Gerrit-Change-Number: 15140 Gerrit-PatchSet: 4 Gerrit-Owner: Quanlong Huang Gerrit-Reviewer: Aman Sinha Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Quanlong Huang Gerrit-Comment-Date: Fri, 31 Jan 2020 07:56:25 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-9348: Add flag to disable column masking
Quanlong Huang has posted comments on this change. ( http://gerrit.cloudera.org:8080/15140 ) Change subject: IMPALA-9348: Add flag to disable column masking .. Patch Set 4: (1 comment) http://gerrit.cloudera.org:8080/#/c/15140/3/fe/src/main/java/org/apache/impala/authorization/ranger/RangerAuthorizationChecker.java File fe/src/main/java/org/apache/impala/authorization/ranger/RangerAuthorizationChecker.java: http://gerrit.cloudera.org:8080/#/c/15140/3/fe/src/main/java/org/apache/impala/authorization/ranger/RangerAuthorizationChecker.java@285 PS3, Line 285: throw new AuthorizationException(String.format( > Thanks for making the change to avoid unnecessary function calls. One mino Yeah, I add back the check for isColumnMaskingEnabled_ anyway. -- To view, visit http://gerrit.cloudera.org:8080/15140 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ife7c98042cfb831ba00d9d2179367cba257b77a0 Gerrit-Change-Number: 15140 Gerrit-PatchSet: 4 Gerrit-Owner: Quanlong Huang Gerrit-Reviewer: Aman Sinha Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Quanlong Huang Gerrit-Comment-Date: Fri, 31 Jan 2020 07:13:02 + Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-9348: Add flag to disable column masking
Hello Aman Sinha, Impala Public Jenkins, I'd like you to reexamine a change. Please visit http://gerrit.cloudera.org:8080/15140 to look at the new patch set (#4). Change subject: IMPALA-9348: Add flag to disable column masking .. IMPALA-9348: Add flag to disable column masking Add flag --enable_column_masking which defaults to false to disable column masking feature until this feature becomes mature. This patch recovers some legacy codes before IMPALA-9009 (mainly inside RangerAuthorizationChecker). When the flag is set to false, the legacy code paths are used. Tests: - Recover legacy FE tests for verifying error messages. Change-Id: Ife7c98042cfb831ba00d9d2179367cba257b77a0 --- M be/src/common/global-flags.cc M be/src/util/backend-gflag-util.cc M common/thrift/BackendGflags.thrift M fe/src/main/java/org/apache/impala/authorization/BaseAuthorizationChecker.java M fe/src/main/java/org/apache/impala/authorization/ranger/RangerAuthorizationChecker.java M fe/src/main/java/org/apache/impala/authorization/ranger/RangerAuthorizationFactory.java M fe/src/main/java/org/apache/impala/service/BackendConfig.java M fe/src/test/java/org/apache/impala/authorization/AuthorizationStmtTest.java M fe/src/test/java/org/apache/impala/authorization/AuthorizationTestBase.java M tests/authorization/test_ranger.py 10 files changed, 170 insertions(+), 8 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/40/15140/4 -- To view, visit http://gerrit.cloudera.org:8080/15140 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: Ife7c98042cfb831ba00d9d2179367cba257b77a0 Gerrit-Change-Number: 15140 Gerrit-PatchSet: 4 Gerrit-Owner: Quanlong Huang Gerrit-Reviewer: Aman Sinha Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Quanlong Huang
[Impala-ASF-CR] IMPALA-9348: Add flag to disable column masking
Aman Sinha has posted comments on this change. ( http://gerrit.cloudera.org:8080/15140 ) Change subject: IMPALA-9348: Add flag to disable column masking .. Patch Set 3: (1 comment) http://gerrit.cloudera.org:8080/#/c/15140/3/fe/src/main/java/org/apache/impala/authorization/ranger/RangerAuthorizationChecker.java File fe/src/main/java/org/apache/impala/authorization/ranger/RangerAuthorizationChecker.java: http://gerrit.cloudera.org:8080/#/c/15140/3/fe/src/main/java/org/apache/impala/authorization/ranger/RangerAuthorizationChecker.java@285 PS3, Line 285: "Column masking is disabled by --enable_column_masking flag. Can't access " + Thanks for making the change to avoid unnecessary function calls. One minor nit: This error message says 'Column masking is disabled by ...' but this assumes that the caller is always calling this function when the flag is false. Another caller could potentially call this function even when the flag is true. Anyways, this is ok for now since I understand the disabling is a temporary thing until the feature is fully tested. -- To view, visit http://gerrit.cloudera.org:8080/15140 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ife7c98042cfb831ba00d9d2179367cba257b77a0 Gerrit-Change-Number: 15140 Gerrit-PatchSet: 3 Gerrit-Owner: Quanlong Huang Gerrit-Reviewer: Aman Sinha Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Quanlong Huang Gerrit-Comment-Date: Fri, 31 Jan 2020 06:06:55 + Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-9348: Add flag to disable column masking
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/15140 ) Change subject: IMPALA-9348: Add flag to disable column masking .. Patch Set 3: Build Successful https://jenkins.impala.io/job/gerrit-code-review-checks/5565/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests. -- To view, visit http://gerrit.cloudera.org:8080/15140 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ife7c98042cfb831ba00d9d2179367cba257b77a0 Gerrit-Change-Number: 15140 Gerrit-PatchSet: 3 Gerrit-Owner: Quanlong Huang Gerrit-Reviewer: Aman Sinha Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Quanlong Huang Gerrit-Comment-Date: Fri, 31 Jan 2020 06:00:16 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-9348: Add flag to disable column masking
Hello Aman Sinha, Impala Public Jenkins, I'd like you to reexamine a change. Please visit http://gerrit.cloudera.org:8080/15140 to look at the new patch set (#3). Change subject: IMPALA-9348: Add flag to disable column masking .. IMPALA-9348: Add flag to disable column masking Add flag --enable_column_masking which defaults to false to disable column masking feature until this feature becomes mature. This patch recovers some legacy codes before IMPALA-9009 (mainly inside RangerAuthorizationChecker). When the flag is set to false, the legacy code paths are used. Tests: - Recover legacy FE tests for verifying error messages. Change-Id: Ife7c98042cfb831ba00d9d2179367cba257b77a0 --- M be/src/common/global-flags.cc M be/src/util/backend-gflag-util.cc M common/thrift/BackendGflags.thrift M fe/src/main/java/org/apache/impala/authorization/BaseAuthorizationChecker.java M fe/src/main/java/org/apache/impala/authorization/ranger/RangerAuthorizationChecker.java M fe/src/main/java/org/apache/impala/authorization/ranger/RangerAuthorizationFactory.java M fe/src/main/java/org/apache/impala/service/BackendConfig.java M fe/src/test/java/org/apache/impala/authorization/AuthorizationStmtTest.java M fe/src/test/java/org/apache/impala/authorization/AuthorizationTestBase.java M tests/authorization/test_ranger.py 10 files changed, 169 insertions(+), 8 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/40/15140/3 -- To view, visit http://gerrit.cloudera.org:8080/15140 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: Ife7c98042cfb831ba00d9d2179367cba257b77a0 Gerrit-Change-Number: 15140 Gerrit-PatchSet: 3 Gerrit-Owner: Quanlong Huang Gerrit-Reviewer: Aman Sinha Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Quanlong Huang
[Impala-ASF-CR] IMPALA-9348: Add flag to disable column masking
Quanlong Huang has posted comments on this change. ( http://gerrit.cloudera.org:8080/15140 ) Change subject: IMPALA-9348: Add flag to disable column masking .. Patch Set 2: (1 comment) http://gerrit.cloudera.org:8080/#/c/15140/2/fe/src/main/java/org/apache/impala/authorization/ranger/RangerAuthorizationChecker.java File fe/src/main/java/org/apache/impala/authorization/ranger/RangerAuthorizationChecker.java: http://gerrit.cloudera.org:8080/#/c/15140/2/fe/src/main/java/org/apache/impala/authorization/ranger/RangerAuthorizationChecker.java@282 PS2, Line 282: if (!isColumnMaskingEnabled_ > Ignore this as I understand that you are reverting to the pre-IMPALA-9009 b Thanks! But we can exit sooner if the flag is True by moving "!isColumnMaskingEnabled_" to the caller to avoid 100 function calls. -- To view, visit http://gerrit.cloudera.org:8080/15140 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ife7c98042cfb831ba00d9d2179367cba257b77a0 Gerrit-Change-Number: 15140 Gerrit-PatchSet: 2 Gerrit-Owner: Quanlong Huang Gerrit-Reviewer: Aman Sinha Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Quanlong Huang Gerrit-Comment-Date: Fri, 31 Jan 2020 05:13:54 + Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-9348: Add flag to disable column masking
Aman Sinha has posted comments on this change. ( http://gerrit.cloudera.org:8080/15140 ) Change subject: IMPALA-9348: Add flag to disable column masking .. Patch Set 2: (1 comment) http://gerrit.cloudera.org:8080/#/c/15140/2/fe/src/main/java/org/apache/impala/authorization/ranger/RangerAuthorizationChecker.java File fe/src/main/java/org/apache/impala/authorization/ranger/RangerAuthorizationChecker.java: http://gerrit.cloudera.org:8080/#/c/15140/2/fe/src/main/java/org/apache/impala/authorization/ranger/RangerAuthorizationChecker.java@282 PS2, Line 282: if (!isColumnMaskingEnabled_ > If there are 100 columns, this check for isColumnMaskingEnabled_ will be do Ignore this as I understand that you are reverting to the pre-IMPALA-9009 behavior. -- To view, visit http://gerrit.cloudera.org:8080/15140 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ife7c98042cfb831ba00d9d2179367cba257b77a0 Gerrit-Change-Number: 15140 Gerrit-PatchSet: 2 Gerrit-Owner: Quanlong Huang Gerrit-Reviewer: Aman Sinha Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Quanlong Huang Gerrit-Comment-Date: Fri, 31 Jan 2020 05:08:46 + Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-9348: Add flag to disable column masking
Aman Sinha has posted comments on this change. ( http://gerrit.cloudera.org:8080/15140 ) Change subject: IMPALA-9348: Add flag to disable column masking .. Patch Set 2: Code-Review+1 > Patch Set 2: > > (1 comment) > > l I see the reasoning for this now..you are reverting to the pre-IMPALA-9009 behavior which was calling the evaluateColumnMask() for each column and you have added the extra check. -- To view, visit http://gerrit.cloudera.org:8080/15140 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ife7c98042cfb831ba00d9d2179367cba257b77a0 Gerrit-Change-Number: 15140 Gerrit-PatchSet: 2 Gerrit-Owner: Quanlong Huang Gerrit-Reviewer: Aman Sinha Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Quanlong Huang Gerrit-Comment-Date: Fri, 31 Jan 2020 05:06:18 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-9348: Add flag to disable column masking
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/15140 ) Change subject: IMPALA-9348: Add flag to disable column masking .. Patch Set 2: Build Successful https://jenkins.impala.io/job/gerrit-code-review-checks/5564/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests. -- To view, visit http://gerrit.cloudera.org:8080/15140 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ife7c98042cfb831ba00d9d2179367cba257b77a0 Gerrit-Change-Number: 15140 Gerrit-PatchSet: 2 Gerrit-Owner: Quanlong Huang Gerrit-Reviewer: Aman Sinha Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Quanlong Huang Gerrit-Comment-Date: Fri, 31 Jan 2020 04:57:36 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-9348: Add flag to disable column masking
Aman Sinha has posted comments on this change. ( http://gerrit.cloudera.org:8080/15140 ) Change subject: IMPALA-9348: Add flag to disable column masking .. Patch Set 2: (1 comment) l http://gerrit.cloudera.org:8080/#/c/15140/2/fe/src/main/java/org/apache/impala/authorization/ranger/RangerAuthorizationChecker.java File fe/src/main/java/org/apache/impala/authorization/ranger/RangerAuthorizationChecker.java: http://gerrit.cloudera.org:8080/#/c/15140/2/fe/src/main/java/org/apache/impala/authorization/ranger/RangerAuthorizationChecker.java@282 PS2, Line 282: if (!isColumnMaskingEnabled_ If there are 100 columns, this check for isColumnMaskingEnabled_ will be done as many times until one of the columns (could be the last one) access throws an exception. Can't we exit sooner since the flag is False anyways ? -- To view, visit http://gerrit.cloudera.org:8080/15140 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ife7c98042cfb831ba00d9d2179367cba257b77a0 Gerrit-Change-Number: 15140 Gerrit-PatchSet: 2 Gerrit-Owner: Quanlong Huang Gerrit-Reviewer: Aman Sinha Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Quanlong Huang Gerrit-Comment-Date: Fri, 31 Jan 2020 04:40:07 + Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-9348: Add flag to disable column masking
Hello Aman Sinha, Impala Public Jenkins, I'd like you to reexamine a change. Please visit http://gerrit.cloudera.org:8080/15140 to look at the new patch set (#2). Change subject: IMPALA-9348: Add flag to disable column masking .. IMPALA-9348: Add flag to disable column masking Add flag --enable_column_masking which defaults to false to disable column masking feature until this feature becomes mature. This patch recovers some legacy codes before IMPALA-9009 (mainly inside RangerAuthorizationChecker). When the flag is set to false, the legacy code paths are used. Tests: - Recover legacy FE tests for verifying error messages. Change-Id: Ife7c98042cfb831ba00d9d2179367cba257b77a0 --- M be/src/common/global-flags.cc M be/src/util/backend-gflag-util.cc M common/thrift/BackendGflags.thrift M fe/src/main/java/org/apache/impala/authorization/BaseAuthorizationChecker.java M fe/src/main/java/org/apache/impala/authorization/ranger/RangerAuthorizationChecker.java M fe/src/main/java/org/apache/impala/authorization/ranger/RangerAuthorizationFactory.java M fe/src/main/java/org/apache/impala/service/BackendConfig.java M fe/src/test/java/org/apache/impala/authorization/AuthorizationStmtTest.java M fe/src/test/java/org/apache/impala/authorization/AuthorizationTestBase.java M tests/authorization/test_ranger.py 10 files changed, 169 insertions(+), 8 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/40/15140/2 -- To view, visit http://gerrit.cloudera.org:8080/15140 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: Ife7c98042cfb831ba00d9d2179367cba257b77a0 Gerrit-Change-Number: 15140 Gerrit-PatchSet: 2 Gerrit-Owner: Quanlong Huang Gerrit-Reviewer: Aman Sinha Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Quanlong Huang
[Impala-ASF-CR] IMPALA-9348: Add flag to disable column masking
Quanlong Huang has posted comments on this change. ( http://gerrit.cloudera.org:8080/15140 ) Change subject: IMPALA-9348: Add flag to disable column masking .. Patch Set 1: (1 comment) http://gerrit.cloudera.org:8080/#/c/15140/1/fe/src/main/java/org/apache/impala/authorization/ranger/RangerAuthorizationChecker.java File fe/src/main/java/org/apache/impala/authorization/ranger/RangerAuthorizationChecker.java: http://gerrit.cloudera.org:8080/#/c/15140/1/fe/src/main/java/org/apache/impala/authorization/ranger/RangerAuthorizationChecker.java@193 PS1, Line 193: if (!isColumnMaskingEnabled_ authorizeColumnMask() is the legacy codes before IMPALA-9009. I just add the new flag inside this function. It doesn't belong to the new code of IMPALA-9009. It will block any access to columns with column masking policies if the flag is false. > BTW, the check for if (!isColumnMaskingEnabled_) is being done twice ..is > that needed ? Oops, I'll remove the check here. -- To view, visit http://gerrit.cloudera.org:8080/15140 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ife7c98042cfb831ba00d9d2179367cba257b77a0 Gerrit-Change-Number: 15140 Gerrit-PatchSet: 1 Gerrit-Owner: Quanlong Huang Gerrit-Reviewer: Aman Sinha Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Quanlong Huang Gerrit-Comment-Date: Fri, 31 Jan 2020 04:05:18 + Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-9348: Add flag to disable column masking
Aman Sinha has posted comments on this change. ( http://gerrit.cloudera.org:8080/15140 ) Change subject: IMPALA-9348: Add flag to disable column masking .. Patch Set 1: (1 comment) http://gerrit.cloudera.org:8080/#/c/15140/1/fe/src/main/java/org/apache/impala/authorization/ranger/RangerAuthorizationChecker.java File fe/src/main/java/org/apache/impala/authorization/ranger/RangerAuthorizationChecker.java: http://gerrit.cloudera.org:8080/#/c/15140/1/fe/src/main/java/org/apache/impala/authorization/ranger/RangerAuthorizationChecker.java@193 PS1, Line 193: if (!isColumnMaskingEnabled_ If column masking is disabled, it's not clear to me why authorizeColumnMask() is being called. Shouldn't the effect of setting this flag to False be that the new code is not called at all ? Unless I misunderstood the intent of the flag. BTW, the check for if (!isColumnMaskingEnabled_) is being done twice ..is that needed ? -- To view, visit http://gerrit.cloudera.org:8080/15140 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ife7c98042cfb831ba00d9d2179367cba257b77a0 Gerrit-Change-Number: 15140 Gerrit-PatchSet: 1 Gerrit-Owner: Quanlong Huang Gerrit-Reviewer: Aman Sinha Gerrit-Reviewer: Impala Public Jenkins Gerrit-Comment-Date: Fri, 31 Jan 2020 03:52:20 + Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-9348: Add flag to disable column masking
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/15140 ) Change subject: IMPALA-9348: Add flag to disable column masking .. Patch Set 1: Build Successful https://jenkins.impala.io/job/gerrit-code-review-checks/5563/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests. -- To view, visit http://gerrit.cloudera.org:8080/15140 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ife7c98042cfb831ba00d9d2179367cba257b77a0 Gerrit-Change-Number: 15140 Gerrit-PatchSet: 1 Gerrit-Owner: Quanlong Huang Gerrit-Reviewer: Impala Public Jenkins Gerrit-Comment-Date: Fri, 31 Jan 2020 03:06:33 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-9348: Add flag to disable column masking
Quanlong Huang has uploaded this change for review. ( http://gerrit.cloudera.org:8080/15140 Change subject: IMPALA-9348: Add flag to disable column masking .. IMPALA-9348: Add flag to disable column masking Add flag --enable_column_masking which defaults to false to disable column masking feature until this feature becomes mature. Tests: - Recover legacy FE tests for verifying error messages. Change-Id: Ife7c98042cfb831ba00d9d2179367cba257b77a0 --- M be/src/common/global-flags.cc M be/src/util/backend-gflag-util.cc M common/thrift/BackendGflags.thrift M fe/src/main/java/org/apache/impala/authorization/BaseAuthorizationChecker.java M fe/src/main/java/org/apache/impala/authorization/ranger/RangerAuthorizationChecker.java M fe/src/main/java/org/apache/impala/authorization/ranger/RangerAuthorizationFactory.java M fe/src/main/java/org/apache/impala/service/BackendConfig.java M fe/src/test/java/org/apache/impala/authorization/AuthorizationStmtTest.java M fe/src/test/java/org/apache/impala/authorization/AuthorizationTestBase.java M tests/authorization/test_ranger.py 10 files changed, 170 insertions(+), 8 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/40/15140/1 -- To view, visit http://gerrit.cloudera.org:8080/15140 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newchange Gerrit-Change-Id: Ife7c98042cfb831ba00d9d2179367cba257b77a0 Gerrit-Change-Number: 15140 Gerrit-PatchSet: 1 Gerrit-Owner: Quanlong Huang
