Fang-Yu Rao has uploaded a new patch set (#14). ( http://gerrit.cloudera.org:8080/15854 )
Change subject: IMPALA-9597: Eliminate redundant Ranger audits for column masking ...................................................................... IMPALA-9597: Eliminate redundant Ranger audits for column masking After IMPALA-9350, Impala is able to produce the corresponding Ranger audits when a query involves policies of column masking. However, redundant audit events could be produced due to the fact that the analysis of the TableRef containing a column involved in a column masking policy could be performed more than once for a query that has to be analyzed more than once. For example, a query consisting of a WithClause or a query that requires a rewrite operation followed by a re-analysis phase would result in RangerImpalaPlugin#evalDataMaskPolicies() being invoked multiple times, each producing an audit log entry for the same column. Moreover, for a query involving column masking policies, the corresponding audit log entries will still be generated even though there is an AuthorizationException thrown in the authorization phase. This patch fixes those two issues described above by adding some post-processing steps after the analysis of a query to deduplicate the List of AuthzAuditEvent's for column masking policies. Specifically, we stash the deduplicated audit events after the analysis of the query and will add back those deduplicated events only if the authorization of the query is successful. On the other hand, this patch also resolves an inconsistency when an "Unmasked" policy is involved in a query that retains the original column value. Specifically, when an "Unmasked" policy is the only column masking policy involved in this query, RangerAuthorizationChecker#createColumnMask() will not be called to produce the corresponding AuthzAuditEvent, whereas createColumnMask() will be invoked to produce the respective AuthzAuditEvent if there are policies of other types. Since an "Unmasked" policy essentially does not change the original column value, we filter out the respective events with mask type equal to "MASK_NONE" which corresponds to an "Unmasked" policy. Testing: - Added three test cases in RangerAuditLogTest#testAuditsForColumnMasking() to make sure the issues above are resolved. - Verified that this patch passes the FE tests in the DEBUG build. Change-Id: I42d60130fba93d63fbc36949f2bf746b7ae2497d --- M fe/src/main/java/org/apache/impala/analysis/AnalysisContext.java M fe/src/main/java/org/apache/impala/authorization/AuthorizationChecker.java M fe/src/main/java/org/apache/impala/authorization/BaseAuthorizationChecker.java M fe/src/main/java/org/apache/impala/authorization/NoopAuthorizationFactory.java M fe/src/main/java/org/apache/impala/authorization/ranger/RangerAuthorizationChecker.java M fe/src/main/java/org/apache/impala/authorization/ranger/RangerAuthorizationContext.java M fe/src/main/java/org/apache/impala/authorization/ranger/RangerImpalaPlugin.java M fe/src/test/java/org/apache/impala/authorization/ranger/RangerAuditLogTest.java M fe/src/test/java/org/apache/impala/common/FrontendTestBase.java 9 files changed, 259 insertions(+), 56 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/54/15854/14 -- To view, visit http://gerrit.cloudera.org:8080/15854 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: I42d60130fba93d63fbc36949f2bf746b7ae2497d Gerrit-Change-Number: 15854 Gerrit-PatchSet: 14 Gerrit-Owner: Fang-Yu Rao <[email protected]> Gerrit-Reviewer: Csaba Ringhofer <[email protected]> Gerrit-Reviewer: Fang-Yu Rao <[email protected]> Gerrit-Reviewer: Impala Public Jenkins <[email protected]> Gerrit-Reviewer: Quanlong Huang <[email protected]>
