Impala Public Jenkins has submitted this change and it was merged. ( http://gerrit.cloudera.org:8080/23908 )
Change subject: IMPALA-14703: Improves finding column masking policies of a table ...................................................................... IMPALA-14703: Improves finding column masking policies of a table When Ranger authorization is enabled, if a user wants to update masked data, it should be blocked. This is done by checking if there are any column masking or row filtering policies on the table enabled for the user. Currently we iterate all the columns of the table and check if a masking policy exists on the column. This is inefficent, especially for wide tables. It also requires metadata of the table is loaded to get the column list, which introduces a performance regression for INVALIDATE and REFRESH statements that previously don't trigger metadata loading. This patch improves the check to make the request on table level with a resource matching scope of SELF_OR_DESCENDANTS. By using this, ranger plugin will return the first matching column masking policy in evalDataMaskPolicies(). As we don't need the column list now, table loading triggered by INVALIDATE and REFRESH statements are also removed. Tests - Ran test_block_metadata_update and data masking tests in test_ranger.py Change-Id: Ic8ab88b7cfd4f7e156c4eead53a2ff3086b1daa6 Reviewed-on: http://gerrit.cloudera.org:8080/23908 Reviewed-by: Csaba Ringhofer <[email protected]> Reviewed-by: Fang-Yu Rao <[email protected]> Tested-by: Impala Public Jenkins <[email protected]> --- M fe/src/main/java/org/apache/impala/analysis/StmtMetadataLoader.java M fe/src/main/java/org/apache/impala/authorization/AuthorizableTable.java M fe/src/main/java/org/apache/impala/authorization/BaseAuthorizationChecker.java M fe/src/main/java/org/apache/impala/authorization/ranger/RangerAuthorizationChecker.java 4 files changed, 22 insertions(+), 81 deletions(-) Approvals: Csaba Ringhofer: Looks good to me, approved Fang-Yu Rao: Looks good to me, approved Impala Public Jenkins: Verified -- To view, visit http://gerrit.cloudera.org:8080/23908 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: merged Gerrit-Change-Id: Ic8ab88b7cfd4f7e156c4eead53a2ff3086b1daa6 Gerrit-Change-Number: 23908 Gerrit-PatchSet: 2 Gerrit-Owner: Quanlong Huang <[email protected]> Gerrit-Reviewer: Csaba Ringhofer <[email protected]> Gerrit-Reviewer: Fang-Yu Rao <[email protected]> Gerrit-Reviewer: Impala Public Jenkins <[email protected]> Gerrit-Reviewer: Quanlong Huang <[email protected]>
