[
https://issues.apache.org/jira/browse/IMPALA-14703?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=18092599#comment-18092599
]
ASF subversion and git services commented on IMPALA-14703:
----------------------------------------------------------
Commit a659afeabbdd48ddda365ba918798f9e47676df2 in impala's branch
refs/heads/master from stiga-huang
[ https://gitbox.apache.org/repos/asf?p=impala.git;h=a659afeab ]
IMPALA-13946: Turn on allow_catalog_cache_op_from_masked_users by default
Most of the users turn this on to resolve the performance regression on
INVALIDATE and REFRESH commands. The regression is due to additional
metadata loading if the table is unloaded in catalogd. Though this is
resolved by IMPALA-14703, i.e. don't need metadata loading anymore,
checking table masking policies is still an additional work. Setting
allow_catalog_cache_op_from_masked_users to true skips this check.
Test Changes:
- Moved the negative test (test_block_metadata_update) to
TestRangerIndependent since it now requires restarting the cluster
with the flag off. Also mirrored it to test on both catalog modes.
- For FE tests that depend on this flag to be false, explicitly set it
before the tests.
Tests
- Ran CORE tests
Change-Id: I2d3be4467c6dddf5d9cbe1f68e1a19555b3f9359
Reviewed-on: http://gerrit.cloudera.org:8080/24510
Reviewed-by: Zoltan Borok-Nagy <[email protected]>
Tested-by: Impala Public Jenkins <[email protected]>
> Improves finding column masking policies of a table for a user
> --------------------------------------------------------------
>
> Key: IMPALA-14703
> URL: https://issues.apache.org/jira/browse/IMPALA-14703
> Project: IMPALA
> Issue Type: Improvement
> Components: Frontend, Security
> Reporter: Quanlong Huang
> Assignee: Quanlong Huang
> Priority: Critical
> Fix For: Impala 5.0.0
>
>
> When Ranger authorization is enabled, to block updates on masked data, we
> check column masking policies on each column of a table:
> {code:java}
> for (String column : columns) {
> RangerAccessResult columnMaskResult = evalColumnMask(user,
> authorizable.getDbName(), authorizable.getTableName(), column,
> /*auditHandler*/null);
> if (columnMaskResult != null && columnMaskResult.isMaskEnabled()) {
> LOG.trace("Deny {} on {} due to column masking policy {}",
> privilege, authorizable.getName(), columnMaskResult.getPolicyId());
> accessResult.setIsAllowed(false);
> accessResult.setPolicyId(columnMaskResult.getPolicyId());
> accessResult.setReason("User does not have access to unmasked column
> values");
> break;{code}
> [https://github.com/apache/impala/blob/00c233cc4fc25d23fc8a7e2f1efdf2d85c29f653/fe/src/main/java/org/apache/impala/authorization/ranger/RangerAuthorizationChecker.java#L737-L747]
> This is inefficient for wide tables. It also requires table metadata is
> loaded to get the correct column list (IMPALA-11281), which introduces a
> performance regression for INVALIDATE and REFRESH on unloaded tables. See
> IMPALA-11501.
> We can consider using table level request with scope
> RangerAccessRequest.ResourceMatchingScope.SELF_OR_DESCENDANTS to find any
> column masking policies. This is the implementation in Hive:
> {code:java}
> RangerHiveResource tblResource = new
> RangerHiveResource(HiveObjectType.TABLE, resource.getDatabase(),
> resource.getTable());
> request.setHiveAccessType(HiveAccessType.SELECT); // filtering/masking
> policies are defined only for SELECT
> request.setResource(tblResource);
> ...
> // check if masking is enabled for any column in the table/view
> request.setResourceMatchingScope(RangerAccessRequest.ResourceMatchingScope.SELF_OR_DESCENDANTS);{code}
> [https://github.com/apache/ranger/blob/d48e3528eb0d5dca965e53bb4a75f18f3b2d24a2/hive-agent/src/main/java/org/apache/ranger/authorization/hive/authorizer/RangerHiveAuthorizer.java#L1028]
> With this, INVALIDATE and REFRESH don't need to trigger metadata loading on
> unloaded tables to get the column list.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
