This is an automated email from the ASF dual-hosted git repository. tarmstrong pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/impala.git
commit 1b863132c6fbe1b45aabea9986653a9ec7817092 Author: Fang-Yu Rao <[email protected]> AuthorDate: Sat Oct 3 12:00:30 2020 -0700 IMPALA-10211 (Part 1): Add support for role-related statements This patch adds the support for the following role-related statements. 1. CREATE ROLE <role_name>. 2. DROP ROLE <role_name>. 3. GRANT ROLE <role_name> TO GROUP <group_name>. 4. REVOKE ROLE <role_name> FROM GROUP <group_name>. 5. GRANT <privilege> ON <resource> TO ROLE <role_name>. 6. REVOKE <privilege> ON <resource> FROM ROLE <role_name>. 7. SHOW GRANT ROLE <role_name> ON <resource>. 8. SHOW ROLES. 9. SHOW CURRENT ROLES. 10. SHOW ROLE GRANT GROUP <group_name>. To support the first 4 statements, we implemented the methods of createRole()/dropRole(), and grantRoleToGroup()/revokeRoleFromGroup() with their respective API calls provided by Ranger. To support the 5th and 6th statements, we modified createGrantRevokeRequest() so that the cases in which the grantee or revokee is a role could be processed. We slightly extended getPrivileges() so as to include the case when the principal is a role for the 7th statement. For the last 3 statements, to make Impala's behavior consistent with that when Sentry was the authorization provider, we based our implementation on SentryImpaladAuthorizationManager#getRoles() at https://gerrit.cloudera.org/c/15833/8/fe/src/main/java/org/apache/impala/authorization/sentry/SentryImpaladAuthorizationManager.java, which was removed in IMPALA-9708 when we dropped the support for Sentry. To test the implemented functionalities, we based our test cases on those at https://gerrit.cloudera.org/c/15833/8/testdata/workloads/functional-query/queries/QueryTest/grant_revoke.test. We note that before our tests could be automatically run in a Kerberized environment (IMPALA-9360), in order to run the statements of CREATE/DROP ROLE <role_name>, GRANT/REVOKE ROLE <role_name> TO/FROM GROUP <group_name>, and SHOW ROLES, we revised security-applicationContext.xml, one of the files needed when the Ranger server is started, so that the corresponding API calls could be performed in a non-Kerberized environment. During the process of adding test cases to grant_revoke.test, we found the following differences in Impala's behavior between the case when Ranger is the authorization provider and that when Sentry is the authorization provider. Specifically, we have the following two major differences. 1. Before dropping a role in Ranger, we have to remove all the privileges granted to the role in advance, which is not the case when Sentry is the authorization provider. 2. The resource has to be specified for the statement of SHOW GRANT ROLE <role_name> ON <resource>, which is different when Sentry is the authorization provider. This could be partly due to the fact that there is no API provided by Ranger that allows Impala to directly retrieve the list of all privileges granted to a specified role. Due to the differences in Impala's behavior described above, we had to revise the test cases in grant_revoke.test accordingly. On the other hand, to include as many test cases that were in the original grant_revoke.test as possible, we had to explicitly add the test section of 'USER' to specify the connecting user to Impala for some queries that require the connecting user to be a Ranger administrator, e.g., CREATE/DROP ROLE <role_name> and GRANT/REVOKE <role_name> TO/FROM GROUP <group_name>. The user has to be 'admin' in the current grant_revoke.test, whereas it could be the default user 'getuser()' in the original grant_revoke.test because previously 'getuser()' was also a Sentry administrator. Moreover, for some test cases, we had to explicitly alter the owner of a resource in the original grant_revoke.test when we would like to prevent the original owner of the resource, e.g., the creator of the resource, from accessing the resource since the original grant_revoke.test was run without object ownership being taken into consideration. We also note that in this patch we added the decorator of @pytest.mark.execute_serially to each test in test_ranger.py since we have observed that in some cases, e.g., if we are only running the E2E tests in the Jenkins environment, some tests do not seem to be executed sequentially. Testing: - Briefly verified that the implemented statements work as expected in a Kerberized cluster. - Verified that test_ranger.py passes in a local development environment. - Verified that the patch passes the exhaustive tests in the DEBUG build. Change-Id: Ic2b204e62a1d8ae1932d955b4efc28be22202860 Reviewed-on: http://gerrit.cloudera.org:8080/16837 Reviewed-by: Quanlong Huang <[email protected]> Tested-by: Impala Public Jenkins <[email protected]> --- bin/create-test-configuration.sh | 17 + .../ranger/RangerCatalogdAuthorizationManager.java | 207 +++- .../ranger/RangerImpaladAuthorizationManager.java | 91 +- .../impala/authorization/ranger/RangerUtil.java | 37 + .../authorization/AuthorizationTestBase.java | 4 +- .../queries/QueryTest/grant_revoke.test | 1274 ++++++++++++++++++++ tests/authorization/test_ranger.py | 119 +- 7 files changed, 1687 insertions(+), 62 deletions(-) diff --git a/bin/create-test-configuration.sh b/bin/create-test-configuration.sh index 83d500b..642d310 100755 --- a/bin/create-test-configuration.sh +++ b/bin/create-test-configuration.sh @@ -235,6 +235,23 @@ cp -f "${RANGER_TEST_CONF_DIR}/ranger-admin-env-logdir.sh" "${RANGER_SERVER_CONF cp -f "${RANGER_TEST_CONF_DIR}/ranger-admin-env-piddir.sh" "${RANGER_SERVER_CONF_DIR}" cp -f "${RANGER_SERVER_CONFDIST_DIR}/security-applicationContext.xml" \ "${RANGER_SERVER_CONF_DIR}" + +# Prepend the following 5 URL's to the line starting with "<intercept-url pattern="/**"". +# Before the end-to-end tests could be performed in a Kerberized environment +# automatically, we need to allow the requests for the following links so that the +# statements like CREATE/DROP ROLE <role_name>, +# GRANT/REVOKE ROLE <role_name> TO/FROM GROUP <group_name>, and SHOW ROLES could work in a +# non-Kerberized environment. It is better to add the allowed links using sed than to use +# a hardcoded configuration file consisting of those links since some other configurations +# could change after CDP_BUILD_NUMBER is bumped up, e.g., the version of jquery. +sed -i '/<intercept-url pattern="\/\*\*"/i \ + <intercept-url pattern="/service/public/v2/api/roles/*" access="permitAll"/> \ + <intercept-url pattern="/service/public/v2/api/roles/name/*" access="permitAll"/> \ + <intercept-url pattern="/service/public/v2/api/roles/grant/*" access="permitAll"/> \ + <intercept-url pattern="/service/public/v2/api/roles/revoke/*" access="permitAll"/> \ + <intercept-url pattern="/service/public/v2/api/roles/names/*" access="permitAll"/>' \ +"${RANGER_SERVER_CONF_DIR}/security-applicationContext.xml" + if [[ -f "${POSTGRES_JDBC_DRIVER}" ]]; then cp -f "${POSTGRES_JDBC_DRIVER}" "${RANGER_SERVER_LIB_DIR}" else diff --git a/fe/src/main/java/org/apache/impala/authorization/ranger/RangerCatalogdAuthorizationManager.java b/fe/src/main/java/org/apache/impala/authorization/ranger/RangerCatalogdAuthorizationManager.java index 6f741ba..0fc7502 100644 --- a/fe/src/main/java/org/apache/impala/authorization/ranger/RangerCatalogdAuthorizationManager.java +++ b/fe/src/main/java/org/apache/impala/authorization/ranger/RangerCatalogdAuthorizationManager.java @@ -40,16 +40,22 @@ import org.apache.impala.thrift.TShowGrantPrincipalParams; import org.apache.impala.thrift.TShowRolesParams; import org.apache.impala.thrift.TShowRolesResult; import org.apache.impala.util.ClassUtil; +import org.apache.ranger.plugin.model.RangerRole; import org.apache.ranger.plugin.util.GrantRevokeRequest; +import org.apache.ranger.plugin.util.GrantRevokeRoleRequest; import org.slf4j.Logger; import org.slf4j.LoggerFactory; import java.util.ArrayList; import java.util.Collections; +import java.util.HashSet; import java.util.List; import java.util.Map; +import java.util.Set; import java.util.function.Function; import java.util.function.Supplier; +import java.util.regex.Pattern; +import java.util.regex.Matcher; /** * An implementation of {@link AuthorizationManager} for Catalogd using Ranger. @@ -76,13 +82,48 @@ public class RangerCatalogdAuthorizationManager implements AuthorizationManager @Override public void createRole(User requestingUser, TCreateDropRoleParams params, TDdlExecResponse response) throws ImpalaException { - throw new UnsupportedFeatureException("CREATE ROLE is not supported by Ranger."); + RangerRole role = new RangerRole(); + role.setName(params.getRole_name()); + role.setCreatedByUser(requestingUser.getShortName()); + + try { + plugin_.get().createRole(role, /*resultProcessor*/ null); + } catch (Exception e) { + LOG.error("Error creating role {} by user {} in Ranger.", params.getRole_name(), + requestingUser.getShortName()); + throw new InternalException("Error creating role " + params.getRole_name() + + " by user " + requestingUser.getShortName() + + " in Ranger. Ranger error message: " + e.getMessage()); + } + // Update the authorization refresh marker so that the Impalads can refresh their + // Ranger caches. + refreshAuthorization(response); } @Override public void dropRole(User requestingUser, TCreateDropRoleParams params, TDdlExecResponse response) throws ImpalaException { - throw new UnsupportedFeatureException("DROP ROLE is not supported by Ranger."); + try { + // We found that when a non-Ranger administrator is trying to remove a role that + // does not exist in Ranger, the error message returned from Ranger would indicate + // that the name of the role does not exist, which reveals the non-existence of the + // role. This should be considered a bug of Ranger. Before the issue is resolved, we + // always call RangerUtil#validateRangerAdmin(). + // TODO: Remove the call to validateRangerAdmin() after the bug is fixed in Ranger. + // RANGER-3125 has been created to keep track of the issue. + RangerUtil.validateRangerAdmin(plugin_.get(), requestingUser.getShortName()); + plugin_.get().dropRole(requestingUser.getShortName(), params.getRole_name(), + /*resultProcessor*/ null); + } catch (Exception e) { + LOG.error("Error dropping role {} by user {} in Ranger.", params.getRole_name(), + requestingUser.getShortName()); + throw new InternalException("Error dropping role " + params.getRole_name() + + " by user " + requestingUser.getShortName() + + " in Ranger. Ranger error message: " + e.getMessage()); + } + // Update the authorization refresh marker so that the Impalads can refresh their + // Ranger caches. + refreshAuthorization(response); } @Override @@ -94,37 +135,125 @@ public class RangerCatalogdAuthorizationManager implements AuthorizationManager @Override public void grantRoleToGroup(User requestingUser, TGrantRevokeRoleParams params, TDdlExecResponse response) throws ImpalaException { - throw new UnsupportedFeatureException( - "GRANT ROLE TO GROUP is not supported by Ranger."); + GrantRevokeRoleRequest request = createGrantRevokeRoleRequest( + requestingUser.getShortName(), new HashSet<>(params.getRole_names()), + new HashSet<>(params.getGroup_names())); + + try { + // We found that granting a role to a group that is already assigned the role would + // actually revoke the role from the group. This should be considered a bug of + // Ranger. In this regard, as a workaround we always revoke the role from the group + // first whether or not the role has been granted to the group. An alternative to + // this solution is to call plugin_.get().getRolesFromUserAndGroups() to retrieve + // the roles currently granted to the group and only call grantRole() when those + // roles do not include the role of params.getRole_names().get(0), which is the + // role to be granted to the group. But since there is no guarantee that the result + // from getRolesFromUserAndGroups() is always up-to-date, we decide to call + // revokeRole() in any case before the bug is fixed. + // TODO: Remove the call to revokeRole() after the bug of Ranger is fixed. + // RANGER-3126 has been created to keep track of the issue. + plugin_.get().revokeRole(request, /*resultProcessor*/ null); + plugin_.get().grantRole(request, /*resultProcessor*/ null); + } catch (Exception e) { + Pattern pattern = Pattern.compile(".*doesn't have permissions.*"); + Matcher matcher = pattern.matcher(e.getMessage()); + if (matcher.matches()) { + // To avoid confusion, we do not use the error message from Ranger directly when + // the grantor does not have the necessary permissions, since in the case when + // the grantor does not have permissions to grant role, the error message from + // Ranger would start with "User doesn't have permissions to revoke role" due to + // the fact that we call revokeRole() first. + // We note that we will also get this message when a Ranger administrator is + // trying to grant a non-existing role to a group whether or not this group + // exists. + LOG.error("Error granting role {} to group {} by user {} in Ranger. " + + "Ranger error message: HTTP 400 Error: User doesn't have permissions to " + + "grant role " + params.getRole_names().get(0), params.getRole_names().get(0), + params.getGroup_names().get(0), requestingUser.getShortName()); + throw new InternalException("Error granting role " + + params.getRole_names().get(0) + " to group " + + params.getGroup_names().get(0) + " by user " + + requestingUser.getShortName() + " in Ranger. " + + "Ranger error message: HTTP 400 Error: User doesn't have permissions to " + + "grant role " + params.getRole_names().get(0)); + } else { + // When a Ranger administrator tries to grant an existing role to a non-existing + // group, we will get this error. + LOG.error("Error granting role {} to group {} by user {} in Ranger. " + + "Ranger error message: " + e.getMessage(), params.getRole_names().get(0), + params.getGroup_names().get(0), requestingUser.getShortName()); + throw new InternalException("Error granting role " + + params.getRole_names().get(0) + " to group " + + params.getGroup_names().get(0) + " by user " + + requestingUser.getShortName() + " in Ranger. " + + "Ranger error message: " + e.getMessage()); + } + } + // Update the authorization refresh marker so that the Impalads can refresh their + // Ranger caches. + refreshAuthorization(response); } @Override public void revokeRoleFromGroup(User requestingUser, TGrantRevokeRoleParams params, TDdlExecResponse response) throws ImpalaException { - throw new UnsupportedFeatureException( - "REVOKE ROLE FROM GROUP is not supported by Ranger."); + GrantRevokeRoleRequest request = createGrantRevokeRoleRequest( + requestingUser.getShortName(), new HashSet<>(params.getRole_names()), + new HashSet<>(params.getGroup_names())); + + try { + plugin_.get().revokeRole(request, /*resultProcessor*/ null); + } catch (Exception e) { + LOG.error("Error revoking role {} from group {} by user {} in Ranger. " + + "Ranger error message: " + e.getMessage(), params.getRole_names().get(0), + params.getGroup_names().get(0), requestingUser.getShortName()); + throw new InternalException("Error revoking role " + + params.getRole_names().get(0) + " from group " + + params.getGroup_names().get(0) + " by user " + + requestingUser.getShortName() + " in Ranger. " + + "Ranger error message: " + e.getMessage()); + } + // Update the authorization refresh marker so that the Impalads can refresh their + // Ranger caches. + refreshAuthorization(response); } @Override public void grantPrivilegeToRole(TCatalogServiceRequestHeader header, TGrantRevokePrivParams params, TDdlExecResponse response) throws ImpalaException { - throw new UnsupportedFeatureException( - "GRANT <privilege> TO ROLE is not supported by Ranger."); + List<GrantRevokeRequest> requests = createGrantRevokeRequests( + new User(header.getRequesting_user()).getShortName(), /*isGrant*/ true, + /*user*/ null, Collections.emptyList(), + Collections.singletonList(params.getPrincipal_name()), + plugin_.get().getClusterName(), header.getClient_ip(), params.getPrivileges()); + + grantPrivilege(requests, header.getRedacted_sql_stmt(), header.getClient_ip()); + // Update the authorization refresh marker so that the Impalads can refresh their + // Ranger caches. + refreshAuthorization(response); } @Override public void revokePrivilegeFromRole(TCatalogServiceRequestHeader header, TGrantRevokePrivParams params, TDdlExecResponse response) throws ImpalaException { - throw new UnsupportedFeatureException( - "REVOKE <privilege> FROM ROLE is not supported by Ranger."); + List<GrantRevokeRequest> requests = createGrantRevokeRequests( + new User(header.getRequesting_user()).getShortName(), /*isGrant*/ false, + /*user*/ null, Collections.emptyList(), + Collections.singletonList(params.getPrincipal_name()), + plugin_.get().getClusterName(), header.getClient_ip(), params.getPrivileges()); + + revokePrivilege(requests, header.getRedacted_sql_stmt(), header.getClient_ip()); + // Update the authorization refresh marker so that the Impalads can refresh their + // Ranger caches. + refreshAuthorization(response); } @Override public void grantPrivilegeToUser(TCatalogServiceRequestHeader header, TGrantRevokePrivParams params, TDdlExecResponse response) throws ImpalaException { List<GrantRevokeRequest> requests = createGrantRevokeRequests( - new User(header.getRequesting_user()).getShortName(), true, - params.getPrincipal_name(), Collections.emptyList(), + new User(header.getRequesting_user()).getShortName(), /*isGrant*/ true, + params.getPrincipal_name(), Collections.emptyList(), Collections.emptyList(), plugin_.get().getClusterName(), header.getClient_ip(), params.getPrivileges()); grantPrivilege(requests, header.getRedacted_sql_stmt(), header.getClient_ip()); @@ -135,8 +264,8 @@ public class RangerCatalogdAuthorizationManager implements AuthorizationManager public void revokePrivilegeFromUser(TCatalogServiceRequestHeader header, TGrantRevokePrivParams params, TDdlExecResponse response) throws ImpalaException { List<GrantRevokeRequest> requests = createGrantRevokeRequests( - new User(header.getRequesting_user()).getShortName(), false, - params.getPrincipal_name(), Collections.emptyList(), + new User(header.getRequesting_user()).getShortName(), /*isGrant*/ false, + params.getPrincipal_name(), Collections.emptyList(), Collections.emptyList(), plugin_.get().getClusterName(), header.getClient_ip(), params.getPrivileges()); revokePrivilege(requests, header.getRedacted_sql_stmt(), header.getClient_ip()); @@ -147,9 +276,10 @@ public class RangerCatalogdAuthorizationManager implements AuthorizationManager public void grantPrivilegeToGroup(TCatalogServiceRequestHeader header, TGrantRevokePrivParams params, TDdlExecResponse response) throws ImpalaException { List<GrantRevokeRequest> requests = createGrantRevokeRequests( - new User(header.getRequesting_user()).getShortName(), true, null, - Collections.singletonList(params.getPrincipal_name()), - plugin_.get().getClusterName(), header.getClient_ip(), params.getPrivileges()); + new User(header.getRequesting_user()).getShortName(), /*isGrant*/ true, + /*user*/ null, Collections.singletonList(params.getPrincipal_name()), + Collections.emptyList(), plugin_.get().getClusterName(), header.getClient_ip(), + params.getPrivileges()); grantPrivilege(requests, header.getRedacted_sql_stmt(), header.getClient_ip()); refreshAuthorization(response); @@ -159,9 +289,10 @@ public class RangerCatalogdAuthorizationManager implements AuthorizationManager public void revokePrivilegeFromGroup(TCatalogServiceRequestHeader header, TGrantRevokePrivParams params, TDdlExecResponse response) throws ImpalaException { List<GrantRevokeRequest> requests = createGrantRevokeRequests( - new User(header.getRequesting_user()).getShortName(), false, null, - Collections.singletonList(params.getPrincipal_name()), - plugin_.get().getClusterName(), header.getClient_ip(), params.getPrivileges()); + new User(header.getRequesting_user()).getShortName(), /*isGrant*/ false, + /*user*/ null, Collections.singletonList(params.getPrincipal_name()), + Collections.emptyList(), plugin_.get().getClusterName(), header.getClient_ip(), + params.getPrivileges()); revokePrivilege(requests, header.getRedacted_sql_stmt(), header.getClient_ip()); // Update the authorization refresh marker so that the Impalads can refresh their @@ -243,14 +374,14 @@ public class RangerCatalogdAuthorizationManager implements AuthorizationManager } public static List<GrantRevokeRequest> createGrantRevokeRequests(String grantor, - boolean isGrant, String user, List<String> groups, String clusterName, - String clientIp, List<TPrivilege> privileges) { + boolean isGrant, String user, List<String> groups, List<String> roles, + String clusterName, String clientIp, List<TPrivilege> privileges) { List<GrantRevokeRequest> requests = new ArrayList<>(); for (TPrivilege p: privileges) { Function<Map<String, String>, GrantRevokeRequest> createRequest = (resource) -> - createGrantRevokeRequest(grantor, user, groups, clusterName, p.has_grant_opt, - isGrant, p.privilege_level, resource, clientIp); + createGrantRevokeRequest(grantor, user, groups, roles, clusterName, + p.has_grant_opt, isGrant, p.privilege_level, resource, clientIp); // Ranger Impala service definition defines 3 resources: // [DB -> Table -> Column] @@ -279,12 +410,18 @@ public class RangerCatalogdAuthorizationManager implements AuthorizationManager } private static GrantRevokeRequest createGrantRevokeRequest(String grantor, String user, - List<String> groups, String clusterName, boolean withGrantOpt, boolean isGrant, - TPrivilegeLevel level, Map<String, String> resource, String clientIp) { + List<String> groups, List<String> roles, String clusterName, boolean withGrantOpt, + boolean isGrant, TPrivilegeLevel level, Map<String, String> resource, + String clientIp) { GrantRevokeRequest request = new GrantRevokeRequest(); request.setGrantor(grantor); + // In a Kerberized environment, we also need to call setGrantorGroups() to provide + // Ranger with the groups 'grantor' belongs to even though it is not required in a + // non-Kerberized environment. + request.setGrantorGroups(RangerUtil.getGroups(grantor)); if (user != null) request.getUsers().add(user); if (!groups.isEmpty()) request.getGroups().addAll(groups); + if (!roles.isEmpty()) request.getRoles().addAll(roles); request.setDelegateAdmin(isGrant && withGrantOpt); request.setEnableAudit(Boolean.TRUE); request.setReplaceExistingPermissions(Boolean.FALSE); @@ -303,4 +440,22 @@ public class RangerCatalogdAuthorizationManager implements AuthorizationManager return request; } + + /** + * The caller of this method calls Ranger's REST API to grant/revoke roles + * corresponding to 'targetRoleNames' to/from groups associated with 'groupNames'. + */ + private static GrantRevokeRoleRequest createGrantRevokeRoleRequest( + String grantor, Set<String> targetRoleNames, Set<String> groupNames) { + GrantRevokeRoleRequest request = new GrantRevokeRoleRequest(); + request.setGrantor(grantor); + request.setTargetRoles(targetRoleNames); + request.setGroups(groupNames); + // We do not set the field of 'grantOption' since WITH GRANT OPTION is not supported + // when granting/revoking roles. By default, 'grantOption' is set to Boolean.FALSE so + // that a user in a group assigned a role is not able to grant/revoke the role to/from + // other groups. + + return request; + } } diff --git a/fe/src/main/java/org/apache/impala/authorization/ranger/RangerImpaladAuthorizationManager.java b/fe/src/main/java/org/apache/impala/authorization/ranger/RangerImpaladAuthorizationManager.java index fc1faad..42f2be2 100644 --- a/fe/src/main/java/org/apache/impala/authorization/ranger/RangerImpaladAuthorizationManager.java +++ b/fe/src/main/java/org/apache/impala/authorization/ranger/RangerImpaladAuthorizationManager.java @@ -17,14 +17,16 @@ package org.apache.impala.authorization.ranger; +import com.google.common.base.Preconditions; +import com.google.common.collect.Lists; import com.google.common.collect.Sets; import org.apache.hadoop.hive.metastore.api.PrincipalType; -import org.apache.hadoop.security.UserGroupInformation; import org.apache.impala.authorization.AuthorizationDelta; import org.apache.impala.authorization.AuthorizationManager; import org.apache.impala.authorization.User; import org.apache.impala.catalog.Type; import org.apache.impala.common.ImpalaException; +import org.apache.impala.common.InternalException; import org.apache.impala.common.UnsupportedFeatureException; import org.apache.impala.thrift.TCatalogServiceRequestHeader; import org.apache.impala.thrift.TColumn; @@ -50,7 +52,9 @@ import org.apache.ranger.plugin.policyengine.RangerAccessResourceImpl; import org.apache.ranger.plugin.policyengine.RangerPolicyEngine; import org.apache.ranger.plugin.policyengine.RangerResourceACLs; import org.apache.ranger.plugin.policyengine.RangerResourceACLs.AccessResult; -import org.apache.ranger.plugin.service.RangerAuthContext; +import org.apache.ranger.plugin.model.RangerRole; +import org.slf4j.Logger; +import org.slf4j.LoggerFactory; import java.util.ArrayList; import java.util.Collections; @@ -73,6 +77,8 @@ import java.util.stream.Collectors; * Operations not supported by Ranger will throw an {@link UnsupportedFeatureException}. */ public class RangerImpaladAuthorizationManager implements AuthorizationManager { + private static final Logger LOG = LoggerFactory.getLogger( + RangerImpaladAuthorizationManager.class); private static final String ANY = "*"; private final Supplier<RangerImpalaPlugin> plugin_; @@ -95,15 +101,62 @@ public class RangerImpaladAuthorizationManager implements AuthorizationManager { "%s is not supported in Impalad", ClassUtil.getMethodName())); } + /** + * This method will be called for the statements of 1) SHOW ROLES, + * 2) SHOW CURRENT ROLES, or 3) SHOW ROLE GRANT GROUP <group_name>. + */ @Override public TShowRolesResult getRoles(TShowRolesParams params) throws ImpalaException { - if (params.getGrant_group() != null) { - throw new UnsupportedFeatureException( - "SHOW ROLE GRANT GROUP is not supported by Ranger."); + try { + TShowRolesResult result = new TShowRolesResult(); + Set<String> groups = RangerUtil.getGroups(params.getRequesting_user()); + + boolean adminOp = + !(groups.contains(params.getGrant_group()) || params.is_show_current_roles); + + if (adminOp) { + RangerUtil.validateRangerAdmin(plugin_.get(), params.getRequesting_user()); + } + + // The branch for SHOW CURRENT ROLES and SHOW ROLE GRANT GROUP. + Set<String> roleNames; + if (params.isIs_show_current_roles() || params.isSetGrant_group()) { + Set<String> groupNames; + if (params.isIs_show_current_roles()) { + groupNames = groups; + } else { + Preconditions.checkState(params.isSetGrant_group()); + groupNames = Sets.newHashSet(params.getGrant_group()); + } + roleNames = plugin_.get().getRolesFromUserAndGroups(null, groupNames); + } else { + // The branch for SHOW ROLES. + Preconditions.checkState(!params.isIs_show_current_roles()); + Set<RangerRole> roles = plugin_.get().getRoles().getRangerRoles(); + roleNames = roles.stream().map(RangerRole::getName).collect(Collectors.toSet()); + } + + // Need to instantiate the field of 'role_names' in 'result' since its field of + // 'role_names' was initialized as null by default. + result.setRole_names(Lists.newArrayList(roleNames)); + Collections.sort(result.getRole_names()); + return result; + } catch (Exception e) { + if (params.is_show_current_roles) { + LOG.error("Error executing SHOW CURRENT ROLES.", e); + throw new InternalException("Error executing SHOW CURRENT ROLES." + + " Ranger error message: " + e.getMessage()); + } else if (params.isSetGrant_group()) { + LOG.error("Error executing SHOW ROLE GRANT GROUP " + params.getGrant_group() + + "."); + throw new InternalException("Error executing SHOW ROLE GRANT GROUP " + + params.getGrant_group() + ". Ranger error message: " + e.getMessage()); + } else { + LOG.error("Error executing SHOW ROLES."); + throw new InternalException("Error executing SHOW ROLES." + + " Ranger error message: " + e.getMessage()); + } } - throw new UnsupportedFeatureException( - String.format("SHOW %sROLES is not supported by Ranger.", - params.is_show_current_roles ? "CURRENT " : "")); } @Override @@ -162,11 +215,6 @@ public class RangerImpaladAuthorizationManager implements AuthorizationManager { "%s is not supported in Impalad", ClassUtil.getMethodName())); } - private static Set<String> getGroups(String principal) { - UserGroupInformation ugi = UserGroupInformation.createRemoteUser(principal); - return Sets.newHashSet(ugi.getGroupNames()); - } - private static Optional<String> getResourceName(String resourceType, String resourceName, AccessResult accessResult) { RangerPolicy.RangerPolicyResource rangerPolicyResource = @@ -196,6 +244,11 @@ public class RangerImpaladAuthorizationManager implements AuthorizationManager { return item.getDelegateAdmin(); } break; + case ROLE: + if (item.getRoles().contains(principal)) { + return item.getDelegateAdmin(); + } + break; default: throw new UnsupportedOperationException(String.format("Unsupported principal " + "type %s", type)); @@ -286,11 +339,6 @@ public class RangerImpaladAuthorizationManager implements AuthorizationManager { @Override public TResultSet getPrivileges(TShowGrantPrincipalParams params) throws ImpalaException { - if (params.principal_type == TPrincipalType.ROLE) { - throw new UnsupportedFeatureException( - "SHOW GRANT ROLE is not supported by Ranger."); - } - List<RangerAccessRequest> requests = buildAccessRequests(params.privilege); Set<TResultRow> resultSet = new TreeSet<>(); TResultSet result = new TResultSet(); @@ -307,7 +355,7 @@ public class RangerImpaladAuthorizationManager implements AuthorizationManager { resultRows = new ArrayList<>(aclToPrivilege( acls.getUserACLs().getOrDefault(params.name, Collections.emptyMap()), params.name, params.privilege, TPrincipalType.USER)); - for (String group : getGroups(params.name)) { + for (String group : RangerUtil.getGroups(params.name)) { resultRows.addAll(aclToPrivilege( acls.getGroupACLs().getOrDefault(group, Collections.emptyMap()), params.name, params.privilege, TPrincipalType.GROUP)); @@ -318,6 +366,11 @@ public class RangerImpaladAuthorizationManager implements AuthorizationManager { acls.getGroupACLs().getOrDefault(params.name, Collections.emptyMap()), params.name, params.privilege, TPrincipalType.GROUP)); break; + case ROLE: + resultRows = new ArrayList<>(aclToPrivilege( + acls.getRoleACLs().getOrDefault(params.name, Collections.emptyMap()), + params.name, params.privilege, TPrincipalType.ROLE)); + break; default: throw new UnsupportedOperationException(String.format("Unsupported principal " + "type %s.", params.principal_type)); diff --git a/fe/src/main/java/org/apache/impala/authorization/ranger/RangerUtil.java b/fe/src/main/java/org/apache/impala/authorization/ranger/RangerUtil.java index 2fa61d9..9abccf0 100644 --- a/fe/src/main/java/org/apache/impala/authorization/ranger/RangerUtil.java +++ b/fe/src/main/java/org/apache/impala/authorization/ranger/RangerUtil.java @@ -17,10 +17,15 @@ package org.apache.impala.authorization.ranger; +import com.google.common.collect.Sets; +import org.apache.hadoop.security.UserGroupInformation; +import org.apache.impala.common.RuntimeEnv; +import org.apache.impala.service.BackendConfig; import org.apache.impala.thrift.TPrivilege; import java.util.HashMap; import java.util.Map; +import java.util.Set; /** * Collection of static functions to support Apache Ranger implementation @@ -70,4 +75,36 @@ public class RangerUtil { private static String getOrAll(String resource) { return (resource == null) ? "*" : resource; } + + /** + * This method returns the groups that 'user' belongs to. By starting impalad and + * catalogd with the argument of "use_customized_user_groups_mapper_for_ranger", + * the customized user-to-groups mapper would be provided, which is useful in the + * testing environment. + */ + public static Set<String> getGroups(String user) { + UserGroupInformation ugi; + if (RuntimeEnv.INSTANCE.isTestEnv() || + BackendConfig.INSTANCE.useCustomizedUserGroupsMapperForRanger()) { + ugi = UserGroupInformation.createUserForTesting(user, + new String[]{user}); + } else { + ugi = UserGroupInformation.createRemoteUser(user); + } + return Sets.newHashSet(ugi.getGroupNames()); + } + + /** + * For now there is no dedicated REST API that allows Impala to determine whether + * 'user' is a Ranger administrator. In this regard, we call + * RangerBasePlugin#getAllRoles(), whose server side method, + * RoleREST#getAllRoleNames(), will call RoleREST#ensureAdminAccess() on 'user' to make + * sure 'user' is a Ranger administrator. An Exception will be thrown if it is not the + * case. + * RANGER-3127 has been created to keep track of the issue. + */ + public static void validateRangerAdmin(RangerImpalaPlugin plugin, String user) + throws Exception { + plugin.getAllRoles(user, null); + } } diff --git a/fe/src/test/java/org/apache/impala/authorization/AuthorizationTestBase.java b/fe/src/test/java/org/apache/impala/authorization/AuthorizationTestBase.java index 523330b..45db49b 100644 --- a/fe/src/test/java/org/apache/impala/authorization/AuthorizationTestBase.java +++ b/fe/src/test/java/org/apache/impala/authorization/AuthorizationTestBase.java @@ -204,7 +204,7 @@ public abstract class AuthorizationTestBase extends FrontendTestBase { // to whether or not we test the query with the requesting user that is the // owner of the resource. getName(), - Collections.emptyList(), + Collections.emptyList(), Collections.emptyList(), rangerImpalaPlugin_.getClusterName(), "127.0.0.1", privileges); } } @@ -217,7 +217,7 @@ public abstract class AuthorizationTestBase extends FrontendTestBase { // We provide the name of the grantee, which is a group in this case, according // to whether or not we test the query with the requesting user that is the // owner of the resource. - (as_owner_ ? OWNER_GROUPS : GROUPS), + (as_owner_ ? OWNER_GROUPS : GROUPS), Collections.emptyList(), // groups, rangerImpalaPlugin_.getClusterName(), "127.0.0.1", privileges); } diff --git a/testdata/workloads/functional-query/queries/QueryTest/grant_revoke.test b/testdata/workloads/functional-query/queries/QueryTest/grant_revoke.test new file mode 100644 index 0000000..c5fbdae --- /dev/null +++ b/testdata/workloads/functional-query/queries/QueryTest/grant_revoke.test @@ -0,0 +1,1274 @@ +==== +---- USER +admin +---- QUERY +create role grant_revoke_test_ALL_SERVER +---- RESULTS +'Role has been created.' +==== +---- USER +admin +---- QUERY +create role grant_revoke_test_ALL_TEST_DB +---- RESULTS +'Role has been created.' +==== +---- USER +admin +---- QUERY +create role grant_revoke_test_SELECT_INSERT_TEST_TBL +---- RESULTS +'Role has been created.' +==== +---- USER +admin +---- QUERY +create role grant_revoke_test_ALL_URI +---- RESULTS +'Role has been created.' +==== +---- USER +admin +---- QUERY +refresh authorization; +show roles; +---- RESULTS: VERIFY_IS_SUBSET +'grant_revoke_test_ALL_SERVER' +'grant_revoke_test_ALL_TEST_DB' +'grant_revoke_test_SELECT_INSERT_TEST_TBL' +'grant_revoke_test_ALL_URI' +==== +---- QUERY +create database grant_rev_db location '$FILESYSTEM_PREFIX/test-warehouse/grant_rev_db.db' +---- CATCH +does not have privileges to execute 'CREATE' on: grant_rev_db +==== +---- USER +admin +---- QUERY +grant all on server to grant_revoke_test_ALL_SERVER +==== +---- USER +admin +---- QUERY +# Grant the privilege to the group `$GROUP_NAME`, to which the user 'getuser()' belongs +# so that the user 'getuser()' could perform the some of the following operations on the +# database 'grant_rev_db'. +# Group name will be replaced with the group of 'getuser()' in the test. +# framework. +grant role grant_revoke_test_ALL_SERVER to group `$GROUP_NAME` +==== +---- USER +admin +---- QUERY +refresh authorization +==== +---- QUERY +show current roles +---- RESULTS: VERIFY_IS_SUBSET +'grant_revoke_test_ALL_SERVER' +---- TYPES +STRING +==== +---- USER +does_not_exist +---- QUERY +# Run this query as a different user and verify no roles show up but the +# stmt does not fail with an authorization error. +show current roles +---- RESULTS: VERIFY_IS_SUBSET +---- TYPES +STRING +==== +---- QUERY +# We have to specify the resource for SHOW GRANT since SHOW GRANT is not supported +# without a defined resource in Ranger. +show grant role grant_revoke_test_ALL_SERVER on server +---- RESULTS: VERIFY_IS_EQUAL_SORTED +'ROLE','grant_revoke_test_ALL_SERVER','','','','*','','all',false,regex:.+ +'ROLE','grant_revoke_test_ALL_SERVER','*','','','','*','all',false,regex:.+ +'ROLE','grant_revoke_test_ALL_SERVER','*','*','*','','','all',false,regex:.+ +---- LABELS +principal_type, principal_name, database, table, column, uri, udf, privilege, grant_option, create_time +---- TYPES +STRING, STRING, STRING, STRING, STRING, STRING, STRING, STRING, BOOLEAN, STRING +==== +---- USER +does_not_exist +---- QUERY +# A user that is not a Ranger administrator should not have privileges to execute +# SHOW ROLES +show roles +---- RESULTS: VERIFY_IS_SUBSET +---- TYPES +STRING +---- CATCH +User does_not_exist does not have permission for this operation +==== +---- USER +does_not_exist +---- QUERY +# A user that is not a Ranger administrator should not have privilege to execute +# SHOW ROLE GRANT GROUP for a group the user does not belong to. +show role grant group non_owner +---- RESULTS: VERIFY_IS_SUBSET +---- TYPES +STRING +---- CATCH +User does_not_exist does not have permission for this operation +==== +---- USER +non_owner +---- QUERY +# The 'non_owner' user doesn't have any roles granted to them, but since it is part of the +# 'non_owner' group, they should have privileges to execute this statement. +# Note that this test case requires that impalad was started with the argument +# '--use_customized_user_groups_mapper_for_ranger' so that a customized user-to-groups +# mapper will be used to retrieve the groups the user 'non_owner' belongs to, which +# consists of exactly one group, the group 'non_owner'. +show role grant group non_owner +---- RESULTS: VERIFY_IS_SUBSET +---- TYPES +STRING +==== +---- QUERY +drop database if exists grant_rev_db +==== +---- QUERY +create database grant_rev_db location '$FILESYSTEM_PREFIX/test-warehouse/grant_rev_db.db' +==== +---- QUERY +show tables in grant_rev_db +---- RESULTS +---- TYPES +STRING +==== +---- QUERY +create table grant_rev_db.test_tbl1(i int) +==== +---- QUERY +show tables in grant_rev_db +---- RESULTS +'test_tbl1' +---- TYPES +STRING +==== +---- QUERY +create function grant_rev_db.fn() RETURNS int +LOCATION '$FILESYSTEM_PREFIX/test-warehouse/libTestUdfs.so' SYMBOL='Fn' +==== +---- QUERY +show functions in grant_rev_db +---- RESULTS +'INT','fn()','NATIVE','true' +---- TYPES +STRING, STRING, STRING, STRING +==== +---- QUERY +show create function grant_rev_db.fn +---- RESULTS: MULTI_LINE +['CREATE FUNCTION grant_rev_db.fn() + RETURNS INT + LOCATION ''$NAMENODE/test-warehouse/libTestUdfs.so'' + SYMBOL=''_Z2FnPN10impala_udf15FunctionContextE'' +'] +---- TYPES +STRING +==== +---- USER +admin +---- QUERY +# To prevent the user 'getuser()' (which belongs to the group '$GROUP_NAME') from +# performing any operation on the database 'grant_rev_db', we also need to alter +# the owner of 'grant_rev_db' since 'getuser()' is the owner of this database, which by +# default is allowed by Ranger to perform any operation on the database. +revoke role grant_revoke_test_ALL_SERVER from group `$GROUP_NAME`; +alter database grant_rev_db set owner user admin; +refresh authorization; +==== +---- QUERY +create database grant_rev_db location '$FILESYSTEM_PREFIX/test-warehouse/grant_rev_db.db' +---- CATCH +does not have privileges to execute 'CREATE' on: grant_rev_db +==== +---- QUERY +show tables in grant_rev_db +---- CATCH +does not have privileges to access: grant_rev_db.* +==== +---- QUERY +show functions in grant_rev_db +---- CATCH +does not have privileges to access: grant_rev_db +==== +---- QUERY +show create function grant_rev_db.fn +---- CATCH +does not have privileges to access: grant_rev_db +==== +---- QUERY +show create function _impala_builtins.sin +---- RESULTS: MULTI_LINE +['CREATE FUNCTION _impala_builtins.sin(DOUBLE) + RETURNS DOUBLE + SYMBOL=''_ZN6impala13MathFunctions3SinEPN10impala_udf15FunctionContextERKNS1_9DoubleValE'' +'] +---- TYPES +STRING +==== +---- USER +admin +---- QUERY +grant role grant_revoke_test_ALL_TEST_DB to group `$GROUP_NAME` +==== +---- USER +admin +---- QUERY +# Should now have all privileges on the test db +grant all on database grant_rev_db to grant_revoke_test_ALL_TEST_DB +==== +---- QUERY +show tables in grant_rev_db +---- RESULTS +'test_tbl1' +---- TYPES +STRING +==== +---- QUERY +# Even though the user has all privileges on the database, they do not have privileges +# on any URIs. The FE tests have additional error message verification. +create table grant_rev_db.test_tbl2(i int) location '$FILESYSTEM_PREFIX/test-warehouse/grant_rev_test_tbl2'; +---- CATCH +does not have privileges to access: $NAMENODE/test-warehouse/grant_rev_test_tbl2 +==== +---- USER +admin +---- QUERY +grant role grant_revoke_test_ALL_URI to group `$GROUP_NAME` +==== +---- USER +admin +---- QUERY +grant all on uri '$FILESYSTEM_PREFIX/test-warehouse/grant_rev_test_tbl2' to grant_revoke_test_ALL_URI +==== +---- QUERY +# Should now have privileges to create the table. +create table grant_rev_db.test_tbl2(i int) location '$FILESYSTEM_PREFIX/test-warehouse/grant_rev_test_tbl2'; +==== +---- USER +admin +---- QUERY +# Running grant on a URI with upper case letters +grant all on uri '$FILESYSTEM_PREFIX/test-warehouse/GRANT_REV_TEST_TBL3' to grant_revoke_test_ALL_URI +==== +---- QUERY +# Should now have privileges to create the table. +create table grant_rev_db.test_tbl_uppercase(i int) location '$FILESYSTEM_PREFIX/test-warehouse/GRANT_REV_TEST_TBL3'; +==== +---- QUERY +# Privileges on the uri '$FILESYSTEM_PREFIX/test-warehouse/GRANT_REV_TEST_TBL3' do not +# imply privileges on any location under +# '$FILESYSTEM_PREFIX/test-warehouse/GRANT_REV_TEST_TBL3'. +create table grant_rev_db.test_tbl2_uppercase(i int) location '$FILESYSTEM_PREFIX/test-warehouse/GRANT_REV_TEST_TBL3/test'; +---- CATCH +does not have privileges to access: $NAMENODE/test-warehouse/GRANT_REV_TEST_TBL3/test +==== +---- QUERY +show tables in grant_rev_db +---- RESULTS +'test_tbl1' +'test_tbl2' +'test_tbl_uppercase' +---- TYPES +STRING +==== +---- QUERY +# IMPALA-1670: User does not have privileges to access URI when adding partitions +create table grant_rev_db.test_tbl_partitioned(i int) partitioned by (j int); +alter table grant_rev_db.test_tbl_partitioned add +partition (j=1) +partition (j=2) location '$FILESYSTEM_PREFIX/test-warehouse/grant_rev_test_prt'; +---- CATCH +does not have privileges to access: $NAMENODE/test-warehouse/grant_rev_test_prt +==== +---- USER +admin +---- QUERY +grant all on uri '$FILESYSTEM_PREFIX/test-warehouse/grant_rev_test_prt' +to grant_revoke_test_ALL_URI; +==== +---- QUERY +# Should now have privileges to add partitions +alter table grant_rev_db.test_tbl_partitioned add +partition (j=1) +partition (j=2) location '$FILESYSTEM_PREFIX/test-warehouse/grant_rev_test_prt'; +show partitions grant_rev_db.test_tbl_partitioned; +---- RESULTS +'1',-1,0,'0B','NOT CACHED','NOT CACHED','TEXT','false',regex:.*/j=1 +'2',-1,0,'0B','NOT CACHED','NOT CACHED','TEXT','false','$NAMENODE/test-warehouse/grant_rev_test_prt' +'Total',-1,0,'0B','0B','','','','' +---- TYPES +STRING, BIGINT, BIGINT, STRING, STRING, STRING, STRING, STRING, STRING +==== +---- QUERY +# We have to specify the resource for SHOW GRANT since SHOW GRANT is not supported +# without a defined resource in Ranger. +show grant role grant_revoke_test_ALL_URI on uri '$NAMENODE/test-warehouse/grant_rev_test_tbl2' +---- RESULTS +'ROLE','grant_revoke_test_ALL_URI','','','','$NAMENODE/test-warehouse/grant_rev_test_tbl2','','all',false,regex:.+ +---- LABELS +principal_type, principal_name, database, table, column, uri, udf, privilege, grant_option, create_time +---- TYPES +STRING, STRING, STRING, STRING, STRING, STRING, STRING, STRING, BOOLEAN, STRING +==== +---- QUERY +# We have to specify the resource for SHOW GRANT since SHOW GRANT is not supported +# without a defined resource in Ranger. +show grant role grant_revoke_test_ALL_URI on uri '$NAMENODE/test-warehouse/GRANT_REV_TEST_TBL3' +---- RESULTS +'ROLE','grant_revoke_test_ALL_URI','','','','$NAMENODE/test-warehouse/GRANT_REV_TEST_TBL3','','all',false,regex:.+ +---- LABELS +principal_type, principal_name, database, table, column, uri, udf, privilege, grant_option, create_time +---- TYPES +STRING, STRING, STRING, STRING, STRING, STRING, STRING, STRING, BOOLEAN, STRING +==== +---- QUERY +# We have to specify the resource for SHOW GRANT since SHOW GRANT is not supported +# without a defined resource in Ranger. +show grant role grant_revoke_test_ALL_URI on uri '$NAMENODE/test-warehouse/grant_rev_test_prt' +---- RESULTS +'ROLE','grant_revoke_test_ALL_URI','','','','$NAMENODE/test-warehouse/grant_rev_test_prt','','all',false,regex:.+ +---- LABELS +principal_type, principal_name, database, table, column, uri, udf, privilege, grant_option, create_time +---- TYPES +STRING, STRING, STRING, STRING, STRING, STRING, STRING, STRING, BOOLEAN, STRING +==== +---- QUERY +# Does not result in an AuthorizationException since '$GROUP_NAME' is still assigned the +# role 'grant_revoke_test_ALL_TEST_DB', which is granted the 'ALL' privilege on the +# database 'grant_rev_db' +# TODO(IMPALA-10401): Investigate whether the privilege on the provided uri should be +# verified. +create database grant_rev_db location '$FILESYSTEM_PREFIX/test-warehouse/grant_rev_db.db' +---- CATCH +Database already exists: grant_rev_db +==== +---- QUERY +# To create a database server-level privileges are required. +create database grant_rev_db2 location '$FILESYSTEM_PREFIX/test-warehouse/grant_rev_db2.db' +---- CATCH +does not have privileges to execute 'CREATE' on: grant_rev_db2 +==== +---- USER +admin +---- QUERY +# Currently dropping the role does not remove its associated privileges. +drop role grant_revoke_test_ALL_TEST_DB +---- CATCH +Role 'grant_revoke_test_ALL_TEST_DB' can not be deleted as it is referenced in one or more policies +==== +---- USER +admin +---- QUERY +# We have to manually remove the privilege associated with the role +# 'grant_revoke_test_ALL_TEST_DB' before removing it. +revoke all on database grant_rev_db from grant_revoke_test_ALL_TEST_DB; +drop role grant_revoke_test_ALL_TEST_DB; +==== +---- QUERY +# Recall that the owner of the database 'grant_rev_db' has been changed to 'admin'. +show tables in grant_rev_db +---- CATCH +does not have privileges to access: grant_rev_db.* +==== +---- QUERY +# The user 'getuser()' can select the data from the table 'grant_rev_db.test_tbl1' +# because 'getuser()' is still the owner of 'grant_rev_db.test_tbl1'. +select * from grant_rev_db.test_tbl1 +---- RESULTS +---- TYPES +INT +==== +---- USER +admin +---- QUERY +# Up to this point, the owner of the tables 'grant_rev_db.test_tbl1' and +# 'grant_rev_db.test_tbl1' is the user 'getuser()'. Changing the owner of these tables to +# the user 'admin' prevents 'getuser()' from accessing the data in those tables. +alter table grant_rev_db.test_tbl1 set owner user admin; +alter table grant_rev_db.test_tbl2 set owner user admin; +==== +---- QUERY +# After changing the owner of the table 'grant_rev_db.test_tbl1' to the user 'admin', +# the user 'getuser()' can no longer select the data from 'grant_rev_db.test_tbl1'. +select * from grant_rev_db.test_tbl1 +---- CATCH +does not have privileges to execute 'SELECT' on: grant_rev_db.test_tbl1 +==== +---- USER +admin +---- QUERY +grant role grant_revoke_test_SELECT_INSERT_TEST_TBL to group `$GROUP_NAME` +==== +---- USER +admin +---- QUERY +GRANT SELECT ON TABLE grant_rev_db.test_tbl1 TO grant_revoke_test_SELECT_INSERT_TEST_TBL +==== +---- QUERY +# This query succeeds since the user 'getuser()' has been granted the role +# 'grant_revoke_test_SELECT_INSERT_TEST_TBL', which in turn has been granted the SELECT +# privilege on the table 'grant_rev_db.test_tbl1'. +select * from grant_rev_db.test_tbl1 +---- RESULTS +---- TYPES +INT +==== +---- QUERY +# The user 'getuser()' should not be able to select the data from the table +# 'grant_rev_db.test_tbl2'. +select * from grant_rev_db.test_tbl2 +---- CATCH +does not have privileges to execute 'SELECT' on: grant_rev_db.test_tbl2 +==== +---- QUERY +insert overwrite grant_rev_db.test_tbl1 select 1 +---- CATCH +does not have privileges to execute 'INSERT' on: grant_rev_db.test_tbl1 +==== +---- USER +admin +---- QUERY +GRANT INSERT ON TABLE grant_rev_db.test_tbl1 TO grant_revoke_test_SELECT_INSERT_TEST_TBL +==== +---- QUERY +show grant role grant_revoke_test_SELECT_INSERT_TEST_TBL on table grant_rev_db.test_tbl1 +---- RESULTS: VERIFY_IS_EQUAL_SORTED +'ROLE','grant_revoke_test_SELECT_INSERT_TEST_TBL','grant_rev_db','test_tbl1','*','','','insert',false,regex:.+ +'ROLE','grant_revoke_test_SELECT_INSERT_TEST_TBL','grant_rev_db','test_tbl1','*','','','select',false,regex:.+ +---- LABELS +principal_type, principal_name, database, table, column, uri, udf, privilege, grant_option, create_time +---- TYPES +STRING, STRING, STRING, STRING, STRING, STRING, STRING, STRING, BOOLEAN, STRING +==== +---- QUERY +insert overwrite grant_rev_db.test_tbl1 select 1 +---- RESULTS +: 1 +==== +---- QUERY +select * from grant_rev_db.test_tbl1 +---- RESULTS +1 +---- TYPES +INT +==== +---- USER +test_user +---- QUERY +create role some_test_role +---- CATCH +User test_user does not have permission for this operation +==== +---- USER +test_user +---- QUERY +drop role grant_revoke_test_ALL_SERVER +---- CATCH +User test_user does not have permission for this operation +==== +---- USER +test_user +---- QUERY +# A non-Ranger administrator should not be able to tell from the result that the role +# does not exist. +drop role grant_revoke_test_NON_EXISTING +---- CATCH +User test_user does not have permission for this operation +==== +---- USER +test_user +---- QUERY +grant role grant_revoke_test_ALL_SERVER to group `$GROUP_NAME` +---- CATCH +User doesn't have permissions to grant role grant_revoke_test_ALL_SERVER +==== +---- USER +test_user +---- QUERY +revoke role grant_revoke_test_ALL_SERVER from group `$GROUP_NAME` +---- CATCH +User doesn't have permissions to revoke role grant_revoke_test_ALL_SERVER +==== +---- USER +non_owner +---- QUERY +# Grantor is required to exist in Ranger and thus we set the user to 'non_owner', which +# is currently a user created in Ranger in Impala's development environment. +grant all on server to grant_revoke_test_ALL_SERVER +---- CATCH +User doesn't have necessary permission to grant access +==== +---- USER +non_owner +---- QUERY +# Grantor is required to exist in Ranger and thus we set the user to 'non_owner', which +# is currently a user created in Ranger in Impala's development environment. +revoke all on server from grant_revoke_test_ALL_SERVER +---- CATCH +User doesn't have necessary permission to revoke access +==== +---- USER +admin +---- QUERY +# Set up a role to test the WITH GRANT OPTION. +# Note that the group 'non_owner' and the user 'non_owner' are created in Ranger in +# Impala's development environment and that the group 'non_owner' is a group to which the +# user 'non_owner' belongs according to the user-to-groups mapper provided for impalad +# and catalogd via the argument of "--use_customized_user_groups_mapper_for_ranger". +create role grant_revoke_test_NON_OWNER; +grant role grant_revoke_test_NON_OWNER to group non_owner; +grant all on database functional to grant_revoke_test_NON_OWNER WITH GRANT OPTION; +==== +---- USER +non_owner +---- QUERY +# There should only be one role that exists for non_owner. +show current roles +---- RESULTS +'grant_revoke_test_NON_OWNER' +---- TYPES +STRING +==== +---- USER +non_owner +---- QUERY +# This privilege is actually active. +show databases +---- RESULTS +'default','Default Hive database' +'functional','' +---- TYPES +STRING,STRING +==== +---- USER +non_owner +---- QUERY +# The non_owner user should be able to grant/revoke child privileges. +grant all on table functional.alltypes to grant_revoke_test_NON_OWNER +==== +---- USER +non_owner +---- QUERY +# We have to specify the resource for SHOW GRANT since SHOW GRANT is not supported +# without a defined resource in Ranger. +show grant role grant_revoke_test_NON_OWNER on database functional +---- RESULTS: VERIFY_IS_EQUAL_SORTED +'ROLE','grant_revoke_test_NON_OWNER','functional','','','','*','all',true,regex:.+ +'ROLE','grant_revoke_test_NON_OWNER','functional','*','*','','','all',true,regex:.+ +---- LABELS +principal_type, principal_name, database, table, column, uri, udf, privilege, grant_option, create_time +---- TYPES +STRING, STRING, STRING, STRING, STRING, STRING, STRING, STRING, BOOLEAN, STRING +==== +---- USER +non_owner +---- QUERY +# We have to specify the resource for SHOW GRANT since SHOW GRANT is not supported +# without a defined resource in Ranger. +# Note that the column of 'grant_option' is false for this privilege. +show grant role grant_revoke_test_NON_OWNER on table functional.alltypes +---- RESULTS: VERIFY_IS_EQUAL_SORTED +'ROLE','grant_revoke_test_NON_OWNER','functional','alltypes','*','','','all',false,regex:.+ +---- LABELS +principal_type, principal_name, database, table, column, uri, udf, privilege, grant_option, create_time +---- TYPES +STRING, STRING, STRING, STRING, STRING, STRING, STRING, STRING, BOOLEAN, STRING +==== +---- USER +admin +---- QUERY +revoke all on table functional.alltypes from grant_revoke_test_NON_OWNER +==== +---- USER +non_owner +---- QUERY +# User should not be able to grant privileges outside of this scope of the database +# 'functional'. +grant all on table functional_seq.alltypes to grant_revoke_test_NON_OWNER +---- CATCH +User doesn't have necessary permission to grant access +==== +---- USER +non_owner +---- QUERY +# Also cannot create/drop/grant roles +create role grant_revoke_test_NON_OWNER2 +---- CATCH +User non_owner does not have permission for this operation +==== +---- USER +non_owner +---- QUERY +# Also cannot create/drop/grant roles +grant role grant_revoke_test_NON_OWNER to group non_owner +---- CATCH +User doesn't have permissions to grant role grant_revoke_test_NON_OWNER +==== +---- USER +admin +---- QUERY +# Revoke the GRANT OPTION and verify the user can no longer GRANT or REVOKE +revoke grant option for all on database functional from grant_revoke_test_NON_OWNER +==== +---- USER +non_owner +---- QUERY +grant all on table functional.alltypes to grant_revoke_test_NON_OWNER +---- CATCH +User doesn't have necessary permission to grant access +==== +---- USER +non_owner +---- QUERY +# This privilege is actually active. +show databases +---- RESULTS +'default','Default Hive database' +'functional','' +---- TYPES +STRING,STRING +==== +---- USER +non_owner +---- QUERY +# We have to specify the resource for SHOW GRANT since SHOW GRANT is not supported +# without a defined resource in Ranger. +# Also, privilege still exists, but grant option is set to false. +show grant role grant_revoke_test_NON_OWNER on database functional +---- RESULTS: VERIFY_IS_EQUAL_SORTED +'ROLE','grant_revoke_test_NON_OWNER','functional','','','','*','all',false,regex:.+ +'ROLE','grant_revoke_test_NON_OWNER','functional','*','*','','','all',false,regex:.+ +---- LABELS +principal_type, principal_name, database, table, column, uri, udf, privilege, grant_option, create_time +---- TYPES +STRING, STRING, STRING, STRING, STRING, STRING, STRING, STRING, BOOLEAN, STRING +==== +---- USER +admin +---- QUERY +# After the following two statements, there is no role assigned to the group +# '$GROUP_NAME' to which the user 'getuser()' belongs. +REVOKE ROLE grant_revoke_test_ALL_URI FROM GROUP `$GROUP_NAME`; +REVOKE ROLE grant_revoke_test_SELECT_INSERT_TEST_TBL FROM GROUP `$GROUP_NAME`; +---- RESULTS +'Role has been revoked.' +==== +---- USER +admin +---- QUERY +GRANT ROLE grant_revoke_test_ALL_SERVER TO GROUP `$GROUP_NAME` +---- RESULTS +'Role has been granted.' +==== +---- QUERY +show current roles +---- RESULTS: VERIFY_IS_SUBSET +'grant_revoke_test_ALL_SERVER' +---- TYPES +STRING +==== +---- QUERY +# Create a table with multiple columns to test column-level security. +create table grant_rev_db.test_tbl3(a int, b int, c int, d int, e int) partitioned by (x int, y int) +---- RESULTS +'Table has been created.' +==== +---- USER +admin +---- QUERY +GRANT SELECT (a, b, x) ON TABLE grant_rev_db.test_tbl3 TO grant_revoke_test_ALL_SERVER +---- RESULTS +'Privilege(s) have been granted.' +==== +---- QUERY +# We have to specify the resource for SHOW GRANT since SHOW GRANT is not supported +# without a defined resource in Ranger. +# Note that after IMPALA-8587, the related privileges with regard to the specified +# resource will be shown as well. +show grant role grant_revoke_test_ALL_SERVER on column grant_rev_db.test_tbl3.a +---- RESULTS: VERIFY_IS_EQUAL_SORTED +'ROLE','grant_revoke_test_ALL_SERVER','*','*','*','','','all',false,regex:.+ +'ROLE','grant_revoke_test_ALL_SERVER','grant_rev_db','test_tbl3','a','','','select',false,regex:.+ +---- LABELS +principal_type, principal_name, database, table, column, uri, udf, privilege, grant_option, create_time +---- TYPES +STRING, STRING, STRING, STRING, STRING, STRING, STRING, STRING, BOOLEAN, STRING +==== +---- QUERY +# We have to specify the resource for SHOW GRANT since SHOW GRANT is not supported +# without a defined resource in Ranger. +show grant role grant_revoke_test_ALL_SERVER on column grant_rev_db.test_tbl3.b +---- RESULTS: VERIFY_IS_EQUAL_SORTED +'ROLE','grant_revoke_test_ALL_SERVER','*','*','*','','','all',false,regex:.+ +'ROLE','grant_revoke_test_ALL_SERVER','grant_rev_db','test_tbl3','b','','','select',false,regex:.+ +---- LABELS +principal_type, principal_name, database, table, column, uri, udf, privilege, grant_option, create_time +---- TYPES +STRING, STRING, STRING, STRING, STRING, STRING, STRING, STRING, BOOLEAN, STRING +==== +---- QUERY +# We have to specify the resource for SHOW GRANT since SHOW GRANT is not supported +# without a defined resource in Ranger. +show grant role grant_revoke_test_ALL_SERVER on column grant_rev_db.test_tbl3.x +---- RESULTS: VERIFY_IS_EQUAL_SORTED +'ROLE','grant_revoke_test_ALL_SERVER','*','*','*','','','all',false,regex:.+ +'ROLE','grant_revoke_test_ALL_SERVER','grant_rev_db','test_tbl3','x','','','select',false,regex:.+ +---- LABELS +principal_type, principal_name, database, table, column, uri, udf, privilege, grant_option, create_time +---- TYPES +STRING, STRING, STRING, STRING, STRING, STRING, STRING, STRING, BOOLEAN, STRING +==== +---- USER +admin +---- QUERY +# After granting the privileges on (c, d, y), we will check if they are indeed granted in +# the following 3 SHOW GRANT statements. +GRANT SELECT (c, d, y) ON TABLE grant_rev_db.test_tbl3 TO grant_revoke_test_ALL_SERVER +---- RESULTS +'Privilege(s) have been granted.' +==== +---- QUERY +# We have to specify the resource for SHOW GRANT since SHOW GRANT is not supported +# without a defined resource in Ranger. +show grant role grant_revoke_test_ALL_SERVER on column grant_rev_db.test_tbl3.c +---- RESULTS: VERIFY_IS_EQUAL_SORTED +'ROLE','grant_revoke_test_ALL_SERVER','*','*','*','','','all',false,regex:.+ +'ROLE','grant_revoke_test_ALL_SERVER','grant_rev_db','test_tbl3','c','','','select',false,regex:.+ +---- LABELS +principal_type, principal_name, database, table, column, uri, udf, privilege, grant_option, create_time +---- TYPES +STRING, STRING, STRING, STRING, STRING, STRING, STRING, STRING, BOOLEAN, STRING +==== +---- QUERY +# We have to specify the resource for SHOW GRANT since SHOW GRANT is not supported +# without a defined resource in Ranger. +show grant role grant_revoke_test_ALL_SERVER on column grant_rev_db.test_tbl3.d +---- RESULTS: VERIFY_IS_EQUAL_SORTED +'ROLE','grant_revoke_test_ALL_SERVER','*','*','*','','','all',false,regex:.+ +'ROLE','grant_revoke_test_ALL_SERVER','grant_rev_db','test_tbl3','d','','','select',false,regex:.+ +---- LABELS +principal_type, principal_name, database, table, column, uri, udf, privilege, grant_option, create_time +---- TYPES +STRING, STRING, STRING, STRING, STRING, STRING, STRING, STRING, BOOLEAN, STRING +==== +---- QUERY +# We have to specify the resource for SHOW GRANT since SHOW GRANT is not supported +# without a defined resource in Ranger. +show grant role grant_revoke_test_ALL_SERVER on column grant_rev_db.test_tbl3.y +---- RESULTS: VERIFY_IS_EQUAL_SORTED +'ROLE','grant_revoke_test_ALL_SERVER','*','*','*','','','all',false,regex:.+ +'ROLE','grant_revoke_test_ALL_SERVER','grant_rev_db','test_tbl3','y','','','select',false,regex:.+ +---- LABELS +principal_type, principal_name, database, table, column, uri, udf, privilege, grant_option, create_time +---- TYPES +STRING, STRING, STRING, STRING, STRING, STRING, STRING, STRING, BOOLEAN, STRING +==== +---- USER +admin +---- QUERY +GRANT SELECT (a, a, e, x) ON TABLE grant_rev_db.test_tbl3 TO grant_revoke_test_ALL_SERVER +---- RESULTS +'Privilege(s) have been granted.' +==== +---- QUERY +# We have to specify the resource for SHOW GRANT since SHOW GRANT is not supported +# without a defined resource in Ranger. +# Verify that the privilege on 'grant_rev_db.test_tbl3.e' is indeed granted. +show grant role grant_revoke_test_ALL_SERVER on column grant_rev_db.test_tbl3.e +---- RESULTS: VERIFY_IS_EQUAL_SORTED +'ROLE','grant_revoke_test_ALL_SERVER','*','*','*','','','all',false,regex:.+ +'ROLE','grant_revoke_test_ALL_SERVER','grant_rev_db','test_tbl3','e','','','select',false,regex:.+ +---- LABELS +principal_type, principal_name, database, table, column, uri, udf, privilege, grant_option, create_time +---- TYPES +STRING, STRING, STRING, STRING, STRING, STRING, STRING, STRING, BOOLEAN, STRING +==== +---- USER +admin +---- QUERY +# Revoke SELECT privileges from columns. +# Verify the privileges on (a, b, y) are indeed revoked in the following 3 queries. +REVOKE SELECT (a, b, b, y) ON TABLE grant_rev_db.test_tbl3 FROM grant_revoke_test_ALL_SERVER +---- RESULTS +'Privilege(s) have been revoked.' +==== +---- QUERY +# We have to specify the resource for SHOW GRANT since SHOW GRANT is not supported +# without a defined resource in Ranger. +show grant role grant_revoke_test_ALL_SERVER on column grant_rev_db.test_tbl3.a +---- RESULTS: VERIFY_IS_EQUAL_SORTED +'ROLE','grant_revoke_test_ALL_SERVER','*','*','*','','','all',false,regex:.+ +---- LABELS +principal_type, principal_name, database, table, column, uri, udf, privilege, grant_option, create_time +---- TYPES +STRING, STRING, STRING, STRING, STRING, STRING, STRING, STRING, BOOLEAN, STRING +==== +---- QUERY +# We have to specify the resource for SHOW GRANT since SHOW GRANT is not supported +# without a defined resource in Ranger. +show grant role grant_revoke_test_ALL_SERVER on column grant_rev_db.test_tbl3.b +---- RESULTS: VERIFY_IS_EQUAL_SORTED +'ROLE','grant_revoke_test_ALL_SERVER','*','*','*','','','all',false,regex:.+ +---- LABELS +principal_type, principal_name, database, table, column, uri, udf, privilege, grant_option, create_time +---- TYPES +STRING, STRING, STRING, STRING, STRING, STRING, STRING, STRING, BOOLEAN, STRING +==== +---- QUERY +# We have to specify the resource for SHOW GRANT since SHOW GRANT is not supported +# without a defined resource in Ranger. +show grant role grant_revoke_test_ALL_SERVER on column grant_rev_db.test_tbl3.y +---- RESULTS: VERIFY_IS_EQUAL_SORTED +'ROLE','grant_revoke_test_ALL_SERVER','*','*','*','','','all',false,regex:.+ +---- LABELS +principal_type, principal_name, database, table, column, uri, udf, privilege, grant_option, create_time +---- TYPES +STRING, STRING, STRING, STRING, STRING, STRING, STRING, STRING, BOOLEAN, STRING +==== +---- USER +admin +---- QUERY +# Verify the privileges on (b, c, x) are indeed revoked in the following 3 queries. +# Recall that the privilege on (a) had been revoked previously. +REVOKE SELECT (a, b, c, x) ON TABLE grant_rev_db.test_tbl3 FROM grant_revoke_test_ALL_SERVER +---- RESULTS +'Privilege(s) have been revoked.' +==== +---- QUERY +# We have to specify the resource for SHOW GRANT since SHOW GRANT is not supported +# without a defined resource in Ranger. +show grant role grant_revoke_test_ALL_SERVER on column grant_rev_db.test_tbl3.b +---- RESULTS: VERIFY_IS_EQUAL_SORTED +'ROLE','grant_revoke_test_ALL_SERVER','*','*','*','','','all',false,regex:.+ +---- LABELS +principal_type, principal_name, database, table, column, uri, udf, privilege, grant_option, create_time +---- TYPES +STRING, STRING, STRING, STRING, STRING, STRING, STRING, STRING, BOOLEAN, STRING +==== +---- QUERY +# We have to specify the resource for SHOW GRANT since SHOW GRANT is not supported +# without a defined resource in Ranger. +show grant role grant_revoke_test_ALL_SERVER on column grant_rev_db.test_tbl3.c +---- RESULTS: VERIFY_IS_EQUAL_SORTED +'ROLE','grant_revoke_test_ALL_SERVER','*','*','*','','','all',false,regex:.+ +---- LABELS +principal_type, principal_name, database, table, column, uri, udf, privilege, grant_option, create_time +---- TYPES +STRING, STRING, STRING, STRING, STRING, STRING, STRING, STRING, BOOLEAN, STRING +==== +---- QUERY +# We have to specify the resource for SHOW GRANT since SHOW GRANT is not supported +# without a defined resource in Ranger. +show grant role grant_revoke_test_ALL_SERVER on column grant_rev_db.test_tbl3.x +---- RESULTS: VERIFY_IS_EQUAL_SORTED +'ROLE','grant_revoke_test_ALL_SERVER','*','*','*','','','all',false,regex:.+ +---- LABELS +principal_type, principal_name, database, table, column, uri, udf, privilege, grant_option, create_time +---- TYPES +STRING, STRING, STRING, STRING, STRING, STRING, STRING, STRING, BOOLEAN, STRING +==== +---- USER +admin +---- QUERY +# Verify the privileges on (d, e) are indeed revoked in the following 2 queries. +# Recall that the privilege on (a) had been revoked previously. +REVOKE SELECT (a, b, c, d, e) ON TABLE grant_rev_db.test_tbl3 FROM grant_revoke_test_ALL_SERVER; +---- RESULTS +'Privilege(s) have been revoked.' +==== +---- QUERY +# We have to specify the resource for SHOW GRANT since SHOW GRANT is not supported +# without a defined resource in Ranger. +show grant role grant_revoke_test_ALL_SERVER on column grant_rev_db.test_tbl3.d +---- RESULTS: VERIFY_IS_EQUAL_SORTED +'ROLE','grant_revoke_test_ALL_SERVER','*','*','*','','','all',false,regex:.+ +---- LABELS +principal_type, principal_name, database, table, column, uri, udf, privilege, grant_option, create_time +---- TYPES +STRING, STRING, STRING, STRING, STRING, STRING, STRING, STRING, BOOLEAN, STRING +==== +---- QUERY +# We have to specify the resource for SHOW GRANT since SHOW GRANT is not supported +# without a defined resource in Ranger. +show grant role grant_revoke_test_ALL_SERVER on column grant_rev_db.test_tbl3.e +---- RESULTS: VERIFY_IS_EQUAL_SORTED +'ROLE','grant_revoke_test_ALL_SERVER','*','*','*','','','all',false,regex:.+ +---- LABELS +principal_type, principal_name, database, table, column, uri, udf, privilege, grant_option, create_time +---- TYPES +STRING, STRING, STRING, STRING, STRING, STRING, STRING, STRING, BOOLEAN, STRING +==== +---- USER +admin +---- QUERY +# Grant SELECT on table to 'non_owner' without 'WITH GRANT' option. +GRANT ROLE grant_revoke_test_NON_OWNER TO GROUP non_owner; +GRANT SELECT ON TABLE grant_rev_db.test_tbl3 TO grant_revoke_test_NON_OWNER; +REVOKE ALL ON DATABASE functional FROM grant_revoke_test_NON_OWNER; +---- RESULTS +'Privilege(s) have been revoked.' +==== +---- USER +non_owner +---- QUERY +# We have to specify the resource for SHOW GRANT since SHOW GRANT is not supported +# without a defined resource in Ranger. +# Verify that the SELECT privilege is indeed granted to the role +# 'grant_revoke_test_NON_OWNER'. +show grant role grant_revoke_test_NON_OWNER on table grant_rev_db.test_tbl3 +---- RESULTS: VERIFY_IS_EQUAL_SORTED +'ROLE','grant_revoke_test_NON_OWNER','grant_rev_db','test_tbl3','*','','','select',false,regex:.+ +---- LABELS +principal_type, principal_name, database, table, column, uri, udf, privilege, grant_option, create_time +---- TYPES +STRING, STRING, STRING, STRING, STRING, STRING, STRING, STRING, BOOLEAN, STRING +==== +---- USER +non_owner +---- QUERY +GRANT SELECT (a) ON TABLE grant_rev_db.test_tbl3 TO grant_revoke_test_NON_OWNER +---- CATCH +User doesn't have necessary permission to grant access +==== +---- USER +admin +---- QUERY +REVOKE SELECT ON TABLE grant_rev_db.test_tbl3 FROM grant_revoke_test_NON_OWNER +---- RESULTS +'Privilege(s) have been revoked.' +==== +---- USER +admin +---- QUERY +# Grant SELECT on table to 'non_owner' with 'WITH GRANT' option. +GRANT SELECT ON TABLE grant_rev_db.test_tbl3 TO grant_revoke_test_NON_OWNER WITH GRANT OPTION +---- RESULTS +'Privilege(s) have been granted.' +==== +---- USER +non_owner +---- QUERY +GRANT SELECT (a) ON TABLE grant_rev_db.test_tbl3 TO grant_revoke_test_NON_OWNER +---- RESULTS +'Privilege(s) have been granted.' +==== +---- USER +non_owner +---- QUERY +# We have to specify the resource for SHOW GRANT since SHOW GRANT is not supported +# without a defined resource in Ranger. +# Verify that the privilege on the column 'grant_rev_db.test_tbl3.a' is indeed granted. +# TODO: Notice that at this point, the role 'grant_revoke_test_NON_OWNER' has also been +# granted the SELECT privilege on the table 'grant_rev_db.test_tbl3'. We need to +# investigate whether this is the expected behavior after IMPALA-8587. +show grant role grant_revoke_test_NON_OWNER on column grant_rev_db.test_tbl3.a +---- RESULTS: VERIFY_IS_EQUAL_SORTED +'ROLE','grant_revoke_test_NON_OWNER','grant_rev_db','test_tbl3','a','','','select',false,regex:.+ +---- LABELS +principal_type, principal_name, database, table, column, uri, udf, privilege, grant_option, create_time +---- TYPES +STRING, STRING, STRING, STRING, STRING, STRING, STRING, STRING, BOOLEAN, STRING +==== +---- USER +admin +---- QUERY +# Grant the SELECT privileges on the columns (a, c, e) and verify in the following 3 +# queries that the privileges are granted with the column of 'grant_option' being true. +GRANT SELECT (a, c, e) ON TABLE grant_rev_db.test_tbl3 TO grant_revoke_test_ALL_SERVER WITH GRANT OPTION +---- RESULTS +'Privilege(s) have been granted.' +==== +---- QUERY +# We have to specify the resource for SHOW GRANT since SHOW GRANT is not supported +# without a defined resource in Ranger. +show grant role grant_revoke_test_ALL_SERVER on column grant_rev_db.test_tbl3.a +---- RESULTS: VERIFY_IS_EQUAL_SORTED +'ROLE','grant_revoke_test_ALL_SERVER','*','*','*','','','all',false,regex:.+ +'ROLE','grant_revoke_test_ALL_SERVER','grant_rev_db','test_tbl3','a','','','select',true,regex:.+ +---- LABELS +principal_type, principal_name, database, table, column, uri, udf, privilege, grant_option, create_time +---- TYPES +STRING, STRING, STRING, STRING, STRING, STRING, STRING, STRING, BOOLEAN, STRING +==== +---- QUERY +# We have to specify the resource for SHOW GRANT since SHOW GRANT is not supported +# without a defined resource in Ranger. +show grant role grant_revoke_test_ALL_SERVER on column grant_rev_db.test_tbl3.c +---- RESULTS: VERIFY_IS_EQUAL_SORTED +'ROLE','grant_revoke_test_ALL_SERVER','*','*','*','','','all',false,regex:.+ +'ROLE','grant_revoke_test_ALL_SERVER','grant_rev_db','test_tbl3','c','','','select',true,regex:.+ +---- LABELS +principal_type, principal_name, database, table, column, uri, udf, privilege, grant_option, create_time +---- TYPES +STRING, STRING, STRING, STRING, STRING, STRING, STRING, STRING, BOOLEAN, STRING +==== +---- QUERY +# We have to specify the resource for SHOW GRANT since SHOW GRANT is not supported +# without a defined resource in Ranger. +show grant role grant_revoke_test_ALL_SERVER on column grant_rev_db.test_tbl3.e +---- RESULTS: VERIFY_IS_EQUAL_SORTED +'ROLE','grant_revoke_test_ALL_SERVER','*','*','*','','','all',false,regex:.+ +'ROLE','grant_revoke_test_ALL_SERVER','grant_rev_db','test_tbl3','e','','','select',true,regex:.+ +---- LABELS +principal_type, principal_name, database, table, column, uri, udf, privilege, grant_option, create_time +---- TYPES +STRING, STRING, STRING, STRING, STRING, STRING, STRING, STRING, BOOLEAN, STRING +==== +---- USER +admin +---- QUERY +# Revoke the granting privilege on the columns (a, c) and veviry in the following 2 +# queries that the column of 'grant_option' is indeed false. +REVOKE GRANT OPTION FOR SELECT (a, c) ON TABLE grant_rev_db.test_tbl3 FROM grant_revoke_test_ALL_SERVER +---- RESULTS +'Privilege(s) have been revoked.' +==== +---- QUERY +# We have to specify the resource for SHOW GRANT since SHOW GRANT is not supported +# without a defined resource in Ranger. +show grant role grant_revoke_test_ALL_SERVER on column grant_rev_db.test_tbl3.a +---- RESULTS: VERIFY_IS_EQUAL_SORTED +'ROLE','grant_revoke_test_ALL_SERVER','*','*','*','','','all',false,regex:.+ +'ROLE','grant_revoke_test_ALL_SERVER','grant_rev_db','test_tbl3','a','','','select',false,regex:.+ +---- LABELS +principal_type, principal_name, database, table, column, uri, udf, privilege, grant_option, create_time +---- TYPES +STRING, STRING, STRING, STRING, STRING, STRING, STRING, STRING, BOOLEAN, STRING +==== +---- QUERY +# We have to specify the resource for SHOW GRANT since SHOW GRANT is not supported +# without a defined resource in Ranger. +show grant role grant_revoke_test_ALL_SERVER on column grant_rev_db.test_tbl3.c +---- RESULTS: VERIFY_IS_EQUAL_SORTED +'ROLE','grant_revoke_test_ALL_SERVER','*','*','*','','','all',false,regex:.+ +'ROLE','grant_revoke_test_ALL_SERVER','grant_rev_db','test_tbl3','c','','','select',false,regex:.+ +---- LABELS +principal_type, principal_name, database, table, column, uri, udf, privilege, grant_option, create_time +---- TYPES +STRING, STRING, STRING, STRING, STRING, STRING, STRING, STRING, BOOLEAN, STRING +==== +---- USER +admin +---- QUERY +# At this point, the group '$GROUP_NAME' is still assigned the role of +# 'grant_revoke_test_ALL_SERVER'. +revoke role grant_revoke_test_ALL_SERVER from group `$GROUP_NAME` +==== +---- USER +admin +---- QUERY +# Test 'grant all on server' with explicit server name specified. +create role grant_revoke_test_ALL_SERVER1 +---- RESULTS +'Role has been created.' +==== +---- USER +admin +---- QUERY +grant all on server server1 to grant_revoke_test_ALL_SERVER1 +==== +---- USER +admin +---- QUERY +# Grant role to the group '$GROUP_NAME' to which the user 'getuser()' belongs. +grant role grant_revoke_test_ALL_SERVER1 to group `$GROUP_NAME` +==== +---- USER +admin +---- QUERY +# We will be testing whether the user 'getuser()' is able to drop and then create the +# database 'grant_rev_db'. Thus, we have to revoke the privileges on the resources under +# the database 'grant_rev_db' in advance. Otherwise, we won't be able to revoke the +# privileges after dropping the database since revoking the privileges on those +# non-existing resources would result AnalysisException's. If we are not able to remove +# these privileges, then we won't be able to drop those roles they reference after the +# execution of this test file, leaving some roles and privileges that are not well-defined. +REVOKE SELECT ON TABLE grant_rev_db.test_tbl1 FROM grant_revoke_test_SELECT_INSERT_TEST_TBL; +REVOKE INSERT ON TABLE grant_rev_db.test_tbl1 FROM grant_revoke_test_SELECT_INSERT_TEST_TBL; +REVOKE SELECT ON TABLE grant_rev_db.test_tbl3 FROM grant_revoke_test_NON_OWNER; +REVOKE SELECT (a) ON TABLE grant_rev_db.test_tbl3 FROM grant_revoke_test_NON_OWNER; +REVOKE SELECT (a, c, e) ON TABLE grant_rev_db.test_tbl3 FROM grant_revoke_test_ALL_SERVER; +==== +---- QUERY +# Verify that the user 'getuser()' is able to drop the database. +drop database grant_rev_db cascade +==== +---- QUERY +create database grant_rev_db location '$FILESYSTEM_PREFIX/test-warehouse/grant_rev_db.db' +==== +---- USER +admin +---- QUERY +# To prevent the user 'getuser()' (which belongs to the group '$GROUP_NAME') from +# performing any operation on the database 'grant_rev_db', we also need to alter +# the owner of 'grant_rev_db' since 'getuser()' is the owner of this database, which by +# default is allowed by Ranger to perform any operation on the database. +revoke role grant_revoke_test_ALL_SERVER1 from group `$GROUP_NAME`; +alter database grant_rev_db set owner user admin; +==== +---- QUERY +create database grant_rev_db location '$FILESYSTEM_PREFIX/test-warehouse/grant_rev_db.db' +---- CATCH +does not have privileges to execute 'CREATE' on: grant_rev_db +==== +---- QUERY +# We have to specify the resource for SHOW GRANT since SHOW GRANT is not supported +# without a defined resource in Ranger. +show grant role grant_revoke_test_ALL_SERVER1 on server +---- RESULTS: VERIFY_IS_EQUAL_SORTED +'ROLE','grant_revoke_test_ALL_SERVER1','','','','*','','all',false,regex:.+ +'ROLE','grant_revoke_test_ALL_SERVER1','*','','','','*','all',false,regex:.+ +'ROLE','grant_revoke_test_ALL_SERVER1','*','*','*','','','all',false,regex:.+ +---- LABELS +principal_type, principal_name, database, table, column, uri, udf, privilege, grant_option, create_time +---- TYPES +STRING, STRING, STRING, STRING, STRING, STRING, STRING, STRING, BOOLEAN, STRING +==== +---- USER +admin +---- QUERY +revoke all on server server1 from grant_revoke_test_ALL_SERVER1 +==== +---- QUERY +# We have to specify the resource for SHOW GRANT since SHOW GRANT is not supported +# without a defined resource in Ranger. +# Verify that the privilege on server is indeed revoked. +show grant role grant_revoke_test_ALL_SERVER1 on server +---- RESULTS: VERIFY_IS_EQUAL_SORTED +---- LABELS +principal_type, principal_name, database, table, column, uri, udf, privilege, grant_option, create_time +---- TYPES +STRING, STRING, STRING, STRING, STRING, STRING, STRING, STRING, BOOLEAN, STRING +==== +---- USER +admin +---- QUERY +# IMPALA-4951: make sure database is visible to a user having only column level access +# to a table in the database +grant role grant_revoke_test_ALL_SERVER to group non_owner +---- RESULTS +'Role has been granted.' +==== +---- USER +admin +---- QUERY +create role grant_revoke_test_COLUMN_PRIV +==== +---- USER +admin +---- QUERY +grant role grant_revoke_test_COLUMN_PRIV to group non_owner +==== +---- USER +non_owner +---- QUERY +create database if not exists grant_rev_db; +==== +---- USER +non_owner +---- QUERY +create table grant_rev_db.test_tbl4 (col1 int, col2 int) +==== +---- USER +admin +---- QUERY +revoke role grant_revoke_test_ALL_SERVER from group non_owner +==== +---- USER +non_owner +---- QUERY +# We have to specify the resource for SHOW GRANT since SHOW GRANT is not supported +# without a defined resource in Ranger. +show grant role grant_revoke_test_COLUMN_PRIV on table grant_rev_db.test_tbl4 +---- RESULTS +---- LABELS +principal_type, principal_name, database, table, column, uri, udf, privilege, grant_option, create_time +---- TYPES +STRING, STRING, STRING, STRING, STRING, STRING, STRING, STRING, BOOLEAN, STRING +==== +---- USER +non_owner +---- QUERY +# grant_rev_db is not visible as user does not have any level of access to it +show databases +---- RESULTS +'default','Default Hive database' +---- TYPES +STRING,STRING +==== +---- USER +admin +---- QUERY +grant select(col1) on table grant_rev_db.test_tbl4 to role grant_revoke_test_COLUMN_PRIV +==== +---- USER +non_owner +---- QUERY +show grant role grant_revoke_test_COLUMN_PRIV on column grant_rev_db.test_tbl4.col1 +---- RESULTS +'ROLE','grant_revoke_test_COLUMN_PRIV','grant_rev_db','test_tbl4','col1','','','select',false,regex:.+ +---- LABELS +principal_type, principal_name, database, table, column, uri, udf, privilege, grant_option, create_time +---- TYPES +STRING, STRING, STRING, STRING, STRING, STRING, STRING, STRING, BOOLEAN, STRING +==== +---- USER +non_owner +---- QUERY +# Verify that the database 'grant_rev_db' is visible to the user 'non_owner'. +show databases +---- RESULTS +'default','Default Hive database' +'grant_rev_db','' +---- TYPES +STRING,STRING +==== +---- USER +admin +---- QUERY +# We will be testing whether the user 'getuser()' is able to drop a database if the group +# 'getuser()' belongs to is assigned the role 'grant_revoke_test_ALL_SERVER', which is +# granted the ALL privilege on the server. Before dropping a database, the user also has +# to revoke every privilege granted on the resources of the database and thus we grant +# the ALL privilege with the grant option. +grant all on server to role grant_revoke_test_ALL_SERVER with grant option; +grant role grant_revoke_test_ALL_SERVER to group `$GROUP_NAME`; +---- RESULTS +'Role has been granted.' +==== +---- QUERY +# The user 'getuser()' has to revoke the privileges on the resources under the database +# 'grant_rev_db' before dropping the database. Otherwise, the revocation would fail due +# to an AnalysisException thrown because 'grant_rev_db' does not exist. +revoke select(col1) on table grant_rev_db.test_tbl4 from role grant_revoke_test_COLUMN_PRIV; +drop database if exists grant_rev_db cascade; +==== +---- USER +admin +---- QUERY +revoke role grant_revoke_test_ALL_SERVER from group `$GROUP_NAME` +---- RESULTS +'Role has been revoked.' +==== +---- USER +admin +---- QUERY +revoke role grant_revoke_test_COLUMN_PRIV from group `$GROUP_NAME` +==== +---- QUERY +# Verify that the user 'getuser()', which belongs to the group '$GROUP_NAME' is not +# assigned any role afterwards. +show current roles +---- RESULTS +==== diff --git a/tests/authorization/test_ranger.py b/tests/authorization/test_ranger.py index 1f7e8a9..b3052e0 100644 --- a/tests/authorization/test_ranger.py +++ b/tests/authorization/test_ranger.py @@ -115,6 +115,7 @@ class TestRanger(CustomClusterTestSuite): admin_client.execute("drop database if exists {0} cascade" .format(unique_database), user=ADMIN) + @pytest.mark.execute_serially @CustomClusterTestSuite.with_args( impalad_args=IMPALAD_ARGS, catalogd_args=CATALOGD_ARGS) def test_grant_option(self, unique_name): @@ -182,6 +183,7 @@ class TestRanger(CustomClusterTestSuite): admin_client.execute("drop database if exists {0} cascade".format(unique_database), user=ADMIN) + @pytest.mark.execute_serially @CustomClusterTestSuite.with_args( impalad_args=IMPALAD_ARGS, catalogd_args=CATALOGD_ARGS) def test_show_grant(self, unique_name): @@ -468,7 +470,7 @@ class TestRanger(CustomClusterTestSuite): admin_client.execute("revoke all on table {0}.{1} from {2} {3}" .format(unique_database, unique_table, kw, id)) - + @pytest.mark.execute_serially @CustomClusterTestSuite.with_args( impalad_args=IMPALAD_ARGS, catalogd_args=CATALOGD_ARGS) def test_grant_revoke_ranger_api(self, unique_name): @@ -529,6 +531,13 @@ class TestRanger(CustomClusterTestSuite): admin_client.execute("drop database if exists {0} cascade".format(unique_db), user=ADMIN) + # TODO(IMPALA-10399, IMPALA-10401): We found that if this test is run after + # test_grant_revoke_with_role() in the exhaustive tests, the test could fail due to an + # empty list returned from the first call to _get_ranger_privileges_db() although a + # list consisting of "lock" and "select" is expected. We suspect there might be + # something wrong with the underlying Ranger API but it requires more thorough + # investigation. + @pytest.mark.execute_serially @CustomClusterTestSuite.with_args( impalad_args=IMPALAD_ARGS, catalogd_args=CATALOGD_ARGS) def test_show_grant_hive_privilege(self, unique_name): @@ -726,27 +735,20 @@ class TestRanger(CustomClusterTestSuite): impala_client, query, user=username, query_options={'sync_ddl': 1}) return self.execute_query_expect_failure(impala_client, query, user=username) + @pytest.mark.execute_serially @CustomClusterTestSuite.with_args( impalad_args=IMPALAD_ARGS, catalogd_args=CATALOGD_ARGS) def test_unsupported_sql(self): """Tests unsupported SQL statements when running with Ranger.""" user = "admin" impala_client = self.create_impala_client() - error_msg = "UnsupportedFeatureException: {0} is not supported by Ranger." - for statement in [("show roles", error_msg.format("SHOW ROLES")), - ("show current roles", error_msg.format("SHOW CURRENT ROLES")), - ("create role foo", error_msg.format("CREATE ROLE")), - ("drop role foo", error_msg.format("DROP ROLE")), - ("grant select on database functional to role foo", - error_msg.format("GRANT <privilege> TO ROLE")), - ("revoke select on database functional from role foo", - error_msg.format("REVOKE <privilege> FROM ROLE")), - ("show grant role foo", error_msg.format("SHOW GRANT ROLE")), - ("show role grant group foo", - error_msg.format("SHOW ROLE GRANT GROUP"))]: - result = self.execute_query_expect_failure(impala_client, statement[0], user=user) - assert statement[1] in str(result) + error_msg = "UnsupportedOperationException: SHOW GRANT is not supported without a " \ + "defined resource in Ranger." + statement = "show grant role foo" + result = self.execute_query_expect_failure(impala_client, statement, user=user) + assert error_msg in str(result) + @pytest.mark.execute_serially @CustomClusterTestSuite.with_args( impalad_args=IMPALAD_ARGS, catalogd_args=CATALOGD_ARGS) def test_grant_revoke_invalid_principal(self): @@ -801,11 +803,13 @@ class TestRanger(CustomClusterTestSuite): assert "Error revoking a privilege in Ranger. Ranger error message: " \ "HTTP 403 Error: Grantee group invalid_group doesn't exist" in str(result) + @pytest.mark.execute_serially @CustomClusterTestSuite.with_args( impalad_args=IMPALAD_ARGS, catalogd_args=CATALOGD_ARGS) def test_legacy_catalog_ownership(self): self._test_ownership() + @pytest.mark.execute_serially @CustomClusterTestSuite.with_args(impalad_args=LOCAL_CATALOG_IMPALAD_ARGS, catalogd_args=LOCAL_CATALOG_CATALOGD_ARGS) def test_local_catalog_ownership(self): @@ -815,6 +819,7 @@ class TestRanger(CustomClusterTestSuite): pytest.xfail("getTableIfCached() faulty behavior, known issue") self._test_ownership() + @pytest.mark.execute_serially @CustomClusterTestSuite.with_args( impalad_args=IMPALAD_ARGS, catalogd_args=CATALOGD_ARGS) def test_show_functions(self, unique_name): @@ -916,6 +921,7 @@ class TestRanger(CustomClusterTestSuite): finally: self._run_query_as_user("drop database {0} cascade".format(test_db), ADMIN, True) + @pytest.mark.execute_serially @CustomClusterTestSuite.with_args( impalad_args=IMPALAD_ARGS, catalogd_args=CATALOGD_ARGS) def test_column_masking(self, vector, unique_name): @@ -988,6 +994,7 @@ class TestRanger(CustomClusterTestSuite): for i in range(policy_cnt): TestRanger._remove_column_masking_policy(unique_name + str(i)) + @pytest.mark.execute_serially @SkipIfABFS.hive @SkipIfADLS.hive @SkipIfIsilon.hive @@ -1011,6 +1018,87 @@ class TestRanger(CustomClusterTestSuite): check_call([script]) TestRanger._remove_column_masking_policy("col_mask_for_hive") + @pytest.mark.execute_serially + @CustomClusterTestSuite.with_args( + # We additionally provide impalad and catalogd with the customized user-to-groups + # mapper since some test cases in grant_revoke.test require Impala to retrieve the + # groups a given user belongs to and such users might not exist in the underlying + # OS in the testing environment, e.g., the user 'non_owner'. + impalad_args="{0} {1}".format(IMPALAD_ARGS, + "--use_customized_user_groups_mapper_for_ranger"), + catalogd_args="{0} {1}".format(CATALOGD_ARGS, + "--use_customized_user_groups_mapper_for_ranger")) + def test_grant_revoke_with_role(self, vector): + """Test grant/revoke with role.""" + admin_client = self.create_impala_client() + try: + self.run_test_case('QueryTest/grant_revoke', vector, use_db="default") + finally: + # Below are the statements that need to be executed in order to clean up the + # privileges granted to the test roles as well as the test roles themselves. + # Note that we need to revoke those previously granted privileges so that each role + # is not referenced by any policy before we delete those roles. + # Moreover, we need to revoke the privilege on the database 'grant_rev_db' before + # dropping 'grant_rev_db'. Otherwise, the revocation would fail due to an + # AnalysisException thrown because 'grant_rev_db' does not exist. + cleanup_statements = [ + "revoke all on database grant_rev_db from grant_revoke_test_ALL_TEST_DB", + "revoke all on server from grant_revoke_test_ALL_SERVER", + "revoke all on table functional.alltypes from grant_revoke_test_NON_OWNER", + "revoke grant option for all on database functional " + "from grant_revoke_test_NON_OWNER", + "REVOKE SELECT (a, b, c, d, e, x, y) ON TABLE grant_rev_db.test_tbl3 " + "FROM grant_revoke_test_ALL_SERVER", + "REVOKE ALL ON DATABASE functional FROM grant_revoke_test_NON_OWNER", + "REVOKE SELECT ON TABLE grant_rev_db.test_tbl3 FROM grant_revoke_test_NON_OWNER", + "REVOKE GRANT OPTION FOR SELECT (a, c) ON TABLE grant_rev_db.test_tbl3 " + "FROM grant_revoke_test_ALL_SERVER", + "REVOKE SELECT ON TABLE grant_rev_db.test_tbl1 " + "FROM grant_revoke_test_SELECT_INSERT_TEST_TBL", + "REVOKE INSERT ON TABLE grant_rev_db.test_tbl1 " + "FROM grant_revoke_test_SELECT_INSERT_TEST_TBL", + "REVOKE SELECT ON TABLE grant_rev_db.test_tbl3 " + "FROM grant_revoke_test_NON_OWNER", + "REVOKE SELECT (a) ON TABLE grant_rev_db.test_tbl3 " + "FROM grant_revoke_test_NON_OWNER", + "REVOKE SELECT (a, c, e) ON TABLE grant_rev_db.test_tbl3 " + "FROM grant_revoke_test_ALL_SERVER", + "revoke all on server server1 from grant_revoke_test_ALL_SERVER1", + "revoke select(col1) on table grant_rev_db.test_tbl4 " + "from role grant_revoke_test_COLUMN_PRIV", + "{0}{1}{2}".format("revoke all on uri '", + os.getenv("FILESYSTEM_PREFIX"), + "/test-warehouse/grant_rev_test_tbl2'" + "from grant_revoke_test_ALL_URI"), + "{0}{1}{2}".format("revoke all on uri '", + os.getenv("FILESYSTEM_PREFIX"), + "/test-warehouse/GRANT_REV_TEST_TBL3'" + "from grant_revoke_test_ALL_URI"), + "{0}{1}{2}".format("revoke all on uri '", + os.getenv("FILESYSTEM_PREFIX"), + "/test-warehouse/grant_rev_test_prt'" + "from grant_revoke_test_ALL_URI"), + "drop role grant_revoke_test_ALL_TEST_DB", + "drop role grant_revoke_test_ALL_SERVER", + "drop role grant_revoke_test_SELECT_INSERT_TEST_TBL", + "drop role grant_revoke_test_ALL_URI", + "drop role grant_revoke_test_NON_OWNER", + "drop role grant_revoke_test_ALL_SERVER1", + "drop role grant_revoke_test_COLUMN_PRIV", + "drop database grant_rev_db cascade" + ] + + for statement in cleanup_statements: + try: + admin_client.execute(statement, user=ADMIN) + except Exception: + # There could be an exception thrown due to the non-existence of the role or + # resource involved in a statement that aims to revoke the privilege on a + # resource from a role, but we do not have to handle such an exception. We only + # need to make sure in the case when the role and the corresponding resource + # exist, the granted privilege is revoked. The same applies to the case when we + # drop a role. + pass class TestRangerColumnMaskingTpchNested(CustomClusterTestSuite): """ @@ -1028,6 +1116,7 @@ class TestRangerColumnMaskingTpchNested(CustomClusterTestSuite): cls.ImpalaTestMatrix.add_constraint( lambda v: v.get_value('table_format').file_format == 'parquet') + @pytest.mark.execute_serially @CustomClusterTestSuite.with_args( impalad_args=IMPALAD_ARGS, catalogd_args=CATALOGD_ARGS) def test_tpch_nested_column_masking(self, vector):
