Re: [PR] [#3963] feat(core): Apache Ranger Hive authorization pushdown [gravitino]
yuqi1129 merged PR #4622: URL: https://github.com/apache/gravitino/pull/4622 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
Re: [PR] [#3963] feat(core): Apache Ranger Hive authorization pushdown [gravitino]
yuqi1129 closed pull request #4622: [#3963] feat(core): Apache Ranger Hive authorization pushdown URL: https://github.com/apache/gravitino/pull/4622 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
Re: [PR] [#3963] feat(core): Apache Ranger Hive authorization pushdown [gravitino]
xunliu merged PR #4515: URL: https://github.com/apache/gravitino/pull/4515 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
Re: [PR] [#3963] feat(core): Apache Ranger Hive authorization pushdown [gravitino]
xunliu commented on code in PR #4515:
URL: https://github.com/apache/gravitino/pull/4515#discussion_r1726208894
##
authorizations/authorization-ranger/src/main/java/org/apache/gravitino/authorization/ranger/RangerAuthorizationPlugin.java:
##
@@ -0,0 +1,1006 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+package org.apache.gravitino.authorization.ranger;
+
+import com.google.common.annotations.VisibleForTesting;
+import com.google.common.collect.ImmutableMap;
+import com.google.common.collect.Lists;
+import com.google.errorprone.annotations.FormatMethod;
+import com.google.errorprone.annotations.FormatString;
+import java.io.IOException;
+import java.time.Instant;
+import java.util.Arrays;
+import java.util.Collections;
+import java.util.HashMap;
+import java.util.HashSet;
+import java.util.List;
+import java.util.Map;
+import java.util.Objects;
+import java.util.Set;
+import java.util.stream.Collectors;
+import org.apache.gravitino.MetadataObject;
+import org.apache.gravitino.authorization.Group;
+import org.apache.gravitino.authorization.Owner;
+import org.apache.gravitino.authorization.Privilege;
+import org.apache.gravitino.authorization.Role;
+import org.apache.gravitino.authorization.RoleChange;
+import org.apache.gravitino.authorization.SecurableObject;
+import org.apache.gravitino.authorization.SecurableObjects;
+import org.apache.gravitino.authorization.User;
+import org.apache.gravitino.authorization.ranger.defines.VXGroup;
+import org.apache.gravitino.authorization.ranger.defines.VXGroupList;
+import org.apache.gravitino.authorization.ranger.defines.VXUser;
+import org.apache.gravitino.authorization.ranger.defines.VXUserList;
+import org.apache.gravitino.connector.AuthorizationPropertiesMeta;
+import org.apache.gravitino.connector.authorization.AuthorizationPlugin;
+import org.apache.gravitino.exceptions.AuthorizationPluginException;
+import org.apache.gravitino.meta.AuditInfo;
+import org.apache.gravitino.meta.GroupEntity;
+import org.apache.gravitino.meta.UserEntity;
+import org.apache.gravitino.utils.PrincipalUtils;
+import org.apache.ranger.RangerServiceException;
+import org.apache.ranger.plugin.model.RangerPolicy;
+import org.apache.ranger.plugin.model.RangerRole;
+import org.apache.ranger.plugin.util.GrantRevokeRoleRequest;
+import org.apache.ranger.plugin.util.SearchFilter;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+/** Ranger authorization operations plugin abstract class. */
+public abstract class RangerAuthorizationPlugin implements AuthorizationPlugin
{
+ private static final Logger LOG =
LoggerFactory.getLogger(RangerAuthorizationPlugin.class);
+
+ protected String catalogProvider;
+ protected RangerClientExtend rangerClient;
+ protected String rangerServiceName;
+ /** Mapping Gravitino privilege name to the underlying authorization system
privileges. */
+ protected Map> mapPrivileges = null;
+ // The owner privileges, the owner can do anything on the metadata object
+ protected Set ownerPrivileges = null;
+
+ /**
+ * Because Ranger doesn't support the precise filter, Ranger will return the
policy meets the
+ * wildcard(*,?) conditions, just like `*.*.*` policy will match
`db1.table1.column1` So we need
+ * to manual precise filter the policies.
+ */
+ // Search Ranger policy filter keys
+ protected List policyFilterKeys = null;
+ // Search Ranger policy precise filter keys
+ protected List policyPreciseFilterKeys = null;
+
+ public static final String MANAGED_BY_GRAVITINO = "MANAGED_BY_GRAVITINO";
+
+ // TODO: Maybe need to move to the configuration in the future
+ public static final String RANGER_ADMIN_NAME = "admin";
+
+ public RangerAuthorizationPlugin(String catalogProvider, Map
config) {
+super();
+this.catalogProvider = catalogProvider;
+String rangerUrl =
config.get(AuthorizationPropertiesMeta.RANGER_ADMIN_URL);
+String authType = config.get(AuthorizationPropertiesMeta.RANGER_AUTH_TYPE);
+String username = config.get(AuthorizationPropertiesMeta.RANGER_USERNAME);
+// Apache Ranger Password should be minimum 8 characters with min one
alphabet and one numeric.
+String password = config.get(AuthorizationProperties
Re: [PR] [#3963] feat(core): Apache Ranger Hive authorization pushdown [gravitino]
xunliu commented on code in PR #4515: URL: https://github.com/apache/gravitino/pull/4515#discussion_r1724849823 ## authorizations/authorization-ranger/src/main/resources/META-INF/services/org.apache.gravitino.connector.authorization.AuthorizationProvider: ## @@ -0,0 +1,19 @@ +# +# Licensed to the Apache Software Foundation (ASF) under one +# or more contributor license agreements. See the NOTICE file +# distributed with this work for additional information +# regarding copyright ownership. The ASF licenses this file +# to you under the Apache License, Version 2.0 (the +# "License"); you may not use this file except in compliance +# with the License. You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, +# software distributed under the License is distributed on an +# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY +# KIND, either express or implied. See the License for the +# specific language governing permissions and limitations +# under the License. +# +org.apache.gravitino.authorization.ranger.RangerAuthorization Review Comment: We can add multiple Authorization in the `authorization.AuthorizationProvider`, just like https://github.com/apache/gravitino/blob/main/core/src/test/resources/META-INF/services/org.apache.gravitino.connector.authorization.AuthorizationProvider#L19 Look more carefully before you jump to conclusions -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
Re: [PR] [#3963] feat(core): Apache Ranger Hive authorization pushdown [gravitino]
xunliu commented on code in PR #4515:
URL: https://github.com/apache/gravitino/pull/4515#discussion_r1726200490
##
authorizations/authorization-ranger/src/main/java/org/apache/gravitino/authorization/ranger/RangerHelper.java:
##
@@ -0,0 +1,533 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+package org.apache.gravitino.authorization.ranger;
+
+import com.google.common.annotations.VisibleForTesting;
+import com.google.common.collect.ImmutableMap;
+import com.google.common.collect.ImmutableSet;
+import com.google.common.collect.Lists;
+import com.google.common.collect.Sets;
+import com.google.errorprone.annotations.FormatMethod;
+import com.google.errorprone.annotations.FormatString;
+import java.util.Arrays;
+import java.util.HashMap;
+import java.util.HashSet;
+import java.util.List;
+import java.util.Map;
+import java.util.Objects;
+import java.util.Set;
+import java.util.stream.Collectors;
+import org.apache.commons.lang.StringUtils;
+import org.apache.gravitino.MetadataObject;
+import org.apache.gravitino.authorization.Owner;
+import org.apache.gravitino.authorization.Privilege;
+import org.apache.gravitino.authorization.SecurableObject;
+import org.apache.gravitino.authorization.SecurableObjects;
+import org.apache.gravitino.authorization.ranger.reference.RangerDefines;
+import org.apache.gravitino.exceptions.AuthorizationPluginException;
+import org.apache.ranger.RangerServiceException;
+import org.apache.ranger.plugin.model.RangerPolicy;
+import org.apache.ranger.plugin.model.RangerRole;
+import org.apache.ranger.plugin.util.GrantRevokeRoleRequest;
+import org.apache.ranger.plugin.util.SearchFilter;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+/**
+ * This class is a helper class for the Ranger authorization plugin. It
provides the ability to
+ * manage the Ranger policies and roles.
+ */
+public class RangerHelper {
+ private static final Logger LOG =
LoggerFactory.getLogger(RangerHelper.class);
+ public static final String MANAGED_BY_GRAVITINO = "MANAGED_BY_GRAVITINO";
+
+ RangerAuthorizationPlugin rangerAuthorizationPlugin;
+
+ /** Mapping Gravitino privilege name to the underlying authorization system
privileges. */
+ protected Map> privilegesMapping = null;
+ /** The owner privileges, the owner can do anything on the metadata object */
+ protected Set ownerPrivileges = null;
+
+ /**
+ * Because Ranger doesn't support the precise search, Ranger will return the
policy meets the
+ * wildcard(*,?) conditions, If you use `db.table` condition to search
policy, the Ranger will
+ * match `db1.table1`, `db1.table2`, `db*.table*`, So we need to manually
precisely filter this
+ * research results.
+ * policySearchKeys: The search Ranger policy condition key defines.
+ * policyPreciseFilterKeys: The precise filter Ranger search results key
defines
+ */
+ protected List policySearchKeys = null;
+
+ protected List policyPreciseFilterKeys = null;
+
+ public RangerHelper(RangerAuthorizationPlugin rangerAuthorizationPlugin,
String catalogProvider) {
+this.rangerAuthorizationPlugin = rangerAuthorizationPlugin;
+switch (catalogProvider) {
+ case "hive":
+initPrivilegesMapping();
+initOwnerPrivileges();
+initPolicySearchKeys();
+initPreciseFilterKeys();
+break;
+ default:
+throw new IllegalArgumentException(
+"Authorization plugin unsupported catalog provider: " +
catalogProvider);
+}
+ }
+
+ /** Initial mapping Gravitino privilege name to the underlying authorization
system privileges. */
+ private void initPrivilegesMapping() {
+privilegesMapping =
+ImmutableMap.>builder()
+.put(
+Privilege.Name.CREATE_SCHEMA,
+ImmutableSet.of(RangerDefines.ACCESS_TYPE_HIVE_CREATE))
+.put(
+Privilege.Name.CREATE_TABLE,
ImmutableSet.of(RangerDefines.ACCESS_TYPE_HIVE_CREATE))
+.put(
+Privilege.Name.MODIFY_TABLE,
+ImmutableSet.of(
+RangerDefines.ACCESS_TYPE_HIVE_UPDATE,
+RangerDefines.ACCESS_TYPE_HIVE_ALTER,
+RangerDefines.A
Re: [PR] [#3963] feat(core): Apache Ranger Hive authorization pushdown [gravitino]
jerqi commented on code in PR #4515:
URL: https://github.com/apache/gravitino/pull/4515#discussion_r1725098417
##
authorizations/authorization-ranger/src/main/java/org/apache/gravitino/authorization/ranger/RangerHelper.java:
##
@@ -0,0 +1,533 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+package org.apache.gravitino.authorization.ranger;
+
+import com.google.common.annotations.VisibleForTesting;
+import com.google.common.collect.ImmutableMap;
+import com.google.common.collect.ImmutableSet;
+import com.google.common.collect.Lists;
+import com.google.common.collect.Sets;
+import com.google.errorprone.annotations.FormatMethod;
+import com.google.errorprone.annotations.FormatString;
+import java.util.Arrays;
+import java.util.HashMap;
+import java.util.HashSet;
+import java.util.List;
+import java.util.Map;
+import java.util.Objects;
+import java.util.Set;
+import java.util.stream.Collectors;
+import org.apache.commons.lang.StringUtils;
+import org.apache.gravitino.MetadataObject;
+import org.apache.gravitino.authorization.Owner;
+import org.apache.gravitino.authorization.Privilege;
+import org.apache.gravitino.authorization.SecurableObject;
+import org.apache.gravitino.authorization.SecurableObjects;
+import org.apache.gravitino.authorization.ranger.reference.RangerDefines;
+import org.apache.gravitino.exceptions.AuthorizationPluginException;
+import org.apache.ranger.RangerServiceException;
+import org.apache.ranger.plugin.model.RangerPolicy;
+import org.apache.ranger.plugin.model.RangerRole;
+import org.apache.ranger.plugin.util.GrantRevokeRoleRequest;
+import org.apache.ranger.plugin.util.SearchFilter;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+/**
+ * This class is a helper class for the Ranger authorization plugin. It
provides the ability to
+ * manage the Ranger policies and roles.
+ */
+public class RangerHelper {
+ private static final Logger LOG =
LoggerFactory.getLogger(RangerHelper.class);
+ public static final String MANAGED_BY_GRAVITINO = "MANAGED_BY_GRAVITINO";
+
+ RangerAuthorizationPlugin rangerAuthorizationPlugin;
+
+ /** Mapping Gravitino privilege name to the underlying authorization system
privileges. */
+ protected Map> privilegesMapping = null;
+ /** The owner privileges, the owner can do anything on the metadata object */
+ protected Set ownerPrivileges = null;
+
+ /**
+ * Because Ranger doesn't support the precise search, Ranger will return the
policy meets the
+ * wildcard(*,?) conditions, If you use `db.table` condition to search
policy, the Ranger will
+ * match `db1.table1`, `db1.table2`, `db*.table*`, So we need to manually
precisely filter this
+ * research results.
+ * policySearchKeys: The search Ranger policy condition key defines.
+ * policyPreciseFilterKeys: The precise filter Ranger search results key
defines
+ */
+ protected List policySearchKeys = null;
+
+ protected List policyPreciseFilterKeys = null;
+
+ public RangerHelper(RangerAuthorizationPlugin rangerAuthorizationPlugin,
String catalogProvider) {
+this.rangerAuthorizationPlugin = rangerAuthorizationPlugin;
+switch (catalogProvider) {
+ case "hive":
+initPrivilegesMapping();
+initOwnerPrivileges();
+initPolicySearchKeys();
+initPreciseFilterKeys();
+break;
+ default:
+throw new IllegalArgumentException(
+"Authorization plugin unsupported catalog provider: " +
catalogProvider);
+}
+ }
+
+ /** Initial mapping Gravitino privilege name to the underlying authorization
system privileges. */
+ private void initPrivilegesMapping() {
+privilegesMapping =
+ImmutableMap.>builder()
+.put(
+Privilege.Name.CREATE_SCHEMA,
+ImmutableSet.of(RangerDefines.ACCESS_TYPE_HIVE_CREATE))
+.put(
+Privilege.Name.CREATE_TABLE,
ImmutableSet.of(RangerDefines.ACCESS_TYPE_HIVE_CREATE))
+.put(
+Privilege.Name.MODIFY_TABLE,
+ImmutableSet.of(
+RangerDefines.ACCESS_TYPE_HIVE_UPDATE,
+RangerDefines.ACCESS_TYPE_HIVE_ALTER,
+RangerDefines.AC
Re: [PR] [#3963] feat(core): Apache Ranger Hive authorization pushdown [gravitino]
xunliu commented on code in PR #4515:
URL: https://github.com/apache/gravitino/pull/4515#discussion_r1724962022
##
authorizations/authorization-ranger/src/test/java/org/apache/gravitino/authorization/ranger/integration/test/RangerITEnv.java:
##
@@ -252,7 +257,7 @@ protected static String updateOrCreateRangerPolicy(
if (!policies.isEmpty()) {
// Because Ranger user the wildcard filter, Ranger will return the
policy meets
// the wildcard(*,?) conditions, just like `*.*.*` policy will match
`db1.table1.column1`
Review Comment:
DONE
##
authorizations/authorization-ranger/src/main/java/org/apache/gravitino/authorization/ranger/RangerHelper.java:
##
@@ -0,0 +1,533 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+package org.apache.gravitino.authorization.ranger;
+
+import com.google.common.annotations.VisibleForTesting;
+import com.google.common.collect.ImmutableMap;
+import com.google.common.collect.ImmutableSet;
+import com.google.common.collect.Lists;
+import com.google.common.collect.Sets;
+import com.google.errorprone.annotations.FormatMethod;
+import com.google.errorprone.annotations.FormatString;
+import java.util.Arrays;
+import java.util.HashMap;
+import java.util.HashSet;
+import java.util.List;
+import java.util.Map;
+import java.util.Objects;
+import java.util.Set;
+import java.util.stream.Collectors;
+import org.apache.commons.lang.StringUtils;
+import org.apache.gravitino.MetadataObject;
+import org.apache.gravitino.authorization.Owner;
+import org.apache.gravitino.authorization.Privilege;
+import org.apache.gravitino.authorization.SecurableObject;
+import org.apache.gravitino.authorization.SecurableObjects;
+import org.apache.gravitino.authorization.ranger.reference.RangerDefines;
+import org.apache.gravitino.exceptions.AuthorizationPluginException;
+import org.apache.ranger.RangerServiceException;
+import org.apache.ranger.plugin.model.RangerPolicy;
+import org.apache.ranger.plugin.model.RangerRole;
+import org.apache.ranger.plugin.util.GrantRevokeRoleRequest;
+import org.apache.ranger.plugin.util.SearchFilter;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+/**
+ * This class is a helper class for the Ranger authorization plugin. It
provides the ability to
+ * manage the Ranger policies and roles.
+ */
+public class RangerHelper {
+ private static final Logger LOG =
LoggerFactory.getLogger(RangerHelper.class);
+ public static final String MANAGED_BY_GRAVITINO = "MANAGED_BY_GRAVITINO";
+
+ RangerAuthorizationPlugin rangerAuthorizationPlugin;
+
+ /** Mapping Gravitino privilege name to the underlying authorization system
privileges. */
+ protected Map> privilegesMapping = null;
+ /** The owner privileges, the owner can do anything on the metadata object */
+ protected Set ownerPrivileges = null;
+
+ /**
+ * Because Ranger doesn't support the precise search, Ranger will return the
policy meets the
+ * wildcard(*,?) conditions, If you use `db.table` condition to search
policy, the Ranger will
+ * match `db1.table1`, `db1.table2`, `db*.table*`, So we need to manually
precisely filter this
+ * research results.
+ * policySearchKeys: The search Ranger policy condition key defines.
+ * policyPreciseFilterKeys: The precise filter Ranger search results key
defines
+ */
+ protected List policySearchKeys = null;
+
+ protected List policyPreciseFilterKeys = null;
+
+ public RangerHelper(RangerAuthorizationPlugin rangerAuthorizationPlugin,
String catalogProvider) {
+this.rangerAuthorizationPlugin = rangerAuthorizationPlugin;
+switch (catalogProvider) {
+ case "hive":
+initPrivilegesMapping();
+initOwnerPrivileges();
+initPolicySearchKeys();
+initPreciseFilterKeys();
+break;
+ default:
+throw new IllegalArgumentException(
+"Authorization plugin unsupported catalog provider: " +
catalogProvider);
+}
+ }
+
+ /** Initial mapping Gravitino privilege name to the underlying authorization
system privileges. */
+ private void initPrivilegesMapping() {
+privilegesMapping =
+ImmutableMap.>builder()
+.put(
+Privil
Re: [PR] [#3963] feat(core): Apache Ranger Hive authorization pushdown [gravitino]
xunliu commented on PR #4515: URL: https://github.com/apache/gravitino/pull/4515#issuecomment-2302034040 > Only one point: > > 1. Some privileges like `use schema`, `use catalog`, `create catalog` could be pushed down underlying system, too. The check logic will throw exception for them. I created EPIC https://github.com/apache/gravitino/pull/4515 to track these issue. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
Re: [PR] [#3963] feat(core): Apache Ranger Hive authorization pushdown [gravitino]
jerqi commented on code in PR #4515:
URL: https://github.com/apache/gravitino/pull/4515#discussion_r1725005020
##
authorizations/authorization-ranger/src/main/java/org/apache/gravitino/authorization/ranger/RangerAuthorizationPlugin.java:
##
@@ -0,0 +1,583 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+package org.apache.gravitino.authorization.ranger;
+
+import com.google.common.annotations.VisibleForTesting;
+import com.google.common.collect.ImmutableMap;
+import com.google.common.collect.Lists;
+import java.io.IOException;
+import java.time.Instant;
+import java.util.Collections;
+import java.util.List;
+import java.util.Map;
+import java.util.Objects;
+import java.util.Set;
+import java.util.stream.Collectors;
+import org.apache.gravitino.MetadataObject;
+import org.apache.gravitino.authorization.Group;
+import org.apache.gravitino.authorization.Owner;
+import org.apache.gravitino.authorization.Privilege;
+import org.apache.gravitino.authorization.Role;
+import org.apache.gravitino.authorization.RoleChange;
+import org.apache.gravitino.authorization.SecurableObjects;
+import org.apache.gravitino.authorization.User;
+import org.apache.gravitino.authorization.ranger.reference.RangerDefines;
+import org.apache.gravitino.authorization.ranger.reference.VXGroup;
+import org.apache.gravitino.authorization.ranger.reference.VXGroupList;
+import org.apache.gravitino.authorization.ranger.reference.VXUser;
+import org.apache.gravitino.authorization.ranger.reference.VXUserList;
+import org.apache.gravitino.connector.AuthorizationPropertiesMeta;
+import org.apache.gravitino.connector.authorization.AuthorizationPlugin;
+import org.apache.gravitino.meta.AuditInfo;
+import org.apache.gravitino.meta.GroupEntity;
+import org.apache.gravitino.meta.UserEntity;
+import org.apache.gravitino.utils.PrincipalUtils;
+import org.apache.ranger.RangerServiceException;
+import org.apache.ranger.plugin.model.RangerPolicy;
+import org.apache.ranger.plugin.util.GrantRevokeRoleRequest;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+/**
+ * Ranger authorization operations plugin abstract class.
+ * 1. For Ranger limit, The same metadata object only has a unique Ranger
policy, So a Ranger policy
+ * maybe contains multiple Gravitino securable objects.
+ * 2. For easy management, each Ranger privilege will create one
RangerPolicyItemAccess in the
+ * policy.
+ * 3. Ranger also have Role concept, and support adds multiple users or groups
in the Ranger Role,
+ * So we can use the Ranger Role to implement the Gravitino Role.
+ * 4. The Ranger policy also supports multiple users and groups, But we only
use a user or group to
+ * implement Gravitino Owner concept.
+ */
+public class RangerAuthorizationPlugin implements AuthorizationPlugin {
+ private static final Logger LOG =
LoggerFactory.getLogger(RangerAuthorizationPlugin.class);
+
+ protected String catalogProvider;
+ protected String rangerServiceName;
+ protected RangerClientExtend rangerClient;
+ private RangerHelper rangerHelper;
+ @VisibleForTesting public final String rangerAdminName;
+
+ public RangerAuthorizationPlugin(String catalogProvider, Map
config) {
+super();
+this.catalogProvider = catalogProvider;
+String rangerUrl =
config.get(AuthorizationPropertiesMeta.RANGER_ADMIN_URL);
+String authType = config.get(AuthorizationPropertiesMeta.RANGER_AUTH_TYPE);
+rangerAdminName = config.get(AuthorizationPropertiesMeta.RANGER_USERNAME);
+// Apache Ranger Password should be minimum 8 characters with min one
alphabet and one numeric.
+String password = config.get(AuthorizationPropertiesMeta.RANGER_PASSWORD);
+rangerServiceName =
config.get(AuthorizationPropertiesMeta.RANGER_SERVICE_NAME);
+RangerHelper.check(rangerUrl != null, "Ranger admin URL is required");
+RangerHelper.check(authType != null, "Ranger auth type is required");
+RangerHelper.check(rangerAdminName != null, "Ranger username is required");
+RangerHelper.check(password != null, "Ranger password is required");
+RangerHelper.check(rangerServiceName != null, "Ranger service name is
required");
+rangerClient = new RangerClientExtend(rangerUrl, authType
Re: [PR] [#3963] feat(core): Apache Ranger Hive authorization pushdown [gravitino]
yuqi1129 commented on code in PR #4515:
URL: https://github.com/apache/gravitino/pull/4515#discussion_r1724993429
##
authorizations/authorization-ranger/src/main/java/org/apache/gravitino/authorization/ranger/RangerAuthorizationPlugin.java:
##
@@ -0,0 +1,556 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+package org.apache.gravitino.authorization.ranger;
+
+import com.google.common.annotations.VisibleForTesting;
+import com.google.common.collect.ImmutableMap;
+import com.google.common.collect.Lists;
+import java.io.IOException;
+import java.time.Instant;
+import java.util.Collections;
+import java.util.List;
+import java.util.Map;
+import java.util.Objects;
+import java.util.Set;
+import java.util.stream.Collectors;
+import org.apache.gravitino.MetadataObject;
+import org.apache.gravitino.authorization.Group;
+import org.apache.gravitino.authorization.Owner;
+import org.apache.gravitino.authorization.Privilege;
+import org.apache.gravitino.authorization.Role;
+import org.apache.gravitino.authorization.RoleChange;
+import org.apache.gravitino.authorization.User;
+import org.apache.gravitino.authorization.ranger.reference.VXGroup;
+import org.apache.gravitino.authorization.ranger.reference.VXGroupList;
+import org.apache.gravitino.authorization.ranger.reference.VXUser;
+import org.apache.gravitino.authorization.ranger.reference.VXUserList;
+import org.apache.gravitino.connector.AuthorizationPropertiesMeta;
+import org.apache.gravitino.connector.authorization.AuthorizationPlugin;
+import org.apache.gravitino.meta.AuditInfo;
+import org.apache.gravitino.meta.GroupEntity;
+import org.apache.gravitino.meta.UserEntity;
+import org.apache.gravitino.utils.PrincipalUtils;
+import org.apache.ranger.RangerServiceException;
+import org.apache.ranger.plugin.model.RangerPolicy;
+import org.apache.ranger.plugin.util.GrantRevokeRoleRequest;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+/**
+ * Ranger authorization operations plugin abstract class.
+ * 1. For Ranger limit, The same metadata object only has a unique Ranger
policy, So a Ranger policy
+ * maybe contains multiple Gravitino securable objects.
+ * 2. For easy management, each Ranger privilege will create one
RangerPolicyItemAccess in the
+ * policy.
+ * 3. Ranger also have Role concept, and support adds multiple users or groups
in the Ranger Role,
+ * So we can use the Ranger Role to implement the Gravitino Role.
+ * 4. The Ranger policy also supports multiple users and groups, But we only
use a user or group to
+ * implement Gravitino Owner concept.
+ */
+public class RangerAuthorizationPlugin implements AuthorizationPlugin {
+ private static final Logger LOG =
LoggerFactory.getLogger(RangerAuthorizationPlugin.class);
+
+ protected String catalogProvider;
+ protected String rangerServiceName;
+ protected RangerClientExtend rangerClient;
+ private RangerHelper rangerHelper;
+ @VisibleForTesting public final String rangerAdminName;
+
+ public RangerAuthorizationPlugin(String catalogProvider, Map
config) {
+super();
+this.catalogProvider = catalogProvider;
+String rangerUrl =
config.get(AuthorizationPropertiesMeta.RANGER_ADMIN_URL);
+String authType = config.get(AuthorizationPropertiesMeta.RANGER_AUTH_TYPE);
+rangerAdminName = config.get(AuthorizationPropertiesMeta.RANGER_USERNAME);
+// Apache Ranger Password should be minimum 8 characters with min one
alphabet and one numeric.
+String password = config.get(AuthorizationPropertiesMeta.RANGER_PASSWORD);
+rangerServiceName =
config.get(AuthorizationPropertiesMeta.RANGER_SERVICE_NAME);
+RangerHelper.check(rangerUrl != null, "Ranger admin URL is required");
+RangerHelper.check(authType != null, "Ranger auth type is required");
+RangerHelper.check(rangerAdminName != null, "Ranger username is required");
+RangerHelper.check(password != null, "Ranger password is required");
+RangerHelper.check(rangerServiceName != null, "Ranger service name is
required");
+rangerClient = new RangerClientExtend(rangerUrl, authType,
rangerAdminName, password);
+rangerHelper = new RangerHelper(this, catalogProvider);
+ }
+
+ /**
+ * Translate the privile
Re: [PR] [#3963] feat(core): Apache Ranger Hive authorization pushdown [gravitino]
jerqi commented on code in PR #4515:
URL: https://github.com/apache/gravitino/pull/4515#discussion_r1724987141
##
authorizations/authorization-ranger/src/main/java/org/apache/gravitino/authorization/ranger/RangerHelper.java:
##
@@ -0,0 +1,539 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+package org.apache.gravitino.authorization.ranger;
+
+import com.google.common.annotations.VisibleForTesting;
+import com.google.common.collect.ImmutableMap;
+import com.google.common.collect.ImmutableSet;
+import com.google.common.collect.Lists;
+import com.google.common.collect.Sets;
+import com.google.errorprone.annotations.FormatMethod;
+import com.google.errorprone.annotations.FormatString;
+import java.util.Arrays;
+import java.util.HashMap;
+import java.util.HashSet;
+import java.util.List;
+import java.util.Map;
+import java.util.Objects;
+import java.util.Set;
+import java.util.stream.Collectors;
+import org.apache.commons.lang.StringUtils;
+import org.apache.gravitino.MetadataObject;
+import org.apache.gravitino.authorization.Owner;
+import org.apache.gravitino.authorization.Privilege;
+import org.apache.gravitino.authorization.SecurableObject;
+import org.apache.gravitino.authorization.SecurableObjects;
+import org.apache.gravitino.authorization.ranger.reference.RangerDefines;
+import org.apache.gravitino.exceptions.AuthorizationPluginException;
+import org.apache.ranger.RangerServiceException;
+import org.apache.ranger.plugin.model.RangerPolicy;
+import org.apache.ranger.plugin.model.RangerRole;
+import org.apache.ranger.plugin.util.GrantRevokeRoleRequest;
+import org.apache.ranger.plugin.util.SearchFilter;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+/**
+ * This class is a helper class for the Ranger authorization plugin. It
provides the ability to
+ * manage the Ranger policies and roles.
+ */
+public class RangerHelper {
+ private static final Logger LOG =
LoggerFactory.getLogger(RangerHelper.class);
+
+ public static final String MANAGED_BY_GRAVITINO = "MANAGED_BY_GRAVITINO";
+ RangerAuthorizationPlugin rangerAuthorizationPlugin;
+
+ /** Mapping Gravitino privilege name to the underlying authorization system
privileges. */
+ protected Map> privilegesMapping = null;
+ /** The owner privileges, the owner can do anything on the metadata object */
+ protected Set ownerPrivileges = null;
+
+ /**
+ * Because Ranger doesn't support the precise search, Ranger will return the
policy meets the
+ * wildcard(*,?) conditions, If you use `db.table` condition to search
policy, the Ranger will
+ * match `db1.table1`, `db1.table2`, `db*.table*`, So we need to manually
precisely filter this
+ * research results.
+ * policySearchKeys: The search Ranger policy condition key defines.
+ * policyPreciseFilterKeys: The precise filter Ranger search results key
defines
+ */
+ protected List policySearchKeys = null;
+
+ protected List policyPreciseFilterKeys = null;
+
+ public RangerHelper(RangerAuthorizationPlugin rangerAuthorizationPlugin,
String catalogProvider) {
+this.rangerAuthorizationPlugin = rangerAuthorizationPlugin;
+switch (catalogProvider) {
+ case "hive":
+initPrivilegesMapping();
+initOwnerPrivileges();
+initPolicySearchKeys();
+initPreciseFilterKeys();
+break;
+ default:
+throw new IllegalArgumentException(
+"Authorization plugin unsupported catalog provider: " +
catalogProvider);
+}
+ }
+
+ /** Initial mapping Gravitino privilege name to the underlying authorization
system privileges. */
+ private void initPrivilegesMapping() {
+privilegesMapping =
+ImmutableMap.>builder()
+.put(
+Privilege.Name.CREATE_SCHEMA,
+ImmutableSet.of(RangerDefines.ACCESS_TYPE_HIVE_CREATE))
+.put(
+Privilege.Name.CREATE_TABLE,
ImmutableSet.of(RangerDefines.ACCESS_TYPE_HIVE_CREATE))
+.put(
+Privilege.Name.MODIFY_TABLE,
+ImmutableSet.of(
+RangerDefines.ACCESS_TYPE_HIVE_UPDATE,
+RangerDefines.ACCESS_TYPE_HIVE_ALTER,
+RangerDefines.AC
Re: [PR] [#3963] feat(core): Apache Ranger Hive authorization pushdown [gravitino]
jerqi commented on code in PR #4515:
URL: https://github.com/apache/gravitino/pull/4515#discussion_r1724906342
##
authorizations/authorization-ranger/src/main/java/org/apache/gravitino/authorization/ranger/RangerHelper.java:
##
@@ -0,0 +1,533 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+package org.apache.gravitino.authorization.ranger;
+
+import com.google.common.annotations.VisibleForTesting;
+import com.google.common.collect.ImmutableMap;
+import com.google.common.collect.ImmutableSet;
+import com.google.common.collect.Lists;
+import com.google.common.collect.Sets;
+import com.google.errorprone.annotations.FormatMethod;
+import com.google.errorprone.annotations.FormatString;
+import java.util.Arrays;
+import java.util.HashMap;
+import java.util.HashSet;
+import java.util.List;
+import java.util.Map;
+import java.util.Objects;
+import java.util.Set;
+import java.util.stream.Collectors;
+import org.apache.commons.lang.StringUtils;
+import org.apache.gravitino.MetadataObject;
+import org.apache.gravitino.authorization.Owner;
+import org.apache.gravitino.authorization.Privilege;
+import org.apache.gravitino.authorization.SecurableObject;
+import org.apache.gravitino.authorization.SecurableObjects;
+import org.apache.gravitino.authorization.ranger.reference.RangerDefines;
+import org.apache.gravitino.exceptions.AuthorizationPluginException;
+import org.apache.ranger.RangerServiceException;
+import org.apache.ranger.plugin.model.RangerPolicy;
+import org.apache.ranger.plugin.model.RangerRole;
+import org.apache.ranger.plugin.util.GrantRevokeRoleRequest;
+import org.apache.ranger.plugin.util.SearchFilter;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+/**
+ * This class is a helper class for the Ranger authorization plugin. It
provides the ability to
+ * manage the Ranger policies and roles.
+ */
+public class RangerHelper {
+ private static final Logger LOG =
LoggerFactory.getLogger(RangerHelper.class);
+ public static final String MANAGED_BY_GRAVITINO = "MANAGED_BY_GRAVITINO";
+
+ RangerAuthorizationPlugin rangerAuthorizationPlugin;
+
+ /** Mapping Gravitino privilege name to the underlying authorization system
privileges. */
+ protected Map> privilegesMapping = null;
+ /** The owner privileges, the owner can do anything on the metadata object */
+ protected Set ownerPrivileges = null;
+
+ /**
+ * Because Ranger doesn't support the precise search, Ranger will return the
policy meets the
+ * wildcard(*,?) conditions, If you use `db.table` condition to search
policy, the Ranger will
+ * match `db1.table1`, `db1.table2`, `db*.table*`, So we need to manually
precisely filter this
+ * research results.
+ * policySearchKeys: The search Ranger policy condition key defines.
+ * policyPreciseFilterKeys: The precise filter Ranger search results key
defines
+ */
+ protected List policySearchKeys = null;
+
+ protected List policyPreciseFilterKeys = null;
+
+ public RangerHelper(RangerAuthorizationPlugin rangerAuthorizationPlugin,
String catalogProvider) {
+this.rangerAuthorizationPlugin = rangerAuthorizationPlugin;
+switch (catalogProvider) {
+ case "hive":
+initPrivilegesMapping();
+initOwnerPrivileges();
+initPolicySearchKeys();
+initPreciseFilterKeys();
+break;
+ default:
+throw new IllegalArgumentException(
+"Authorization plugin unsupported catalog provider: " +
catalogProvider);
+}
+ }
+
+ /** Initial mapping Gravitino privilege name to the underlying authorization
system privileges. */
+ private void initPrivilegesMapping() {
+privilegesMapping =
+ImmutableMap.>builder()
+.put(
+Privilege.Name.CREATE_SCHEMA,
+ImmutableSet.of(RangerDefines.ACCESS_TYPE_HIVE_CREATE))
+.put(
+Privilege.Name.CREATE_TABLE,
ImmutableSet.of(RangerDefines.ACCESS_TYPE_HIVE_CREATE))
+.put(
+Privilege.Name.MODIFY_TABLE,
+ImmutableSet.of(
+RangerDefines.ACCESS_TYPE_HIVE_UPDATE,
+RangerDefines.ACCESS_TYPE_HIVE_ALTER,
+RangerDefines.AC
Re: [PR] [#3963] feat(core): Apache Ranger Hive authorization pushdown [gravitino]
jerqi commented on code in PR #4515:
URL: https://github.com/apache/gravitino/pull/4515#discussion_r1724906342
##
authorizations/authorization-ranger/src/main/java/org/apache/gravitino/authorization/ranger/RangerHelper.java:
##
@@ -0,0 +1,533 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+package org.apache.gravitino.authorization.ranger;
+
+import com.google.common.annotations.VisibleForTesting;
+import com.google.common.collect.ImmutableMap;
+import com.google.common.collect.ImmutableSet;
+import com.google.common.collect.Lists;
+import com.google.common.collect.Sets;
+import com.google.errorprone.annotations.FormatMethod;
+import com.google.errorprone.annotations.FormatString;
+import java.util.Arrays;
+import java.util.HashMap;
+import java.util.HashSet;
+import java.util.List;
+import java.util.Map;
+import java.util.Objects;
+import java.util.Set;
+import java.util.stream.Collectors;
+import org.apache.commons.lang.StringUtils;
+import org.apache.gravitino.MetadataObject;
+import org.apache.gravitino.authorization.Owner;
+import org.apache.gravitino.authorization.Privilege;
+import org.apache.gravitino.authorization.SecurableObject;
+import org.apache.gravitino.authorization.SecurableObjects;
+import org.apache.gravitino.authorization.ranger.reference.RangerDefines;
+import org.apache.gravitino.exceptions.AuthorizationPluginException;
+import org.apache.ranger.RangerServiceException;
+import org.apache.ranger.plugin.model.RangerPolicy;
+import org.apache.ranger.plugin.model.RangerRole;
+import org.apache.ranger.plugin.util.GrantRevokeRoleRequest;
+import org.apache.ranger.plugin.util.SearchFilter;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+/**
+ * This class is a helper class for the Ranger authorization plugin. It
provides the ability to
+ * manage the Ranger policies and roles.
+ */
+public class RangerHelper {
+ private static final Logger LOG =
LoggerFactory.getLogger(RangerHelper.class);
+ public static final String MANAGED_BY_GRAVITINO = "MANAGED_BY_GRAVITINO";
+
+ RangerAuthorizationPlugin rangerAuthorizationPlugin;
+
+ /** Mapping Gravitino privilege name to the underlying authorization system
privileges. */
+ protected Map> privilegesMapping = null;
+ /** The owner privileges, the owner can do anything on the metadata object */
+ protected Set ownerPrivileges = null;
+
+ /**
+ * Because Ranger doesn't support the precise search, Ranger will return the
policy meets the
+ * wildcard(*,?) conditions, If you use `db.table` condition to search
policy, the Ranger will
+ * match `db1.table1`, `db1.table2`, `db*.table*`, So we need to manually
precisely filter this
+ * research results.
+ * policySearchKeys: The search Ranger policy condition key defines.
+ * policyPreciseFilterKeys: The precise filter Ranger search results key
defines
+ */
+ protected List policySearchKeys = null;
+
+ protected List policyPreciseFilterKeys = null;
+
+ public RangerHelper(RangerAuthorizationPlugin rangerAuthorizationPlugin,
String catalogProvider) {
+this.rangerAuthorizationPlugin = rangerAuthorizationPlugin;
+switch (catalogProvider) {
+ case "hive":
+initPrivilegesMapping();
+initOwnerPrivileges();
+initPolicySearchKeys();
+initPreciseFilterKeys();
+break;
+ default:
+throw new IllegalArgumentException(
+"Authorization plugin unsupported catalog provider: " +
catalogProvider);
+}
+ }
+
+ /** Initial mapping Gravitino privilege name to the underlying authorization
system privileges. */
+ private void initPrivilegesMapping() {
+privilegesMapping =
+ImmutableMap.>builder()
+.put(
+Privilege.Name.CREATE_SCHEMA,
+ImmutableSet.of(RangerDefines.ACCESS_TYPE_HIVE_CREATE))
+.put(
+Privilege.Name.CREATE_TABLE,
ImmutableSet.of(RangerDefines.ACCESS_TYPE_HIVE_CREATE))
+.put(
+Privilege.Name.MODIFY_TABLE,
+ImmutableSet.of(
+RangerDefines.ACCESS_TYPE_HIVE_UPDATE,
+RangerDefines.ACCESS_TYPE_HIVE_ALTER,
+RangerDefines.AC
Re: [PR] [#3963] feat(core): Apache Ranger Hive authorization pushdown [gravitino]
yuqi1129 commented on code in PR #4515:
URL: https://github.com/apache/gravitino/pull/4515#discussion_r1724903813
##
authorizations/authorization-ranger/src/main/java/org/apache/gravitino/authorization/ranger/RangerAuthorizationPlugin.java:
##
@@ -0,0 +1,583 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+package org.apache.gravitino.authorization.ranger;
+
+import com.google.common.annotations.VisibleForTesting;
+import com.google.common.collect.ImmutableMap;
+import com.google.common.collect.Lists;
+import java.io.IOException;
+import java.time.Instant;
+import java.util.Collections;
+import java.util.List;
+import java.util.Map;
+import java.util.Objects;
+import java.util.Set;
+import java.util.stream.Collectors;
+import org.apache.gravitino.MetadataObject;
+import org.apache.gravitino.authorization.Group;
+import org.apache.gravitino.authorization.Owner;
+import org.apache.gravitino.authorization.Privilege;
+import org.apache.gravitino.authorization.Role;
+import org.apache.gravitino.authorization.RoleChange;
+import org.apache.gravitino.authorization.SecurableObjects;
+import org.apache.gravitino.authorization.User;
+import org.apache.gravitino.authorization.ranger.reference.RangerDefines;
+import org.apache.gravitino.authorization.ranger.reference.VXGroup;
+import org.apache.gravitino.authorization.ranger.reference.VXGroupList;
+import org.apache.gravitino.authorization.ranger.reference.VXUser;
+import org.apache.gravitino.authorization.ranger.reference.VXUserList;
+import org.apache.gravitino.connector.AuthorizationPropertiesMeta;
+import org.apache.gravitino.connector.authorization.AuthorizationPlugin;
+import org.apache.gravitino.meta.AuditInfo;
+import org.apache.gravitino.meta.GroupEntity;
+import org.apache.gravitino.meta.UserEntity;
+import org.apache.gravitino.utils.PrincipalUtils;
+import org.apache.ranger.RangerServiceException;
+import org.apache.ranger.plugin.model.RangerPolicy;
+import org.apache.ranger.plugin.util.GrantRevokeRoleRequest;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+/**
+ * Ranger authorization operations plugin abstract class.
+ * 1. For Ranger limit, The same metadata object only has a unique Ranger
policy, So a Ranger policy
+ * maybe contains multiple Gravitino securable objects.
+ * 2. For easy management, each Ranger privilege will create one
RangerPolicyItemAccess in the
+ * policy.
+ * 3. Ranger also have Role concept, and support adds multiple users or groups
in the Ranger Role,
+ * So we can use the Ranger Role to implement the Gravitino Role.
+ * 4. The Ranger policy also supports multiple users and groups, But we only
use a user or group to
+ * implement Gravitino Owner concept.
+ */
+public class RangerAuthorizationPlugin implements AuthorizationPlugin {
+ private static final Logger LOG =
LoggerFactory.getLogger(RangerAuthorizationPlugin.class);
+
+ protected String catalogProvider;
+ protected String rangerServiceName;
+ protected RangerClientExtend rangerClient;
+ private RangerHelper rangerHelper;
+ @VisibleForTesting public final String rangerAdminName;
+
+ public RangerAuthorizationPlugin(String catalogProvider, Map
config) {
+super();
+this.catalogProvider = catalogProvider;
+String rangerUrl =
config.get(AuthorizationPropertiesMeta.RANGER_ADMIN_URL);
+String authType = config.get(AuthorizationPropertiesMeta.RANGER_AUTH_TYPE);
+rangerAdminName = config.get(AuthorizationPropertiesMeta.RANGER_USERNAME);
+// Apache Ranger Password should be minimum 8 characters with min one
alphabet and one numeric.
+String password = config.get(AuthorizationPropertiesMeta.RANGER_PASSWORD);
+rangerServiceName =
config.get(AuthorizationPropertiesMeta.RANGER_SERVICE_NAME);
+RangerHelper.check(rangerUrl != null, "Ranger admin URL is required");
+RangerHelper.check(authType != null, "Ranger auth type is required");
+RangerHelper.check(rangerAdminName != null, "Ranger username is required");
+RangerHelper.check(password != null, "Ranger password is required");
+RangerHelper.check(rangerServiceName != null, "Ranger service name is
required");
+rangerClient = new RangerClientExtend(rangerUrl, authT
Re: [PR] [#3963] feat(core): Apache Ranger Hive authorization pushdown [gravitino]
jerqi commented on code in PR #4515:
URL: https://github.com/apache/gravitino/pull/4515#discussion_r1724902782
##
authorizations/authorization-ranger/src/main/java/org/apache/gravitino/authorization/ranger/RangerAuthorizationPlugin.java:
##
@@ -0,0 +1,583 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+package org.apache.gravitino.authorization.ranger;
+
+import com.google.common.annotations.VisibleForTesting;
+import com.google.common.collect.ImmutableMap;
+import com.google.common.collect.Lists;
+import java.io.IOException;
+import java.time.Instant;
+import java.util.Collections;
+import java.util.List;
+import java.util.Map;
+import java.util.Objects;
+import java.util.Set;
+import java.util.stream.Collectors;
+import org.apache.gravitino.MetadataObject;
+import org.apache.gravitino.authorization.Group;
+import org.apache.gravitino.authorization.Owner;
+import org.apache.gravitino.authorization.Privilege;
+import org.apache.gravitino.authorization.Role;
+import org.apache.gravitino.authorization.RoleChange;
+import org.apache.gravitino.authorization.SecurableObjects;
+import org.apache.gravitino.authorization.User;
+import org.apache.gravitino.authorization.ranger.reference.RangerDefines;
+import org.apache.gravitino.authorization.ranger.reference.VXGroup;
+import org.apache.gravitino.authorization.ranger.reference.VXGroupList;
+import org.apache.gravitino.authorization.ranger.reference.VXUser;
+import org.apache.gravitino.authorization.ranger.reference.VXUserList;
+import org.apache.gravitino.connector.AuthorizationPropertiesMeta;
+import org.apache.gravitino.connector.authorization.AuthorizationPlugin;
+import org.apache.gravitino.meta.AuditInfo;
+import org.apache.gravitino.meta.GroupEntity;
+import org.apache.gravitino.meta.UserEntity;
+import org.apache.gravitino.utils.PrincipalUtils;
+import org.apache.ranger.RangerServiceException;
+import org.apache.ranger.plugin.model.RangerPolicy;
+import org.apache.ranger.plugin.util.GrantRevokeRoleRequest;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+/**
+ * Ranger authorization operations plugin abstract class.
+ * 1. For Ranger limit, The same metadata object only has a unique Ranger
policy, So a Ranger policy
+ * maybe contains multiple Gravitino securable objects.
+ * 2. For easy management, each Ranger privilege will create one
RangerPolicyItemAccess in the
+ * policy.
+ * 3. Ranger also have Role concept, and support adds multiple users or groups
in the Ranger Role,
+ * So we can use the Ranger Role to implement the Gravitino Role.
+ * 4. The Ranger policy also supports multiple users and groups, But we only
use a user or group to
+ * implement Gravitino Owner concept.
+ */
+public class RangerAuthorizationPlugin implements AuthorizationPlugin {
+ private static final Logger LOG =
LoggerFactory.getLogger(RangerAuthorizationPlugin.class);
+
+ protected String catalogProvider;
+ protected String rangerServiceName;
+ protected RangerClientExtend rangerClient;
+ private RangerHelper rangerHelper;
+ @VisibleForTesting public final String rangerAdminName;
+
+ public RangerAuthorizationPlugin(String catalogProvider, Map
config) {
+super();
+this.catalogProvider = catalogProvider;
+String rangerUrl =
config.get(AuthorizationPropertiesMeta.RANGER_ADMIN_URL);
+String authType = config.get(AuthorizationPropertiesMeta.RANGER_AUTH_TYPE);
+rangerAdminName = config.get(AuthorizationPropertiesMeta.RANGER_USERNAME);
+// Apache Ranger Password should be minimum 8 characters with min one
alphabet and one numeric.
+String password = config.get(AuthorizationPropertiesMeta.RANGER_PASSWORD);
+rangerServiceName =
config.get(AuthorizationPropertiesMeta.RANGER_SERVICE_NAME);
+RangerHelper.check(rangerUrl != null, "Ranger admin URL is required");
+RangerHelper.check(authType != null, "Ranger auth type is required");
+RangerHelper.check(rangerAdminName != null, "Ranger username is required");
+RangerHelper.check(password != null, "Ranger password is required");
+RangerHelper.check(rangerServiceName != null, "Ranger service name is
required");
+rangerClient = new RangerClientExtend(rangerUrl, authType
Re: [PR] [#3963] feat(core): Apache Ranger Hive authorization pushdown [gravitino]
jerqi commented on code in PR #4515:
URL: https://github.com/apache/gravitino/pull/4515#discussion_r1724898605
##
authorizations/authorization-ranger/src/test/java/org/apache/gravitino/authorization/ranger/integration/test/RangerITEnv.java:
##
@@ -252,7 +257,7 @@ protected static String updateOrCreateRangerPolicy(
if (!policies.isEmpty()) {
// Because Ranger user the wildcard filter, Ranger will return the
policy meets
// the wildcard(*,?) conditions, just like `*.*.*` policy will match
`db1.table1.column1`
Review Comment:
Could you modify the comment to align to other places?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
Re: [PR] [#3963] feat(core): Apache Ranger Hive authorization pushdown [gravitino]
xunliu commented on code in PR #4515:
URL: https://github.com/apache/gravitino/pull/4515#discussion_r1724893759
##
authorizations/authorization-ranger/src/main/java/org/apache/gravitino/authorization/ranger/RangerAuthorizationPlugin.java:
##
@@ -0,0 +1,583 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+package org.apache.gravitino.authorization.ranger;
+
+import com.google.common.annotations.VisibleForTesting;
+import com.google.common.collect.ImmutableMap;
+import com.google.common.collect.Lists;
+import java.io.IOException;
+import java.time.Instant;
+import java.util.Collections;
+import java.util.List;
+import java.util.Map;
+import java.util.Objects;
+import java.util.Set;
+import java.util.stream.Collectors;
+import org.apache.gravitino.MetadataObject;
+import org.apache.gravitino.authorization.Group;
+import org.apache.gravitino.authorization.Owner;
+import org.apache.gravitino.authorization.Privilege;
+import org.apache.gravitino.authorization.Role;
+import org.apache.gravitino.authorization.RoleChange;
+import org.apache.gravitino.authorization.SecurableObjects;
+import org.apache.gravitino.authorization.User;
+import org.apache.gravitino.authorization.ranger.reference.RangerDefines;
+import org.apache.gravitino.authorization.ranger.reference.VXGroup;
+import org.apache.gravitino.authorization.ranger.reference.VXGroupList;
+import org.apache.gravitino.authorization.ranger.reference.VXUser;
+import org.apache.gravitino.authorization.ranger.reference.VXUserList;
+import org.apache.gravitino.connector.AuthorizationPropertiesMeta;
+import org.apache.gravitino.connector.authorization.AuthorizationPlugin;
+import org.apache.gravitino.meta.AuditInfo;
+import org.apache.gravitino.meta.GroupEntity;
+import org.apache.gravitino.meta.UserEntity;
+import org.apache.gravitino.utils.PrincipalUtils;
+import org.apache.ranger.RangerServiceException;
+import org.apache.ranger.plugin.model.RangerPolicy;
+import org.apache.ranger.plugin.util.GrantRevokeRoleRequest;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+/**
+ * Ranger authorization operations plugin abstract class.
+ * 1. For Ranger limit, The same metadata object only has a unique Ranger
policy, So a Ranger policy
+ * maybe contains multiple Gravitino securable objects.
+ * 2. For easy management, each Ranger privilege will create one
RangerPolicyItemAccess in the
+ * policy.
+ * 3. Ranger also have Role concept, and support adds multiple users or groups
in the Ranger Role,
+ * So we can use the Ranger Role to implement the Gravitino Role.
+ * 4. The Ranger policy also supports multiple users and groups, But we only
use a user or group to
+ * implement Gravitino Owner concept.
+ */
+public class RangerAuthorizationPlugin implements AuthorizationPlugin {
+ private static final Logger LOG =
LoggerFactory.getLogger(RangerAuthorizationPlugin.class);
+
+ protected String catalogProvider;
+ protected String rangerServiceName;
+ protected RangerClientExtend rangerClient;
+ private RangerHelper rangerHelper;
+ @VisibleForTesting public final String rangerAdminName;
+
+ public RangerAuthorizationPlugin(String catalogProvider, Map
config) {
+super();
+this.catalogProvider = catalogProvider;
+String rangerUrl =
config.get(AuthorizationPropertiesMeta.RANGER_ADMIN_URL);
+String authType = config.get(AuthorizationPropertiesMeta.RANGER_AUTH_TYPE);
+rangerAdminName = config.get(AuthorizationPropertiesMeta.RANGER_USERNAME);
+// Apache Ranger Password should be minimum 8 characters with min one
alphabet and one numeric.
+String password = config.get(AuthorizationPropertiesMeta.RANGER_PASSWORD);
+rangerServiceName =
config.get(AuthorizationPropertiesMeta.RANGER_SERVICE_NAME);
+RangerHelper.check(rangerUrl != null, "Ranger admin URL is required");
+RangerHelper.check(authType != null, "Ranger auth type is required");
+RangerHelper.check(rangerAdminName != null, "Ranger username is required");
+RangerHelper.check(password != null, "Ranger password is required");
+RangerHelper.check(rangerServiceName != null, "Ranger service name is
required");
+rangerClient = new RangerClientExtend(rangerUrl, authTyp
Re: [PR] [#3963] feat(core): Apache Ranger Hive authorization pushdown [gravitino]
jerqi commented on code in PR #4515:
URL: https://github.com/apache/gravitino/pull/4515#discussion_r1724888182
##
authorizations/authorization-ranger/src/main/java/org/apache/gravitino/authorization/ranger/RangerAuthorizationPlugin.java:
##
@@ -0,0 +1,583 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+package org.apache.gravitino.authorization.ranger;
+
+import com.google.common.annotations.VisibleForTesting;
+import com.google.common.collect.ImmutableMap;
+import com.google.common.collect.Lists;
+import java.io.IOException;
+import java.time.Instant;
+import java.util.Collections;
+import java.util.List;
+import java.util.Map;
+import java.util.Objects;
+import java.util.Set;
+import java.util.stream.Collectors;
+import org.apache.gravitino.MetadataObject;
+import org.apache.gravitino.authorization.Group;
+import org.apache.gravitino.authorization.Owner;
+import org.apache.gravitino.authorization.Privilege;
+import org.apache.gravitino.authorization.Role;
+import org.apache.gravitino.authorization.RoleChange;
+import org.apache.gravitino.authorization.SecurableObjects;
+import org.apache.gravitino.authorization.User;
+import org.apache.gravitino.authorization.ranger.reference.RangerDefines;
+import org.apache.gravitino.authorization.ranger.reference.VXGroup;
+import org.apache.gravitino.authorization.ranger.reference.VXGroupList;
+import org.apache.gravitino.authorization.ranger.reference.VXUser;
+import org.apache.gravitino.authorization.ranger.reference.VXUserList;
+import org.apache.gravitino.connector.AuthorizationPropertiesMeta;
+import org.apache.gravitino.connector.authorization.AuthorizationPlugin;
+import org.apache.gravitino.meta.AuditInfo;
+import org.apache.gravitino.meta.GroupEntity;
+import org.apache.gravitino.meta.UserEntity;
+import org.apache.gravitino.utils.PrincipalUtils;
+import org.apache.ranger.RangerServiceException;
+import org.apache.ranger.plugin.model.RangerPolicy;
+import org.apache.ranger.plugin.util.GrantRevokeRoleRequest;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+/**
+ * Ranger authorization operations plugin abstract class.
+ * 1. For Ranger limit, The same metadata object only has a unique Ranger
policy, So a Ranger policy
+ * maybe contains multiple Gravitino securable objects.
+ * 2. For easy management, each Ranger privilege will create one
RangerPolicyItemAccess in the
+ * policy.
+ * 3. Ranger also have Role concept, and support adds multiple users or groups
in the Ranger Role,
+ * So we can use the Ranger Role to implement the Gravitino Role.
+ * 4. The Ranger policy also supports multiple users and groups, But we only
use a user or group to
+ * implement Gravitino Owner concept.
+ */
+public class RangerAuthorizationPlugin implements AuthorizationPlugin {
+ private static final Logger LOG =
LoggerFactory.getLogger(RangerAuthorizationPlugin.class);
+
+ protected String catalogProvider;
+ protected String rangerServiceName;
+ protected RangerClientExtend rangerClient;
+ private RangerHelper rangerHelper;
+ @VisibleForTesting public final String rangerAdminName;
+
+ public RangerAuthorizationPlugin(String catalogProvider, Map
config) {
+super();
+this.catalogProvider = catalogProvider;
+String rangerUrl =
config.get(AuthorizationPropertiesMeta.RANGER_ADMIN_URL);
+String authType = config.get(AuthorizationPropertiesMeta.RANGER_AUTH_TYPE);
+rangerAdminName = config.get(AuthorizationPropertiesMeta.RANGER_USERNAME);
+// Apache Ranger Password should be minimum 8 characters with min one
alphabet and one numeric.
+String password = config.get(AuthorizationPropertiesMeta.RANGER_PASSWORD);
+rangerServiceName =
config.get(AuthorizationPropertiesMeta.RANGER_SERVICE_NAME);
+RangerHelper.check(rangerUrl != null, "Ranger admin URL is required");
+RangerHelper.check(authType != null, "Ranger auth type is required");
+RangerHelper.check(rangerAdminName != null, "Ranger username is required");
+RangerHelper.check(password != null, "Ranger password is required");
+RangerHelper.check(rangerServiceName != null, "Ranger service name is
required");
+rangerClient = new RangerClientExtend(rangerUrl, authType
Re: [PR] [#3963] feat(core): Apache Ranger Hive authorization pushdown [gravitino]
jerqi commented on code in PR #4515:
URL: https://github.com/apache/gravitino/pull/4515#discussion_r1724887242
##
authorizations/authorization-ranger/src/main/java/org/apache/gravitino/authorization/ranger/RangerAuthorizationPlugin.java:
##
@@ -0,0 +1,583 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+package org.apache.gravitino.authorization.ranger;
+
+import com.google.common.annotations.VisibleForTesting;
+import com.google.common.collect.ImmutableMap;
+import com.google.common.collect.Lists;
+import java.io.IOException;
+import java.time.Instant;
+import java.util.Collections;
+import java.util.List;
+import java.util.Map;
+import java.util.Objects;
+import java.util.Set;
+import java.util.stream.Collectors;
+import org.apache.gravitino.MetadataObject;
+import org.apache.gravitino.authorization.Group;
+import org.apache.gravitino.authorization.Owner;
+import org.apache.gravitino.authorization.Privilege;
+import org.apache.gravitino.authorization.Role;
+import org.apache.gravitino.authorization.RoleChange;
+import org.apache.gravitino.authorization.SecurableObjects;
+import org.apache.gravitino.authorization.User;
+import org.apache.gravitino.authorization.ranger.reference.RangerDefines;
+import org.apache.gravitino.authorization.ranger.reference.VXGroup;
+import org.apache.gravitino.authorization.ranger.reference.VXGroupList;
+import org.apache.gravitino.authorization.ranger.reference.VXUser;
+import org.apache.gravitino.authorization.ranger.reference.VXUserList;
+import org.apache.gravitino.connector.AuthorizationPropertiesMeta;
+import org.apache.gravitino.connector.authorization.AuthorizationPlugin;
+import org.apache.gravitino.meta.AuditInfo;
+import org.apache.gravitino.meta.GroupEntity;
+import org.apache.gravitino.meta.UserEntity;
+import org.apache.gravitino.utils.PrincipalUtils;
+import org.apache.ranger.RangerServiceException;
+import org.apache.ranger.plugin.model.RangerPolicy;
+import org.apache.ranger.plugin.util.GrantRevokeRoleRequest;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+/**
+ * Ranger authorization operations plugin abstract class.
+ * 1. For Ranger limit, The same metadata object only has a unique Ranger
policy, So a Ranger policy
+ * maybe contains multiple Gravitino securable objects.
+ * 2. For easy management, each Ranger privilege will create one
RangerPolicyItemAccess in the
+ * policy.
+ * 3. Ranger also have Role concept, and support adds multiple users or groups
in the Ranger Role,
+ * So we can use the Ranger Role to implement the Gravitino Role.
+ * 4. The Ranger policy also supports multiple users and groups, But we only
use a user or group to
+ * implement Gravitino Owner concept.
+ */
+public class RangerAuthorizationPlugin implements AuthorizationPlugin {
+ private static final Logger LOG =
LoggerFactory.getLogger(RangerAuthorizationPlugin.class);
+
+ protected String catalogProvider;
+ protected String rangerServiceName;
+ protected RangerClientExtend rangerClient;
+ private RangerHelper rangerHelper;
+ @VisibleForTesting public final String rangerAdminName;
+
+ public RangerAuthorizationPlugin(String catalogProvider, Map
config) {
+super();
+this.catalogProvider = catalogProvider;
+String rangerUrl =
config.get(AuthorizationPropertiesMeta.RANGER_ADMIN_URL);
+String authType = config.get(AuthorizationPropertiesMeta.RANGER_AUTH_TYPE);
+rangerAdminName = config.get(AuthorizationPropertiesMeta.RANGER_USERNAME);
+// Apache Ranger Password should be minimum 8 characters with min one
alphabet and one numeric.
+String password = config.get(AuthorizationPropertiesMeta.RANGER_PASSWORD);
+rangerServiceName =
config.get(AuthorizationPropertiesMeta.RANGER_SERVICE_NAME);
+RangerHelper.check(rangerUrl != null, "Ranger admin URL is required");
+RangerHelper.check(authType != null, "Ranger auth type is required");
+RangerHelper.check(rangerAdminName != null, "Ranger username is required");
+RangerHelper.check(password != null, "Ranger password is required");
+RangerHelper.check(rangerServiceName != null, "Ranger service name is
required");
+rangerClient = new RangerClientExtend(rangerUrl, authType
Re: [PR] [#3963] feat(core): Apache Ranger Hive authorization pushdown [gravitino]
jerqi commented on code in PR #4515:
URL: https://github.com/apache/gravitino/pull/4515#discussion_r1724884494
##
authorizations/authorization-ranger/src/main/java/org/apache/gravitino/authorization/ranger/RangerHelper.java:
##
@@ -0,0 +1,533 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+package org.apache.gravitino.authorization.ranger;
+
+import com.google.common.annotations.VisibleForTesting;
+import com.google.common.collect.ImmutableMap;
+import com.google.common.collect.ImmutableSet;
+import com.google.common.collect.Lists;
+import com.google.common.collect.Sets;
+import com.google.errorprone.annotations.FormatMethod;
+import com.google.errorprone.annotations.FormatString;
+import java.util.Arrays;
+import java.util.HashMap;
+import java.util.HashSet;
+import java.util.List;
+import java.util.Map;
+import java.util.Objects;
+import java.util.Set;
+import java.util.stream.Collectors;
+import org.apache.commons.lang.StringUtils;
+import org.apache.gravitino.MetadataObject;
+import org.apache.gravitino.authorization.Owner;
+import org.apache.gravitino.authorization.Privilege;
+import org.apache.gravitino.authorization.SecurableObject;
+import org.apache.gravitino.authorization.SecurableObjects;
+import org.apache.gravitino.authorization.ranger.reference.RangerDefines;
+import org.apache.gravitino.exceptions.AuthorizationPluginException;
+import org.apache.ranger.RangerServiceException;
+import org.apache.ranger.plugin.model.RangerPolicy;
+import org.apache.ranger.plugin.model.RangerRole;
+import org.apache.ranger.plugin.util.GrantRevokeRoleRequest;
+import org.apache.ranger.plugin.util.SearchFilter;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+/**
+ * This class is a helper class for the Ranger authorization plugin. It
provides the ability to
+ * manage the Ranger policies and roles.
+ */
+public class RangerHelper {
+ private static final Logger LOG =
LoggerFactory.getLogger(RangerHelper.class);
+ public static final String MANAGED_BY_GRAVITINO = "MANAGED_BY_GRAVITINO";
+
+ RangerAuthorizationPlugin rangerAuthorizationPlugin;
+
+ /** Mapping Gravitino privilege name to the underlying authorization system
privileges. */
+ protected Map> privilegesMapping = null;
+ /** The owner privileges, the owner can do anything on the metadata object */
+ protected Set ownerPrivileges = null;
+
+ /**
+ * Because Ranger doesn't support the precise search, Ranger will return the
policy meets the
+ * wildcard(*,?) conditions, If you use `db.table` condition to search
policy, the Ranger will
+ * match `db1.table1`, `db1.table2`, `db*.table*`, So we need to manually
precisely filter this
+ * research results.
+ * policySearchKeys: The search Ranger policy condition key defines.
+ * policyPreciseFilterKeys: The precise filter Ranger search results key
defines
+ */
+ protected List policySearchKeys = null;
+
+ protected List policyPreciseFilterKeys = null;
+
+ public RangerHelper(RangerAuthorizationPlugin rangerAuthorizationPlugin,
String catalogProvider) {
+this.rangerAuthorizationPlugin = rangerAuthorizationPlugin;
+switch (catalogProvider) {
+ case "hive":
+initPrivilegesMapping();
+initOwnerPrivileges();
+initPolicySearchKeys();
+initPreciseFilterKeys();
+break;
+ default:
+throw new IllegalArgumentException(
+"Authorization plugin unsupported catalog provider: " +
catalogProvider);
+}
+ }
+
+ /** Initial mapping Gravitino privilege name to the underlying authorization
system privileges. */
+ private void initPrivilegesMapping() {
+privilegesMapping =
+ImmutableMap.>builder()
+.put(
+Privilege.Name.CREATE_SCHEMA,
+ImmutableSet.of(RangerDefines.ACCESS_TYPE_HIVE_CREATE))
+.put(
+Privilege.Name.CREATE_TABLE,
ImmutableSet.of(RangerDefines.ACCESS_TYPE_HIVE_CREATE))
+.put(
+Privilege.Name.MODIFY_TABLE,
+ImmutableSet.of(
+RangerDefines.ACCESS_TYPE_HIVE_UPDATE,
+RangerDefines.ACCESS_TYPE_HIVE_ALTER,
+RangerDefines.AC
Re: [PR] [#3963] feat(core): Apache Ranger Hive authorization pushdown [gravitino]
yuqi1129 commented on code in PR #4515:
URL: https://github.com/apache/gravitino/pull/4515#discussion_r1724881433
##
authorizations/authorization-ranger/src/main/java/org/apache/gravitino/authorization/ranger/RangerAuthorizationPlugin.java:
##
@@ -0,0 +1,583 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+package org.apache.gravitino.authorization.ranger;
+
+import com.google.common.annotations.VisibleForTesting;
+import com.google.common.collect.ImmutableMap;
+import com.google.common.collect.Lists;
+import java.io.IOException;
+import java.time.Instant;
+import java.util.Collections;
+import java.util.List;
+import java.util.Map;
+import java.util.Objects;
+import java.util.Set;
+import java.util.stream.Collectors;
+import org.apache.gravitino.MetadataObject;
+import org.apache.gravitino.authorization.Group;
+import org.apache.gravitino.authorization.Owner;
+import org.apache.gravitino.authorization.Privilege;
+import org.apache.gravitino.authorization.Role;
+import org.apache.gravitino.authorization.RoleChange;
+import org.apache.gravitino.authorization.SecurableObjects;
+import org.apache.gravitino.authorization.User;
+import org.apache.gravitino.authorization.ranger.reference.RangerDefines;
+import org.apache.gravitino.authorization.ranger.reference.VXGroup;
+import org.apache.gravitino.authorization.ranger.reference.VXGroupList;
+import org.apache.gravitino.authorization.ranger.reference.VXUser;
+import org.apache.gravitino.authorization.ranger.reference.VXUserList;
+import org.apache.gravitino.connector.AuthorizationPropertiesMeta;
+import org.apache.gravitino.connector.authorization.AuthorizationPlugin;
+import org.apache.gravitino.meta.AuditInfo;
+import org.apache.gravitino.meta.GroupEntity;
+import org.apache.gravitino.meta.UserEntity;
+import org.apache.gravitino.utils.PrincipalUtils;
+import org.apache.ranger.RangerServiceException;
+import org.apache.ranger.plugin.model.RangerPolicy;
+import org.apache.ranger.plugin.util.GrantRevokeRoleRequest;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+/**
+ * Ranger authorization operations plugin abstract class.
+ * 1. For Ranger limit, The same metadata object only has a unique Ranger
policy, So a Ranger policy
+ * maybe contains multiple Gravitino securable objects.
+ * 2. For easy management, each Ranger privilege will create one
RangerPolicyItemAccess in the
+ * policy.
+ * 3. Ranger also have Role concept, and support adds multiple users or groups
in the Ranger Role,
+ * So we can use the Ranger Role to implement the Gravitino Role.
+ * 4. The Ranger policy also supports multiple users and groups, But we only
use a user or group to
+ * implement Gravitino Owner concept.
+ */
+public class RangerAuthorizationPlugin implements AuthorizationPlugin {
+ private static final Logger LOG =
LoggerFactory.getLogger(RangerAuthorizationPlugin.class);
+
+ protected String catalogProvider;
+ protected String rangerServiceName;
+ protected RangerClientExtend rangerClient;
+ private RangerHelper rangerHelper;
+ @VisibleForTesting public final String rangerAdminName;
+
+ public RangerAuthorizationPlugin(String catalogProvider, Map
config) {
+super();
+this.catalogProvider = catalogProvider;
+String rangerUrl =
config.get(AuthorizationPropertiesMeta.RANGER_ADMIN_URL);
+String authType = config.get(AuthorizationPropertiesMeta.RANGER_AUTH_TYPE);
+rangerAdminName = config.get(AuthorizationPropertiesMeta.RANGER_USERNAME);
+// Apache Ranger Password should be minimum 8 characters with min one
alphabet and one numeric.
+String password = config.get(AuthorizationPropertiesMeta.RANGER_PASSWORD);
+rangerServiceName =
config.get(AuthorizationPropertiesMeta.RANGER_SERVICE_NAME);
+RangerHelper.check(rangerUrl != null, "Ranger admin URL is required");
+RangerHelper.check(authType != null, "Ranger auth type is required");
+RangerHelper.check(rangerAdminName != null, "Ranger username is required");
+RangerHelper.check(password != null, "Ranger password is required");
+RangerHelper.check(rangerServiceName != null, "Ranger service name is
required");
+rangerClient = new RangerClientExtend(rangerUrl, authT
Re: [PR] [#3963] feat(core): Apache Ranger Hive authorization pushdown [gravitino]
jerqi commented on code in PR #4515:
URL: https://github.com/apache/gravitino/pull/4515#discussion_r1724882507
##
authorizations/authorization-ranger/src/main/java/org/apache/gravitino/authorization/ranger/RangerHelper.java:
##
@@ -0,0 +1,533 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+package org.apache.gravitino.authorization.ranger;
+
+import com.google.common.annotations.VisibleForTesting;
+import com.google.common.collect.ImmutableMap;
+import com.google.common.collect.ImmutableSet;
+import com.google.common.collect.Lists;
+import com.google.common.collect.Sets;
+import com.google.errorprone.annotations.FormatMethod;
+import com.google.errorprone.annotations.FormatString;
+import java.util.Arrays;
+import java.util.HashMap;
+import java.util.HashSet;
+import java.util.List;
+import java.util.Map;
+import java.util.Objects;
+import java.util.Set;
+import java.util.stream.Collectors;
+import org.apache.commons.lang.StringUtils;
+import org.apache.gravitino.MetadataObject;
+import org.apache.gravitino.authorization.Owner;
+import org.apache.gravitino.authorization.Privilege;
+import org.apache.gravitino.authorization.SecurableObject;
+import org.apache.gravitino.authorization.SecurableObjects;
+import org.apache.gravitino.authorization.ranger.reference.RangerDefines;
+import org.apache.gravitino.exceptions.AuthorizationPluginException;
+import org.apache.ranger.RangerServiceException;
+import org.apache.ranger.plugin.model.RangerPolicy;
+import org.apache.ranger.plugin.model.RangerRole;
+import org.apache.ranger.plugin.util.GrantRevokeRoleRequest;
+import org.apache.ranger.plugin.util.SearchFilter;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+/**
+ * This class is a helper class for the Ranger authorization plugin. It
provides the ability to
+ * manage the Ranger policies and roles.
+ */
+public class RangerHelper {
+ private static final Logger LOG =
LoggerFactory.getLogger(RangerHelper.class);
+ public static final String MANAGED_BY_GRAVITINO = "MANAGED_BY_GRAVITINO";
+
+ RangerAuthorizationPlugin rangerAuthorizationPlugin;
+
+ /** Mapping Gravitino privilege name to the underlying authorization system
privileges. */
+ protected Map> privilegesMapping = null;
+ /** The owner privileges, the owner can do anything on the metadata object */
+ protected Set ownerPrivileges = null;
+
+ /**
+ * Because Ranger doesn't support the precise search, Ranger will return the
policy meets the
+ * wildcard(*,?) conditions, If you use `db.table` condition to search
policy, the Ranger will
+ * match `db1.table1`, `db1.table2`, `db*.table*`, So we need to manually
precisely filter this
+ * research results.
+ * policySearchKeys: The search Ranger policy condition key defines.
+ * policyPreciseFilterKeys: The precise filter Ranger search results key
defines
+ */
+ protected List policySearchKeys = null;
+
+ protected List policyPreciseFilterKeys = null;
+
+ public RangerHelper(RangerAuthorizationPlugin rangerAuthorizationPlugin,
String catalogProvider) {
+this.rangerAuthorizationPlugin = rangerAuthorizationPlugin;
+switch (catalogProvider) {
+ case "hive":
+initPrivilegesMapping();
+initOwnerPrivileges();
+initPolicySearchKeys();
+initPreciseFilterKeys();
+break;
+ default:
+throw new IllegalArgumentException(
+"Authorization plugin unsupported catalog provider: " +
catalogProvider);
+}
+ }
+
+ /** Initial mapping Gravitino privilege name to the underlying authorization
system privileges. */
+ private void initPrivilegesMapping() {
+privilegesMapping =
+ImmutableMap.>builder()
+.put(
+Privilege.Name.CREATE_SCHEMA,
+ImmutableSet.of(RangerDefines.ACCESS_TYPE_HIVE_CREATE))
+.put(
+Privilege.Name.CREATE_TABLE,
ImmutableSet.of(RangerDefines.ACCESS_TYPE_HIVE_CREATE))
+.put(
+Privilege.Name.MODIFY_TABLE,
+ImmutableSet.of(
+RangerDefines.ACCESS_TYPE_HIVE_UPDATE,
+RangerDefines.ACCESS_TYPE_HIVE_ALTER,
+RangerDefines.AC
Re: [PR] [#3963] feat(core): Apache Ranger Hive authorization pushdown [gravitino]
jerqi commented on code in PR #4515:
URL: https://github.com/apache/gravitino/pull/4515#discussion_r1724879222
##
authorizations/authorization-ranger/src/main/java/org/apache/gravitino/authorization/ranger/RangerHelper.java:
##
@@ -0,0 +1,533 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+package org.apache.gravitino.authorization.ranger;
+
+import com.google.common.annotations.VisibleForTesting;
+import com.google.common.collect.ImmutableMap;
+import com.google.common.collect.ImmutableSet;
+import com.google.common.collect.Lists;
+import com.google.common.collect.Sets;
+import com.google.errorprone.annotations.FormatMethod;
+import com.google.errorprone.annotations.FormatString;
+import java.util.Arrays;
+import java.util.HashMap;
+import java.util.HashSet;
+import java.util.List;
+import java.util.Map;
+import java.util.Objects;
+import java.util.Set;
+import java.util.stream.Collectors;
+import org.apache.commons.lang.StringUtils;
+import org.apache.gravitino.MetadataObject;
+import org.apache.gravitino.authorization.Owner;
+import org.apache.gravitino.authorization.Privilege;
+import org.apache.gravitino.authorization.SecurableObject;
+import org.apache.gravitino.authorization.SecurableObjects;
+import org.apache.gravitino.authorization.ranger.reference.RangerDefines;
+import org.apache.gravitino.exceptions.AuthorizationPluginException;
+import org.apache.ranger.RangerServiceException;
+import org.apache.ranger.plugin.model.RangerPolicy;
+import org.apache.ranger.plugin.model.RangerRole;
+import org.apache.ranger.plugin.util.GrantRevokeRoleRequest;
+import org.apache.ranger.plugin.util.SearchFilter;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+/**
+ * This class is a helper class for the Ranger authorization plugin. It
provides the ability to
+ * manage the Ranger policies and roles.
+ */
+public class RangerHelper {
+ private static final Logger LOG =
LoggerFactory.getLogger(RangerHelper.class);
+ public static final String MANAGED_BY_GRAVITINO = "MANAGED_BY_GRAVITINO";
+
+ RangerAuthorizationPlugin rangerAuthorizationPlugin;
+
+ /** Mapping Gravitino privilege name to the underlying authorization system
privileges. */
+ protected Map> privilegesMapping = null;
+ /** The owner privileges, the owner can do anything on the metadata object */
+ protected Set ownerPrivileges = null;
+
+ /**
+ * Because Ranger doesn't support the precise search, Ranger will return the
policy meets the
+ * wildcard(*,?) conditions, If you use `db.table` condition to search
policy, the Ranger will
+ * match `db1.table1`, `db1.table2`, `db*.table*`, So we need to manually
precisely filter this
+ * research results.
+ * policySearchKeys: The search Ranger policy condition key defines.
+ * policyPreciseFilterKeys: The precise filter Ranger search results key
defines
+ */
+ protected List policySearchKeys = null;
+
+ protected List policyPreciseFilterKeys = null;
+
+ public RangerHelper(RangerAuthorizationPlugin rangerAuthorizationPlugin,
String catalogProvider) {
+this.rangerAuthorizationPlugin = rangerAuthorizationPlugin;
+switch (catalogProvider) {
+ case "hive":
+initPrivilegesMapping();
+initOwnerPrivileges();
+initPolicySearchKeys();
+initPreciseFilterKeys();
+break;
+ default:
+throw new IllegalArgumentException(
+"Authorization plugin unsupported catalog provider: " +
catalogProvider);
+}
+ }
+
+ /** Initial mapping Gravitino privilege name to the underlying authorization
system privileges. */
+ private void initPrivilegesMapping() {
+privilegesMapping =
+ImmutableMap.>builder()
+.put(
+Privilege.Name.CREATE_SCHEMA,
+ImmutableSet.of(RangerDefines.ACCESS_TYPE_HIVE_CREATE))
+.put(
+Privilege.Name.CREATE_TABLE,
ImmutableSet.of(RangerDefines.ACCESS_TYPE_HIVE_CREATE))
+.put(
+Privilege.Name.MODIFY_TABLE,
+ImmutableSet.of(
+RangerDefines.ACCESS_TYPE_HIVE_UPDATE,
+RangerDefines.ACCESS_TYPE_HIVE_ALTER,
+RangerDefines.AC
Re: [PR] [#3963] feat(core): Apache Ranger Hive authorization pushdown [gravitino]
jerqi commented on code in PR #4515:
URL: https://github.com/apache/gravitino/pull/4515#discussion_r1724878878
##
authorizations/authorization-ranger/src/main/java/org/apache/gravitino/authorization/ranger/RangerHelper.java:
##
@@ -0,0 +1,533 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+package org.apache.gravitino.authorization.ranger;
+
+import com.google.common.annotations.VisibleForTesting;
+import com.google.common.collect.ImmutableMap;
+import com.google.common.collect.ImmutableSet;
+import com.google.common.collect.Lists;
+import com.google.common.collect.Sets;
+import com.google.errorprone.annotations.FormatMethod;
+import com.google.errorprone.annotations.FormatString;
+import java.util.Arrays;
+import java.util.HashMap;
+import java.util.HashSet;
+import java.util.List;
+import java.util.Map;
+import java.util.Objects;
+import java.util.Set;
+import java.util.stream.Collectors;
+import org.apache.commons.lang.StringUtils;
+import org.apache.gravitino.MetadataObject;
+import org.apache.gravitino.authorization.Owner;
+import org.apache.gravitino.authorization.Privilege;
+import org.apache.gravitino.authorization.SecurableObject;
+import org.apache.gravitino.authorization.SecurableObjects;
+import org.apache.gravitino.authorization.ranger.reference.RangerDefines;
+import org.apache.gravitino.exceptions.AuthorizationPluginException;
+import org.apache.ranger.RangerServiceException;
+import org.apache.ranger.plugin.model.RangerPolicy;
+import org.apache.ranger.plugin.model.RangerRole;
+import org.apache.ranger.plugin.util.GrantRevokeRoleRequest;
+import org.apache.ranger.plugin.util.SearchFilter;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+/**
+ * This class is a helper class for the Ranger authorization plugin. It
provides the ability to
+ * manage the Ranger policies and roles.
+ */
+public class RangerHelper {
+ private static final Logger LOG =
LoggerFactory.getLogger(RangerHelper.class);
+ public static final String MANAGED_BY_GRAVITINO = "MANAGED_BY_GRAVITINO";
+
+ RangerAuthorizationPlugin rangerAuthorizationPlugin;
+
+ /** Mapping Gravitino privilege name to the underlying authorization system
privileges. */
+ protected Map> privilegesMapping = null;
+ /** The owner privileges, the owner can do anything on the metadata object */
+ protected Set ownerPrivileges = null;
+
+ /**
+ * Because Ranger doesn't support the precise search, Ranger will return the
policy meets the
+ * wildcard(*,?) conditions, If you use `db.table` condition to search
policy, the Ranger will
+ * match `db1.table1`, `db1.table2`, `db*.table*`, So we need to manually
precisely filter this
+ * research results.
+ * policySearchKeys: The search Ranger policy condition key defines.
+ * policyPreciseFilterKeys: The precise filter Ranger search results key
defines
+ */
+ protected List policySearchKeys = null;
+
+ protected List policyPreciseFilterKeys = null;
+
+ public RangerHelper(RangerAuthorizationPlugin rangerAuthorizationPlugin,
String catalogProvider) {
+this.rangerAuthorizationPlugin = rangerAuthorizationPlugin;
+switch (catalogProvider) {
+ case "hive":
+initPrivilegesMapping();
+initOwnerPrivileges();
+initPolicySearchKeys();
+initPreciseFilterKeys();
+break;
+ default:
+throw new IllegalArgumentException(
+"Authorization plugin unsupported catalog provider: " +
catalogProvider);
+}
+ }
+
+ /** Initial mapping Gravitino privilege name to the underlying authorization
system privileges. */
+ private void initPrivilegesMapping() {
+privilegesMapping =
+ImmutableMap.>builder()
+.put(
+Privilege.Name.CREATE_SCHEMA,
+ImmutableSet.of(RangerDefines.ACCESS_TYPE_HIVE_CREATE))
+.put(
+Privilege.Name.CREATE_TABLE,
ImmutableSet.of(RangerDefines.ACCESS_TYPE_HIVE_CREATE))
+.put(
+Privilege.Name.MODIFY_TABLE,
+ImmutableSet.of(
+RangerDefines.ACCESS_TYPE_HIVE_UPDATE,
+RangerDefines.ACCESS_TYPE_HIVE_ALTER,
+RangerDefines.AC
Re: [PR] [#3963] feat(core): Apache Ranger Hive authorization pushdown [gravitino]
jerqi commented on PR #4515: URL: https://github.com/apache/gravitino/pull/4515#issuecomment-2301785058 Only one point: 1. Some privileges like `use schema`, `use catalog`, `create catalog` could be pushed down underlying system, too. The check logic will throw exception for them. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
Re: [PR] [#3963] feat(core): Apache Ranger Hive authorization pushdown [gravitino]
jerqi commented on code in PR #4515: URL: https://github.com/apache/gravitino/pull/4515#discussion_r1724863560 ## authorizations/authorization-ranger/src/main/resources/META-INF/services/org.apache.gravitino.connector.authorization.AuthorizationProvider: ## @@ -0,0 +1,19 @@ +# +# Licensed to the Apache Software Foundation (ASF) under one +# or more contributor license agreements. See the NOTICE file +# distributed with this work for additional information +# regarding copyright ownership. The ASF licenses this file +# to you under the Apache License, Version 2.0 (the +# "License"); you may not use this file except in compliance +# with the License. You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, +# software distributed under the License is distributed on an +# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY +# KIND, either express or implied. See the License for the +# specific language governing permissions and limitations +# under the License. +# +org.apache.gravitino.authorization.ranger.RangerAuthorization Review Comment: OK, I got it. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
Re: [PR] [#3963] feat(core): Apache Ranger Hive authorization pushdown [gravitino]
xunliu commented on code in PR #4515:
URL: https://github.com/apache/gravitino/pull/4515#discussion_r1724857748
##
authorizations/authorization-ranger/src/main/java/org/apache/gravitino/authorization/ranger/RangerAuthorizationPlugin.java:
##
@@ -0,0 +1,583 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+package org.apache.gravitino.authorization.ranger;
+
+import com.google.common.annotations.VisibleForTesting;
+import com.google.common.collect.ImmutableMap;
+import com.google.common.collect.Lists;
+import java.io.IOException;
+import java.time.Instant;
+import java.util.Collections;
+import java.util.List;
+import java.util.Map;
+import java.util.Objects;
+import java.util.Set;
+import java.util.stream.Collectors;
+import org.apache.gravitino.MetadataObject;
+import org.apache.gravitino.authorization.Group;
+import org.apache.gravitino.authorization.Owner;
+import org.apache.gravitino.authorization.Privilege;
+import org.apache.gravitino.authorization.Role;
+import org.apache.gravitino.authorization.RoleChange;
+import org.apache.gravitino.authorization.SecurableObjects;
+import org.apache.gravitino.authorization.User;
+import org.apache.gravitino.authorization.ranger.reference.RangerDefines;
+import org.apache.gravitino.authorization.ranger.reference.VXGroup;
+import org.apache.gravitino.authorization.ranger.reference.VXGroupList;
+import org.apache.gravitino.authorization.ranger.reference.VXUser;
+import org.apache.gravitino.authorization.ranger.reference.VXUserList;
+import org.apache.gravitino.connector.AuthorizationPropertiesMeta;
+import org.apache.gravitino.connector.authorization.AuthorizationPlugin;
+import org.apache.gravitino.meta.AuditInfo;
+import org.apache.gravitino.meta.GroupEntity;
+import org.apache.gravitino.meta.UserEntity;
+import org.apache.gravitino.utils.PrincipalUtils;
+import org.apache.ranger.RangerServiceException;
+import org.apache.ranger.plugin.model.RangerPolicy;
+import org.apache.ranger.plugin.util.GrantRevokeRoleRequest;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+/**
+ * Ranger authorization operations plugin abstract class.
+ * 1. For Ranger limit, The same metadata object only has a unique Ranger
policy, So a Ranger policy
+ * maybe contains multiple Gravitino securable objects.
+ * 2. For easy management, each Ranger privilege will create one
RangerPolicyItemAccess in the
+ * policy.
+ * 3. Ranger also have Role concept, and support adds multiple users or groups
in the Ranger Role,
+ * So we can use the Ranger Role to implement the Gravitino Role.
+ * 4. The Ranger policy also supports multiple users and groups, But we only
use a user or group to
+ * implement Gravitino Owner concept.
+ */
+public class RangerAuthorizationPlugin implements AuthorizationPlugin {
+ private static final Logger LOG =
LoggerFactory.getLogger(RangerAuthorizationPlugin.class);
+
+ protected String catalogProvider;
+ protected String rangerServiceName;
+ protected RangerClientExtend rangerClient;
+ private RangerHelper rangerHelper;
+ @VisibleForTesting public final String rangerAdminName;
+
+ public RangerAuthorizationPlugin(String catalogProvider, Map
config) {
+super();
+this.catalogProvider = catalogProvider;
+String rangerUrl =
config.get(AuthorizationPropertiesMeta.RANGER_ADMIN_URL);
+String authType = config.get(AuthorizationPropertiesMeta.RANGER_AUTH_TYPE);
+rangerAdminName = config.get(AuthorizationPropertiesMeta.RANGER_USERNAME);
+// Apache Ranger Password should be minimum 8 characters with min one
alphabet and one numeric.
+String password = config.get(AuthorizationPropertiesMeta.RANGER_PASSWORD);
+rangerServiceName =
config.get(AuthorizationPropertiesMeta.RANGER_SERVICE_NAME);
+RangerHelper.check(rangerUrl != null, "Ranger admin URL is required");
+RangerHelper.check(authType != null, "Ranger auth type is required");
+RangerHelper.check(rangerAdminName != null, "Ranger username is required");
+RangerHelper.check(password != null, "Ranger password is required");
+RangerHelper.check(rangerServiceName != null, "Ranger service name is
required");
+rangerClient = new RangerClientExtend(rangerUrl, authTyp
Re: [PR] [#3963] feat(core): Apache Ranger Hive authorization pushdown [gravitino]
xunliu commented on code in PR #4515: URL: https://github.com/apache/gravitino/pull/4515#discussion_r1724849823 ## authorizations/authorization-ranger/src/main/resources/META-INF/services/org.apache.gravitino.connector.authorization.AuthorizationProvider: ## @@ -0,0 +1,19 @@ +# +# Licensed to the Apache Software Foundation (ASF) under one +# or more contributor license agreements. See the NOTICE file +# distributed with this work for additional information +# regarding copyright ownership. The ASF licenses this file +# to you under the Apache License, Version 2.0 (the +# "License"); you may not use this file except in compliance +# with the License. You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, +# software distributed under the License is distributed on an +# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY +# KIND, either express or implied. See the License for the +# specific language governing permissions and limitations +# under the License. +# +org.apache.gravitino.authorization.ranger.RangerAuthorization Review Comment: We can add multiple Authorization in the `authorization.AuthorizationProvider`, just like https://github.com/apache/gravitino/blob/main/core/src/test/resources/META-INF/services/org.apache.gravitino.connector.authorization.AuthorizationProvider#L19 Look more carefully before you jump to conclusions！！！ -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
Re: [PR] [#3963] feat(core): Apache Ranger Hive authorization pushdown [gravitino]
yuqi1129 commented on code in PR #4515:
URL: https://github.com/apache/gravitino/pull/4515#discussion_r1724790047
##
authorizations/authorization-ranger/src/test/java/org/apache/gravitino/authorization/ranger/integration/test/RangerITEnv.java:
##
@@ -311,4 +314,11 @@ protected static void cleanAllPolicy(String serviceName) {
throw new RuntimeException(e);
}
}
+
+ /**
+ * Didn't call this function in the Lambda function body, It will return a
random function name
Review Comment:
Don't or Do not
##
authorizations/authorization-ranger/src/main/java/org/apache/gravitino/authorization/ranger/RangerAuthorizationPlugin.java:
##
@@ -0,0 +1,583 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+package org.apache.gravitino.authorization.ranger;
+
+import com.google.common.annotations.VisibleForTesting;
+import com.google.common.collect.ImmutableMap;
+import com.google.common.collect.Lists;
+import java.io.IOException;
+import java.time.Instant;
+import java.util.Collections;
+import java.util.List;
+import java.util.Map;
+import java.util.Objects;
+import java.util.Set;
+import java.util.stream.Collectors;
+import org.apache.gravitino.MetadataObject;
+import org.apache.gravitino.authorization.Group;
+import org.apache.gravitino.authorization.Owner;
+import org.apache.gravitino.authorization.Privilege;
+import org.apache.gravitino.authorization.Role;
+import org.apache.gravitino.authorization.RoleChange;
+import org.apache.gravitino.authorization.SecurableObjects;
+import org.apache.gravitino.authorization.User;
+import org.apache.gravitino.authorization.ranger.reference.RangerDefines;
+import org.apache.gravitino.authorization.ranger.reference.VXGroup;
+import org.apache.gravitino.authorization.ranger.reference.VXGroupList;
+import org.apache.gravitino.authorization.ranger.reference.VXUser;
+import org.apache.gravitino.authorization.ranger.reference.VXUserList;
+import org.apache.gravitino.connector.AuthorizationPropertiesMeta;
+import org.apache.gravitino.connector.authorization.AuthorizationPlugin;
+import org.apache.gravitino.meta.AuditInfo;
+import org.apache.gravitino.meta.GroupEntity;
+import org.apache.gravitino.meta.UserEntity;
+import org.apache.gravitino.utils.PrincipalUtils;
+import org.apache.ranger.RangerServiceException;
+import org.apache.ranger.plugin.model.RangerPolicy;
+import org.apache.ranger.plugin.util.GrantRevokeRoleRequest;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+/**
+ * Ranger authorization operations plugin abstract class.
+ * 1. For Ranger limit, The same metadata object only has a unique Ranger
policy, So a Ranger policy
+ * maybe contains multiple Gravitino securable objects.
+ * 2. For easy management, each Ranger privilege will create one
RangerPolicyItemAccess in the
+ * policy.
+ * 3. Ranger also have Role concept, and support adds multiple users or groups
in the Ranger Role,
+ * So we can use the Ranger Role to implement the Gravitino Role.
+ * 4. The Ranger policy also supports multiple users and groups, But we only
use a user or group to
+ * implement Gravitino Owner concept.
+ */
+public class RangerAuthorizationPlugin implements AuthorizationPlugin {
+ private static final Logger LOG =
LoggerFactory.getLogger(RangerAuthorizationPlugin.class);
+
+ protected String catalogProvider;
+ protected String rangerServiceName;
+ protected RangerClientExtend rangerClient;
+ private RangerHelper rangerHelper;
+ @VisibleForTesting public final String rangerAdminName;
+
+ public RangerAuthorizationPlugin(String catalogProvider, Map
config) {
+super();
+this.catalogProvider = catalogProvider;
+String rangerUrl =
config.get(AuthorizationPropertiesMeta.RANGER_ADMIN_URL);
+String authType = config.get(AuthorizationPropertiesMeta.RANGER_AUTH_TYPE);
+rangerAdminName = config.get(AuthorizationPropertiesMeta.RANGER_USERNAME);
+// Apache Ranger Password should be minimum 8 characters with min one
alphabet and one numeric.
+String password = config.get(AuthorizationPropertiesMeta.RANGER_PASSWORD);
+rangerServiceName =
config.get(AuthorizationPropertiesMeta.RANGER_SERVICE_NAME);
+RangerHelper.check(r
Re: [PR] [#3963] feat(core): Apache Ranger Hive authorization pushdown [gravitino]
jerqi commented on code in PR #4515: URL: https://github.com/apache/gravitino/pull/4515#discussion_r1724800112 ## authorizations/authorization-ranger/src/main/resources/META-INF/services/org.apache.gravitino.connector.authorization.AuthorizationProvider: ## @@ -0,0 +1,19 @@ +# +# Licensed to the Apache Software Foundation (ASF) under one +# or more contributor license agreements. See the NOTICE file +# distributed with this work for additional information +# regarding copyright ownership. The ASF licenses this file +# to you under the Apache License, Version 2.0 (the +# "License"); you may not use this file except in compliance +# with the License. You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, +# software distributed under the License is distributed on an +# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY +# KIND, either express or implied. See the License for the +# specific language governing permissions and limitations +# under the License. +# +org.apache.gravitino.authorization.ranger.RangerAuthorization Review Comment: We could only use RangerAuthorization for all catalogs. This seems not ok if we only support partial catalog authorization. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
Re: [PR] [#3963] feat(core): Apache Ranger Hive authorization pushdown [gravitino]
xunliu commented on code in PR #4515:
URL: https://github.com/apache/gravitino/pull/4515#discussion_r1724783782
##
authorizations/authorization-ranger/src/main/java/org/apache/gravitino/authorization/ranger/RangerHelper.java:
##
@@ -0,0 +1,533 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+package org.apache.gravitino.authorization.ranger;
+
+import com.google.common.annotations.VisibleForTesting;
+import com.google.common.collect.ImmutableMap;
+import com.google.common.collect.ImmutableSet;
+import com.google.common.collect.Lists;
+import com.google.common.collect.Sets;
+import com.google.errorprone.annotations.FormatMethod;
+import com.google.errorprone.annotations.FormatString;
+import java.util.Arrays;
+import java.util.HashMap;
+import java.util.HashSet;
+import java.util.List;
+import java.util.Map;
+import java.util.Objects;
+import java.util.Set;
+import java.util.stream.Collectors;
+import org.apache.commons.lang.StringUtils;
+import org.apache.gravitino.MetadataObject;
+import org.apache.gravitino.authorization.Owner;
+import org.apache.gravitino.authorization.Privilege;
+import org.apache.gravitino.authorization.SecurableObject;
+import org.apache.gravitino.authorization.SecurableObjects;
+import org.apache.gravitino.authorization.ranger.reference.RangerDefines;
+import org.apache.gravitino.exceptions.AuthorizationPluginException;
+import org.apache.ranger.RangerServiceException;
+import org.apache.ranger.plugin.model.RangerPolicy;
+import org.apache.ranger.plugin.model.RangerRole;
+import org.apache.ranger.plugin.util.GrantRevokeRoleRequest;
+import org.apache.ranger.plugin.util.SearchFilter;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+/**
+ * This class is a helper class for the Ranger authorization plugin. It
provides the ability to
+ * manage the Ranger policies and roles.
+ */
+public class RangerHelper {
+ private static final Logger LOG =
LoggerFactory.getLogger(RangerHelper.class);
+ public static final String MANAGED_BY_GRAVITINO = "MANAGED_BY_GRAVITINO";
+
+ RangerAuthorizationPlugin rangerAuthorizationPlugin;
+
+ /** Mapping Gravitino privilege name to the underlying authorization system
privileges. */
+ protected Map> privilegesMapping = null;
+ /** The owner privileges, the owner can do anything on the metadata object */
+ protected Set ownerPrivileges = null;
+
+ /**
+ * Because Ranger doesn't support the precise search, Ranger will return the
policy meets the
+ * wildcard(*,?) conditions, If you use `db.table` condition to search
policy, the Ranger will
+ * match `db1.table1`, `db1.table2`, `db*.table*`, So we need to manually
precisely filter this
+ * research results.
+ * policySearchKeys: The search Ranger policy condition key defines.
+ * policyPreciseFilterKeys: The precise filter Ranger search results key
defines
+ */
+ protected List policySearchKeys = null;
+
+ protected List policyPreciseFilterKeys = null;
+
+ public RangerHelper(RangerAuthorizationPlugin rangerAuthorizationPlugin,
String catalogProvider) {
+this.rangerAuthorizationPlugin = rangerAuthorizationPlugin;
+switch (catalogProvider) {
+ case "hive":
+initPrivilegesMapping();
+initOwnerPrivileges();
+initPolicySearchKeys();
+initPreciseFilterKeys();
+break;
+ default:
+throw new IllegalArgumentException(
+"Authorization plugin unsupported catalog provider: " +
catalogProvider);
+}
+ }
+
+ /** Initial mapping Gravitino privilege name to the underlying authorization
system privileges. */
+ private void initPrivilegesMapping() {
+privilegesMapping =
+ImmutableMap.>builder()
+.put(
+Privilege.Name.CREATE_SCHEMA,
+ImmutableSet.of(RangerDefines.ACCESS_TYPE_HIVE_CREATE))
+.put(
+Privilege.Name.CREATE_TABLE,
ImmutableSet.of(RangerDefines.ACCESS_TYPE_HIVE_CREATE))
+.put(
+Privilege.Name.MODIFY_TABLE,
+ImmutableSet.of(
+RangerDefines.ACCESS_TYPE_HIVE_UPDATE,
+RangerDefines.ACCESS_TYPE_HIVE_ALTER,
+RangerDefines.A
Re: [PR] [#3963] feat(core): Apache Ranger Hive authorization pushdown [gravitino]
xunliu commented on PR #4515: URL: https://github.com/apache/gravitino/pull/4515#issuecomment-2301657509 > > > Does it support multiple Hive catalog uses one Ranger? > > > > > > Yes, we support. Because each Hive catalog have a separate AuthorizationPlugin instance. > > All the authorization plugin uses the same label. Won't it influence each other? They use the same property names but have different values. The key point is `RANGER_SERVICE_NAME` has different values. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
Re: [PR] [#3963] feat(core): Apache Ranger Hive authorization pushdown [gravitino]
jerqi commented on PR #4515: URL: https://github.com/apache/gravitino/pull/4515#issuecomment-2301655274 Thanks for your hard work! The code seems good shape. There are some points to discuss with you. 1. Could you configure the catalog with other authorization plugin instead of RangerPlugin? 2. Some privileges like `use schema`, `use catalog`, `create catalog` could be pushed down underlying system, too. The check logic will throw exception for them. 3. We should reduce the duplicated code further as possible as we can. 4. If we use a label for Gravitino, but we don't have any catalog information. If we have two hive catalogs, catalog1 and catalog2, they have the same name schema schema1, will the policies influence each other? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
Re: [PR] [#3963] feat(core): Apache Ranger Hive authorization pushdown [gravitino]
jerqi commented on code in PR #4515:
URL: https://github.com/apache/gravitino/pull/4515#discussion_r1724775759
##
authorizations/authorization-ranger/src/main/java/org/apache/gravitino/authorization/ranger/RangerAuthorizationPlugin.java:
##
@@ -0,0 +1,1006 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+package org.apache.gravitino.authorization.ranger;
+
+import com.google.common.annotations.VisibleForTesting;
+import com.google.common.collect.ImmutableMap;
+import com.google.common.collect.Lists;
+import com.google.errorprone.annotations.FormatMethod;
+import com.google.errorprone.annotations.FormatString;
+import java.io.IOException;
+import java.time.Instant;
+import java.util.Arrays;
+import java.util.Collections;
+import java.util.HashMap;
+import java.util.HashSet;
+import java.util.List;
+import java.util.Map;
+import java.util.Objects;
+import java.util.Set;
+import java.util.stream.Collectors;
+import org.apache.gravitino.MetadataObject;
+import org.apache.gravitino.authorization.Group;
+import org.apache.gravitino.authorization.Owner;
+import org.apache.gravitino.authorization.Privilege;
+import org.apache.gravitino.authorization.Role;
+import org.apache.gravitino.authorization.RoleChange;
+import org.apache.gravitino.authorization.SecurableObject;
+import org.apache.gravitino.authorization.SecurableObjects;
+import org.apache.gravitino.authorization.User;
+import org.apache.gravitino.authorization.ranger.defines.VXGroup;
+import org.apache.gravitino.authorization.ranger.defines.VXGroupList;
+import org.apache.gravitino.authorization.ranger.defines.VXUser;
+import org.apache.gravitino.authorization.ranger.defines.VXUserList;
+import org.apache.gravitino.connector.AuthorizationPropertiesMeta;
+import org.apache.gravitino.connector.authorization.AuthorizationPlugin;
+import org.apache.gravitino.exceptions.AuthorizationPluginException;
+import org.apache.gravitino.meta.AuditInfo;
+import org.apache.gravitino.meta.GroupEntity;
+import org.apache.gravitino.meta.UserEntity;
+import org.apache.gravitino.utils.PrincipalUtils;
+import org.apache.ranger.RangerServiceException;
+import org.apache.ranger.plugin.model.RangerPolicy;
+import org.apache.ranger.plugin.model.RangerRole;
+import org.apache.ranger.plugin.util.GrantRevokeRoleRequest;
+import org.apache.ranger.plugin.util.SearchFilter;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+/** Ranger authorization operations plugin abstract class. */
+public abstract class RangerAuthorizationPlugin implements AuthorizationPlugin
{
+ private static final Logger LOG =
LoggerFactory.getLogger(RangerAuthorizationPlugin.class);
+
+ protected String catalogProvider;
+ protected RangerClientExtend rangerClient;
+ protected String rangerServiceName;
+ /** Mapping Gravitino privilege name to the underlying authorization system
privileges. */
+ protected Map> privilegesMapping = null;
+ // The owner privileges, the owner can do anything on the metadata object
+ protected Set ownerPrivileges = null;
+
+ /**
+ * Because Ranger doesn't support the precise filter, Ranger will return the
policy meets the
+ * wildcard(*,?) conditions, just like `*.*.*` policy will match
`db1.table1.column1` So we need
+ * to manually precisely filter the policies.
+ */
+ // Search Ranger policy filter keys
+ protected List policyFilterKeys = null;
+ // Search Ranger policy precise filter keys
+ protected List policyPreciseFilterKeys = null;
+
+ public static final String MANAGED_BY_GRAVITINO = "MANAGED_BY_GRAVITINO";
+
+ // TODO: Maybe need to move to the configuration in the future
+ public static final String RANGER_ADMIN_NAME = "admin";
+
+ public RangerAuthorizationPlugin(String catalogProvider, Map
config) {
+super();
+this.catalogProvider = catalogProvider;
+String rangerUrl =
config.get(AuthorizationPropertiesMeta.RANGER_ADMIN_URL);
+String authType = config.get(AuthorizationPropertiesMeta.RANGER_AUTH_TYPE);
+String username = config.get(AuthorizationPropertiesMeta.RANGER_USERNAME);
+// Apache Ranger Password should be minimum 8 characters with min one
alphabet and one numeric.
+String password = config.get(AuthorizationPro
Re: [PR] [#3963] feat(core): Apache Ranger Hive authorization pushdown [gravitino]
jerqi commented on code in PR #4515:
URL: https://github.com/apache/gravitino/pull/4515#discussion_r1724774456
##
authorizations/authorization-ranger/src/main/java/org/apache/gravitino/authorization/ranger/RangerAuthorizationPlugin.java:
##
@@ -0,0 +1,1006 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+package org.apache.gravitino.authorization.ranger;
+
+import com.google.common.annotations.VisibleForTesting;
+import com.google.common.collect.ImmutableMap;
+import com.google.common.collect.Lists;
+import com.google.errorprone.annotations.FormatMethod;
+import com.google.errorprone.annotations.FormatString;
+import java.io.IOException;
+import java.time.Instant;
+import java.util.Arrays;
+import java.util.Collections;
+import java.util.HashMap;
+import java.util.HashSet;
+import java.util.List;
+import java.util.Map;
+import java.util.Objects;
+import java.util.Set;
+import java.util.stream.Collectors;
+import org.apache.gravitino.MetadataObject;
+import org.apache.gravitino.authorization.Group;
+import org.apache.gravitino.authorization.Owner;
+import org.apache.gravitino.authorization.Privilege;
+import org.apache.gravitino.authorization.Role;
+import org.apache.gravitino.authorization.RoleChange;
+import org.apache.gravitino.authorization.SecurableObject;
+import org.apache.gravitino.authorization.SecurableObjects;
+import org.apache.gravitino.authorization.User;
+import org.apache.gravitino.authorization.ranger.defines.VXGroup;
+import org.apache.gravitino.authorization.ranger.defines.VXGroupList;
+import org.apache.gravitino.authorization.ranger.defines.VXUser;
+import org.apache.gravitino.authorization.ranger.defines.VXUserList;
+import org.apache.gravitino.connector.AuthorizationPropertiesMeta;
+import org.apache.gravitino.connector.authorization.AuthorizationPlugin;
+import org.apache.gravitino.exceptions.AuthorizationPluginException;
+import org.apache.gravitino.meta.AuditInfo;
+import org.apache.gravitino.meta.GroupEntity;
+import org.apache.gravitino.meta.UserEntity;
+import org.apache.gravitino.utils.PrincipalUtils;
+import org.apache.ranger.RangerServiceException;
+import org.apache.ranger.plugin.model.RangerPolicy;
+import org.apache.ranger.plugin.model.RangerRole;
+import org.apache.ranger.plugin.util.GrantRevokeRoleRequest;
+import org.apache.ranger.plugin.util.SearchFilter;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+/** Ranger authorization operations plugin abstract class. */
+public abstract class RangerAuthorizationPlugin implements AuthorizationPlugin
{
+ private static final Logger LOG =
LoggerFactory.getLogger(RangerAuthorizationPlugin.class);
+
+ protected String catalogProvider;
+ protected RangerClientExtend rangerClient;
+ protected String rangerServiceName;
+ /** Mapping Gravitino privilege name to the underlying authorization system
privileges. */
+ protected Map> mapPrivileges = null;
+ // The owner privileges, the owner can do anything on the metadata object
+ protected Set ownerPrivileges = null;
+
+ /**
+ * Because Ranger doesn't support the precise filter, Ranger will return the
policy meets the
+ * wildcard(*,?) conditions, just like `*.*.*` policy will match
`db1.table1.column1` So we need
+ * to manual precise filter the policies.
+ */
+ // Search Ranger policy filter keys
+ protected List policyFilterKeys = null;
Review Comment:
This is resolved.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
Re: [PR] [#3963] feat(core): Apache Ranger Hive authorization pushdown [gravitino]
jerqi commented on code in PR #4515:
URL: https://github.com/apache/gravitino/pull/4515#discussion_r1724774008
##
authorizations/authorization-ranger/src/main/java/org/apache/gravitino/authorization/ranger/RangerAuthorizationPlugin.java:
##
@@ -0,0 +1,1006 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+package org.apache.gravitino.authorization.ranger;
+
+import com.google.common.annotations.VisibleForTesting;
+import com.google.common.collect.ImmutableMap;
+import com.google.common.collect.Lists;
+import com.google.errorprone.annotations.FormatMethod;
+import com.google.errorprone.annotations.FormatString;
+import java.io.IOException;
+import java.time.Instant;
+import java.util.Arrays;
+import java.util.Collections;
+import java.util.HashMap;
+import java.util.HashSet;
+import java.util.List;
+import java.util.Map;
+import java.util.Objects;
+import java.util.Set;
+import java.util.stream.Collectors;
+import org.apache.gravitino.MetadataObject;
+import org.apache.gravitino.authorization.Group;
+import org.apache.gravitino.authorization.Owner;
+import org.apache.gravitino.authorization.Privilege;
+import org.apache.gravitino.authorization.Role;
+import org.apache.gravitino.authorization.RoleChange;
+import org.apache.gravitino.authorization.SecurableObject;
+import org.apache.gravitino.authorization.SecurableObjects;
+import org.apache.gravitino.authorization.User;
+import org.apache.gravitino.authorization.ranger.defines.VXGroup;
+import org.apache.gravitino.authorization.ranger.defines.VXGroupList;
+import org.apache.gravitino.authorization.ranger.defines.VXUser;
+import org.apache.gravitino.authorization.ranger.defines.VXUserList;
+import org.apache.gravitino.connector.AuthorizationPropertiesMeta;
+import org.apache.gravitino.connector.authorization.AuthorizationPlugin;
+import org.apache.gravitino.exceptions.AuthorizationPluginException;
+import org.apache.gravitino.meta.AuditInfo;
+import org.apache.gravitino.meta.GroupEntity;
+import org.apache.gravitino.meta.UserEntity;
+import org.apache.gravitino.utils.PrincipalUtils;
+import org.apache.ranger.RangerServiceException;
+import org.apache.ranger.plugin.model.RangerPolicy;
+import org.apache.ranger.plugin.model.RangerRole;
+import org.apache.ranger.plugin.util.GrantRevokeRoleRequest;
+import org.apache.ranger.plugin.util.SearchFilter;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+/** Ranger authorization operations plugin abstract class. */
+public abstract class RangerAuthorizationPlugin implements AuthorizationPlugin
{
+ private static final Logger LOG =
LoggerFactory.getLogger(RangerAuthorizationPlugin.class);
+
+ protected String catalogProvider;
+ protected RangerClientExtend rangerClient;
+ protected String rangerServiceName;
+ /** Mapping Gravitino privilege name to the underlying authorization system
privileges. */
+ protected Map> mapPrivileges = null;
+ // The owner privileges, the owner can do anything on the metadata object
+ protected Set ownerPrivileges = null;
+
+ /**
+ * Because Ranger doesn't support the precise filter, Ranger will return the
policy meets the
Review Comment:
This is resolved.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
Re: [PR] [#3963] feat(core): Apache Ranger Hive authorization pushdown [gravitino]
jerqi commented on code in PR #4515:
URL: https://github.com/apache/gravitino/pull/4515#discussion_r1724773530
##
authorizations/authorization-ranger/src/main/java/org/apache/gravitino/authorization/ranger/RangerAuthorizationPlugin.java:
##
@@ -0,0 +1,1006 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+package org.apache.gravitino.authorization.ranger;
+
+import com.google.common.annotations.VisibleForTesting;
+import com.google.common.collect.ImmutableMap;
+import com.google.common.collect.Lists;
+import com.google.errorprone.annotations.FormatMethod;
+import com.google.errorprone.annotations.FormatString;
+import java.io.IOException;
+import java.time.Instant;
+import java.util.Arrays;
+import java.util.Collections;
+import java.util.HashMap;
+import java.util.HashSet;
+import java.util.List;
+import java.util.Map;
+import java.util.Objects;
+import java.util.Set;
+import java.util.stream.Collectors;
+import org.apache.gravitino.MetadataObject;
+import org.apache.gravitino.authorization.Group;
+import org.apache.gravitino.authorization.Owner;
+import org.apache.gravitino.authorization.Privilege;
+import org.apache.gravitino.authorization.Role;
+import org.apache.gravitino.authorization.RoleChange;
+import org.apache.gravitino.authorization.SecurableObject;
+import org.apache.gravitino.authorization.SecurableObjects;
+import org.apache.gravitino.authorization.User;
+import org.apache.gravitino.authorization.ranger.defines.VXGroup;
+import org.apache.gravitino.authorization.ranger.defines.VXGroupList;
+import org.apache.gravitino.authorization.ranger.defines.VXUser;
+import org.apache.gravitino.authorization.ranger.defines.VXUserList;
+import org.apache.gravitino.connector.AuthorizationPropertiesMeta;
+import org.apache.gravitino.connector.authorization.AuthorizationPlugin;
+import org.apache.gravitino.exceptions.AuthorizationPluginException;
+import org.apache.gravitino.meta.AuditInfo;
+import org.apache.gravitino.meta.GroupEntity;
+import org.apache.gravitino.meta.UserEntity;
+import org.apache.gravitino.utils.PrincipalUtils;
+import org.apache.ranger.RangerServiceException;
+import org.apache.ranger.plugin.model.RangerPolicy;
+import org.apache.ranger.plugin.model.RangerRole;
+import org.apache.ranger.plugin.util.GrantRevokeRoleRequest;
+import org.apache.ranger.plugin.util.SearchFilter;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+/** Ranger authorization operations plugin abstract class. */
+public abstract class RangerAuthorizationPlugin implements AuthorizationPlugin
{
+ private static final Logger LOG =
LoggerFactory.getLogger(RangerAuthorizationPlugin.class);
+
+ protected String catalogProvider;
+ protected RangerClientExtend rangerClient;
+ protected String rangerServiceName;
+ /** Mapping Gravitino privilege name to the underlying authorization system
privileges. */
+ protected Map> mapPrivileges = null;
+ // The owner privileges, the owner can do anything on the metadata object
+ protected Set ownerPrivileges = null;
+
+ /**
+ * Because Ranger doesn't support the precise filter, Ranger will return the
policy meets the
+ * wildcard(*,?) conditions, just like `*.*.*` policy will match
`db1.table1.column1` So we need
+ * to manual precise filter the policies.
+ */
+ // Search Ranger policy filter keys
+ protected List policyFilterKeys = null;
+ // Search Ranger policy precise filter keys
+ protected List policyPreciseFilterKeys = null;
+
+ public static final String MANAGED_BY_GRAVITINO = "MANAGED_BY_GRAVITINO";
+
+ // TODO: Maybe need to move to the configuration in the future
+ public static final String RANGER_ADMIN_NAME = "admin";
+
+ public RangerAuthorizationPlugin(String catalogProvider, Map
config) {
+super();
+this.catalogProvider = catalogProvider;
+String rangerUrl =
config.get(AuthorizationPropertiesMeta.RANGER_ADMIN_URL);
+String authType = config.get(AuthorizationPropertiesMeta.RANGER_AUTH_TYPE);
+String username = config.get(AuthorizationPropertiesMeta.RANGER_USERNAME);
+// Apache Ranger Password should be minimum 8 characters with min one
alphabet and one numeric.
+String password = config.get(AuthorizationPropertiesM
Re: [PR] [#3963] feat(core): Apache Ranger Hive authorization pushdown [gravitino]
jerqi commented on PR #4515: URL: https://github.com/apache/gravitino/pull/4515#issuecomment-2301631113 > > Does it support multiple Hive catalog uses one Ranger? > > Yes, we support. Because each Hive catalog have a separate AuthorizationPlugin instance. All the authorization plugin uses the same label. Won't it influence each other? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
Re: [PR] [#3963] feat(core): Apache Ranger Hive authorization pushdown [gravitino]
jerqi commented on code in PR #4515:
URL: https://github.com/apache/gravitino/pull/4515#discussion_r1724766504
##
authorizations/authorization-ranger/src/main/java/org/apache/gravitino/authorization/ranger/RangerHelper.java:
##
@@ -0,0 +1,533 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+package org.apache.gravitino.authorization.ranger;
+
+import com.google.common.annotations.VisibleForTesting;
+import com.google.common.collect.ImmutableMap;
+import com.google.common.collect.ImmutableSet;
+import com.google.common.collect.Lists;
+import com.google.common.collect.Sets;
+import com.google.errorprone.annotations.FormatMethod;
+import com.google.errorprone.annotations.FormatString;
+import java.util.Arrays;
+import java.util.HashMap;
+import java.util.HashSet;
+import java.util.List;
+import java.util.Map;
+import java.util.Objects;
+import java.util.Set;
+import java.util.stream.Collectors;
+import org.apache.commons.lang.StringUtils;
+import org.apache.gravitino.MetadataObject;
+import org.apache.gravitino.authorization.Owner;
+import org.apache.gravitino.authorization.Privilege;
+import org.apache.gravitino.authorization.SecurableObject;
+import org.apache.gravitino.authorization.SecurableObjects;
+import org.apache.gravitino.authorization.ranger.reference.RangerDefines;
+import org.apache.gravitino.exceptions.AuthorizationPluginException;
+import org.apache.ranger.RangerServiceException;
+import org.apache.ranger.plugin.model.RangerPolicy;
+import org.apache.ranger.plugin.model.RangerRole;
+import org.apache.ranger.plugin.util.GrantRevokeRoleRequest;
+import org.apache.ranger.plugin.util.SearchFilter;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+/**
+ * This class is a helper class for the Ranger authorization plugin. It
provides the ability to
+ * manage the Ranger policies and roles.
+ */
+public class RangerHelper {
+ private static final Logger LOG =
LoggerFactory.getLogger(RangerHelper.class);
+ public static final String MANAGED_BY_GRAVITINO = "MANAGED_BY_GRAVITINO";
+
+ RangerAuthorizationPlugin rangerAuthorizationPlugin;
+
+ /** Mapping Gravitino privilege name to the underlying authorization system
privileges. */
+ protected Map> privilegesMapping = null;
+ /** The owner privileges, the owner can do anything on the metadata object */
+ protected Set ownerPrivileges = null;
+
+ /**
+ * Because Ranger doesn't support the precise search, Ranger will return the
policy meets the
+ * wildcard(*,?) conditions, If you use `db.table` condition to search
policy, the Ranger will
+ * match `db1.table1`, `db1.table2`, `db*.table*`, So we need to manually
precisely filter this
+ * research results.
+ * policySearchKeys: The search Ranger policy condition key defines.
+ * policyPreciseFilterKeys: The precise filter Ranger search results key
defines
+ */
+ protected List policySearchKeys = null;
+
+ protected List policyPreciseFilterKeys = null;
+
+ public RangerHelper(RangerAuthorizationPlugin rangerAuthorizationPlugin,
String catalogProvider) {
+this.rangerAuthorizationPlugin = rangerAuthorizationPlugin;
+switch (catalogProvider) {
+ case "hive":
+initPrivilegesMapping();
+initOwnerPrivileges();
+initPolicySearchKeys();
+initPreciseFilterKeys();
+break;
+ default:
+throw new IllegalArgumentException(
+"Authorization plugin unsupported catalog provider: " +
catalogProvider);
+}
+ }
+
+ /** Initial mapping Gravitino privilege name to the underlying authorization
system privileges. */
+ private void initPrivilegesMapping() {
+privilegesMapping =
+ImmutableMap.>builder()
+.put(
+Privilege.Name.CREATE_SCHEMA,
+ImmutableSet.of(RangerDefines.ACCESS_TYPE_HIVE_CREATE))
+.put(
+Privilege.Name.CREATE_TABLE,
ImmutableSet.of(RangerDefines.ACCESS_TYPE_HIVE_CREATE))
+.put(
+Privilege.Name.MODIFY_TABLE,
+ImmutableSet.of(
+RangerDefines.ACCESS_TYPE_HIVE_UPDATE,
+RangerDefines.ACCESS_TYPE_HIVE_ALTER,
+RangerDefines.AC
Re: [PR] [#3963] feat(core): Apache Ranger Hive authorization pushdown [gravitino]
jerqi commented on code in PR #4515:
URL: https://github.com/apache/gravitino/pull/4515#discussion_r1724759506
##
authorizations/authorization-ranger/src/main/java/org/apache/gravitino/authorization/ranger/RangerAuthorizationPlugin.java:
##
@@ -0,0 +1,583 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+package org.apache.gravitino.authorization.ranger;
+
+import com.google.common.annotations.VisibleForTesting;
+import com.google.common.collect.ImmutableMap;
+import com.google.common.collect.Lists;
+import java.io.IOException;
+import java.time.Instant;
+import java.util.Collections;
+import java.util.List;
+import java.util.Map;
+import java.util.Objects;
+import java.util.Set;
+import java.util.stream.Collectors;
+import org.apache.gravitino.MetadataObject;
+import org.apache.gravitino.authorization.Group;
+import org.apache.gravitino.authorization.Owner;
+import org.apache.gravitino.authorization.Privilege;
+import org.apache.gravitino.authorization.Role;
+import org.apache.gravitino.authorization.RoleChange;
+import org.apache.gravitino.authorization.SecurableObjects;
+import org.apache.gravitino.authorization.User;
+import org.apache.gravitino.authorization.ranger.reference.RangerDefines;
+import org.apache.gravitino.authorization.ranger.reference.VXGroup;
+import org.apache.gravitino.authorization.ranger.reference.VXGroupList;
+import org.apache.gravitino.authorization.ranger.reference.VXUser;
+import org.apache.gravitino.authorization.ranger.reference.VXUserList;
+import org.apache.gravitino.connector.AuthorizationPropertiesMeta;
+import org.apache.gravitino.connector.authorization.AuthorizationPlugin;
+import org.apache.gravitino.meta.AuditInfo;
+import org.apache.gravitino.meta.GroupEntity;
+import org.apache.gravitino.meta.UserEntity;
+import org.apache.gravitino.utils.PrincipalUtils;
+import org.apache.ranger.RangerServiceException;
+import org.apache.ranger.plugin.model.RangerPolicy;
+import org.apache.ranger.plugin.util.GrantRevokeRoleRequest;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+/**
+ * Ranger authorization operations plugin abstract class.
+ * 1. For Ranger limit, The same metadata object only has a unique Ranger
policy, So a Ranger policy
+ * maybe contains multiple Gravitino securable objects.
+ * 2. For easy management, each Ranger privilege will create one
RangerPolicyItemAccess in the
+ * policy.
+ * 3. Ranger also have Role concept, and support adds multiple users or groups
in the Ranger Role,
+ * So we can use the Ranger Role to implement the Gravitino Role.
+ * 4. The Ranger policy also supports multiple users and groups, But we only
use a user or group to
+ * implement Gravitino Owner concept.
+ */
+public class RangerAuthorizationPlugin implements AuthorizationPlugin {
+ private static final Logger LOG =
LoggerFactory.getLogger(RangerAuthorizationPlugin.class);
+
+ protected String catalogProvider;
+ protected String rangerServiceName;
+ protected RangerClientExtend rangerClient;
+ private RangerHelper rangerHelper;
+ @VisibleForTesting public final String rangerAdminName;
+
+ public RangerAuthorizationPlugin(String catalogProvider, Map
config) {
+super();
+this.catalogProvider = catalogProvider;
+String rangerUrl =
config.get(AuthorizationPropertiesMeta.RANGER_ADMIN_URL);
+String authType = config.get(AuthorizationPropertiesMeta.RANGER_AUTH_TYPE);
+rangerAdminName = config.get(AuthorizationPropertiesMeta.RANGER_USERNAME);
+// Apache Ranger Password should be minimum 8 characters with min one
alphabet and one numeric.
+String password = config.get(AuthorizationPropertiesMeta.RANGER_PASSWORD);
+rangerServiceName =
config.get(AuthorizationPropertiesMeta.RANGER_SERVICE_NAME);
+RangerHelper.check(rangerUrl != null, "Ranger admin URL is required");
+RangerHelper.check(authType != null, "Ranger auth type is required");
+RangerHelper.check(rangerAdminName != null, "Ranger username is required");
+RangerHelper.check(password != null, "Ranger password is required");
+RangerHelper.check(rangerServiceName != null, "Ranger service name is
required");
+rangerClient = new RangerClientExtend(rangerUrl, authType
Re: [PR] [#3963] feat(core): Apache Ranger Hive authorization pushdown [gravitino]
jerqi commented on code in PR #4515:
URL: https://github.com/apache/gravitino/pull/4515#discussion_r1724755880
##
authorizations/authorization-ranger/src/main/java/org/apache/gravitino/authorization/ranger/RangerAuthorizationPlugin.java:
##
@@ -0,0 +1,998 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+package org.apache.gravitino.authorization.ranger;
+
+import com.google.common.annotations.VisibleForTesting;
+import com.google.common.collect.ImmutableMap;
+import com.google.common.collect.Lists;
+import com.google.common.collect.Sets;
+import com.google.errorprone.annotations.FormatMethod;
+import com.google.errorprone.annotations.FormatString;
+import java.io.IOException;
+import java.time.Instant;
+import java.util.Collections;
+import java.util.HashMap;
+import java.util.List;
+import java.util.Map;
+import java.util.Objects;
+import java.util.Set;
+import java.util.stream.Collectors;
+import org.apache.commons.lang3.StringUtils;
+import org.apache.gravitino.MetadataObject;
+import org.apache.gravitino.authorization.Group;
+import org.apache.gravitino.authorization.Owner;
+import org.apache.gravitino.authorization.Privilege;
+import org.apache.gravitino.authorization.Role;
+import org.apache.gravitino.authorization.RoleChange;
+import org.apache.gravitino.authorization.SecurableObject;
+import org.apache.gravitino.authorization.SecurableObjects;
+import org.apache.gravitino.authorization.User;
+import org.apache.gravitino.authorization.ranger.defines.VXGroup;
+import org.apache.gravitino.authorization.ranger.defines.VXGroupList;
+import org.apache.gravitino.authorization.ranger.defines.VXUser;
+import org.apache.gravitino.authorization.ranger.defines.VXUserList;
+import org.apache.gravitino.connector.AuthorizationPropertiesMeta;
+import org.apache.gravitino.connector.authorization.AuthorizationPlugin;
+import org.apache.gravitino.exceptions.AuthorizationPluginException;
+import org.apache.gravitino.meta.AuditInfo;
+import org.apache.gravitino.meta.GroupEntity;
+import org.apache.gravitino.meta.UserEntity;
+import org.apache.gravitino.utils.PrincipalUtils;
+import org.apache.ranger.RangerServiceException;
+import org.apache.ranger.plugin.model.RangerPolicy;
+import org.apache.ranger.plugin.model.RangerPolicy.RangerPolicyItem;
+import org.apache.ranger.plugin.model.RangerRole;
+import org.apache.ranger.plugin.util.GrantRevokeRoleRequest;
+import org.apache.ranger.plugin.util.SearchFilter;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+/** Ranger authorization operations plugin abstract class. */
+public abstract class RangerAuthorizationPlugin implements AuthorizationPlugin
{
+ private static final Logger LOG =
LoggerFactory.getLogger(RangerAuthorizationPlugin.class);
+
+ protected String catalogProvider;
+ protected RangerClientExtend rangerClient;
+ protected String rangerServiceName;
+ /** Mapping Gravitino privilege name to the underlying authorization system
privileges. */
+ protected Map> privilegesMapping = null;
+ // The owner privileges, the owner can do anything on the metadata object
+ protected Set ownerPrivileges = null;
+
+ /**
+ * Because Ranger doesn't support the precise filter, Ranger will return the
policy meets the
+ * wildcard(*,?) conditions, just like `*.*.*` policy will match
`db1.table1.column1` So we need
+ * to manually precisely filter the policies.
+ */
+ // Search Ranger policy filter keys
+ protected List policyFilterKeys = null;
+ // Search Ranger policy precise filter keys
+ protected List policyPreciseFilterKeys = null;
+
+ public static final String MANAGED_BY_GRAVITINO = "MANAGED_BY_GRAVITINO";
+
+ // TODO: Maybe need to move to the configuration in the future
+ public static final String RANGER_ADMIN_NAME = "admin";
+
+ public RangerAuthorizationPlugin(String catalogProvider, Map
config) {
+super();
+this.catalogProvider = catalogProvider;
+String rangerUrl =
config.get(AuthorizationPropertiesMeta.RANGER_ADMIN_URL);
+String authType = config.get(AuthorizationPropertiesMeta.RANGER_AUTH_TYPE);
+String username = config.get(AuthorizationPropertiesMeta.RANGER_USERNAME);
+// Apache Ranger Password should be minimum 8
Re: [PR] [#3963] feat(core): Apache Ranger Hive authorization pushdown [gravitino]
jerqi commented on code in PR #4515:
URL: https://github.com/apache/gravitino/pull/4515#discussion_r1724753819
##
authorizations/authorization-ranger/src/main/java/org/apache/gravitino/authorization/ranger/RangerAuthorizationPlugin.java:
##
@@ -0,0 +1,583 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+package org.apache.gravitino.authorization.ranger;
+
+import com.google.common.annotations.VisibleForTesting;
+import com.google.common.collect.ImmutableMap;
+import com.google.common.collect.Lists;
+import java.io.IOException;
+import java.time.Instant;
+import java.util.Collections;
+import java.util.List;
+import java.util.Map;
+import java.util.Objects;
+import java.util.Set;
+import java.util.stream.Collectors;
+import org.apache.gravitino.MetadataObject;
+import org.apache.gravitino.authorization.Group;
+import org.apache.gravitino.authorization.Owner;
+import org.apache.gravitino.authorization.Privilege;
+import org.apache.gravitino.authorization.Role;
+import org.apache.gravitino.authorization.RoleChange;
+import org.apache.gravitino.authorization.SecurableObjects;
+import org.apache.gravitino.authorization.User;
+import org.apache.gravitino.authorization.ranger.reference.RangerDefines;
+import org.apache.gravitino.authorization.ranger.reference.VXGroup;
+import org.apache.gravitino.authorization.ranger.reference.VXGroupList;
+import org.apache.gravitino.authorization.ranger.reference.VXUser;
+import org.apache.gravitino.authorization.ranger.reference.VXUserList;
+import org.apache.gravitino.connector.AuthorizationPropertiesMeta;
+import org.apache.gravitino.connector.authorization.AuthorizationPlugin;
+import org.apache.gravitino.meta.AuditInfo;
+import org.apache.gravitino.meta.GroupEntity;
+import org.apache.gravitino.meta.UserEntity;
+import org.apache.gravitino.utils.PrincipalUtils;
+import org.apache.ranger.RangerServiceException;
+import org.apache.ranger.plugin.model.RangerPolicy;
+import org.apache.ranger.plugin.util.GrantRevokeRoleRequest;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+/**
+ * Ranger authorization operations plugin abstract class.
+ * 1. For Ranger limit, The same metadata object only has a unique Ranger
policy, So a Ranger policy
+ * maybe contains multiple Gravitino securable objects.
+ * 2. For easy management, each Ranger privilege will create one
RangerPolicyItemAccess in the
+ * policy.
+ * 3. Ranger also have Role concept, and support adds multiple users or groups
in the Ranger Role,
+ * So we can use the Ranger Role to implement the Gravitino Role.
+ * 4. The Ranger policy also supports multiple users and groups, But we only
use a user or group to
+ * implement Gravitino Owner concept.
+ */
+public class RangerAuthorizationPlugin implements AuthorizationPlugin {
+ private static final Logger LOG =
LoggerFactory.getLogger(RangerAuthorizationPlugin.class);
+
+ protected String catalogProvider;
+ protected String rangerServiceName;
+ protected RangerClientExtend rangerClient;
+ private RangerHelper rangerHelper;
+ @VisibleForTesting public final String rangerAdminName;
+
+ public RangerAuthorizationPlugin(String catalogProvider, Map
config) {
+super();
+this.catalogProvider = catalogProvider;
+String rangerUrl =
config.get(AuthorizationPropertiesMeta.RANGER_ADMIN_URL);
+String authType = config.get(AuthorizationPropertiesMeta.RANGER_AUTH_TYPE);
+rangerAdminName = config.get(AuthorizationPropertiesMeta.RANGER_USERNAME);
+// Apache Ranger Password should be minimum 8 characters with min one
alphabet and one numeric.
+String password = config.get(AuthorizationPropertiesMeta.RANGER_PASSWORD);
+rangerServiceName =
config.get(AuthorizationPropertiesMeta.RANGER_SERVICE_NAME);
+RangerHelper.check(rangerUrl != null, "Ranger admin URL is required");
+RangerHelper.check(authType != null, "Ranger auth type is required");
+RangerHelper.check(rangerAdminName != null, "Ranger username is required");
+RangerHelper.check(password != null, "Ranger password is required");
+RangerHelper.check(rangerServiceName != null, "Ranger service name is
required");
+rangerClient = new RangerClientExtend(rangerUrl, authType
Re: [PR] [#3963] feat(core): Apache Ranger Hive authorization pushdown [gravitino]
jerqi commented on code in PR #4515:
URL: https://github.com/apache/gravitino/pull/4515#discussion_r1724753495
##
authorizations/authorization-ranger/src/main/java/org/apache/gravitino/authorization/ranger/RangerAuthorizationPlugin.java:
##
@@ -0,0 +1,583 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+package org.apache.gravitino.authorization.ranger;
+
+import com.google.common.annotations.VisibleForTesting;
+import com.google.common.collect.ImmutableMap;
+import com.google.common.collect.Lists;
+import java.io.IOException;
+import java.time.Instant;
+import java.util.Collections;
+import java.util.List;
+import java.util.Map;
+import java.util.Objects;
+import java.util.Set;
+import java.util.stream.Collectors;
+import org.apache.gravitino.MetadataObject;
+import org.apache.gravitino.authorization.Group;
+import org.apache.gravitino.authorization.Owner;
+import org.apache.gravitino.authorization.Privilege;
+import org.apache.gravitino.authorization.Role;
+import org.apache.gravitino.authorization.RoleChange;
+import org.apache.gravitino.authorization.SecurableObjects;
+import org.apache.gravitino.authorization.User;
+import org.apache.gravitino.authorization.ranger.reference.RangerDefines;
+import org.apache.gravitino.authorization.ranger.reference.VXGroup;
+import org.apache.gravitino.authorization.ranger.reference.VXGroupList;
+import org.apache.gravitino.authorization.ranger.reference.VXUser;
+import org.apache.gravitino.authorization.ranger.reference.VXUserList;
+import org.apache.gravitino.connector.AuthorizationPropertiesMeta;
+import org.apache.gravitino.connector.authorization.AuthorizationPlugin;
+import org.apache.gravitino.meta.AuditInfo;
+import org.apache.gravitino.meta.GroupEntity;
+import org.apache.gravitino.meta.UserEntity;
+import org.apache.gravitino.utils.PrincipalUtils;
+import org.apache.ranger.RangerServiceException;
+import org.apache.ranger.plugin.model.RangerPolicy;
+import org.apache.ranger.plugin.util.GrantRevokeRoleRequest;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+/**
+ * Ranger authorization operations plugin abstract class.
+ * 1. For Ranger limit, The same metadata object only has a unique Ranger
policy, So a Ranger policy
+ * maybe contains multiple Gravitino securable objects.
+ * 2. For easy management, each Ranger privilege will create one
RangerPolicyItemAccess in the
+ * policy.
+ * 3. Ranger also have Role concept, and support adds multiple users or groups
in the Ranger Role,
+ * So we can use the Ranger Role to implement the Gravitino Role.
+ * 4. The Ranger policy also supports multiple users and groups, But we only
use a user or group to
+ * implement Gravitino Owner concept.
+ */
+public class RangerAuthorizationPlugin implements AuthorizationPlugin {
+ private static final Logger LOG =
LoggerFactory.getLogger(RangerAuthorizationPlugin.class);
+
+ protected String catalogProvider;
+ protected String rangerServiceName;
+ protected RangerClientExtend rangerClient;
+ private RangerHelper rangerHelper;
+ @VisibleForTesting public final String rangerAdminName;
+
+ public RangerAuthorizationPlugin(String catalogProvider, Map
config) {
+super();
+this.catalogProvider = catalogProvider;
+String rangerUrl =
config.get(AuthorizationPropertiesMeta.RANGER_ADMIN_URL);
+String authType = config.get(AuthorizationPropertiesMeta.RANGER_AUTH_TYPE);
+rangerAdminName = config.get(AuthorizationPropertiesMeta.RANGER_USERNAME);
+// Apache Ranger Password should be minimum 8 characters with min one
alphabet and one numeric.
+String password = config.get(AuthorizationPropertiesMeta.RANGER_PASSWORD);
+rangerServiceName =
config.get(AuthorizationPropertiesMeta.RANGER_SERVICE_NAME);
+RangerHelper.check(rangerUrl != null, "Ranger admin URL is required");
+RangerHelper.check(authType != null, "Ranger auth type is required");
+RangerHelper.check(rangerAdminName != null, "Ranger username is required");
+RangerHelper.check(password != null, "Ranger password is required");
+RangerHelper.check(rangerServiceName != null, "Ranger service name is
required");
+rangerClient = new RangerClientExtend(rangerUrl, authType
Re: [PR] [#3963] feat(core): Apache Ranger Hive authorization pushdown [gravitino]
jerqi commented on code in PR #4515:
URL: https://github.com/apache/gravitino/pull/4515#discussion_r1724753165
##
authorizations/authorization-ranger/src/main/java/org/apache/gravitino/authorization/ranger/RangerAuthorizationPlugin.java:
##
@@ -0,0 +1,583 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+package org.apache.gravitino.authorization.ranger;
+
+import com.google.common.annotations.VisibleForTesting;
+import com.google.common.collect.ImmutableMap;
+import com.google.common.collect.Lists;
+import java.io.IOException;
+import java.time.Instant;
+import java.util.Collections;
+import java.util.List;
+import java.util.Map;
+import java.util.Objects;
+import java.util.Set;
+import java.util.stream.Collectors;
+import org.apache.gravitino.MetadataObject;
+import org.apache.gravitino.authorization.Group;
+import org.apache.gravitino.authorization.Owner;
+import org.apache.gravitino.authorization.Privilege;
+import org.apache.gravitino.authorization.Role;
+import org.apache.gravitino.authorization.RoleChange;
+import org.apache.gravitino.authorization.SecurableObjects;
+import org.apache.gravitino.authorization.User;
+import org.apache.gravitino.authorization.ranger.reference.RangerDefines;
+import org.apache.gravitino.authorization.ranger.reference.VXGroup;
+import org.apache.gravitino.authorization.ranger.reference.VXGroupList;
+import org.apache.gravitino.authorization.ranger.reference.VXUser;
+import org.apache.gravitino.authorization.ranger.reference.VXUserList;
+import org.apache.gravitino.connector.AuthorizationPropertiesMeta;
+import org.apache.gravitino.connector.authorization.AuthorizationPlugin;
+import org.apache.gravitino.meta.AuditInfo;
+import org.apache.gravitino.meta.GroupEntity;
+import org.apache.gravitino.meta.UserEntity;
+import org.apache.gravitino.utils.PrincipalUtils;
+import org.apache.ranger.RangerServiceException;
+import org.apache.ranger.plugin.model.RangerPolicy;
+import org.apache.ranger.plugin.util.GrantRevokeRoleRequest;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+/**
+ * Ranger authorization operations plugin abstract class.
+ * 1. For Ranger limit, The same metadata object only has a unique Ranger
policy, So a Ranger policy
+ * maybe contains multiple Gravitino securable objects.
+ * 2. For easy management, each Ranger privilege will create one
RangerPolicyItemAccess in the
+ * policy.
+ * 3. Ranger also have Role concept, and support adds multiple users or groups
in the Ranger Role,
+ * So we can use the Ranger Role to implement the Gravitino Role.
+ * 4. The Ranger policy also supports multiple users and groups, But we only
use a user or group to
+ * implement Gravitino Owner concept.
+ */
+public class RangerAuthorizationPlugin implements AuthorizationPlugin {
+ private static final Logger LOG =
LoggerFactory.getLogger(RangerAuthorizationPlugin.class);
+
+ protected String catalogProvider;
+ protected String rangerServiceName;
+ protected RangerClientExtend rangerClient;
+ private RangerHelper rangerHelper;
+ @VisibleForTesting public final String rangerAdminName;
+
+ public RangerAuthorizationPlugin(String catalogProvider, Map
config) {
+super();
+this.catalogProvider = catalogProvider;
+String rangerUrl =
config.get(AuthorizationPropertiesMeta.RANGER_ADMIN_URL);
+String authType = config.get(AuthorizationPropertiesMeta.RANGER_AUTH_TYPE);
+rangerAdminName = config.get(AuthorizationPropertiesMeta.RANGER_USERNAME);
+// Apache Ranger Password should be minimum 8 characters with min one
alphabet and one numeric.
+String password = config.get(AuthorizationPropertiesMeta.RANGER_PASSWORD);
+rangerServiceName =
config.get(AuthorizationPropertiesMeta.RANGER_SERVICE_NAME);
+RangerHelper.check(rangerUrl != null, "Ranger admin URL is required");
+RangerHelper.check(authType != null, "Ranger auth type is required");
+RangerHelper.check(rangerAdminName != null, "Ranger username is required");
+RangerHelper.check(password != null, "Ranger password is required");
+RangerHelper.check(rangerServiceName != null, "Ranger service name is
required");
+rangerClient = new RangerClientExtend(rangerUrl, authType
Re: [PR] [#3963] feat(core): Apache Ranger Hive authorization pushdown [gravitino]
jerqi commented on code in PR #4515:
URL: https://github.com/apache/gravitino/pull/4515#discussion_r1724752624
##
authorizations/authorization-ranger/src/main/java/org/apache/gravitino/authorization/ranger/RangerAuthorizationPlugin.java:
##
@@ -0,0 +1,583 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+package org.apache.gravitino.authorization.ranger;
+
+import com.google.common.annotations.VisibleForTesting;
+import com.google.common.collect.ImmutableMap;
+import com.google.common.collect.Lists;
+import java.io.IOException;
+import java.time.Instant;
+import java.util.Collections;
+import java.util.List;
+import java.util.Map;
+import java.util.Objects;
+import java.util.Set;
+import java.util.stream.Collectors;
+import org.apache.gravitino.MetadataObject;
+import org.apache.gravitino.authorization.Group;
+import org.apache.gravitino.authorization.Owner;
+import org.apache.gravitino.authorization.Privilege;
+import org.apache.gravitino.authorization.Role;
+import org.apache.gravitino.authorization.RoleChange;
+import org.apache.gravitino.authorization.SecurableObjects;
+import org.apache.gravitino.authorization.User;
+import org.apache.gravitino.authorization.ranger.reference.RangerDefines;
+import org.apache.gravitino.authorization.ranger.reference.VXGroup;
+import org.apache.gravitino.authorization.ranger.reference.VXGroupList;
+import org.apache.gravitino.authorization.ranger.reference.VXUser;
+import org.apache.gravitino.authorization.ranger.reference.VXUserList;
+import org.apache.gravitino.connector.AuthorizationPropertiesMeta;
+import org.apache.gravitino.connector.authorization.AuthorizationPlugin;
+import org.apache.gravitino.meta.AuditInfo;
+import org.apache.gravitino.meta.GroupEntity;
+import org.apache.gravitino.meta.UserEntity;
+import org.apache.gravitino.utils.PrincipalUtils;
+import org.apache.ranger.RangerServiceException;
+import org.apache.ranger.plugin.model.RangerPolicy;
+import org.apache.ranger.plugin.util.GrantRevokeRoleRequest;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+/**
+ * Ranger authorization operations plugin abstract class.
+ * 1. For Ranger limit, The same metadata object only has a unique Ranger
policy, So a Ranger policy
+ * maybe contains multiple Gravitino securable objects.
+ * 2. For easy management, each Ranger privilege will create one
RangerPolicyItemAccess in the
+ * policy.
+ * 3. Ranger also have Role concept, and support adds multiple users or groups
in the Ranger Role,
+ * So we can use the Ranger Role to implement the Gravitino Role.
+ * 4. The Ranger policy also supports multiple users and groups, But we only
use a user or group to
+ * implement Gravitino Owner concept.
+ */
+public class RangerAuthorizationPlugin implements AuthorizationPlugin {
+ private static final Logger LOG =
LoggerFactory.getLogger(RangerAuthorizationPlugin.class);
+
+ protected String catalogProvider;
+ protected String rangerServiceName;
+ protected RangerClientExtend rangerClient;
+ private RangerHelper rangerHelper;
+ @VisibleForTesting public final String rangerAdminName;
+
+ public RangerAuthorizationPlugin(String catalogProvider, Map
config) {
+super();
+this.catalogProvider = catalogProvider;
+String rangerUrl =
config.get(AuthorizationPropertiesMeta.RANGER_ADMIN_URL);
+String authType = config.get(AuthorizationPropertiesMeta.RANGER_AUTH_TYPE);
+rangerAdminName = config.get(AuthorizationPropertiesMeta.RANGER_USERNAME);
+// Apache Ranger Password should be minimum 8 characters with min one
alphabet and one numeric.
+String password = config.get(AuthorizationPropertiesMeta.RANGER_PASSWORD);
+rangerServiceName =
config.get(AuthorizationPropertiesMeta.RANGER_SERVICE_NAME);
+RangerHelper.check(rangerUrl != null, "Ranger admin URL is required");
+RangerHelper.check(authType != null, "Ranger auth type is required");
+RangerHelper.check(rangerAdminName != null, "Ranger username is required");
+RangerHelper.check(password != null, "Ranger password is required");
+RangerHelper.check(rangerServiceName != null, "Ranger service name is
required");
+rangerClient = new RangerClientExtend(rangerUrl, authType
Re: [PR] [#3963] feat(core): Apache Ranger Hive authorization pushdown [gravitino]
jerqi commented on code in PR #4515:
URL: https://github.com/apache/gravitino/pull/4515#discussion_r1724747227
##
authorizations/authorization-ranger/src/main/java/org/apache/gravitino/authorization/ranger/RangerClientExtend.java:
##
@@ -0,0 +1,184 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+package org.apache.gravitino.authorization.ranger;
+
+import com.google.common.collect.ImmutableMap;
+import com.sun.jersey.api.client.GenericType;
+import com.sun.jersey.api.client.UniformInterfaceException;
+import java.lang.reflect.InvocationTargetException;
+import java.lang.reflect.Method;
+import java.util.Map;
+import javax.ws.rs.HttpMethod;
+import javax.ws.rs.core.Response;
+import org.apache.gravitino.authorization.ranger.defines.VXGroup;
+import org.apache.gravitino.authorization.ranger.defines.VXGroupList;
+import org.apache.gravitino.authorization.ranger.defines.VXUser;
+import org.apache.gravitino.authorization.ranger.defines.VXUserList;
+import org.apache.ranger.RangerClient;
+import org.apache.ranger.RangerServiceException;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+/**
+ * Apache Ranger client extension
+ * The class extends the RangerClient class and provides additional methods to
create, search and
+ * delete users and groups
+ */
+public class RangerClientExtend extends RangerClient {
+ private static final Logger LOG =
LoggerFactory.getLogger(RangerClientExtend.class);
+ private static final String URI_USER_BASE = "/service/xusers/users";
+ private static final String URI_USER_BY_ID = URI_USER_BASE + "/%d";
+ private static final String URI_GROUP_BASE = "/service/xusers/groups";
+ private static final String URI_GROUP_BY_ID = URI_GROUP_BASE + "/%d";
+ private static final String URI_CREATE_EXTERNAL_USER = URI_USER_BASE +
"/external";
+
+ // Ranger user APIs
+ private static final API SEARCH_USER = new API(URI_USER_BASE,
HttpMethod.GET, Response.Status.OK);
+ private static final API CREATE_EXTERNAL_USER =
+ new API(URI_CREATE_EXTERNAL_USER, HttpMethod.POST, Response.Status.OK);
+ private static final API DELETE_USER =
+ new API(URI_USER_BY_ID, HttpMethod.DELETE, Response.Status.NO_CONTENT);
+
+ // Ranger group APIs
+ private static final API CREATE_GROUP =
+ new API(URI_GROUP_BASE, HttpMethod.POST, Response.Status.OK);
+ private static final API SEARCH_GROUP =
+ new API(URI_GROUP_BASE, HttpMethod.GET, Response.Status.OK);
+ // private static final API GET_GROUP = new API(URI_GROUP_BY_ID,
HttpMethod.GET,
+ // Response.Status.OK);
+ private static final API DELETE_GROUP =
+ new API(URI_GROUP_BY_ID, HttpMethod.DELETE, Response.Status.NO_CONTENT);
+
+ // apache/ranger/intg/src/main/java/org/apache/ranger/RangerClient.java
+ // The private method callAPI of Ranger is called by reflection
+ // private T callAPI(API api, Map params, Object
request, GenericType
+ // responseType) throws RangerServiceException
+ private Method callAPIMethodGenericResponseType;
+
+ // private T callAPI(API api, Map params, Object
request, Class
+ // responseType) throws RangerServiceException
+ private Method callAPIMethodClassResponseType;
+ // private void callAPI(API api, Map params) throws
RangerServiceException
+ private Method callAPIMethodNonResponse;
+
+ public RangerClientExtend(String hostName, String authType, String username,
String password) {
+super(hostName, authType, username, password, null);
+
+// initialize callAPI method
+try {
+ callAPIMethodGenericResponseType =
+ RangerClient.class.getDeclaredMethod(
+ "callAPI", API.class, Map.class, Object.class,
GenericType.class);
+ callAPIMethodGenericResponseType.setAccessible(true);
+
+ callAPIMethodNonResponse =
+ RangerClient.class.getDeclaredMethod("callAPI", API.class,
Map.class);
+ callAPIMethodNonResponse.setAccessible(true);
+
+ callAPIMethodClassResponseType =
+ RangerClient.class.getDeclaredMethod(
+ "callAPI", API.class, Map.class, Object.class, Class.class);
+ callAPIMethodClassResponseType.setAccessible(true);
+} catch (NoSuchMethodException e) {
+ throw new RuntimeException
Re: [PR] [#3963] feat(core): Apache Ranger Hive authorization pushdown [gravitino]
xunliu commented on PR #4515: URL: https://github.com/apache/gravitino/pull/4515#issuecomment-2301488589 > Does it support multiple Hive catalog uses one Ranger? Yes, we support. Because each Hive catalog have a separate AuthorizationPlugin instance. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
Re: [PR] [#3963] feat(core): Apache Ranger Hive authorization pushdown [gravitino]
xunliu commented on PR #4515:
URL: https://github.com/apache/gravitino/pull/4515#issuecomment-2301431331
@yuqi1129
> Policies can't be null, please see
>
> // Only return the policies that are managed by Gravitino.
> if (policies.size() > 1) {
> throw new AuthorizationPluginException(
> "Each metadata object only have one Gravitino management
enable policies.");
> }
But policies cloud be equal to 1 or 0.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
Re: [PR] [#3963] feat(core): Apache Ranger Hive authorization pushdown [gravitino]
jerqi commented on code in PR #4515:
URL: https://github.com/apache/gravitino/pull/4515#discussion_r1724574374
##
authorizations/authorization-ranger/src/main/java/org/apache/gravitino/authorization/ranger/RangerAuthorizationPlugin.java:
##
@@ -0,0 +1,1006 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+package org.apache.gravitino.authorization.ranger;
+
+import com.google.common.annotations.VisibleForTesting;
+import com.google.common.collect.ImmutableMap;
+import com.google.common.collect.Lists;
+import com.google.errorprone.annotations.FormatMethod;
+import com.google.errorprone.annotations.FormatString;
+import java.io.IOException;
+import java.time.Instant;
+import java.util.Arrays;
+import java.util.Collections;
+import java.util.HashMap;
+import java.util.HashSet;
+import java.util.List;
+import java.util.Map;
+import java.util.Objects;
+import java.util.Set;
+import java.util.stream.Collectors;
+import org.apache.gravitino.MetadataObject;
+import org.apache.gravitino.authorization.Group;
+import org.apache.gravitino.authorization.Owner;
+import org.apache.gravitino.authorization.Privilege;
+import org.apache.gravitino.authorization.Role;
+import org.apache.gravitino.authorization.RoleChange;
+import org.apache.gravitino.authorization.SecurableObject;
+import org.apache.gravitino.authorization.SecurableObjects;
+import org.apache.gravitino.authorization.User;
+import org.apache.gravitino.authorization.ranger.defines.VXGroup;
+import org.apache.gravitino.authorization.ranger.defines.VXGroupList;
+import org.apache.gravitino.authorization.ranger.defines.VXUser;
+import org.apache.gravitino.authorization.ranger.defines.VXUserList;
+import org.apache.gravitino.connector.AuthorizationPropertiesMeta;
+import org.apache.gravitino.connector.authorization.AuthorizationPlugin;
+import org.apache.gravitino.exceptions.AuthorizationPluginException;
+import org.apache.gravitino.meta.AuditInfo;
+import org.apache.gravitino.meta.GroupEntity;
+import org.apache.gravitino.meta.UserEntity;
+import org.apache.gravitino.utils.PrincipalUtils;
+import org.apache.ranger.RangerServiceException;
+import org.apache.ranger.plugin.model.RangerPolicy;
+import org.apache.ranger.plugin.model.RangerRole;
+import org.apache.ranger.plugin.util.GrantRevokeRoleRequest;
+import org.apache.ranger.plugin.util.SearchFilter;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+/** Ranger authorization operations plugin abstract class. */
+public abstract class RangerAuthorizationPlugin implements AuthorizationPlugin
{
+ private static final Logger LOG =
LoggerFactory.getLogger(RangerAuthorizationPlugin.class);
+
+ protected String catalogProvider;
+ protected RangerClientExtend rangerClient;
+ protected String rangerServiceName;
+ /** Mapping Gravitino privilege name to the underlying authorization system
privileges. */
+ protected Map> mapPrivileges = null;
+ // The owner privileges, the owner can do anything on the metadata object
+ protected Set ownerPrivileges = null;
+
+ /**
+ * Because Ranger doesn't support the precise filter, Ranger will return the
policy meets the
+ * wildcard(*,?) conditions, just like `*.*.*` policy will match
`db1.table1.column1` So we need
+ * to manual precise filter the policies.
+ */
+ // Search Ranger policy filter keys
+ protected List policyFilterKeys = null;
Review Comment:
Should we give the policyFilterKeys a better name? Maybe
policyPrefixFilterKeys maybe better.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
Re: [PR] [#3963] feat(core): Apache Ranger Hive authorization pushdown [gravitino]
jerqi commented on code in PR #4515:
URL: https://github.com/apache/gravitino/pull/4515#discussion_r1724574374
##
authorizations/authorization-ranger/src/main/java/org/apache/gravitino/authorization/ranger/RangerAuthorizationPlugin.java:
##
@@ -0,0 +1,1006 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+package org.apache.gravitino.authorization.ranger;
+
+import com.google.common.annotations.VisibleForTesting;
+import com.google.common.collect.ImmutableMap;
+import com.google.common.collect.Lists;
+import com.google.errorprone.annotations.FormatMethod;
+import com.google.errorprone.annotations.FormatString;
+import java.io.IOException;
+import java.time.Instant;
+import java.util.Arrays;
+import java.util.Collections;
+import java.util.HashMap;
+import java.util.HashSet;
+import java.util.List;
+import java.util.Map;
+import java.util.Objects;
+import java.util.Set;
+import java.util.stream.Collectors;
+import org.apache.gravitino.MetadataObject;
+import org.apache.gravitino.authorization.Group;
+import org.apache.gravitino.authorization.Owner;
+import org.apache.gravitino.authorization.Privilege;
+import org.apache.gravitino.authorization.Role;
+import org.apache.gravitino.authorization.RoleChange;
+import org.apache.gravitino.authorization.SecurableObject;
+import org.apache.gravitino.authorization.SecurableObjects;
+import org.apache.gravitino.authorization.User;
+import org.apache.gravitino.authorization.ranger.defines.VXGroup;
+import org.apache.gravitino.authorization.ranger.defines.VXGroupList;
+import org.apache.gravitino.authorization.ranger.defines.VXUser;
+import org.apache.gravitino.authorization.ranger.defines.VXUserList;
+import org.apache.gravitino.connector.AuthorizationPropertiesMeta;
+import org.apache.gravitino.connector.authorization.AuthorizationPlugin;
+import org.apache.gravitino.exceptions.AuthorizationPluginException;
+import org.apache.gravitino.meta.AuditInfo;
+import org.apache.gravitino.meta.GroupEntity;
+import org.apache.gravitino.meta.UserEntity;
+import org.apache.gravitino.utils.PrincipalUtils;
+import org.apache.ranger.RangerServiceException;
+import org.apache.ranger.plugin.model.RangerPolicy;
+import org.apache.ranger.plugin.model.RangerRole;
+import org.apache.ranger.plugin.util.GrantRevokeRoleRequest;
+import org.apache.ranger.plugin.util.SearchFilter;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+/** Ranger authorization operations plugin abstract class. */
+public abstract class RangerAuthorizationPlugin implements AuthorizationPlugin
{
+ private static final Logger LOG =
LoggerFactory.getLogger(RangerAuthorizationPlugin.class);
+
+ protected String catalogProvider;
+ protected RangerClientExtend rangerClient;
+ protected String rangerServiceName;
+ /** Mapping Gravitino privilege name to the underlying authorization system
privileges. */
+ protected Map> mapPrivileges = null;
+ // The owner privileges, the owner can do anything on the metadata object
+ protected Set ownerPrivileges = null;
+
+ /**
+ * Because Ranger doesn't support the precise filter, Ranger will return the
policy meets the
+ * wildcard(*,?) conditions, just like `*.*.*` policy will match
`db1.table1.column1` So we need
+ * to manual precise filter the policies.
+ */
+ // Search Ranger policy filter keys
+ protected List policyFilterKeys = null;
Review Comment:
Should we give the policyFilterKeys a better name?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
Re: [PR] [#3963] feat(core): Apache Ranger Hive authorization pushdown [gravitino]
jerqi commented on code in PR #4515: URL: https://github.com/apache/gravitino/pull/4515#discussion_r1724555745 ## authorizations/authorization-ranger/src/main/resources/META-INF/services/org.apache.gravitino.connector.authorization.AuthorizationProvider: ## @@ -0,0 +1,19 @@ +# +# Licensed to the Apache Software Foundation (ASF) under one +# or more contributor license agreements. See the NOTICE file +# distributed with this work for additional information +# regarding copyright ownership. The ASF licenses this file +# to you under the Apache License, Version 2.0 (the +# "License"); you may not use this file except in compliance +# with the License. You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, +# software distributed under the License is distributed on an +# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY +# KIND, either express or implied. See the License for the +# specific language governing permissions and limitations +# under the License. +# +org.apache.gravitino.authorization.ranger.RangerAuthorization Review Comment: RangerAuthorization is abstract class. Is it ok to use it here? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
Re: [PR] [#3963] feat(core): Apache Ranger Hive authorization pushdown [gravitino]
jerqi commented on code in PR #4515:
URL: https://github.com/apache/gravitino/pull/4515#discussion_r1724551608
##
authorizations/authorization-ranger/src/main/java/org/apache/gravitino/authorization/ranger/RangerClientExtend.java:
##
@@ -0,0 +1,184 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+package org.apache.gravitino.authorization.ranger;
+
+import com.google.common.collect.ImmutableMap;
+import com.sun.jersey.api.client.GenericType;
+import com.sun.jersey.api.client.UniformInterfaceException;
+import java.lang.reflect.InvocationTargetException;
+import java.lang.reflect.Method;
+import java.util.Map;
+import javax.ws.rs.HttpMethod;
+import javax.ws.rs.core.Response;
+import org.apache.gravitino.authorization.ranger.defines.VXGroup;
+import org.apache.gravitino.authorization.ranger.defines.VXGroupList;
+import org.apache.gravitino.authorization.ranger.defines.VXUser;
+import org.apache.gravitino.authorization.ranger.defines.VXUserList;
+import org.apache.ranger.RangerClient;
+import org.apache.ranger.RangerServiceException;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+/**
+ * Apache Ranger client extension
+ * The class extends the RangerClient class and provides additional methods to
create, search and
+ * delete users and groups
+ */
+public class RangerClientExtend extends RangerClient {
+ private static final Logger LOG =
LoggerFactory.getLogger(RangerClientExtend.class);
+ private static final String URI_USER_BASE = "/service/xusers/users";
+ private static final String URI_USER_BY_ID = URI_USER_BASE + "/%d";
+ private static final String URI_GROUP_BASE = "/service/xusers/groups";
+ private static final String URI_GROUP_BY_ID = URI_GROUP_BASE + "/%d";
+ private static final String URI_CREATE_EXTERNAL_USER = URI_USER_BASE +
"/external";
+
+ // Ranger user APIs
+ private static final API SEARCH_USER = new API(URI_USER_BASE,
HttpMethod.GET, Response.Status.OK);
+ private static final API CREATE_EXTERNAL_USER =
+ new API(URI_CREATE_EXTERNAL_USER, HttpMethod.POST, Response.Status.OK);
+ private static final API DELETE_USER =
+ new API(URI_USER_BY_ID, HttpMethod.DELETE, Response.Status.NO_CONTENT);
+
+ // Ranger group APIs
+ private static final API CREATE_GROUP =
+ new API(URI_GROUP_BASE, HttpMethod.POST, Response.Status.OK);
+ private static final API SEARCH_GROUP =
+ new API(URI_GROUP_BASE, HttpMethod.GET, Response.Status.OK);
+ // private static final API GET_GROUP = new API(URI_GROUP_BY_ID,
HttpMethod.GET,
+ // Response.Status.OK);
+ private static final API DELETE_GROUP =
+ new API(URI_GROUP_BY_ID, HttpMethod.DELETE, Response.Status.NO_CONTENT);
+
+ // apache/ranger/intg/src/main/java/org/apache/ranger/RangerClient.java
+ // The private method callAPI of Ranger is called by reflection
+ // private T callAPI(API api, Map params, Object
request, GenericType
+ // responseType) throws RangerServiceException
+ private Method callAPIMethodGenericResponseType;
+
+ // private T callAPI(API api, Map params, Object
request, Class
+ // responseType) throws RangerServiceException
+ private Method callAPIMethodClassResponseType;
+ // private void callAPI(API api, Map params) throws
RangerServiceException
+ private Method callAPIMethodNonResponse;
+
+ public RangerClientExtend(String hostName, String authType, String username,
String password) {
+super(hostName, authType, username, password, null);
+
+// initialize callAPI method
+try {
+ callAPIMethodGenericResponseType =
+ RangerClient.class.getDeclaredMethod(
+ "callAPI", API.class, Map.class, Object.class,
GenericType.class);
+ callAPIMethodGenericResponseType.setAccessible(true);
+
+ callAPIMethodNonResponse =
+ RangerClient.class.getDeclaredMethod("callAPI", API.class,
Map.class);
+ callAPIMethodNonResponse.setAccessible(true);
+
+ callAPIMethodClassResponseType =
+ RangerClient.class.getDeclaredMethod(
+ "callAPI", API.class, Map.class, Object.class, Class.class);
+ callAPIMethodClassResponseType.setAccessible(true);
+} catch (NoSuchMethodException e) {
+ throw new RuntimeException
Re: [PR] [#3963] feat(core): Apache Ranger Hive authorization pushdown [gravitino]
jerqi commented on code in PR #4515:
URL: https://github.com/apache/gravitino/pull/4515#discussion_r1724546784
##
authorizations/authorization-ranger/src/main/java/org/apache/gravitino/authorization/ranger/RangerAuthorizationPlugin.java:
##
@@ -0,0 +1,998 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+package org.apache.gravitino.authorization.ranger;
+
+import com.google.common.annotations.VisibleForTesting;
+import com.google.common.collect.ImmutableMap;
+import com.google.common.collect.Lists;
+import com.google.common.collect.Sets;
+import com.google.errorprone.annotations.FormatMethod;
+import com.google.errorprone.annotations.FormatString;
+import java.io.IOException;
+import java.time.Instant;
+import java.util.Collections;
+import java.util.HashMap;
+import java.util.List;
+import java.util.Map;
+import java.util.Objects;
+import java.util.Set;
+import java.util.stream.Collectors;
+import org.apache.commons.lang3.StringUtils;
+import org.apache.gravitino.MetadataObject;
+import org.apache.gravitino.authorization.Group;
+import org.apache.gravitino.authorization.Owner;
+import org.apache.gravitino.authorization.Privilege;
+import org.apache.gravitino.authorization.Role;
+import org.apache.gravitino.authorization.RoleChange;
+import org.apache.gravitino.authorization.SecurableObject;
+import org.apache.gravitino.authorization.SecurableObjects;
+import org.apache.gravitino.authorization.User;
+import org.apache.gravitino.authorization.ranger.defines.VXGroup;
+import org.apache.gravitino.authorization.ranger.defines.VXGroupList;
+import org.apache.gravitino.authorization.ranger.defines.VXUser;
+import org.apache.gravitino.authorization.ranger.defines.VXUserList;
+import org.apache.gravitino.connector.AuthorizationPropertiesMeta;
+import org.apache.gravitino.connector.authorization.AuthorizationPlugin;
+import org.apache.gravitino.exceptions.AuthorizationPluginException;
+import org.apache.gravitino.meta.AuditInfo;
+import org.apache.gravitino.meta.GroupEntity;
+import org.apache.gravitino.meta.UserEntity;
+import org.apache.gravitino.utils.PrincipalUtils;
+import org.apache.ranger.RangerServiceException;
+import org.apache.ranger.plugin.model.RangerPolicy;
+import org.apache.ranger.plugin.model.RangerPolicy.RangerPolicyItem;
+import org.apache.ranger.plugin.model.RangerRole;
+import org.apache.ranger.plugin.util.GrantRevokeRoleRequest;
+import org.apache.ranger.plugin.util.SearchFilter;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+/** Ranger authorization operations plugin abstract class. */
+public abstract class RangerAuthorizationPlugin implements AuthorizationPlugin
{
+ private static final Logger LOG =
LoggerFactory.getLogger(RangerAuthorizationPlugin.class);
+
+ protected String catalogProvider;
+ protected RangerClientExtend rangerClient;
+ protected String rangerServiceName;
+ /** Mapping Gravitino privilege name to the underlying authorization system
privileges. */
+ protected Map> privilegesMapping = null;
+ // The owner privileges, the owner can do anything on the metadata object
+ protected Set ownerPrivileges = null;
+
+ /**
+ * Because Ranger doesn't support the precise filter, Ranger will return the
policy meets the
+ * wildcard(*,?) conditions, just like `*.*.*` policy will match
`db1.table1.column1` So we need
+ * to manually precisely filter the policies.
+ */
+ // Search Ranger policy filter keys
+ protected List policyFilterKeys = null;
+ // Search Ranger policy precise filter keys
+ protected List policyPreciseFilterKeys = null;
+
+ public static final String MANAGED_BY_GRAVITINO = "MANAGED_BY_GRAVITINO";
+
+ // TODO: Maybe need to move to the configuration in the future
+ public static final String RANGER_ADMIN_NAME = "admin";
+
+ public RangerAuthorizationPlugin(String catalogProvider, Map
config) {
+super();
+this.catalogProvider = catalogProvider;
+String rangerUrl =
config.get(AuthorizationPropertiesMeta.RANGER_ADMIN_URL);
+String authType = config.get(AuthorizationPropertiesMeta.RANGER_AUTH_TYPE);
+String username = config.get(AuthorizationPropertiesMeta.RANGER_USERNAME);
+// Apache Ranger Password should be minimum 8
Re: [PR] [#3963] feat(core): Apache Ranger Hive authorization pushdown [gravitino]
jerqi commented on code in PR #4515:
URL: https://github.com/apache/gravitino/pull/4515#discussion_r1724536521
##
authorizations/authorization-ranger/src/main/java/org/apache/gravitino/authorization/ranger/RangerAuthorizationPlugin.java:
##
@@ -0,0 +1,998 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+package org.apache.gravitino.authorization.ranger;
+
+import com.google.common.annotations.VisibleForTesting;
+import com.google.common.collect.ImmutableMap;
+import com.google.common.collect.Lists;
+import com.google.common.collect.Sets;
+import com.google.errorprone.annotations.FormatMethod;
+import com.google.errorprone.annotations.FormatString;
+import java.io.IOException;
+import java.time.Instant;
+import java.util.Collections;
+import java.util.HashMap;
+import java.util.List;
+import java.util.Map;
+import java.util.Objects;
+import java.util.Set;
+import java.util.stream.Collectors;
+import org.apache.commons.lang3.StringUtils;
+import org.apache.gravitino.MetadataObject;
+import org.apache.gravitino.authorization.Group;
+import org.apache.gravitino.authorization.Owner;
+import org.apache.gravitino.authorization.Privilege;
+import org.apache.gravitino.authorization.Role;
+import org.apache.gravitino.authorization.RoleChange;
+import org.apache.gravitino.authorization.SecurableObject;
+import org.apache.gravitino.authorization.SecurableObjects;
+import org.apache.gravitino.authorization.User;
+import org.apache.gravitino.authorization.ranger.defines.VXGroup;
+import org.apache.gravitino.authorization.ranger.defines.VXGroupList;
+import org.apache.gravitino.authorization.ranger.defines.VXUser;
+import org.apache.gravitino.authorization.ranger.defines.VXUserList;
+import org.apache.gravitino.connector.AuthorizationPropertiesMeta;
+import org.apache.gravitino.connector.authorization.AuthorizationPlugin;
+import org.apache.gravitino.exceptions.AuthorizationPluginException;
+import org.apache.gravitino.meta.AuditInfo;
+import org.apache.gravitino.meta.GroupEntity;
+import org.apache.gravitino.meta.UserEntity;
+import org.apache.gravitino.utils.PrincipalUtils;
+import org.apache.ranger.RangerServiceException;
+import org.apache.ranger.plugin.model.RangerPolicy;
+import org.apache.ranger.plugin.model.RangerPolicy.RangerPolicyItem;
+import org.apache.ranger.plugin.model.RangerRole;
+import org.apache.ranger.plugin.util.GrantRevokeRoleRequest;
+import org.apache.ranger.plugin.util.SearchFilter;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+/** Ranger authorization operations plugin abstract class. */
+public abstract class RangerAuthorizationPlugin implements AuthorizationPlugin
{
+ private static final Logger LOG =
LoggerFactory.getLogger(RangerAuthorizationPlugin.class);
+
+ protected String catalogProvider;
+ protected RangerClientExtend rangerClient;
+ protected String rangerServiceName;
+ /** Mapping Gravitino privilege name to the underlying authorization system
privileges. */
+ protected Map> privilegesMapping = null;
+ // The owner privileges, the owner can do anything on the metadata object
+ protected Set ownerPrivileges = null;
+
+ /**
+ * Because Ranger doesn't support the precise filter, Ranger will return the
policy meets the
+ * wildcard(*,?) conditions, just like `*.*.*` policy will match
`db1.table1.column1` So we need
+ * to manually precisely filter the policies.
+ */
+ // Search Ranger policy filter keys
+ protected List policyFilterKeys = null;
+ // Search Ranger policy precise filter keys
+ protected List policyPreciseFilterKeys = null;
+
+ public static final String MANAGED_BY_GRAVITINO = "MANAGED_BY_GRAVITINO";
+
+ // TODO: Maybe need to move to the configuration in the future
+ public static final String RANGER_ADMIN_NAME = "admin";
+
+ public RangerAuthorizationPlugin(String catalogProvider, Map
config) {
+super();
+this.catalogProvider = catalogProvider;
+String rangerUrl =
config.get(AuthorizationPropertiesMeta.RANGER_ADMIN_URL);
+String authType = config.get(AuthorizationPropertiesMeta.RANGER_AUTH_TYPE);
+String username = config.get(AuthorizationPropertiesMeta.RANGER_USERNAME);
+// Apache Ranger Password should be minimum 8
Re: [PR] [#3963] feat(core): Apache Ranger Hive authorization pushdown [gravitino]
jerqi commented on code in PR #4515:
URL: https://github.com/apache/gravitino/pull/4515#discussion_r1724534814
##
authorizations/authorization-ranger/src/main/java/org/apache/gravitino/authorization/ranger/RangerAuthorizationPlugin.java:
##
@@ -0,0 +1,998 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+package org.apache.gravitino.authorization.ranger;
+
+import com.google.common.annotations.VisibleForTesting;
+import com.google.common.collect.ImmutableMap;
+import com.google.common.collect.Lists;
+import com.google.common.collect.Sets;
+import com.google.errorprone.annotations.FormatMethod;
+import com.google.errorprone.annotations.FormatString;
+import java.io.IOException;
+import java.time.Instant;
+import java.util.Collections;
+import java.util.HashMap;
+import java.util.List;
+import java.util.Map;
+import java.util.Objects;
+import java.util.Set;
+import java.util.stream.Collectors;
+import org.apache.commons.lang3.StringUtils;
+import org.apache.gravitino.MetadataObject;
+import org.apache.gravitino.authorization.Group;
+import org.apache.gravitino.authorization.Owner;
+import org.apache.gravitino.authorization.Privilege;
+import org.apache.gravitino.authorization.Role;
+import org.apache.gravitino.authorization.RoleChange;
+import org.apache.gravitino.authorization.SecurableObject;
+import org.apache.gravitino.authorization.SecurableObjects;
+import org.apache.gravitino.authorization.User;
+import org.apache.gravitino.authorization.ranger.defines.VXGroup;
+import org.apache.gravitino.authorization.ranger.defines.VXGroupList;
+import org.apache.gravitino.authorization.ranger.defines.VXUser;
+import org.apache.gravitino.authorization.ranger.defines.VXUserList;
+import org.apache.gravitino.connector.AuthorizationPropertiesMeta;
+import org.apache.gravitino.connector.authorization.AuthorizationPlugin;
+import org.apache.gravitino.exceptions.AuthorizationPluginException;
+import org.apache.gravitino.meta.AuditInfo;
+import org.apache.gravitino.meta.GroupEntity;
+import org.apache.gravitino.meta.UserEntity;
+import org.apache.gravitino.utils.PrincipalUtils;
+import org.apache.ranger.RangerServiceException;
+import org.apache.ranger.plugin.model.RangerPolicy;
+import org.apache.ranger.plugin.model.RangerPolicy.RangerPolicyItem;
+import org.apache.ranger.plugin.model.RangerRole;
+import org.apache.ranger.plugin.util.GrantRevokeRoleRequest;
+import org.apache.ranger.plugin.util.SearchFilter;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+/** Ranger authorization operations plugin abstract class. */
+public abstract class RangerAuthorizationPlugin implements AuthorizationPlugin
{
+ private static final Logger LOG =
LoggerFactory.getLogger(RangerAuthorizationPlugin.class);
+
+ protected String catalogProvider;
+ protected RangerClientExtend rangerClient;
+ protected String rangerServiceName;
+ /** Mapping Gravitino privilege name to the underlying authorization system
privileges. */
+ protected Map> privilegesMapping = null;
+ // The owner privileges, the owner can do anything on the metadata object
+ protected Set ownerPrivileges = null;
+
+ /**
+ * Because Ranger doesn't support the precise filter, Ranger will return the
policy meets the
+ * wildcard(*,?) conditions, just like `*.*.*` policy will match
`db1.table1.column1` So we need
+ * to manually precisely filter the policies.
+ */
+ // Search Ranger policy filter keys
+ protected List policyFilterKeys = null;
+ // Search Ranger policy precise filter keys
+ protected List policyPreciseFilterKeys = null;
+
+ public static final String MANAGED_BY_GRAVITINO = "MANAGED_BY_GRAVITINO";
+
+ // TODO: Maybe need to move to the configuration in the future
+ public static final String RANGER_ADMIN_NAME = "admin";
+
+ public RangerAuthorizationPlugin(String catalogProvider, Map
config) {
+super();
+this.catalogProvider = catalogProvider;
+String rangerUrl =
config.get(AuthorizationPropertiesMeta.RANGER_ADMIN_URL);
+String authType = config.get(AuthorizationPropertiesMeta.RANGER_AUTH_TYPE);
+String username = config.get(AuthorizationPropertiesMeta.RANGER_USERNAME);
+// Apache Ranger Password should be minimum 8
Re: [PR] [#3963] feat(core): Apache Ranger Hive authorization pushdown [gravitino]
jerqi commented on code in PR #4515:
URL: https://github.com/apache/gravitino/pull/4515#discussion_r1724530797
##
authorizations/authorization-ranger/src/main/java/org/apache/gravitino/authorization/ranger/RangerAuthorizationPlugin.java:
##
@@ -0,0 +1,1027 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+package org.apache.gravitino.authorization.ranger;
+
+import com.google.common.annotations.VisibleForTesting;
+import com.google.common.collect.ImmutableMap;
+import com.google.common.collect.Lists;
+import com.google.errorprone.annotations.FormatMethod;
+import com.google.errorprone.annotations.FormatString;
+import java.io.IOException;
+import java.util.Arrays;
+import java.util.Collections;
+import java.util.HashMap;
+import java.util.HashSet;
+import java.util.List;
+import java.util.Map;
+import java.util.Objects;
+import java.util.Set;
+import java.util.concurrent.atomic.AtomicReference;
+import java.util.stream.Collectors;
+import org.apache.gravitino.MetadataObject;
+import org.apache.gravitino.authorization.Group;
+import org.apache.gravitino.authorization.Owner;
+import org.apache.gravitino.authorization.Privilege;
+import org.apache.gravitino.authorization.Role;
+import org.apache.gravitino.authorization.RoleChange;
+import org.apache.gravitino.authorization.SecurableObject;
+import org.apache.gravitino.authorization.SecurableObjects;
+import org.apache.gravitino.authorization.User;
+import org.apache.gravitino.authorization.ranger.defines.VXGroup;
+import org.apache.gravitino.authorization.ranger.defines.VXGroupList;
+import org.apache.gravitino.authorization.ranger.defines.VXUser;
+import org.apache.gravitino.authorization.ranger.defines.VXUserList;
+import org.apache.gravitino.connector.AuthorizationPropertiesMeta;
+import org.apache.gravitino.connector.authorization.AuthorizationPlugin;
+import org.apache.gravitino.exceptions.AuthorizationPluginException;
+import org.apache.ranger.RangerServiceException;
+import org.apache.ranger.plugin.model.RangerPolicy;
+import org.apache.ranger.plugin.model.RangerRole;
+import org.apache.ranger.plugin.util.GrantRevokeRoleRequest;
+import org.apache.ranger.plugin.util.SearchFilter;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+/** Ranger authorization operations plugin abstract class. */
+public abstract class RangerAuthorizationPlugin implements AuthorizationPlugin
{
+ private static final Logger LOG =
LoggerFactory.getLogger(RangerAuthorizationPlugin.class);
+
+ protected String catalogProvider;
+ protected RangerClientExt rangerClient;
+ protected String rangerServiceName;
+ /** Mapping Gravitino privilege name to the underlying authorization system
privileges. */
+ protected Map> mapPrivileges = null;
+ // The owner privileges, the owner can do anything on the metadata object
+ protected Set ownerPrivileges = null;
+
+ /**
+ * Because Ranger doesn't support the precise filter, Ranger will return the
policy meets the
+ * wildcard(*,?) conditions, just like `*.*.*` policy will match
`db1.table1.column1` So we need
+ * to manual precise filter the policies.
+ */
+ // Search Ranger policy filter keys
+ protected List policyFilterKeys = null;
+ // Search Ranger policy precise filter keys
+ protected List policyPreciseFilterKeys = null;
+
+ public static final String MANAGED_BY_GRAVITINO = "MANAGED_BY_GRAVITINO";
+
+ // TODO: Maybe need to move to the configuration in the future
+ public static final String RANGER_ADMIN_NAME = "admin";
+
+ public RangerAuthorizationPlugin(String catalogProvider, Map
config) {
+super();
+this.catalogProvider = catalogProvider;
+String rangerUrl =
config.get(AuthorizationPropertiesMeta.RANGER_ADMIN_URL);
+String authType = config.get(AuthorizationPropertiesMeta.RANGER_AUTH_TYPE);
+String username = config.get(AuthorizationPropertiesMeta.RANGER_USERNAME);
+// Apache Ranger Password should be minimum 8 characters with min one
alphabet and one numeric.
+String password = config.get(AuthorizationPropertiesMeta.RANGER_PASSWORD);
+rangerServiceName =
config.get(AuthorizationPropertiesMeta.RANGER_SERVICE_NAME);
+check(rangerUrl != null, "Ranger admin URL is requir
Re: [PR] [#3963] feat(core): Apache Ranger Hive authorization pushdown [gravitino]
jerqi commented on code in PR #4515:
URL: https://github.com/apache/gravitino/pull/4515#discussion_r1724525054
##
authorizations/authorization-ranger/src/main/java/org/apache/gravitino/authorization/ranger/RangerAuthorizationPlugin.java:
##
@@ -0,0 +1,998 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+package org.apache.gravitino.authorization.ranger;
+
+import com.google.common.annotations.VisibleForTesting;
+import com.google.common.collect.ImmutableMap;
+import com.google.common.collect.Lists;
+import com.google.common.collect.Sets;
+import com.google.errorprone.annotations.FormatMethod;
+import com.google.errorprone.annotations.FormatString;
+import java.io.IOException;
+import java.time.Instant;
+import java.util.Collections;
+import java.util.HashMap;
+import java.util.List;
+import java.util.Map;
+import java.util.Objects;
+import java.util.Set;
+import java.util.stream.Collectors;
+import org.apache.commons.lang3.StringUtils;
+import org.apache.gravitino.MetadataObject;
+import org.apache.gravitino.authorization.Group;
+import org.apache.gravitino.authorization.Owner;
+import org.apache.gravitino.authorization.Privilege;
+import org.apache.gravitino.authorization.Role;
+import org.apache.gravitino.authorization.RoleChange;
+import org.apache.gravitino.authorization.SecurableObject;
+import org.apache.gravitino.authorization.SecurableObjects;
+import org.apache.gravitino.authorization.User;
+import org.apache.gravitino.authorization.ranger.defines.VXGroup;
+import org.apache.gravitino.authorization.ranger.defines.VXGroupList;
+import org.apache.gravitino.authorization.ranger.defines.VXUser;
+import org.apache.gravitino.authorization.ranger.defines.VXUserList;
+import org.apache.gravitino.connector.AuthorizationPropertiesMeta;
+import org.apache.gravitino.connector.authorization.AuthorizationPlugin;
+import org.apache.gravitino.exceptions.AuthorizationPluginException;
+import org.apache.gravitino.meta.AuditInfo;
+import org.apache.gravitino.meta.GroupEntity;
+import org.apache.gravitino.meta.UserEntity;
+import org.apache.gravitino.utils.PrincipalUtils;
+import org.apache.ranger.RangerServiceException;
+import org.apache.ranger.plugin.model.RangerPolicy;
+import org.apache.ranger.plugin.model.RangerPolicy.RangerPolicyItem;
+import org.apache.ranger.plugin.model.RangerRole;
+import org.apache.ranger.plugin.util.GrantRevokeRoleRequest;
+import org.apache.ranger.plugin.util.SearchFilter;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+/** Ranger authorization operations plugin abstract class. */
+public abstract class RangerAuthorizationPlugin implements AuthorizationPlugin
{
+ private static final Logger LOG =
LoggerFactory.getLogger(RangerAuthorizationPlugin.class);
+
+ protected String catalogProvider;
+ protected RangerClientExtend rangerClient;
+ protected String rangerServiceName;
+ /** Mapping Gravitino privilege name to the underlying authorization system
privileges. */
+ protected Map> privilegesMapping = null;
+ // The owner privileges, the owner can do anything on the metadata object
+ protected Set ownerPrivileges = null;
+
+ /**
+ * Because Ranger doesn't support the precise filter, Ranger will return the
policy meets the
+ * wildcard(*,?) conditions, just like `*.*.*` policy will match
`db1.table1.column1` So we need
+ * to manually precisely filter the policies.
+ */
+ // Search Ranger policy filter keys
+ protected List policyFilterKeys = null;
+ // Search Ranger policy precise filter keys
+ protected List policyPreciseFilterKeys = null;
+
+ public static final String MANAGED_BY_GRAVITINO = "MANAGED_BY_GRAVITINO";
+
+ // TODO: Maybe need to move to the configuration in the future
+ public static final String RANGER_ADMIN_NAME = "admin";
+
+ public RangerAuthorizationPlugin(String catalogProvider, Map
config) {
+super();
+this.catalogProvider = catalogProvider;
+String rangerUrl =
config.get(AuthorizationPropertiesMeta.RANGER_ADMIN_URL);
+String authType = config.get(AuthorizationPropertiesMeta.RANGER_AUTH_TYPE);
+String username = config.get(AuthorizationPropertiesMeta.RANGER_USERNAME);
+// Apache Ranger Password should be minimum 8
Re: [PR] [#3963] feat(core): Apache Ranger Hive authorization pushdown [gravitino]
jerqi commented on code in PR #4515:
URL: https://github.com/apache/gravitino/pull/4515#discussion_r1724520020
##
authorizations/authorization-ranger/src/main/java/org/apache/gravitino/authorization/ranger/RangerAuthorizationPlugin.java:
##
@@ -0,0 +1,998 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+package org.apache.gravitino.authorization.ranger;
+
+import com.google.common.annotations.VisibleForTesting;
+import com.google.common.collect.ImmutableMap;
+import com.google.common.collect.Lists;
+import com.google.common.collect.Sets;
+import com.google.errorprone.annotations.FormatMethod;
+import com.google.errorprone.annotations.FormatString;
+import java.io.IOException;
+import java.time.Instant;
+import java.util.Collections;
+import java.util.HashMap;
+import java.util.List;
+import java.util.Map;
+import java.util.Objects;
+import java.util.Set;
+import java.util.stream.Collectors;
+import org.apache.commons.lang3.StringUtils;
+import org.apache.gravitino.MetadataObject;
+import org.apache.gravitino.authorization.Group;
+import org.apache.gravitino.authorization.Owner;
+import org.apache.gravitino.authorization.Privilege;
+import org.apache.gravitino.authorization.Role;
+import org.apache.gravitino.authorization.RoleChange;
+import org.apache.gravitino.authorization.SecurableObject;
+import org.apache.gravitino.authorization.SecurableObjects;
+import org.apache.gravitino.authorization.User;
+import org.apache.gravitino.authorization.ranger.defines.VXGroup;
+import org.apache.gravitino.authorization.ranger.defines.VXGroupList;
+import org.apache.gravitino.authorization.ranger.defines.VXUser;
+import org.apache.gravitino.authorization.ranger.defines.VXUserList;
+import org.apache.gravitino.connector.AuthorizationPropertiesMeta;
+import org.apache.gravitino.connector.authorization.AuthorizationPlugin;
+import org.apache.gravitino.exceptions.AuthorizationPluginException;
+import org.apache.gravitino.meta.AuditInfo;
+import org.apache.gravitino.meta.GroupEntity;
+import org.apache.gravitino.meta.UserEntity;
+import org.apache.gravitino.utils.PrincipalUtils;
+import org.apache.ranger.RangerServiceException;
+import org.apache.ranger.plugin.model.RangerPolicy;
+import org.apache.ranger.plugin.model.RangerPolicy.RangerPolicyItem;
+import org.apache.ranger.plugin.model.RangerRole;
+import org.apache.ranger.plugin.util.GrantRevokeRoleRequest;
+import org.apache.ranger.plugin.util.SearchFilter;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+/** Ranger authorization operations plugin abstract class. */
+public abstract class RangerAuthorizationPlugin implements AuthorizationPlugin
{
+ private static final Logger LOG =
LoggerFactory.getLogger(RangerAuthorizationPlugin.class);
+
+ protected String catalogProvider;
+ protected RangerClientExtend rangerClient;
+ protected String rangerServiceName;
+ /** Mapping Gravitino privilege name to the underlying authorization system
privileges. */
+ protected Map> privilegesMapping = null;
+ // The owner privileges, the owner can do anything on the metadata object
+ protected Set ownerPrivileges = null;
+
+ /**
+ * Because Ranger doesn't support the precise filter, Ranger will return the
policy meets the
+ * wildcard(*,?) conditions, just like `*.*.*` policy will match
`db1.table1.column1` So we need
+ * to manually precisely filter the policies.
+ */
+ // Search Ranger policy filter keys
+ protected List policyFilterKeys = null;
+ // Search Ranger policy precise filter keys
+ protected List policyPreciseFilterKeys = null;
+
+ public static final String MANAGED_BY_GRAVITINO = "MANAGED_BY_GRAVITINO";
+
+ // TODO: Maybe need to move to the configuration in the future
+ public static final String RANGER_ADMIN_NAME = "admin";
+
+ public RangerAuthorizationPlugin(String catalogProvider, Map
config) {
+super();
+this.catalogProvider = catalogProvider;
+String rangerUrl =
config.get(AuthorizationPropertiesMeta.RANGER_ADMIN_URL);
+String authType = config.get(AuthorizationPropertiesMeta.RANGER_AUTH_TYPE);
+String username = config.get(AuthorizationPropertiesMeta.RANGER_USERNAME);
+// Apache Ranger Password should be minimum 8
Re: [PR] [#3963] feat(core): Apache Ranger Hive authorization pushdown [gravitino]
jerqi commented on code in PR #4515:
URL: https://github.com/apache/gravitino/pull/4515#discussion_r1724520020
##
authorizations/authorization-ranger/src/main/java/org/apache/gravitino/authorization/ranger/RangerAuthorizationPlugin.java:
##
@@ -0,0 +1,998 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+package org.apache.gravitino.authorization.ranger;
+
+import com.google.common.annotations.VisibleForTesting;
+import com.google.common.collect.ImmutableMap;
+import com.google.common.collect.Lists;
+import com.google.common.collect.Sets;
+import com.google.errorprone.annotations.FormatMethod;
+import com.google.errorprone.annotations.FormatString;
+import java.io.IOException;
+import java.time.Instant;
+import java.util.Collections;
+import java.util.HashMap;
+import java.util.List;
+import java.util.Map;
+import java.util.Objects;
+import java.util.Set;
+import java.util.stream.Collectors;
+import org.apache.commons.lang3.StringUtils;
+import org.apache.gravitino.MetadataObject;
+import org.apache.gravitino.authorization.Group;
+import org.apache.gravitino.authorization.Owner;
+import org.apache.gravitino.authorization.Privilege;
+import org.apache.gravitino.authorization.Role;
+import org.apache.gravitino.authorization.RoleChange;
+import org.apache.gravitino.authorization.SecurableObject;
+import org.apache.gravitino.authorization.SecurableObjects;
+import org.apache.gravitino.authorization.User;
+import org.apache.gravitino.authorization.ranger.defines.VXGroup;
+import org.apache.gravitino.authorization.ranger.defines.VXGroupList;
+import org.apache.gravitino.authorization.ranger.defines.VXUser;
+import org.apache.gravitino.authorization.ranger.defines.VXUserList;
+import org.apache.gravitino.connector.AuthorizationPropertiesMeta;
+import org.apache.gravitino.connector.authorization.AuthorizationPlugin;
+import org.apache.gravitino.exceptions.AuthorizationPluginException;
+import org.apache.gravitino.meta.AuditInfo;
+import org.apache.gravitino.meta.GroupEntity;
+import org.apache.gravitino.meta.UserEntity;
+import org.apache.gravitino.utils.PrincipalUtils;
+import org.apache.ranger.RangerServiceException;
+import org.apache.ranger.plugin.model.RangerPolicy;
+import org.apache.ranger.plugin.model.RangerPolicy.RangerPolicyItem;
+import org.apache.ranger.plugin.model.RangerRole;
+import org.apache.ranger.plugin.util.GrantRevokeRoleRequest;
+import org.apache.ranger.plugin.util.SearchFilter;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+/** Ranger authorization operations plugin abstract class. */
+public abstract class RangerAuthorizationPlugin implements AuthorizationPlugin
{
+ private static final Logger LOG =
LoggerFactory.getLogger(RangerAuthorizationPlugin.class);
+
+ protected String catalogProvider;
+ protected RangerClientExtend rangerClient;
+ protected String rangerServiceName;
+ /** Mapping Gravitino privilege name to the underlying authorization system
privileges. */
+ protected Map> privilegesMapping = null;
+ // The owner privileges, the owner can do anything on the metadata object
+ protected Set ownerPrivileges = null;
+
+ /**
+ * Because Ranger doesn't support the precise filter, Ranger will return the
policy meets the
+ * wildcard(*,?) conditions, just like `*.*.*` policy will match
`db1.table1.column1` So we need
+ * to manually precisely filter the policies.
+ */
+ // Search Ranger policy filter keys
+ protected List policyFilterKeys = null;
+ // Search Ranger policy precise filter keys
+ protected List policyPreciseFilterKeys = null;
+
+ public static final String MANAGED_BY_GRAVITINO = "MANAGED_BY_GRAVITINO";
+
+ // TODO: Maybe need to move to the configuration in the future
+ public static final String RANGER_ADMIN_NAME = "admin";
+
+ public RangerAuthorizationPlugin(String catalogProvider, Map
config) {
+super();
+this.catalogProvider = catalogProvider;
+String rangerUrl =
config.get(AuthorizationPropertiesMeta.RANGER_ADMIN_URL);
+String authType = config.get(AuthorizationPropertiesMeta.RANGER_AUTH_TYPE);
+String username = config.get(AuthorizationPropertiesMeta.RANGER_USERNAME);
+// Apache Ranger Password should be minimum 8
Re: [PR] [#3963] feat(core): Apache Ranger Hive authorization pushdown [gravitino]
jerqi commented on code in PR #4515:
URL: https://github.com/apache/gravitino/pull/4515#discussion_r1724518162
##
authorizations/authorization-ranger/src/main/java/org/apache/gravitino/authorization/ranger/RangerAuthorizationPlugin.java:
##
@@ -0,0 +1,998 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+package org.apache.gravitino.authorization.ranger;
+
+import com.google.common.annotations.VisibleForTesting;
+import com.google.common.collect.ImmutableMap;
+import com.google.common.collect.Lists;
+import com.google.common.collect.Sets;
+import com.google.errorprone.annotations.FormatMethod;
+import com.google.errorprone.annotations.FormatString;
+import java.io.IOException;
+import java.time.Instant;
+import java.util.Collections;
+import java.util.HashMap;
+import java.util.List;
+import java.util.Map;
+import java.util.Objects;
+import java.util.Set;
+import java.util.stream.Collectors;
+import org.apache.commons.lang3.StringUtils;
+import org.apache.gravitino.MetadataObject;
+import org.apache.gravitino.authorization.Group;
+import org.apache.gravitino.authorization.Owner;
+import org.apache.gravitino.authorization.Privilege;
+import org.apache.gravitino.authorization.Role;
+import org.apache.gravitino.authorization.RoleChange;
+import org.apache.gravitino.authorization.SecurableObject;
+import org.apache.gravitino.authorization.SecurableObjects;
+import org.apache.gravitino.authorization.User;
+import org.apache.gravitino.authorization.ranger.defines.VXGroup;
+import org.apache.gravitino.authorization.ranger.defines.VXGroupList;
+import org.apache.gravitino.authorization.ranger.defines.VXUser;
+import org.apache.gravitino.authorization.ranger.defines.VXUserList;
+import org.apache.gravitino.connector.AuthorizationPropertiesMeta;
+import org.apache.gravitino.connector.authorization.AuthorizationPlugin;
+import org.apache.gravitino.exceptions.AuthorizationPluginException;
+import org.apache.gravitino.meta.AuditInfo;
+import org.apache.gravitino.meta.GroupEntity;
+import org.apache.gravitino.meta.UserEntity;
+import org.apache.gravitino.utils.PrincipalUtils;
+import org.apache.ranger.RangerServiceException;
+import org.apache.ranger.plugin.model.RangerPolicy;
+import org.apache.ranger.plugin.model.RangerPolicy.RangerPolicyItem;
+import org.apache.ranger.plugin.model.RangerRole;
+import org.apache.ranger.plugin.util.GrantRevokeRoleRequest;
+import org.apache.ranger.plugin.util.SearchFilter;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+/** Ranger authorization operations plugin abstract class. */
+public abstract class RangerAuthorizationPlugin implements AuthorizationPlugin
{
+ private static final Logger LOG =
LoggerFactory.getLogger(RangerAuthorizationPlugin.class);
+
+ protected String catalogProvider;
+ protected RangerClientExtend rangerClient;
+ protected String rangerServiceName;
+ /** Mapping Gravitino privilege name to the underlying authorization system
privileges. */
+ protected Map> privilegesMapping = null;
+ // The owner privileges, the owner can do anything on the metadata object
+ protected Set ownerPrivileges = null;
+
+ /**
+ * Because Ranger doesn't support the precise filter, Ranger will return the
policy meets the
+ * wildcard(*,?) conditions, just like `*.*.*` policy will match
`db1.table1.column1` So we need
+ * to manually precisely filter the policies.
+ */
+ // Search Ranger policy filter keys
+ protected List policyFilterKeys = null;
+ // Search Ranger policy precise filter keys
+ protected List policyPreciseFilterKeys = null;
+
+ public static final String MANAGED_BY_GRAVITINO = "MANAGED_BY_GRAVITINO";
+
+ // TODO: Maybe need to move to the configuration in the future
+ public static final String RANGER_ADMIN_NAME = "admin";
+
+ public RangerAuthorizationPlugin(String catalogProvider, Map
config) {
+super();
+this.catalogProvider = catalogProvider;
+String rangerUrl =
config.get(AuthorizationPropertiesMeta.RANGER_ADMIN_URL);
+String authType = config.get(AuthorizationPropertiesMeta.RANGER_AUTH_TYPE);
+String username = config.get(AuthorizationPropertiesMeta.RANGER_USERNAME);
+// Apache Ranger Password should be minimum 8
Re: [PR] [#3963] feat(core): Apache Ranger Hive authorization pushdown [gravitino]
jerqi commented on code in PR #4515:
URL: https://github.com/apache/gravitino/pull/4515#discussion_r1724517579
##
authorizations/authorization-ranger/src/main/java/org/apache/gravitino/authorization/ranger/RangerAuthorizationPlugin.java:
##
@@ -0,0 +1,998 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+package org.apache.gravitino.authorization.ranger;
+
+import com.google.common.annotations.VisibleForTesting;
+import com.google.common.collect.ImmutableMap;
+import com.google.common.collect.Lists;
+import com.google.common.collect.Sets;
+import com.google.errorprone.annotations.FormatMethod;
+import com.google.errorprone.annotations.FormatString;
+import java.io.IOException;
+import java.time.Instant;
+import java.util.Collections;
+import java.util.HashMap;
+import java.util.List;
+import java.util.Map;
+import java.util.Objects;
+import java.util.Set;
+import java.util.stream.Collectors;
+import org.apache.commons.lang3.StringUtils;
+import org.apache.gravitino.MetadataObject;
+import org.apache.gravitino.authorization.Group;
+import org.apache.gravitino.authorization.Owner;
+import org.apache.gravitino.authorization.Privilege;
+import org.apache.gravitino.authorization.Role;
+import org.apache.gravitino.authorization.RoleChange;
+import org.apache.gravitino.authorization.SecurableObject;
+import org.apache.gravitino.authorization.SecurableObjects;
+import org.apache.gravitino.authorization.User;
+import org.apache.gravitino.authorization.ranger.defines.VXGroup;
+import org.apache.gravitino.authorization.ranger.defines.VXGroupList;
+import org.apache.gravitino.authorization.ranger.defines.VXUser;
+import org.apache.gravitino.authorization.ranger.defines.VXUserList;
+import org.apache.gravitino.connector.AuthorizationPropertiesMeta;
+import org.apache.gravitino.connector.authorization.AuthorizationPlugin;
+import org.apache.gravitino.exceptions.AuthorizationPluginException;
+import org.apache.gravitino.meta.AuditInfo;
+import org.apache.gravitino.meta.GroupEntity;
+import org.apache.gravitino.meta.UserEntity;
+import org.apache.gravitino.utils.PrincipalUtils;
+import org.apache.ranger.RangerServiceException;
+import org.apache.ranger.plugin.model.RangerPolicy;
+import org.apache.ranger.plugin.model.RangerPolicy.RangerPolicyItem;
+import org.apache.ranger.plugin.model.RangerRole;
+import org.apache.ranger.plugin.util.GrantRevokeRoleRequest;
+import org.apache.ranger.plugin.util.SearchFilter;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+/** Ranger authorization operations plugin abstract class. */
+public abstract class RangerAuthorizationPlugin implements AuthorizationPlugin
{
+ private static final Logger LOG =
LoggerFactory.getLogger(RangerAuthorizationPlugin.class);
+
+ protected String catalogProvider;
+ protected RangerClientExtend rangerClient;
+ protected String rangerServiceName;
+ /** Mapping Gravitino privilege name to the underlying authorization system
privileges. */
+ protected Map> privilegesMapping = null;
+ // The owner privileges, the owner can do anything on the metadata object
+ protected Set ownerPrivileges = null;
+
+ /**
+ * Because Ranger doesn't support the precise filter, Ranger will return the
policy meets the
+ * wildcard(*,?) conditions, just like `*.*.*` policy will match
`db1.table1.column1` So we need
+ * to manually precisely filter the policies.
+ */
+ // Search Ranger policy filter keys
+ protected List policyFilterKeys = null;
+ // Search Ranger policy precise filter keys
+ protected List policyPreciseFilterKeys = null;
+
+ public static final String MANAGED_BY_GRAVITINO = "MANAGED_BY_GRAVITINO";
+
+ // TODO: Maybe need to move to the configuration in the future
+ public static final String RANGER_ADMIN_NAME = "admin";
+
+ public RangerAuthorizationPlugin(String catalogProvider, Map
config) {
+super();
+this.catalogProvider = catalogProvider;
+String rangerUrl =
config.get(AuthorizationPropertiesMeta.RANGER_ADMIN_URL);
+String authType = config.get(AuthorizationPropertiesMeta.RANGER_AUTH_TYPE);
+String username = config.get(AuthorizationPropertiesMeta.RANGER_USERNAME);
+// Apache Ranger Password should be minimum 8
Re: [PR] [#3963] feat(core): Apache Ranger Hive authorization pushdown [gravitino]
jerqi commented on code in PR #4515:
URL: https://github.com/apache/gravitino/pull/4515#discussion_r1724516409
##
authorizations/authorization-ranger/src/main/java/org/apache/gravitino/authorization/ranger/RangerAuthorizationPlugin.java:
##
@@ -0,0 +1,998 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+package org.apache.gravitino.authorization.ranger;
+
+import com.google.common.annotations.VisibleForTesting;
+import com.google.common.collect.ImmutableMap;
+import com.google.common.collect.Lists;
+import com.google.common.collect.Sets;
+import com.google.errorprone.annotations.FormatMethod;
+import com.google.errorprone.annotations.FormatString;
+import java.io.IOException;
+import java.time.Instant;
+import java.util.Collections;
+import java.util.HashMap;
+import java.util.List;
+import java.util.Map;
+import java.util.Objects;
+import java.util.Set;
+import java.util.stream.Collectors;
+import org.apache.commons.lang3.StringUtils;
+import org.apache.gravitino.MetadataObject;
+import org.apache.gravitino.authorization.Group;
+import org.apache.gravitino.authorization.Owner;
+import org.apache.gravitino.authorization.Privilege;
+import org.apache.gravitino.authorization.Role;
+import org.apache.gravitino.authorization.RoleChange;
+import org.apache.gravitino.authorization.SecurableObject;
+import org.apache.gravitino.authorization.SecurableObjects;
+import org.apache.gravitino.authorization.User;
+import org.apache.gravitino.authorization.ranger.defines.VXGroup;
+import org.apache.gravitino.authorization.ranger.defines.VXGroupList;
+import org.apache.gravitino.authorization.ranger.defines.VXUser;
+import org.apache.gravitino.authorization.ranger.defines.VXUserList;
+import org.apache.gravitino.connector.AuthorizationPropertiesMeta;
+import org.apache.gravitino.connector.authorization.AuthorizationPlugin;
+import org.apache.gravitino.exceptions.AuthorizationPluginException;
+import org.apache.gravitino.meta.AuditInfo;
+import org.apache.gravitino.meta.GroupEntity;
+import org.apache.gravitino.meta.UserEntity;
+import org.apache.gravitino.utils.PrincipalUtils;
+import org.apache.ranger.RangerServiceException;
+import org.apache.ranger.plugin.model.RangerPolicy;
+import org.apache.ranger.plugin.model.RangerPolicy.RangerPolicyItem;
+import org.apache.ranger.plugin.model.RangerRole;
+import org.apache.ranger.plugin.util.GrantRevokeRoleRequest;
+import org.apache.ranger.plugin.util.SearchFilter;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+/** Ranger authorization operations plugin abstract class. */
+public abstract class RangerAuthorizationPlugin implements AuthorizationPlugin
{
+ private static final Logger LOG =
LoggerFactory.getLogger(RangerAuthorizationPlugin.class);
+
+ protected String catalogProvider;
+ protected RangerClientExtend rangerClient;
+ protected String rangerServiceName;
+ /** Mapping Gravitino privilege name to the underlying authorization system
privileges. */
+ protected Map> privilegesMapping = null;
+ // The owner privileges, the owner can do anything on the metadata object
+ protected Set ownerPrivileges = null;
+
+ /**
+ * Because Ranger doesn't support the precise filter, Ranger will return the
policy meets the
+ * wildcard(*,?) conditions, just like `*.*.*` policy will match
`db1.table1.column1` So we need
+ * to manually precisely filter the policies.
+ */
+ // Search Ranger policy filter keys
+ protected List policyFilterKeys = null;
+ // Search Ranger policy precise filter keys
+ protected List policyPreciseFilterKeys = null;
+
+ public static final String MANAGED_BY_GRAVITINO = "MANAGED_BY_GRAVITINO";
+
+ // TODO: Maybe need to move to the configuration in the future
+ public static final String RANGER_ADMIN_NAME = "admin";
+
+ public RangerAuthorizationPlugin(String catalogProvider, Map
config) {
+super();
+this.catalogProvider = catalogProvider;
+String rangerUrl =
config.get(AuthorizationPropertiesMeta.RANGER_ADMIN_URL);
+String authType = config.get(AuthorizationPropertiesMeta.RANGER_AUTH_TYPE);
+String username = config.get(AuthorizationPropertiesMeta.RANGER_USERNAME);
+// Apache Ranger Password should be minimum 8
Re: [PR] [#3963] feat(core): Apache Ranger Hive authorization pushdown [gravitino]
jerqi commented on code in PR #4515:
URL: https://github.com/apache/gravitino/pull/4515#discussion_r1724515308
##
authorizations/authorization-ranger/src/main/java/org/apache/gravitino/authorization/ranger/RangerAuthorizationPlugin.java:
##
@@ -0,0 +1,998 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+package org.apache.gravitino.authorization.ranger;
+
+import com.google.common.annotations.VisibleForTesting;
+import com.google.common.collect.ImmutableMap;
+import com.google.common.collect.Lists;
+import com.google.common.collect.Sets;
+import com.google.errorprone.annotations.FormatMethod;
+import com.google.errorprone.annotations.FormatString;
+import java.io.IOException;
+import java.time.Instant;
+import java.util.Collections;
+import java.util.HashMap;
+import java.util.List;
+import java.util.Map;
+import java.util.Objects;
+import java.util.Set;
+import java.util.stream.Collectors;
+import org.apache.commons.lang3.StringUtils;
+import org.apache.gravitino.MetadataObject;
+import org.apache.gravitino.authorization.Group;
+import org.apache.gravitino.authorization.Owner;
+import org.apache.gravitino.authorization.Privilege;
+import org.apache.gravitino.authorization.Role;
+import org.apache.gravitino.authorization.RoleChange;
+import org.apache.gravitino.authorization.SecurableObject;
+import org.apache.gravitino.authorization.SecurableObjects;
+import org.apache.gravitino.authorization.User;
+import org.apache.gravitino.authorization.ranger.defines.VXGroup;
+import org.apache.gravitino.authorization.ranger.defines.VXGroupList;
+import org.apache.gravitino.authorization.ranger.defines.VXUser;
+import org.apache.gravitino.authorization.ranger.defines.VXUserList;
+import org.apache.gravitino.connector.AuthorizationPropertiesMeta;
+import org.apache.gravitino.connector.authorization.AuthorizationPlugin;
+import org.apache.gravitino.exceptions.AuthorizationPluginException;
+import org.apache.gravitino.meta.AuditInfo;
+import org.apache.gravitino.meta.GroupEntity;
+import org.apache.gravitino.meta.UserEntity;
+import org.apache.gravitino.utils.PrincipalUtils;
+import org.apache.ranger.RangerServiceException;
+import org.apache.ranger.plugin.model.RangerPolicy;
+import org.apache.ranger.plugin.model.RangerPolicy.RangerPolicyItem;
+import org.apache.ranger.plugin.model.RangerRole;
+import org.apache.ranger.plugin.util.GrantRevokeRoleRequest;
+import org.apache.ranger.plugin.util.SearchFilter;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+/** Ranger authorization operations plugin abstract class. */
+public abstract class RangerAuthorizationPlugin implements AuthorizationPlugin
{
+ private static final Logger LOG =
LoggerFactory.getLogger(RangerAuthorizationPlugin.class);
+
+ protected String catalogProvider;
+ protected RangerClientExtend rangerClient;
+ protected String rangerServiceName;
+ /** Mapping Gravitino privilege name to the underlying authorization system
privileges. */
+ protected Map> privilegesMapping = null;
+ // The owner privileges, the owner can do anything on the metadata object
+ protected Set ownerPrivileges = null;
+
+ /**
+ * Because Ranger doesn't support the precise filter, Ranger will return the
policy meets the
+ * wildcard(*,?) conditions, just like `*.*.*` policy will match
`db1.table1.column1` So we need
+ * to manually precisely filter the policies.
+ */
+ // Search Ranger policy filter keys
+ protected List policyFilterKeys = null;
+ // Search Ranger policy precise filter keys
+ protected List policyPreciseFilterKeys = null;
+
+ public static final String MANAGED_BY_GRAVITINO = "MANAGED_BY_GRAVITINO";
+
+ // TODO: Maybe need to move to the configuration in the future
+ public static final String RANGER_ADMIN_NAME = "admin";
+
+ public RangerAuthorizationPlugin(String catalogProvider, Map
config) {
+super();
+this.catalogProvider = catalogProvider;
+String rangerUrl =
config.get(AuthorizationPropertiesMeta.RANGER_ADMIN_URL);
+String authType = config.get(AuthorizationPropertiesMeta.RANGER_AUTH_TYPE);
+String username = config.get(AuthorizationPropertiesMeta.RANGER_USERNAME);
+// Apache Ranger Password should be minimum 8
Re: [PR] [#3963] feat(core): Apache Ranger Hive authorization pushdown [gravitino]
jerqi commented on code in PR #4515:
URL: https://github.com/apache/gravitino/pull/4515#discussion_r1724512320
##
authorizations/authorization-ranger/src/main/java/org/apache/gravitino/authorization/ranger/RangerAuthorizationPlugin.java:
##
@@ -0,0 +1,998 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+package org.apache.gravitino.authorization.ranger;
+
+import com.google.common.annotations.VisibleForTesting;
+import com.google.common.collect.ImmutableMap;
+import com.google.common.collect.Lists;
+import com.google.common.collect.Sets;
+import com.google.errorprone.annotations.FormatMethod;
+import com.google.errorprone.annotations.FormatString;
+import java.io.IOException;
+import java.time.Instant;
+import java.util.Collections;
+import java.util.HashMap;
+import java.util.List;
+import java.util.Map;
+import java.util.Objects;
+import java.util.Set;
+import java.util.stream.Collectors;
+import org.apache.commons.lang3.StringUtils;
+import org.apache.gravitino.MetadataObject;
+import org.apache.gravitino.authorization.Group;
+import org.apache.gravitino.authorization.Owner;
+import org.apache.gravitino.authorization.Privilege;
+import org.apache.gravitino.authorization.Role;
+import org.apache.gravitino.authorization.RoleChange;
+import org.apache.gravitino.authorization.SecurableObject;
+import org.apache.gravitino.authorization.SecurableObjects;
+import org.apache.gravitino.authorization.User;
+import org.apache.gravitino.authorization.ranger.defines.VXGroup;
+import org.apache.gravitino.authorization.ranger.defines.VXGroupList;
+import org.apache.gravitino.authorization.ranger.defines.VXUser;
+import org.apache.gravitino.authorization.ranger.defines.VXUserList;
+import org.apache.gravitino.connector.AuthorizationPropertiesMeta;
+import org.apache.gravitino.connector.authorization.AuthorizationPlugin;
+import org.apache.gravitino.exceptions.AuthorizationPluginException;
+import org.apache.gravitino.meta.AuditInfo;
+import org.apache.gravitino.meta.GroupEntity;
+import org.apache.gravitino.meta.UserEntity;
+import org.apache.gravitino.utils.PrincipalUtils;
+import org.apache.ranger.RangerServiceException;
+import org.apache.ranger.plugin.model.RangerPolicy;
+import org.apache.ranger.plugin.model.RangerPolicy.RangerPolicyItem;
+import org.apache.ranger.plugin.model.RangerRole;
+import org.apache.ranger.plugin.util.GrantRevokeRoleRequest;
+import org.apache.ranger.plugin.util.SearchFilter;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+/** Ranger authorization operations plugin abstract class. */
+public abstract class RangerAuthorizationPlugin implements AuthorizationPlugin
{
+ private static final Logger LOG =
LoggerFactory.getLogger(RangerAuthorizationPlugin.class);
+
+ protected String catalogProvider;
+ protected RangerClientExtend rangerClient;
+ protected String rangerServiceName;
+ /** Mapping Gravitino privilege name to the underlying authorization system
privileges. */
+ protected Map> privilegesMapping = null;
+ // The owner privileges, the owner can do anything on the metadata object
+ protected Set ownerPrivileges = null;
+
+ /**
+ * Because Ranger doesn't support the precise filter, Ranger will return the
policy meets the
+ * wildcard(*,?) conditions, just like `*.*.*` policy will match
`db1.table1.column1` So we need
+ * to manually precisely filter the policies.
+ */
+ // Search Ranger policy filter keys
+ protected List policyFilterKeys = null;
+ // Search Ranger policy precise filter keys
+ protected List policyPreciseFilterKeys = null;
+
+ public static final String MANAGED_BY_GRAVITINO = "MANAGED_BY_GRAVITINO";
+
+ // TODO: Maybe need to move to the configuration in the future
+ public static final String RANGER_ADMIN_NAME = "admin";
+
+ public RangerAuthorizationPlugin(String catalogProvider, Map
config) {
+super();
+this.catalogProvider = catalogProvider;
+String rangerUrl =
config.get(AuthorizationPropertiesMeta.RANGER_ADMIN_URL);
+String authType = config.get(AuthorizationPropertiesMeta.RANGER_AUTH_TYPE);
+String username = config.get(AuthorizationPropertiesMeta.RANGER_USERNAME);
+// Apache Ranger Password should be minimum 8
Re: [PR] [#3963] feat(core): Apache Ranger Hive authorization pushdown [gravitino]
jerqi commented on code in PR #4515:
URL: https://github.com/apache/gravitino/pull/4515#discussion_r1724512836
##
authorizations/authorization-ranger/src/main/java/org/apache/gravitino/authorization/ranger/RangerAuthorizationPlugin.java:
##
@@ -0,0 +1,998 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+package org.apache.gravitino.authorization.ranger;
+
+import com.google.common.annotations.VisibleForTesting;
+import com.google.common.collect.ImmutableMap;
+import com.google.common.collect.Lists;
+import com.google.common.collect.Sets;
+import com.google.errorprone.annotations.FormatMethod;
+import com.google.errorprone.annotations.FormatString;
+import java.io.IOException;
+import java.time.Instant;
+import java.util.Collections;
+import java.util.HashMap;
+import java.util.List;
+import java.util.Map;
+import java.util.Objects;
+import java.util.Set;
+import java.util.stream.Collectors;
+import org.apache.commons.lang3.StringUtils;
+import org.apache.gravitino.MetadataObject;
+import org.apache.gravitino.authorization.Group;
+import org.apache.gravitino.authorization.Owner;
+import org.apache.gravitino.authorization.Privilege;
+import org.apache.gravitino.authorization.Role;
+import org.apache.gravitino.authorization.RoleChange;
+import org.apache.gravitino.authorization.SecurableObject;
+import org.apache.gravitino.authorization.SecurableObjects;
+import org.apache.gravitino.authorization.User;
+import org.apache.gravitino.authorization.ranger.defines.VXGroup;
+import org.apache.gravitino.authorization.ranger.defines.VXGroupList;
+import org.apache.gravitino.authorization.ranger.defines.VXUser;
+import org.apache.gravitino.authorization.ranger.defines.VXUserList;
+import org.apache.gravitino.connector.AuthorizationPropertiesMeta;
+import org.apache.gravitino.connector.authorization.AuthorizationPlugin;
+import org.apache.gravitino.exceptions.AuthorizationPluginException;
+import org.apache.gravitino.meta.AuditInfo;
+import org.apache.gravitino.meta.GroupEntity;
+import org.apache.gravitino.meta.UserEntity;
+import org.apache.gravitino.utils.PrincipalUtils;
+import org.apache.ranger.RangerServiceException;
+import org.apache.ranger.plugin.model.RangerPolicy;
+import org.apache.ranger.plugin.model.RangerPolicy.RangerPolicyItem;
+import org.apache.ranger.plugin.model.RangerRole;
+import org.apache.ranger.plugin.util.GrantRevokeRoleRequest;
+import org.apache.ranger.plugin.util.SearchFilter;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+/** Ranger authorization operations plugin abstract class. */
+public abstract class RangerAuthorizationPlugin implements AuthorizationPlugin
{
+ private static final Logger LOG =
LoggerFactory.getLogger(RangerAuthorizationPlugin.class);
+
+ protected String catalogProvider;
+ protected RangerClientExtend rangerClient;
+ protected String rangerServiceName;
+ /** Mapping Gravitino privilege name to the underlying authorization system
privileges. */
+ protected Map> privilegesMapping = null;
+ // The owner privileges, the owner can do anything on the metadata object
+ protected Set ownerPrivileges = null;
+
+ /**
+ * Because Ranger doesn't support the precise filter, Ranger will return the
policy meets the
+ * wildcard(*,?) conditions, just like `*.*.*` policy will match
`db1.table1.column1` So we need
+ * to manually precisely filter the policies.
+ */
+ // Search Ranger policy filter keys
+ protected List policyFilterKeys = null;
+ // Search Ranger policy precise filter keys
+ protected List policyPreciseFilterKeys = null;
+
+ public static final String MANAGED_BY_GRAVITINO = "MANAGED_BY_GRAVITINO";
+
+ // TODO: Maybe need to move to the configuration in the future
+ public static final String RANGER_ADMIN_NAME = "admin";
+
+ public RangerAuthorizationPlugin(String catalogProvider, Map
config) {
+super();
+this.catalogProvider = catalogProvider;
+String rangerUrl =
config.get(AuthorizationPropertiesMeta.RANGER_ADMIN_URL);
+String authType = config.get(AuthorizationPropertiesMeta.RANGER_AUTH_TYPE);
+String username = config.get(AuthorizationPropertiesMeta.RANGER_USERNAME);
+// Apache Ranger Password should be minimum 8
Re: [PR] [#3963] feat(core): Apache Ranger Hive authorization pushdown [gravitino]
yuqi1129 commented on code in PR #4515:
URL: https://github.com/apache/gravitino/pull/4515#discussion_r1724485563
##
authorizations/authorization-ranger/src/main/java/org/apache/gravitino/authorization/ranger/RangerAuthorizationPlugin.java:
##
@@ -0,0 +1,1027 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+package org.apache.gravitino.authorization.ranger;
+
+import com.google.common.annotations.VisibleForTesting;
+import com.google.common.collect.ImmutableMap;
+import com.google.common.collect.Lists;
+import com.google.errorprone.annotations.FormatMethod;
+import com.google.errorprone.annotations.FormatString;
+import java.io.IOException;
+import java.util.Arrays;
+import java.util.Collections;
+import java.util.HashMap;
+import java.util.HashSet;
+import java.util.List;
+import java.util.Map;
+import java.util.Objects;
+import java.util.Set;
+import java.util.concurrent.atomic.AtomicReference;
+import java.util.stream.Collectors;
+import org.apache.gravitino.MetadataObject;
+import org.apache.gravitino.authorization.Group;
+import org.apache.gravitino.authorization.Owner;
+import org.apache.gravitino.authorization.Privilege;
+import org.apache.gravitino.authorization.Role;
+import org.apache.gravitino.authorization.RoleChange;
+import org.apache.gravitino.authorization.SecurableObject;
+import org.apache.gravitino.authorization.SecurableObjects;
+import org.apache.gravitino.authorization.User;
+import org.apache.gravitino.authorization.ranger.defines.VXGroup;
+import org.apache.gravitino.authorization.ranger.defines.VXGroupList;
+import org.apache.gravitino.authorization.ranger.defines.VXUser;
+import org.apache.gravitino.authorization.ranger.defines.VXUserList;
+import org.apache.gravitino.connector.AuthorizationPropertiesMeta;
+import org.apache.gravitino.connector.authorization.AuthorizationPlugin;
+import org.apache.gravitino.exceptions.AuthorizationPluginException;
+import org.apache.ranger.RangerServiceException;
+import org.apache.ranger.plugin.model.RangerPolicy;
+import org.apache.ranger.plugin.model.RangerRole;
+import org.apache.ranger.plugin.util.GrantRevokeRoleRequest;
+import org.apache.ranger.plugin.util.SearchFilter;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+/** Ranger authorization operations plugin abstract class. */
+public abstract class RangerAuthorizationPlugin implements AuthorizationPlugin
{
+ private static final Logger LOG =
LoggerFactory.getLogger(RangerAuthorizationPlugin.class);
+
+ protected String catalogProvider;
+ protected RangerClientExt rangerClient;
+ protected String rangerServiceName;
+ /** Mapping Gravitino privilege name to the underlying authorization system
privileges. */
+ protected Map> mapPrivileges = null;
+ // The owner privileges, the owner can do anything on the metadata object
+ protected Set ownerPrivileges = null;
+
+ /**
+ * Because Ranger doesn't support the precise filter, Ranger will return the
policy meets the
+ * wildcard(*,?) conditions, just like `*.*.*` policy will match
`db1.table1.column1` So we need
+ * to manual precise filter the policies.
+ */
+ // Search Ranger policy filter keys
+ protected List policyFilterKeys = null;
+ // Search Ranger policy precise filter keys
+ protected List policyPreciseFilterKeys = null;
+
+ public static final String MANAGED_BY_GRAVITINO = "MANAGED_BY_GRAVITINO";
+
+ // TODO: Maybe need to move to the configuration in the future
+ public static final String RANGER_ADMIN_NAME = "admin";
+
+ public RangerAuthorizationPlugin(String catalogProvider, Map
config) {
+super();
+this.catalogProvider = catalogProvider;
+String rangerUrl =
config.get(AuthorizationPropertiesMeta.RANGER_ADMIN_URL);
+String authType = config.get(AuthorizationPropertiesMeta.RANGER_AUTH_TYPE);
+String username = config.get(AuthorizationPropertiesMeta.RANGER_USERNAME);
+// Apache Ranger Password should be minimum 8 characters with min one
alphabet and one numeric.
+String password = config.get(AuthorizationPropertiesMeta.RANGER_PASSWORD);
+rangerServiceName =
config.get(AuthorizationPropertiesMeta.RANGER_SERVICE_NAME);
+check(rangerUrl != null, "Ranger admin URL is req
Re: [PR] [#3963] feat(core): Apache Ranger Hive authorization pushdown [gravitino]
jerqi commented on code in PR #4515:
URL: https://github.com/apache/gravitino/pull/4515#discussion_r1724485802
##
authorizations/authorization-ranger/src/main/java/org/apache/gravitino/authorization/ranger/RangerAuthorizationPlugin.java:
##
@@ -992,12 +985,11 @@ private boolean checkSecurableObject(SecurableObject
securableObject) {
securableObject
.privileges()
.forEach(
-privilege -> {
- check(
- checkPrivilege(privilege.name()),
- "This privilege %s is not support in the Ranger hive
authorization",
- privilege.name());
-});
+privilege ->
+check(
+checkPrivilege(privilege.name()),
+"This privilege %s is not support in the Ranger hive
authorization",
Review Comment:
```suggestion
"This privilege %s is not supported in the Ranger hive
authorization",
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
Re: [PR] [#3963] feat(core): Apache Ranger Hive authorization pushdown [gravitino]
jerqi commented on code in PR #4515:
URL: https://github.com/apache/gravitino/pull/4515#discussion_r1724471028
##
authorizations/authorization-ranger/src/main/java/org/apache/gravitino/authorization/ranger/RangerClientExtend.java:
##
@@ -0,0 +1,184 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+package org.apache.gravitino.authorization.ranger;
+
+import com.google.common.collect.ImmutableMap;
+import com.sun.jersey.api.client.GenericType;
+import com.sun.jersey.api.client.UniformInterfaceException;
+import java.lang.reflect.InvocationTargetException;
+import java.lang.reflect.Method;
+import java.util.Map;
+import javax.ws.rs.HttpMethod;
+import javax.ws.rs.core.Response;
+import org.apache.gravitino.authorization.ranger.defines.VXGroup;
+import org.apache.gravitino.authorization.ranger.defines.VXGroupList;
+import org.apache.gravitino.authorization.ranger.defines.VXUser;
+import org.apache.gravitino.authorization.ranger.defines.VXUserList;
+import org.apache.ranger.RangerClient;
+import org.apache.ranger.RangerServiceException;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+/**
+ * Apache Ranger client extension
+ * The class extends the RangerClient class and provides additional methods to
create, search and
+ * delete users and groups
+ */
+public class RangerClientExtend extends RangerClient {
+ private static final Logger LOG =
LoggerFactory.getLogger(RangerClientExtend.class);
+ private static final String URI_USER_BASE = "/service/xusers/users";
+ private static final String URI_USER_BY_ID = URI_USER_BASE + "/%d";
+ private static final String URI_GROUP_BASE = "/service/xusers/groups";
+ private static final String URI_GROUP_BY_ID = URI_GROUP_BASE + "/%d";
+ private static final String URI_CREATE_EXTERNAL_USER = URI_USER_BASE +
"/external";
+
+ // Ranger user APIs
+ private static final API SEARCH_USER = new API(URI_USER_BASE,
HttpMethod.GET, Response.Status.OK);
+ private static final API CREATE_EXTERNAL_USER =
+ new API(URI_CREATE_EXTERNAL_USER, HttpMethod.POST, Response.Status.OK);
+ private static final API DELETE_USER =
+ new API(URI_USER_BY_ID, HttpMethod.DELETE, Response.Status.NO_CONTENT);
+
+ // Ranger group APIs
+ private static final API CREATE_GROUP =
+ new API(URI_GROUP_BASE, HttpMethod.POST, Response.Status.OK);
+ private static final API SEARCH_GROUP =
+ new API(URI_GROUP_BASE, HttpMethod.GET, Response.Status.OK);
+ // private static final API GET_GROUP = new API(URI_GROUP_BY_ID,
HttpMethod.GET,
Review Comment:
Do we need remove unused comment?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
Re: [PR] [#3963] feat(core): Apache Ranger Hive authorization pushdown [gravitino]
jerqi commented on PR #4515: URL: https://github.com/apache/gravitino/pull/4515#issuecomment-2301237304 Does it support multiple Hive catalog uses one Ranger? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
Re: [PR] [#3963] feat(core): Apache Ranger Hive authorization pushdown [gravitino]
jerqi commented on code in PR #4515:
URL: https://github.com/apache/gravitino/pull/4515#discussion_r1724468740
##
authorizations/authorization-ranger/src/main/java/org/apache/gravitino/authorization/ranger/RangerAuthorizationPlugin.java:
##
@@ -0,0 +1,1006 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+package org.apache.gravitino.authorization.ranger;
+
+import com.google.common.annotations.VisibleForTesting;
+import com.google.common.collect.ImmutableMap;
+import com.google.common.collect.Lists;
+import com.google.errorprone.annotations.FormatMethod;
+import com.google.errorprone.annotations.FormatString;
+import java.io.IOException;
+import java.time.Instant;
+import java.util.Arrays;
+import java.util.Collections;
+import java.util.HashMap;
+import java.util.HashSet;
+import java.util.List;
+import java.util.Map;
+import java.util.Objects;
+import java.util.Set;
+import java.util.stream.Collectors;
+import org.apache.gravitino.MetadataObject;
+import org.apache.gravitino.authorization.Group;
+import org.apache.gravitino.authorization.Owner;
+import org.apache.gravitino.authorization.Privilege;
+import org.apache.gravitino.authorization.Role;
+import org.apache.gravitino.authorization.RoleChange;
+import org.apache.gravitino.authorization.SecurableObject;
+import org.apache.gravitino.authorization.SecurableObjects;
+import org.apache.gravitino.authorization.User;
+import org.apache.gravitino.authorization.ranger.defines.VXGroup;
+import org.apache.gravitino.authorization.ranger.defines.VXGroupList;
+import org.apache.gravitino.authorization.ranger.defines.VXUser;
+import org.apache.gravitino.authorization.ranger.defines.VXUserList;
+import org.apache.gravitino.connector.AuthorizationPropertiesMeta;
+import org.apache.gravitino.connector.authorization.AuthorizationPlugin;
+import org.apache.gravitino.exceptions.AuthorizationPluginException;
+import org.apache.gravitino.meta.AuditInfo;
+import org.apache.gravitino.meta.GroupEntity;
+import org.apache.gravitino.meta.UserEntity;
+import org.apache.gravitino.utils.PrincipalUtils;
+import org.apache.ranger.RangerServiceException;
+import org.apache.ranger.plugin.model.RangerPolicy;
+import org.apache.ranger.plugin.model.RangerRole;
+import org.apache.ranger.plugin.util.GrantRevokeRoleRequest;
+import org.apache.ranger.plugin.util.SearchFilter;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+/** Ranger authorization operations plugin abstract class. */
+public abstract class RangerAuthorizationPlugin implements AuthorizationPlugin
{
+ private static final Logger LOG =
LoggerFactory.getLogger(RangerAuthorizationPlugin.class);
+
+ protected String catalogProvider;
+ protected RangerClientExtend rangerClient;
+ protected String rangerServiceName;
+ /** Mapping Gravitino privilege name to the underlying authorization system
privileges. */
+ protected Map> mapPrivileges = null;
+ // The owner privileges, the owner can do anything on the metadata object
+ protected Set ownerPrivileges = null;
+
+ /**
+ * Because Ranger doesn't support the precise filter, Ranger will return the
policy meets the
+ * wildcard(*,?) conditions, just like `*.*.*` policy will match
`db1.table1.column1` So we need
+ * to manual precise filter the policies.
+ */
+ // Search Ranger policy filter keys
+ protected List policyFilterKeys = null;
+ // Search Ranger policy precise filter keys
+ protected List policyPreciseFilterKeys = null;
+
+ public static final String MANAGED_BY_GRAVITINO = "MANAGED_BY_GRAVITINO";
+
+ // TODO: Maybe need to move to the configuration in the future
+ public static final String RANGER_ADMIN_NAME = "admin";
+
+ public RangerAuthorizationPlugin(String catalogProvider, Map
config) {
+super();
+this.catalogProvider = catalogProvider;
+String rangerUrl =
config.get(AuthorizationPropertiesMeta.RANGER_ADMIN_URL);
+String authType = config.get(AuthorizationPropertiesMeta.RANGER_AUTH_TYPE);
+String username = config.get(AuthorizationPropertiesMeta.RANGER_USERNAME);
+// Apache Ranger Password should be minimum 8 characters with min one
alphabet and one numeric.
+String password = config.get(AuthorizationPropertiesM
Re: [PR] [#3963] feat(core): Apache Ranger Hive authorization pushdown [gravitino]
jerqi commented on code in PR #4515:
URL: https://github.com/apache/gravitino/pull/4515#discussion_r1724462431
##
integration-test-common/src/test/java/org/apache/gravitino/integration/test/container/RangerContainer.java:
##
@@ -40,15 +40,18 @@ public class RangerContainer extends BaseContainer {
public static final int RANGER_SERVER_PORT = 6080;
public RangerClient rangerClient;
private String rangerUrl;
- private static final String username = "admin";
- // Apache Ranger Password should be minimum 8 characters with min one
alphabet and one numeric.
- private static final String password = "rangerR0cks!";
/**
- * for kerberos authentication: authType = "kerberos" username = principal
password = path of the
- * keytab file
+ * for kerberos authentication:
+ * authType = "kerberos"
+ * username = principal
+ * password = path of the keytab file
*/
- private static final String authType = "simple";
+ public static final String authType = "simple";
+
+ public static final String rangerUserName = "admin";
+ // Apache Ranger Password should be minimum 8 characters with min one
alphabet and one numeric.
Review Comment:
OK.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
Re: [PR] [#3963] feat(core): Apache Ranger Hive authorization pushdown [gravitino]
jerqi commented on code in PR #4515: URL: https://github.com/apache/gravitino/pull/4515#discussion_r1724349879 ## authorizations/authorization-ranger/src/main/java/org/apache/gravitino/authorization/ranger/RangerAuthorizationPlugin.java: ## @@ -0,0 +1,1006 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, + * software distributed under the License is distributed on an + * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY + * KIND, either express or implied. See the License for the + * specific language governing permissions and limitations + * under the License. + */ +package org.apache.gravitino.authorization.ranger; + +import com.google.common.annotations.VisibleForTesting; +import com.google.common.collect.ImmutableMap; +import com.google.common.collect.Lists; +import com.google.errorprone.annotations.FormatMethod; +import com.google.errorprone.annotations.FormatString; +import java.io.IOException; +import java.time.Instant; +import java.util.Arrays; +import java.util.Collections; +import java.util.HashMap; +import java.util.HashSet; +import java.util.List; +import java.util.Map; +import java.util.Objects; +import java.util.Set; +import java.util.stream.Collectors; +import org.apache.gravitino.MetadataObject; +import org.apache.gravitino.authorization.Group; +import org.apache.gravitino.authorization.Owner; +import org.apache.gravitino.authorization.Privilege; +import org.apache.gravitino.authorization.Role; +import org.apache.gravitino.authorization.RoleChange; +import org.apache.gravitino.authorization.SecurableObject; +import org.apache.gravitino.authorization.SecurableObjects; +import org.apache.gravitino.authorization.User; +import org.apache.gravitino.authorization.ranger.defines.VXGroup; +import org.apache.gravitino.authorization.ranger.defines.VXGroupList; +import org.apache.gravitino.authorization.ranger.defines.VXUser; +import org.apache.gravitino.authorization.ranger.defines.VXUserList; +import org.apache.gravitino.connector.AuthorizationPropertiesMeta; +import org.apache.gravitino.connector.authorization.AuthorizationPlugin; +import org.apache.gravitino.exceptions.AuthorizationPluginException; +import org.apache.gravitino.meta.AuditInfo; +import org.apache.gravitino.meta.GroupEntity; +import org.apache.gravitino.meta.UserEntity; +import org.apache.gravitino.utils.PrincipalUtils; +import org.apache.ranger.RangerServiceException; +import org.apache.ranger.plugin.model.RangerPolicy; +import org.apache.ranger.plugin.model.RangerRole; +import org.apache.ranger.plugin.util.GrantRevokeRoleRequest; +import org.apache.ranger.plugin.util.SearchFilter; +import org.slf4j.Logger; +import org.slf4j.LoggerFactory; + +/** Ranger authorization operations plugin abstract class. */ Review Comment: Add more comments about this class. 1. What's precise filter. 2. How to map the securable object to policies. We should give a rough mind about this class. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
Re: [PR] [#3963] feat(core): Apache Ranger Hive authorization pushdown [gravitino]
jerqi commented on code in PR #4515:
URL: https://github.com/apache/gravitino/pull/4515#discussion_r1724343686
##
authorizations/authorization-ranger/src/main/java/org/apache/gravitino/authorization/ranger/RangerAuthorizationPlugin.java:
##
@@ -0,0 +1,1027 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+package org.apache.gravitino.authorization.ranger;
+
+import com.google.common.annotations.VisibleForTesting;
+import com.google.common.collect.ImmutableMap;
+import com.google.common.collect.Lists;
+import com.google.errorprone.annotations.FormatMethod;
+import com.google.errorprone.annotations.FormatString;
+import java.io.IOException;
+import java.util.Arrays;
+import java.util.Collections;
+import java.util.HashMap;
+import java.util.HashSet;
+import java.util.List;
+import java.util.Map;
+import java.util.Objects;
+import java.util.Set;
+import java.util.concurrent.atomic.AtomicReference;
+import java.util.stream.Collectors;
+import org.apache.gravitino.MetadataObject;
+import org.apache.gravitino.authorization.Group;
+import org.apache.gravitino.authorization.Owner;
+import org.apache.gravitino.authorization.Privilege;
+import org.apache.gravitino.authorization.Role;
+import org.apache.gravitino.authorization.RoleChange;
+import org.apache.gravitino.authorization.SecurableObject;
+import org.apache.gravitino.authorization.SecurableObjects;
+import org.apache.gravitino.authorization.User;
+import org.apache.gravitino.authorization.ranger.defines.VXGroup;
+import org.apache.gravitino.authorization.ranger.defines.VXGroupList;
+import org.apache.gravitino.authorization.ranger.defines.VXUser;
+import org.apache.gravitino.authorization.ranger.defines.VXUserList;
+import org.apache.gravitino.connector.AuthorizationPropertiesMeta;
+import org.apache.gravitino.connector.authorization.AuthorizationPlugin;
+import org.apache.gravitino.exceptions.AuthorizationPluginException;
+import org.apache.ranger.RangerServiceException;
+import org.apache.ranger.plugin.model.RangerPolicy;
+import org.apache.ranger.plugin.model.RangerRole;
+import org.apache.ranger.plugin.util.GrantRevokeRoleRequest;
+import org.apache.ranger.plugin.util.SearchFilter;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+/** Ranger authorization operations plugin abstract class. */
+public abstract class RangerAuthorizationPlugin implements AuthorizationPlugin
{
+ private static final Logger LOG =
LoggerFactory.getLogger(RangerAuthorizationPlugin.class);
+
+ protected String catalogProvider;
+ protected RangerClientExt rangerClient;
+ protected String rangerServiceName;
+ /** Mapping Gravitino privilege name to the underlying authorization system
privileges. */
+ protected Map> mapPrivileges = null;
+ // The owner privileges, the owner can do anything on the metadata object
+ protected Set ownerPrivileges = null;
+
+ /**
+ * Because Ranger doesn't support the precise filter, Ranger will return the
policy meets the
+ * wildcard(*,?) conditions, just like `*.*.*` policy will match
`db1.table1.column1` So we need
+ * to manual precise filter the policies.
+ */
+ // Search Ranger policy filter keys
+ protected List policyFilterKeys = null;
+ // Search Ranger policy precise filter keys
+ protected List policyPreciseFilterKeys = null;
+
+ public static final String MANAGED_BY_GRAVITINO = "MANAGED_BY_GRAVITINO";
+
+ // TODO: Maybe need to move to the configuration in the future
+ public static final String RANGER_ADMIN_NAME = "admin";
+
+ public RangerAuthorizationPlugin(String catalogProvider, Map
config) {
+super();
+this.catalogProvider = catalogProvider;
+String rangerUrl =
config.get(AuthorizationPropertiesMeta.RANGER_ADMIN_URL);
+String authType = config.get(AuthorizationPropertiesMeta.RANGER_AUTH_TYPE);
+String username = config.get(AuthorizationPropertiesMeta.RANGER_USERNAME);
+// Apache Ranger Password should be minimum 8 characters with min one
alphabet and one numeric.
+String password = config.get(AuthorizationPropertiesMeta.RANGER_PASSWORD);
+rangerServiceName =
config.get(AuthorizationPropertiesMeta.RANGER_SERVICE_NAME);
+check(rangerUrl != null, "Ranger admin URL is requir
Re: [PR] [#3963] feat(core): Apache Ranger Hive authorization pushdown [gravitino]
jerqi commented on code in PR #4515:
URL: https://github.com/apache/gravitino/pull/4515#discussion_r172423
##
authorizations/authorization-ranger/src/main/java/org/apache/gravitino/authorization/ranger/RangerAuthorizationPlugin.java:
##
@@ -0,0 +1,1027 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+package org.apache.gravitino.authorization.ranger;
+
+import com.google.common.annotations.VisibleForTesting;
+import com.google.common.collect.ImmutableMap;
+import com.google.common.collect.Lists;
+import com.google.errorprone.annotations.FormatMethod;
+import com.google.errorprone.annotations.FormatString;
+import java.io.IOException;
+import java.util.Arrays;
+import java.util.Collections;
+import java.util.HashMap;
+import java.util.HashSet;
+import java.util.List;
+import java.util.Map;
+import java.util.Objects;
+import java.util.Set;
+import java.util.concurrent.atomic.AtomicReference;
+import java.util.stream.Collectors;
+import org.apache.gravitino.MetadataObject;
+import org.apache.gravitino.authorization.Group;
+import org.apache.gravitino.authorization.Owner;
+import org.apache.gravitino.authorization.Privilege;
+import org.apache.gravitino.authorization.Role;
+import org.apache.gravitino.authorization.RoleChange;
+import org.apache.gravitino.authorization.SecurableObject;
+import org.apache.gravitino.authorization.SecurableObjects;
+import org.apache.gravitino.authorization.User;
+import org.apache.gravitino.authorization.ranger.defines.VXGroup;
+import org.apache.gravitino.authorization.ranger.defines.VXGroupList;
+import org.apache.gravitino.authorization.ranger.defines.VXUser;
+import org.apache.gravitino.authorization.ranger.defines.VXUserList;
+import org.apache.gravitino.connector.AuthorizationPropertiesMeta;
+import org.apache.gravitino.connector.authorization.AuthorizationPlugin;
+import org.apache.gravitino.exceptions.AuthorizationPluginException;
+import org.apache.ranger.RangerServiceException;
+import org.apache.ranger.plugin.model.RangerPolicy;
+import org.apache.ranger.plugin.model.RangerRole;
+import org.apache.ranger.plugin.util.GrantRevokeRoleRequest;
+import org.apache.ranger.plugin.util.SearchFilter;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+/** Ranger authorization operations plugin abstract class. */
+public abstract class RangerAuthorizationPlugin implements AuthorizationPlugin
{
+ private static final Logger LOG =
LoggerFactory.getLogger(RangerAuthorizationPlugin.class);
+
+ protected String catalogProvider;
+ protected RangerClientExt rangerClient;
+ protected String rangerServiceName;
+ /** Mapping Gravitino privilege name to the underlying authorization system
privileges. */
+ protected Map> mapPrivileges = null;
+ // The owner privileges, the owner can do anything on the metadata object
+ protected Set ownerPrivileges = null;
+
+ /**
+ * Because Ranger doesn't support the precise filter, Ranger will return the
policy meets the
+ * wildcard(*,?) conditions, just like `*.*.*` policy will match
`db1.table1.column1` So we need
+ * to manual precise filter the policies.
+ */
+ // Search Ranger policy filter keys
+ protected List policyFilterKeys = null;
+ // Search Ranger policy precise filter keys
+ protected List policyPreciseFilterKeys = null;
+
+ public static final String MANAGED_BY_GRAVITINO = "MANAGED_BY_GRAVITINO";
+
+ // TODO: Maybe need to move to the configuration in the future
+ public static final String RANGER_ADMIN_NAME = "admin";
+
+ public RangerAuthorizationPlugin(String catalogProvider, Map
config) {
+super();
+this.catalogProvider = catalogProvider;
+String rangerUrl =
config.get(AuthorizationPropertiesMeta.RANGER_ADMIN_URL);
+String authType = config.get(AuthorizationPropertiesMeta.RANGER_AUTH_TYPE);
+String username = config.get(AuthorizationPropertiesMeta.RANGER_USERNAME);
+// Apache Ranger Password should be minimum 8 characters with min one
alphabet and one numeric.
+String password = config.get(AuthorizationPropertiesMeta.RANGER_PASSWORD);
+rangerServiceName =
config.get(AuthorizationPropertiesMeta.RANGER_SERVICE_NAME);
+check(rangerUrl != null, "Ranger admin URL is requir
Re: [PR] [#3963] feat(core): Apache Ranger Hive authorization pushdown [gravitino]
jerqi commented on code in PR #4515:
URL: https://github.com/apache/gravitino/pull/4515#discussion_r1724338147
##
authorizations/authorization-ranger/src/main/java/org/apache/gravitino/authorization/ranger/RangerAuthorizationPlugin.java:
##
@@ -0,0 +1,1006 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+package org.apache.gravitino.authorization.ranger;
+
+import com.google.common.annotations.VisibleForTesting;
+import com.google.common.collect.ImmutableMap;
+import com.google.common.collect.Lists;
+import com.google.errorprone.annotations.FormatMethod;
+import com.google.errorprone.annotations.FormatString;
+import java.io.IOException;
+import java.time.Instant;
+import java.util.Arrays;
+import java.util.Collections;
+import java.util.HashMap;
+import java.util.HashSet;
+import java.util.List;
+import java.util.Map;
+import java.util.Objects;
+import java.util.Set;
+import java.util.stream.Collectors;
+import org.apache.gravitino.MetadataObject;
+import org.apache.gravitino.authorization.Group;
+import org.apache.gravitino.authorization.Owner;
+import org.apache.gravitino.authorization.Privilege;
+import org.apache.gravitino.authorization.Role;
+import org.apache.gravitino.authorization.RoleChange;
+import org.apache.gravitino.authorization.SecurableObject;
+import org.apache.gravitino.authorization.SecurableObjects;
+import org.apache.gravitino.authorization.User;
+import org.apache.gravitino.authorization.ranger.defines.VXGroup;
+import org.apache.gravitino.authorization.ranger.defines.VXGroupList;
+import org.apache.gravitino.authorization.ranger.defines.VXUser;
+import org.apache.gravitino.authorization.ranger.defines.VXUserList;
+import org.apache.gravitino.connector.AuthorizationPropertiesMeta;
+import org.apache.gravitino.connector.authorization.AuthorizationPlugin;
+import org.apache.gravitino.exceptions.AuthorizationPluginException;
+import org.apache.gravitino.meta.AuditInfo;
+import org.apache.gravitino.meta.GroupEntity;
+import org.apache.gravitino.meta.UserEntity;
+import org.apache.gravitino.utils.PrincipalUtils;
+import org.apache.ranger.RangerServiceException;
+import org.apache.ranger.plugin.model.RangerPolicy;
+import org.apache.ranger.plugin.model.RangerRole;
+import org.apache.ranger.plugin.util.GrantRevokeRoleRequest;
+import org.apache.ranger.plugin.util.SearchFilter;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+/** Ranger authorization operations plugin abstract class. */
+public abstract class RangerAuthorizationPlugin implements AuthorizationPlugin
{
+ private static final Logger LOG =
LoggerFactory.getLogger(RangerAuthorizationPlugin.class);
+
+ protected String catalogProvider;
+ protected RangerClientExtend rangerClient;
+ protected String rangerServiceName;
+ /** Mapping Gravitino privilege name to the underlying authorization system
privileges. */
+ protected Map> privilegesMapping = null;
+ // The owner privileges, the owner can do anything on the metadata object
+ protected Set ownerPrivileges = null;
+
+ /**
+ * Because Ranger doesn't support the precise filter, Ranger will return the
policy meets the
+ * wildcard(*,?) conditions, just like `*.*.*` policy will match
`db1.table1.column1` So we need
+ * to manually precisely filter the policies.
+ */
+ // Search Ranger policy filter keys
+ protected List policyFilterKeys = null;
+ // Search Ranger policy precise filter keys
+ protected List policyPreciseFilterKeys = null;
+
+ public static final String MANAGED_BY_GRAVITINO = "MANAGED_BY_GRAVITINO";
+
+ // TODO: Maybe need to move to the configuration in the future
+ public static final String RANGER_ADMIN_NAME = "admin";
+
+ public RangerAuthorizationPlugin(String catalogProvider, Map
config) {
+super();
+this.catalogProvider = catalogProvider;
+String rangerUrl =
config.get(AuthorizationPropertiesMeta.RANGER_ADMIN_URL);
+String authType = config.get(AuthorizationPropertiesMeta.RANGER_AUTH_TYPE);
+String username = config.get(AuthorizationPropertiesMeta.RANGER_USERNAME);
+// Apache Ranger Password should be minimum 8 characters with min one
alphabet and one numeric.
+String password = config.get(AuthorizationPro
Re: [PR] [#3963] feat(core): Apache Ranger Hive authorization pushdown [gravitino]
jerqi commented on code in PR #4515:
URL: https://github.com/apache/gravitino/pull/4515#discussion_r172423
##
authorizations/authorization-ranger/src/main/java/org/apache/gravitino/authorization/ranger/RangerAuthorizationPlugin.java:
##
@@ -0,0 +1,1027 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+package org.apache.gravitino.authorization.ranger;
+
+import com.google.common.annotations.VisibleForTesting;
+import com.google.common.collect.ImmutableMap;
+import com.google.common.collect.Lists;
+import com.google.errorprone.annotations.FormatMethod;
+import com.google.errorprone.annotations.FormatString;
+import java.io.IOException;
+import java.util.Arrays;
+import java.util.Collections;
+import java.util.HashMap;
+import java.util.HashSet;
+import java.util.List;
+import java.util.Map;
+import java.util.Objects;
+import java.util.Set;
+import java.util.concurrent.atomic.AtomicReference;
+import java.util.stream.Collectors;
+import org.apache.gravitino.MetadataObject;
+import org.apache.gravitino.authorization.Group;
+import org.apache.gravitino.authorization.Owner;
+import org.apache.gravitino.authorization.Privilege;
+import org.apache.gravitino.authorization.Role;
+import org.apache.gravitino.authorization.RoleChange;
+import org.apache.gravitino.authorization.SecurableObject;
+import org.apache.gravitino.authorization.SecurableObjects;
+import org.apache.gravitino.authorization.User;
+import org.apache.gravitino.authorization.ranger.defines.VXGroup;
+import org.apache.gravitino.authorization.ranger.defines.VXGroupList;
+import org.apache.gravitino.authorization.ranger.defines.VXUser;
+import org.apache.gravitino.authorization.ranger.defines.VXUserList;
+import org.apache.gravitino.connector.AuthorizationPropertiesMeta;
+import org.apache.gravitino.connector.authorization.AuthorizationPlugin;
+import org.apache.gravitino.exceptions.AuthorizationPluginException;
+import org.apache.ranger.RangerServiceException;
+import org.apache.ranger.plugin.model.RangerPolicy;
+import org.apache.ranger.plugin.model.RangerRole;
+import org.apache.ranger.plugin.util.GrantRevokeRoleRequest;
+import org.apache.ranger.plugin.util.SearchFilter;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+/** Ranger authorization operations plugin abstract class. */
+public abstract class RangerAuthorizationPlugin implements AuthorizationPlugin
{
+ private static final Logger LOG =
LoggerFactory.getLogger(RangerAuthorizationPlugin.class);
+
+ protected String catalogProvider;
+ protected RangerClientExt rangerClient;
+ protected String rangerServiceName;
+ /** Mapping Gravitino privilege name to the underlying authorization system
privileges. */
+ protected Map> mapPrivileges = null;
+ // The owner privileges, the owner can do anything on the metadata object
+ protected Set ownerPrivileges = null;
+
+ /**
+ * Because Ranger doesn't support the precise filter, Ranger will return the
policy meets the
+ * wildcard(*,?) conditions, just like `*.*.*` policy will match
`db1.table1.column1` So we need
+ * to manual precise filter the policies.
+ */
+ // Search Ranger policy filter keys
+ protected List policyFilterKeys = null;
+ // Search Ranger policy precise filter keys
+ protected List policyPreciseFilterKeys = null;
+
+ public static final String MANAGED_BY_GRAVITINO = "MANAGED_BY_GRAVITINO";
+
+ // TODO: Maybe need to move to the configuration in the future
+ public static final String RANGER_ADMIN_NAME = "admin";
+
+ public RangerAuthorizationPlugin(String catalogProvider, Map
config) {
+super();
+this.catalogProvider = catalogProvider;
+String rangerUrl =
config.get(AuthorizationPropertiesMeta.RANGER_ADMIN_URL);
+String authType = config.get(AuthorizationPropertiesMeta.RANGER_AUTH_TYPE);
+String username = config.get(AuthorizationPropertiesMeta.RANGER_USERNAME);
+// Apache Ranger Password should be minimum 8 characters with min one
alphabet and one numeric.
+String password = config.get(AuthorizationPropertiesMeta.RANGER_PASSWORD);
+rangerServiceName =
config.get(AuthorizationPropertiesMeta.RANGER_SERVICE_NAME);
+check(rangerUrl != null, "Ranger admin URL is requir
Re: [PR] [#3963] feat(core): Apache Ranger Hive authorization pushdown [gravitino]
jerqi commented on code in PR #4515:
URL: https://github.com/apache/gravitino/pull/4515#discussion_r1724331813
##
authorizations/authorization-ranger/src/main/java/org/apache/gravitino/authorization/ranger/RangerAuthorizationPlugin.java:
##
@@ -0,0 +1,1006 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+package org.apache.gravitino.authorization.ranger;
+
+import com.google.common.annotations.VisibleForTesting;
+import com.google.common.collect.ImmutableMap;
+import com.google.common.collect.Lists;
+import com.google.errorprone.annotations.FormatMethod;
+import com.google.errorprone.annotations.FormatString;
+import java.io.IOException;
+import java.time.Instant;
+import java.util.Arrays;
+import java.util.Collections;
+import java.util.HashMap;
+import java.util.HashSet;
+import java.util.List;
+import java.util.Map;
+import java.util.Objects;
+import java.util.Set;
+import java.util.stream.Collectors;
+import org.apache.gravitino.MetadataObject;
+import org.apache.gravitino.authorization.Group;
+import org.apache.gravitino.authorization.Owner;
+import org.apache.gravitino.authorization.Privilege;
+import org.apache.gravitino.authorization.Role;
+import org.apache.gravitino.authorization.RoleChange;
+import org.apache.gravitino.authorization.SecurableObject;
+import org.apache.gravitino.authorization.SecurableObjects;
+import org.apache.gravitino.authorization.User;
+import org.apache.gravitino.authorization.ranger.defines.VXGroup;
+import org.apache.gravitino.authorization.ranger.defines.VXGroupList;
+import org.apache.gravitino.authorization.ranger.defines.VXUser;
+import org.apache.gravitino.authorization.ranger.defines.VXUserList;
+import org.apache.gravitino.connector.AuthorizationPropertiesMeta;
+import org.apache.gravitino.connector.authorization.AuthorizationPlugin;
+import org.apache.gravitino.exceptions.AuthorizationPluginException;
+import org.apache.gravitino.meta.AuditInfo;
+import org.apache.gravitino.meta.GroupEntity;
+import org.apache.gravitino.meta.UserEntity;
+import org.apache.gravitino.utils.PrincipalUtils;
+import org.apache.ranger.RangerServiceException;
+import org.apache.ranger.plugin.model.RangerPolicy;
+import org.apache.ranger.plugin.model.RangerRole;
+import org.apache.ranger.plugin.util.GrantRevokeRoleRequest;
+import org.apache.ranger.plugin.util.SearchFilter;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+/** Ranger authorization operations plugin abstract class. */
+public abstract class RangerAuthorizationPlugin implements AuthorizationPlugin
{
+ private static final Logger LOG =
LoggerFactory.getLogger(RangerAuthorizationPlugin.class);
+
+ protected String catalogProvider;
+ protected RangerClientExtend rangerClient;
+ protected String rangerServiceName;
+ /** Mapping Gravitino privilege name to the underlying authorization system
privileges. */
+ protected Map> privilegesMapping = null;
+ // The owner privileges, the owner can do anything on the metadata object
+ protected Set ownerPrivileges = null;
+
+ /**
+ * Because Ranger doesn't support the precise filter, Ranger will return the
policy meets the
+ * wildcard(*,?) conditions, just like `*.*.*` policy will match
`db1.table1.column1` So we need
+ * to manually precisely filter the policies.
+ */
+ // Search Ranger policy filter keys
+ protected List policyFilterKeys = null;
+ // Search Ranger policy precise filter keys
+ protected List policyPreciseFilterKeys = null;
+
+ public static final String MANAGED_BY_GRAVITINO = "MANAGED_BY_GRAVITINO";
+
+ // TODO: Maybe need to move to the configuration in the future
+ public static final String RANGER_ADMIN_NAME = "admin";
+
+ public RangerAuthorizationPlugin(String catalogProvider, Map
config) {
+super();
+this.catalogProvider = catalogProvider;
+String rangerUrl =
config.get(AuthorizationPropertiesMeta.RANGER_ADMIN_URL);
+String authType = config.get(AuthorizationPropertiesMeta.RANGER_AUTH_TYPE);
+String username = config.get(AuthorizationPropertiesMeta.RANGER_USERNAME);
+// Apache Ranger Password should be minimum 8 characters with min one
alphabet and one numeric.
+String password = config.get(AuthorizationPro
Re: [PR] [#3963] feat(core): Apache Ranger Hive authorization pushdown [gravitino]
xunliu commented on code in PR #4515:
URL: https://github.com/apache/gravitino/pull/4515#discussion_r1724283105
##
integration-test-common/src/test/java/org/apache/gravitino/integration/test/container/RangerContainer.java:
##
@@ -40,15 +40,18 @@ public class RangerContainer extends BaseContainer {
public static final int RANGER_SERVER_PORT = 6080;
public RangerClient rangerClient;
private String rangerUrl;
- private static final String username = "admin";
- // Apache Ranger Password should be minimum 8 characters with min one
alphabet and one numeric.
- private static final String password = "rangerR0cks!";
/**
- * for kerberos authentication: authType = "kerberos" username = principal
password = path of the
- * keytab file
+ * for kerberos authentication:
+ * authType = "kerberos"
+ * username = principal
+ * password = path of the keytab file
*/
- private static final String authType = "simple";
+ public static final String authType = "simple";
+
+ public static final String rangerUserName = "admin";
+ // Apache Ranger Password should be minimum 8 characters with min one
alphabet and one numeric.
Review Comment:
This is Apache Ranger original comments,
https://github.com/apache/ranger/blob/master/security-admin/scripts/install.properties#L84
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
Re: [PR] [#3963] feat(core): Apache Ranger Hive authorization pushdown [gravitino]
xunliu commented on code in PR #4515:
URL: https://github.com/apache/gravitino/pull/4515#discussion_r1724283105
##
integration-test-common/src/test/java/org/apache/gravitino/integration/test/container/RangerContainer.java:
##
@@ -40,15 +40,18 @@ public class RangerContainer extends BaseContainer {
public static final int RANGER_SERVER_PORT = 6080;
public RangerClient rangerClient;
private String rangerUrl;
- private static final String username = "admin";
- // Apache Ranger Password should be minimum 8 characters with min one
alphabet and one numeric.
- private static final String password = "rangerR0cks!";
/**
- * for kerberos authentication: authType = "kerberos" username = principal
password = path of the
- * keytab file
+ * for kerberos authentication:
+ * authType = "kerberos"
+ * username = principal
+ * password = path of the keytab file
*/
- private static final String authType = "simple";
+ public static final String authType = "simple";
+
+ public static final String rangerUserName = "admin";
+ // Apache Ranger Password should be minimum 8 characters with min one
alphabet and one numeric.
Review Comment:
This is Apache Ranger origion comments,
https://github.com/apache/ranger/blob/master/security-admin/scripts/install.properties#L84
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
Re: [PR] [#3963] feat(core): Apache Ranger Hive authorization pushdown [gravitino]
jerqi commented on code in PR #4515:
URL: https://github.com/apache/gravitino/pull/4515#discussion_r1724216553
##
integration-test-common/src/test/java/org/apache/gravitino/integration/test/container/RangerContainer.java:
##
@@ -40,15 +40,18 @@ public class RangerContainer extends BaseContainer {
public static final int RANGER_SERVER_PORT = 6080;
public RangerClient rangerClient;
private String rangerUrl;
- private static final String username = "admin";
- // Apache Ranger Password should be minimum 8 characters with min one
alphabet and one numeric.
- private static final String password = "rangerR0cks!";
/**
- * for kerberos authentication: authType = "kerberos" username = principal
password = path of the
- * keytab file
+ * for kerberos authentication:
+ * authType = "kerberos"
+ * username = principal
+ * password = path of the keytab file
*/
- private static final String authType = "simple";
+ public static final String authType = "simple";
+
+ public static final String rangerUserName = "admin";
+ // Apache Ranger Password should be minimum 8 characters with min one
alphabet and one numeric.
Review Comment:
```suggestion
// Apache Ranger Password should have the character size bigger than 8
with at least one alphabet and one numeric.
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
Re: [PR] [#3963] feat(core): Apache Ranger Hive authorization pushdown [gravitino]
jerqi commented on code in PR #4515:
URL: https://github.com/apache/gravitino/pull/4515#discussion_r1724205839
##
authorizations/authorization-ranger/src/main/java/org/apache/gravitino/authorization/ranger/RangerAuthorizationPlugin.java:
##
@@ -0,0 +1,1006 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+package org.apache.gravitino.authorization.ranger;
+
+import com.google.common.annotations.VisibleForTesting;
+import com.google.common.collect.ImmutableMap;
+import com.google.common.collect.Lists;
+import com.google.errorprone.annotations.FormatMethod;
+import com.google.errorprone.annotations.FormatString;
+import java.io.IOException;
+import java.time.Instant;
+import java.util.Arrays;
+import java.util.Collections;
+import java.util.HashMap;
+import java.util.HashSet;
+import java.util.List;
+import java.util.Map;
+import java.util.Objects;
+import java.util.Set;
+import java.util.stream.Collectors;
+import org.apache.gravitino.MetadataObject;
+import org.apache.gravitino.authorization.Group;
+import org.apache.gravitino.authorization.Owner;
+import org.apache.gravitino.authorization.Privilege;
+import org.apache.gravitino.authorization.Role;
+import org.apache.gravitino.authorization.RoleChange;
+import org.apache.gravitino.authorization.SecurableObject;
+import org.apache.gravitino.authorization.SecurableObjects;
+import org.apache.gravitino.authorization.User;
+import org.apache.gravitino.authorization.ranger.defines.VXGroup;
+import org.apache.gravitino.authorization.ranger.defines.VXGroupList;
+import org.apache.gravitino.authorization.ranger.defines.VXUser;
+import org.apache.gravitino.authorization.ranger.defines.VXUserList;
+import org.apache.gravitino.connector.AuthorizationPropertiesMeta;
+import org.apache.gravitino.connector.authorization.AuthorizationPlugin;
+import org.apache.gravitino.exceptions.AuthorizationPluginException;
+import org.apache.gravitino.meta.AuditInfo;
+import org.apache.gravitino.meta.GroupEntity;
+import org.apache.gravitino.meta.UserEntity;
+import org.apache.gravitino.utils.PrincipalUtils;
+import org.apache.ranger.RangerServiceException;
+import org.apache.ranger.plugin.model.RangerPolicy;
+import org.apache.ranger.plugin.model.RangerRole;
+import org.apache.ranger.plugin.util.GrantRevokeRoleRequest;
+import org.apache.ranger.plugin.util.SearchFilter;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+/** Ranger authorization operations plugin abstract class. */
+public abstract class RangerAuthorizationPlugin implements AuthorizationPlugin
{
+ private static final Logger LOG =
LoggerFactory.getLogger(RangerAuthorizationPlugin.class);
+
+ protected String catalogProvider;
+ protected RangerClientExtend rangerClient;
+ protected String rangerServiceName;
+ /** Mapping Gravitino privilege name to the underlying authorization system
privileges. */
+ protected Map> privilegesMapping = null;
+ // The owner privileges, the owner can do anything on the metadata object
+ protected Set ownerPrivileges = null;
+
+ /**
+ * Because Ranger doesn't support the precise filter, Ranger will return the
policy meets the
+ * wildcard(*,?) conditions, just like `*.*.*` policy will match
`db1.table1.column1` So we need
+ * to manually precisely filter the policies.
+ */
+ // Search Ranger policy filter keys
+ protected List policyFilterKeys = null;
+ // Search Ranger policy precise filter keys
+ protected List policyPreciseFilterKeys = null;
+
+ public static final String MANAGED_BY_GRAVITINO = "MANAGED_BY_GRAVITINO";
+
+ // TODO: Maybe need to move to the configuration in the future
+ public static final String RANGER_ADMIN_NAME = "admin";
+
+ public RangerAuthorizationPlugin(String catalogProvider, Map
config) {
+super();
+this.catalogProvider = catalogProvider;
+String rangerUrl =
config.get(AuthorizationPropertiesMeta.RANGER_ADMIN_URL);
+String authType = config.get(AuthorizationPropertiesMeta.RANGER_AUTH_TYPE);
+String username = config.get(AuthorizationPropertiesMeta.RANGER_USERNAME);
+// Apache Ranger Password should be minimum 8 characters with min one
alphabet and one numeric.
+String password = config.get(AuthorizationPro
Re: [PR] [#3963] feat(core): Apache Ranger Hive authorization pushdown [gravitino]
jerqi commented on code in PR #4515:
URL: https://github.com/apache/gravitino/pull/4515#discussion_r1724198950
##
authorizations/authorization-ranger/src/main/java/org/apache/gravitino/authorization/ranger/RangerAuthorizationPlugin.java:
##
@@ -0,0 +1,1006 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+package org.apache.gravitino.authorization.ranger;
+
+import com.google.common.annotations.VisibleForTesting;
+import com.google.common.collect.ImmutableMap;
+import com.google.common.collect.Lists;
+import com.google.errorprone.annotations.FormatMethod;
+import com.google.errorprone.annotations.FormatString;
+import java.io.IOException;
+import java.time.Instant;
+import java.util.Arrays;
+import java.util.Collections;
+import java.util.HashMap;
+import java.util.HashSet;
+import java.util.List;
+import java.util.Map;
+import java.util.Objects;
+import java.util.Set;
+import java.util.stream.Collectors;
+import org.apache.gravitino.MetadataObject;
+import org.apache.gravitino.authorization.Group;
+import org.apache.gravitino.authorization.Owner;
+import org.apache.gravitino.authorization.Privilege;
+import org.apache.gravitino.authorization.Role;
+import org.apache.gravitino.authorization.RoleChange;
+import org.apache.gravitino.authorization.SecurableObject;
+import org.apache.gravitino.authorization.SecurableObjects;
+import org.apache.gravitino.authorization.User;
+import org.apache.gravitino.authorization.ranger.defines.VXGroup;
+import org.apache.gravitino.authorization.ranger.defines.VXGroupList;
+import org.apache.gravitino.authorization.ranger.defines.VXUser;
+import org.apache.gravitino.authorization.ranger.defines.VXUserList;
+import org.apache.gravitino.connector.AuthorizationPropertiesMeta;
+import org.apache.gravitino.connector.authorization.AuthorizationPlugin;
+import org.apache.gravitino.exceptions.AuthorizationPluginException;
+import org.apache.gravitino.meta.AuditInfo;
+import org.apache.gravitino.meta.GroupEntity;
+import org.apache.gravitino.meta.UserEntity;
+import org.apache.gravitino.utils.PrincipalUtils;
+import org.apache.ranger.RangerServiceException;
+import org.apache.ranger.plugin.model.RangerPolicy;
+import org.apache.ranger.plugin.model.RangerRole;
+import org.apache.ranger.plugin.util.GrantRevokeRoleRequest;
+import org.apache.ranger.plugin.util.SearchFilter;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+/** Ranger authorization operations plugin abstract class. */
+public abstract class RangerAuthorizationPlugin implements AuthorizationPlugin
{
+ private static final Logger LOG =
LoggerFactory.getLogger(RangerAuthorizationPlugin.class);
+
+ protected String catalogProvider;
+ protected RangerClientExtend rangerClient;
+ protected String rangerServiceName;
+ /** Mapping Gravitino privilege name to the underlying authorization system
privileges. */
+ protected Map> privilegesMapping = null;
+ // The owner privileges, the owner can do anything on the metadata object
+ protected Set ownerPrivileges = null;
+
+ /**
+ * Because Ranger doesn't support the precise filter, Ranger will return the
policy meets the
+ * wildcard(*,?) conditions, just like `*.*.*` policy will match
`db1.table1.column1` So we need
+ * to manually precisely filter the policies.
+ */
+ // Search Ranger policy filter keys
+ protected List policyFilterKeys = null;
+ // Search Ranger policy precise filter keys
+ protected List policyPreciseFilterKeys = null;
+
+ public static final String MANAGED_BY_GRAVITINO = "MANAGED_BY_GRAVITINO";
+
+ // TODO: Maybe need to move to the configuration in the future
+ public static final String RANGER_ADMIN_NAME = "admin";
+
+ public RangerAuthorizationPlugin(String catalogProvider, Map
config) {
+super();
+this.catalogProvider = catalogProvider;
+String rangerUrl =
config.get(AuthorizationPropertiesMeta.RANGER_ADMIN_URL);
+String authType = config.get(AuthorizationPropertiesMeta.RANGER_AUTH_TYPE);
+String username = config.get(AuthorizationPropertiesMeta.RANGER_USERNAME);
+// Apache Ranger Password should be minimum 8 characters with min one
alphabet and one numeric.
+String password = config.get(AuthorizationPro
Re: [PR] [#3963] feat(core): Apache Ranger Hive authorization pushdown [gravitino]
jerqi commented on code in PR #4515:
URL: https://github.com/apache/gravitino/pull/4515#discussion_r1723297695
##
authorizations/authorization-ranger/src/main/java/org/apache/gravitino/authorization/ranger/RangerAuthorizationPlugin.java:
##
@@ -0,0 +1,1006 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+package org.apache.gravitino.authorization.ranger;
+
+import com.google.common.annotations.VisibleForTesting;
+import com.google.common.collect.ImmutableMap;
+import com.google.common.collect.Lists;
+import com.google.errorprone.annotations.FormatMethod;
+import com.google.errorprone.annotations.FormatString;
+import java.io.IOException;
+import java.time.Instant;
+import java.util.Arrays;
+import java.util.Collections;
+import java.util.HashMap;
+import java.util.HashSet;
+import java.util.List;
+import java.util.Map;
+import java.util.Objects;
+import java.util.Set;
+import java.util.stream.Collectors;
+import org.apache.gravitino.MetadataObject;
+import org.apache.gravitino.authorization.Group;
+import org.apache.gravitino.authorization.Owner;
+import org.apache.gravitino.authorization.Privilege;
+import org.apache.gravitino.authorization.Role;
+import org.apache.gravitino.authorization.RoleChange;
+import org.apache.gravitino.authorization.SecurableObject;
+import org.apache.gravitino.authorization.SecurableObjects;
+import org.apache.gravitino.authorization.User;
+import org.apache.gravitino.authorization.ranger.defines.VXGroup;
+import org.apache.gravitino.authorization.ranger.defines.VXGroupList;
+import org.apache.gravitino.authorization.ranger.defines.VXUser;
+import org.apache.gravitino.authorization.ranger.defines.VXUserList;
+import org.apache.gravitino.connector.AuthorizationPropertiesMeta;
+import org.apache.gravitino.connector.authorization.AuthorizationPlugin;
+import org.apache.gravitino.exceptions.AuthorizationPluginException;
+import org.apache.gravitino.meta.AuditInfo;
+import org.apache.gravitino.meta.GroupEntity;
+import org.apache.gravitino.meta.UserEntity;
+import org.apache.gravitino.utils.PrincipalUtils;
+import org.apache.ranger.RangerServiceException;
+import org.apache.ranger.plugin.model.RangerPolicy;
+import org.apache.ranger.plugin.model.RangerRole;
+import org.apache.ranger.plugin.util.GrantRevokeRoleRequest;
+import org.apache.ranger.plugin.util.SearchFilter;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+/** Ranger authorization operations plugin abstract class. */
+public abstract class RangerAuthorizationPlugin implements AuthorizationPlugin
{
+ private static final Logger LOG =
LoggerFactory.getLogger(RangerAuthorizationPlugin.class);
+
+ protected String catalogProvider;
+ protected RangerClientExtend rangerClient;
+ protected String rangerServiceName;
+ /** Mapping Gravitino privilege name to the underlying authorization system
privileges. */
+ protected Map> privilegesMapping = null;
+ // The owner privileges, the owner can do anything on the metadata object
+ protected Set ownerPrivileges = null;
+
+ /**
+ * Because Ranger doesn't support the precise filter, Ranger will return the
policy meets the
+ * wildcard(*,?) conditions, just like `*.*.*` policy will match
`db1.table1.column1` So we need
+ * to manually precisely filter the policies.
+ */
+ // Search Ranger policy filter keys
+ protected List policyFilterKeys = null;
+ // Search Ranger policy precise filter keys
+ protected List policyPreciseFilterKeys = null;
+
+ public static final String MANAGED_BY_GRAVITINO = "MANAGED_BY_GRAVITINO";
+
+ // TODO: Maybe need to move to the configuration in the future
+ public static final String RANGER_ADMIN_NAME = "admin";
+
+ public RangerAuthorizationPlugin(String catalogProvider, Map
config) {
+super();
+this.catalogProvider = catalogProvider;
+String rangerUrl =
config.get(AuthorizationPropertiesMeta.RANGER_ADMIN_URL);
+String authType = config.get(AuthorizationPropertiesMeta.RANGER_AUTH_TYPE);
+String username = config.get(AuthorizationPropertiesMeta.RANGER_USERNAME);
+// Apache Ranger Password should be minimum 8 characters with min one
alphabet and one numeric.
+String password = config.get(AuthorizationPro
Re: [PR] [#3963] feat(core): Apache Ranger Hive authorization pushdown [gravitino]
jerqi commented on code in PR #4515:
URL: https://github.com/apache/gravitino/pull/4515#discussion_r1723290279
##
authorizations/authorization-ranger/src/main/java/org/apache/gravitino/authorization/ranger/RangerAuthorizationPlugin.java:
##
@@ -0,0 +1,1006 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+package org.apache.gravitino.authorization.ranger;
+
+import com.google.common.annotations.VisibleForTesting;
+import com.google.common.collect.ImmutableMap;
+import com.google.common.collect.Lists;
+import com.google.errorprone.annotations.FormatMethod;
+import com.google.errorprone.annotations.FormatString;
+import java.io.IOException;
+import java.time.Instant;
+import java.util.Arrays;
+import java.util.Collections;
+import java.util.HashMap;
+import java.util.HashSet;
+import java.util.List;
+import java.util.Map;
+import java.util.Objects;
+import java.util.Set;
+import java.util.stream.Collectors;
+import org.apache.gravitino.MetadataObject;
+import org.apache.gravitino.authorization.Group;
+import org.apache.gravitino.authorization.Owner;
+import org.apache.gravitino.authorization.Privilege;
+import org.apache.gravitino.authorization.Role;
+import org.apache.gravitino.authorization.RoleChange;
+import org.apache.gravitino.authorization.SecurableObject;
+import org.apache.gravitino.authorization.SecurableObjects;
+import org.apache.gravitino.authorization.User;
+import org.apache.gravitino.authorization.ranger.defines.VXGroup;
+import org.apache.gravitino.authorization.ranger.defines.VXGroupList;
+import org.apache.gravitino.authorization.ranger.defines.VXUser;
+import org.apache.gravitino.authorization.ranger.defines.VXUserList;
+import org.apache.gravitino.connector.AuthorizationPropertiesMeta;
+import org.apache.gravitino.connector.authorization.AuthorizationPlugin;
+import org.apache.gravitino.exceptions.AuthorizationPluginException;
+import org.apache.gravitino.meta.AuditInfo;
+import org.apache.gravitino.meta.GroupEntity;
+import org.apache.gravitino.meta.UserEntity;
+import org.apache.gravitino.utils.PrincipalUtils;
+import org.apache.ranger.RangerServiceException;
+import org.apache.ranger.plugin.model.RangerPolicy;
+import org.apache.ranger.plugin.model.RangerRole;
+import org.apache.ranger.plugin.util.GrantRevokeRoleRequest;
+import org.apache.ranger.plugin.util.SearchFilter;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+/** Ranger authorization operations plugin abstract class. */
+public abstract class RangerAuthorizationPlugin implements AuthorizationPlugin
{
+ private static final Logger LOG =
LoggerFactory.getLogger(RangerAuthorizationPlugin.class);
+
+ protected String catalogProvider;
+ protected RangerClientExtend rangerClient;
+ protected String rangerServiceName;
+ /** Mapping Gravitino privilege name to the underlying authorization system
privileges. */
+ protected Map> privilegesMapping = null;
+ // The owner privileges, the owner can do anything on the metadata object
+ protected Set ownerPrivileges = null;
+
+ /**
+ * Because Ranger doesn't support the precise filter, Ranger will return the
policy meets the
+ * wildcard(*,?) conditions, just like `*.*.*` policy will match
`db1.table1.column1` So we need
+ * to manually precisely filter the policies.
+ */
+ // Search Ranger policy filter keys
+ protected List policyFilterKeys = null;
+ // Search Ranger policy precise filter keys
+ protected List policyPreciseFilterKeys = null;
+
+ public static final String MANAGED_BY_GRAVITINO = "MANAGED_BY_GRAVITINO";
+
+ // TODO: Maybe need to move to the configuration in the future
+ public static final String RANGER_ADMIN_NAME = "admin";
+
+ public RangerAuthorizationPlugin(String catalogProvider, Map
config) {
+super();
+this.catalogProvider = catalogProvider;
+String rangerUrl =
config.get(AuthorizationPropertiesMeta.RANGER_ADMIN_URL);
+String authType = config.get(AuthorizationPropertiesMeta.RANGER_AUTH_TYPE);
+String username = config.get(AuthorizationPropertiesMeta.RANGER_USERNAME);
+// Apache Ranger Password should be minimum 8 characters with min one
alphabet and one numeric.
+String password = config.get(AuthorizationPro
Re: [PR] [#3963] feat(core): Apache Ranger Hive authorization pushdown [gravitino]
jerqi commented on code in PR #4515:
URL: https://github.com/apache/gravitino/pull/4515#discussion_r1723288336
##
authorizations/authorization-ranger/src/main/java/org/apache/gravitino/authorization/ranger/RangerAuthorizationPlugin.java:
##
@@ -0,0 +1,1006 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+package org.apache.gravitino.authorization.ranger;
+
+import com.google.common.annotations.VisibleForTesting;
+import com.google.common.collect.ImmutableMap;
+import com.google.common.collect.Lists;
+import com.google.errorprone.annotations.FormatMethod;
+import com.google.errorprone.annotations.FormatString;
+import java.io.IOException;
+import java.time.Instant;
+import java.util.Arrays;
+import java.util.Collections;
+import java.util.HashMap;
+import java.util.HashSet;
+import java.util.List;
+import java.util.Map;
+import java.util.Objects;
+import java.util.Set;
+import java.util.stream.Collectors;
+import org.apache.gravitino.MetadataObject;
+import org.apache.gravitino.authorization.Group;
+import org.apache.gravitino.authorization.Owner;
+import org.apache.gravitino.authorization.Privilege;
+import org.apache.gravitino.authorization.Role;
+import org.apache.gravitino.authorization.RoleChange;
+import org.apache.gravitino.authorization.SecurableObject;
+import org.apache.gravitino.authorization.SecurableObjects;
+import org.apache.gravitino.authorization.User;
+import org.apache.gravitino.authorization.ranger.defines.VXGroup;
+import org.apache.gravitino.authorization.ranger.defines.VXGroupList;
+import org.apache.gravitino.authorization.ranger.defines.VXUser;
+import org.apache.gravitino.authorization.ranger.defines.VXUserList;
+import org.apache.gravitino.connector.AuthorizationPropertiesMeta;
+import org.apache.gravitino.connector.authorization.AuthorizationPlugin;
+import org.apache.gravitino.exceptions.AuthorizationPluginException;
+import org.apache.gravitino.meta.AuditInfo;
+import org.apache.gravitino.meta.GroupEntity;
+import org.apache.gravitino.meta.UserEntity;
+import org.apache.gravitino.utils.PrincipalUtils;
+import org.apache.ranger.RangerServiceException;
+import org.apache.ranger.plugin.model.RangerPolicy;
+import org.apache.ranger.plugin.model.RangerRole;
+import org.apache.ranger.plugin.util.GrantRevokeRoleRequest;
+import org.apache.ranger.plugin.util.SearchFilter;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+/** Ranger authorization operations plugin abstract class. */
+public abstract class RangerAuthorizationPlugin implements AuthorizationPlugin
{
+ private static final Logger LOG =
LoggerFactory.getLogger(RangerAuthorizationPlugin.class);
+
+ protected String catalogProvider;
+ protected RangerClientExtend rangerClient;
+ protected String rangerServiceName;
+ /** Mapping Gravitino privilege name to the underlying authorization system
privileges. */
+ protected Map> privilegesMapping = null;
+ // The owner privileges, the owner can do anything on the metadata object
+ protected Set ownerPrivileges = null;
+
+ /**
+ * Because Ranger doesn't support the precise filter, Ranger will return the
policy meets the
+ * wildcard(*,?) conditions, just like `*.*.*` policy will match
`db1.table1.column1` So we need
+ * to manually precisely filter the policies.
+ */
+ // Search Ranger policy filter keys
+ protected List policyFilterKeys = null;
+ // Search Ranger policy precise filter keys
+ protected List policyPreciseFilterKeys = null;
+
+ public static final String MANAGED_BY_GRAVITINO = "MANAGED_BY_GRAVITINO";
+
+ // TODO: Maybe need to move to the configuration in the future
+ public static final String RANGER_ADMIN_NAME = "admin";
+
+ public RangerAuthorizationPlugin(String catalogProvider, Map
config) {
+super();
+this.catalogProvider = catalogProvider;
+String rangerUrl =
config.get(AuthorizationPropertiesMeta.RANGER_ADMIN_URL);
+String authType = config.get(AuthorizationPropertiesMeta.RANGER_AUTH_TYPE);
+String username = config.get(AuthorizationPropertiesMeta.RANGER_USERNAME);
+// Apache Ranger Password should be minimum 8 characters with min one
alphabet and one numeric.
+String password = config.get(AuthorizationPro
Re: [PR] [#3963] feat(core): Apache Ranger Hive authorization pushdown [gravitino]
jerqi commented on code in PR #4515:
URL: https://github.com/apache/gravitino/pull/4515#discussion_r1723289584
##
authorizations/authorization-ranger/src/main/java/org/apache/gravitino/authorization/ranger/RangerAuthorizationPlugin.java:
##
@@ -0,0 +1,1006 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+package org.apache.gravitino.authorization.ranger;
+
+import com.google.common.annotations.VisibleForTesting;
+import com.google.common.collect.ImmutableMap;
+import com.google.common.collect.Lists;
+import com.google.errorprone.annotations.FormatMethod;
+import com.google.errorprone.annotations.FormatString;
+import java.io.IOException;
+import java.time.Instant;
+import java.util.Arrays;
+import java.util.Collections;
+import java.util.HashMap;
+import java.util.HashSet;
+import java.util.List;
+import java.util.Map;
+import java.util.Objects;
+import java.util.Set;
+import java.util.stream.Collectors;
+import org.apache.gravitino.MetadataObject;
+import org.apache.gravitino.authorization.Group;
+import org.apache.gravitino.authorization.Owner;
+import org.apache.gravitino.authorization.Privilege;
+import org.apache.gravitino.authorization.Role;
+import org.apache.gravitino.authorization.RoleChange;
+import org.apache.gravitino.authorization.SecurableObject;
+import org.apache.gravitino.authorization.SecurableObjects;
+import org.apache.gravitino.authorization.User;
+import org.apache.gravitino.authorization.ranger.defines.VXGroup;
+import org.apache.gravitino.authorization.ranger.defines.VXGroupList;
+import org.apache.gravitino.authorization.ranger.defines.VXUser;
+import org.apache.gravitino.authorization.ranger.defines.VXUserList;
+import org.apache.gravitino.connector.AuthorizationPropertiesMeta;
+import org.apache.gravitino.connector.authorization.AuthorizationPlugin;
+import org.apache.gravitino.exceptions.AuthorizationPluginException;
+import org.apache.gravitino.meta.AuditInfo;
+import org.apache.gravitino.meta.GroupEntity;
+import org.apache.gravitino.meta.UserEntity;
+import org.apache.gravitino.utils.PrincipalUtils;
+import org.apache.ranger.RangerServiceException;
+import org.apache.ranger.plugin.model.RangerPolicy;
+import org.apache.ranger.plugin.model.RangerRole;
+import org.apache.ranger.plugin.util.GrantRevokeRoleRequest;
+import org.apache.ranger.plugin.util.SearchFilter;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+/** Ranger authorization operations plugin abstract class. */
+public abstract class RangerAuthorizationPlugin implements AuthorizationPlugin
{
+ private static final Logger LOG =
LoggerFactory.getLogger(RangerAuthorizationPlugin.class);
+
+ protected String catalogProvider;
+ protected RangerClientExtend rangerClient;
+ protected String rangerServiceName;
+ /** Mapping Gravitino privilege name to the underlying authorization system
privileges. */
+ protected Map> privilegesMapping = null;
+ // The owner privileges, the owner can do anything on the metadata object
+ protected Set ownerPrivileges = null;
+
+ /**
+ * Because Ranger doesn't support the precise filter, Ranger will return the
policy meets the
+ * wildcard(*,?) conditions, just like `*.*.*` policy will match
`db1.table1.column1` So we need
+ * to manually precisely filter the policies.
+ */
+ // Search Ranger policy filter keys
+ protected List policyFilterKeys = null;
+ // Search Ranger policy precise filter keys
+ protected List policyPreciseFilterKeys = null;
+
+ public static final String MANAGED_BY_GRAVITINO = "MANAGED_BY_GRAVITINO";
+
+ // TODO: Maybe need to move to the configuration in the future
+ public static final String RANGER_ADMIN_NAME = "admin";
+
+ public RangerAuthorizationPlugin(String catalogProvider, Map
config) {
+super();
+this.catalogProvider = catalogProvider;
+String rangerUrl =
config.get(AuthorizationPropertiesMeta.RANGER_ADMIN_URL);
+String authType = config.get(AuthorizationPropertiesMeta.RANGER_AUTH_TYPE);
+String username = config.get(AuthorizationPropertiesMeta.RANGER_USERNAME);
+// Apache Ranger Password should be minimum 8 characters with min one
alphabet and one numeric.
+String password = config.get(AuthorizationPro
Re: [PR] [#3963] feat(core): Apache Ranger Hive authorization pushdown [gravitino]
jerqi commented on code in PR #4515:
URL: https://github.com/apache/gravitino/pull/4515#discussion_r1723289010
##
authorizations/authorization-ranger/src/main/java/org/apache/gravitino/authorization/ranger/RangerAuthorizationPlugin.java:
##
@@ -0,0 +1,1006 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+package org.apache.gravitino.authorization.ranger;
+
+import com.google.common.annotations.VisibleForTesting;
+import com.google.common.collect.ImmutableMap;
+import com.google.common.collect.Lists;
+import com.google.errorprone.annotations.FormatMethod;
+import com.google.errorprone.annotations.FormatString;
+import java.io.IOException;
+import java.time.Instant;
+import java.util.Arrays;
+import java.util.Collections;
+import java.util.HashMap;
+import java.util.HashSet;
+import java.util.List;
+import java.util.Map;
+import java.util.Objects;
+import java.util.Set;
+import java.util.stream.Collectors;
+import org.apache.gravitino.MetadataObject;
+import org.apache.gravitino.authorization.Group;
+import org.apache.gravitino.authorization.Owner;
+import org.apache.gravitino.authorization.Privilege;
+import org.apache.gravitino.authorization.Role;
+import org.apache.gravitino.authorization.RoleChange;
+import org.apache.gravitino.authorization.SecurableObject;
+import org.apache.gravitino.authorization.SecurableObjects;
+import org.apache.gravitino.authorization.User;
+import org.apache.gravitino.authorization.ranger.defines.VXGroup;
+import org.apache.gravitino.authorization.ranger.defines.VXGroupList;
+import org.apache.gravitino.authorization.ranger.defines.VXUser;
+import org.apache.gravitino.authorization.ranger.defines.VXUserList;
+import org.apache.gravitino.connector.AuthorizationPropertiesMeta;
+import org.apache.gravitino.connector.authorization.AuthorizationPlugin;
+import org.apache.gravitino.exceptions.AuthorizationPluginException;
+import org.apache.gravitino.meta.AuditInfo;
+import org.apache.gravitino.meta.GroupEntity;
+import org.apache.gravitino.meta.UserEntity;
+import org.apache.gravitino.utils.PrincipalUtils;
+import org.apache.ranger.RangerServiceException;
+import org.apache.ranger.plugin.model.RangerPolicy;
+import org.apache.ranger.plugin.model.RangerRole;
+import org.apache.ranger.plugin.util.GrantRevokeRoleRequest;
+import org.apache.ranger.plugin.util.SearchFilter;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+/** Ranger authorization operations plugin abstract class. */
+public abstract class RangerAuthorizationPlugin implements AuthorizationPlugin
{
+ private static final Logger LOG =
LoggerFactory.getLogger(RangerAuthorizationPlugin.class);
+
+ protected String catalogProvider;
+ protected RangerClientExtend rangerClient;
+ protected String rangerServiceName;
+ /** Mapping Gravitino privilege name to the underlying authorization system
privileges. */
+ protected Map> privilegesMapping = null;
+ // The owner privileges, the owner can do anything on the metadata object
+ protected Set ownerPrivileges = null;
+
+ /**
+ * Because Ranger doesn't support the precise filter, Ranger will return the
policy meets the
+ * wildcard(*,?) conditions, just like `*.*.*` policy will match
`db1.table1.column1` So we need
+ * to manually precisely filter the policies.
+ */
+ // Search Ranger policy filter keys
+ protected List policyFilterKeys = null;
+ // Search Ranger policy precise filter keys
+ protected List policyPreciseFilterKeys = null;
+
+ public static final String MANAGED_BY_GRAVITINO = "MANAGED_BY_GRAVITINO";
+
+ // TODO: Maybe need to move to the configuration in the future
+ public static final String RANGER_ADMIN_NAME = "admin";
+
+ public RangerAuthorizationPlugin(String catalogProvider, Map
config) {
+super();
+this.catalogProvider = catalogProvider;
+String rangerUrl =
config.get(AuthorizationPropertiesMeta.RANGER_ADMIN_URL);
+String authType = config.get(AuthorizationPropertiesMeta.RANGER_AUTH_TYPE);
+String username = config.get(AuthorizationPropertiesMeta.RANGER_USERNAME);
+// Apache Ranger Password should be minimum 8 characters with min one
alphabet and one numeric.
+String password = config.get(AuthorizationPro
Re: [PR] [#3963] feat(core): Apache Ranger Hive authorization pushdown [gravitino]
jerqi commented on code in PR #4515:
URL: https://github.com/apache/gravitino/pull/4515#discussion_r1723286789
##
authorizations/authorization-ranger/src/main/java/org/apache/gravitino/authorization/ranger/RangerAuthorizationPlugin.java:
##
@@ -0,0 +1,1006 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+package org.apache.gravitino.authorization.ranger;
+
+import com.google.common.annotations.VisibleForTesting;
+import com.google.common.collect.ImmutableMap;
+import com.google.common.collect.Lists;
+import com.google.errorprone.annotations.FormatMethod;
+import com.google.errorprone.annotations.FormatString;
+import java.io.IOException;
+import java.time.Instant;
+import java.util.Arrays;
+import java.util.Collections;
+import java.util.HashMap;
+import java.util.HashSet;
+import java.util.List;
+import java.util.Map;
+import java.util.Objects;
+import java.util.Set;
+import java.util.stream.Collectors;
+import org.apache.gravitino.MetadataObject;
+import org.apache.gravitino.authorization.Group;
+import org.apache.gravitino.authorization.Owner;
+import org.apache.gravitino.authorization.Privilege;
+import org.apache.gravitino.authorization.Role;
+import org.apache.gravitino.authorization.RoleChange;
+import org.apache.gravitino.authorization.SecurableObject;
+import org.apache.gravitino.authorization.SecurableObjects;
+import org.apache.gravitino.authorization.User;
+import org.apache.gravitino.authorization.ranger.defines.VXGroup;
+import org.apache.gravitino.authorization.ranger.defines.VXGroupList;
+import org.apache.gravitino.authorization.ranger.defines.VXUser;
+import org.apache.gravitino.authorization.ranger.defines.VXUserList;
+import org.apache.gravitino.connector.AuthorizationPropertiesMeta;
+import org.apache.gravitino.connector.authorization.AuthorizationPlugin;
+import org.apache.gravitino.exceptions.AuthorizationPluginException;
+import org.apache.gravitino.meta.AuditInfo;
+import org.apache.gravitino.meta.GroupEntity;
+import org.apache.gravitino.meta.UserEntity;
+import org.apache.gravitino.utils.PrincipalUtils;
+import org.apache.ranger.RangerServiceException;
+import org.apache.ranger.plugin.model.RangerPolicy;
+import org.apache.ranger.plugin.model.RangerRole;
+import org.apache.ranger.plugin.util.GrantRevokeRoleRequest;
+import org.apache.ranger.plugin.util.SearchFilter;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+/** Ranger authorization operations plugin abstract class. */
+public abstract class RangerAuthorizationPlugin implements AuthorizationPlugin
{
+ private static final Logger LOG =
LoggerFactory.getLogger(RangerAuthorizationPlugin.class);
+
+ protected String catalogProvider;
+ protected RangerClientExtend rangerClient;
+ protected String rangerServiceName;
+ /** Mapping Gravitino privilege name to the underlying authorization system
privileges. */
+ protected Map> privilegesMapping = null;
+ // The owner privileges, the owner can do anything on the metadata object
+ protected Set ownerPrivileges = null;
+
+ /**
+ * Because Ranger doesn't support the precise filter, Ranger will return the
policy meets the
+ * wildcard(*,?) conditions, just like `*.*.*` policy will match
`db1.table1.column1` So we need
+ * to manually precisely filter the policies.
+ */
+ // Search Ranger policy filter keys
+ protected List policyFilterKeys = null;
+ // Search Ranger policy precise filter keys
+ protected List policyPreciseFilterKeys = null;
+
+ public static final String MANAGED_BY_GRAVITINO = "MANAGED_BY_GRAVITINO";
+
+ // TODO: Maybe need to move to the configuration in the future
+ public static final String RANGER_ADMIN_NAME = "admin";
+
+ public RangerAuthorizationPlugin(String catalogProvider, Map
config) {
+super();
+this.catalogProvider = catalogProvider;
+String rangerUrl =
config.get(AuthorizationPropertiesMeta.RANGER_ADMIN_URL);
+String authType = config.get(AuthorizationPropertiesMeta.RANGER_AUTH_TYPE);
+String username = config.get(AuthorizationPropertiesMeta.RANGER_USERNAME);
+// Apache Ranger Password should be minimum 8 characters with min one
alphabet and one numeric.
+String password = config.get(AuthorizationPro
Re: [PR] [#3963] feat(core): Apache Ranger Hive authorization pushdown [gravitino]
jerqi commented on code in PR #4515:
URL: https://github.com/apache/gravitino/pull/4515#discussion_r1723284628
##
authorizations/authorization-ranger/src/main/java/org/apache/gravitino/authorization/ranger/RangerAuthorizationPlugin.java:
##
@@ -0,0 +1,1006 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+package org.apache.gravitino.authorization.ranger;
+
+import com.google.common.annotations.VisibleForTesting;
+import com.google.common.collect.ImmutableMap;
+import com.google.common.collect.Lists;
+import com.google.errorprone.annotations.FormatMethod;
+import com.google.errorprone.annotations.FormatString;
+import java.io.IOException;
+import java.time.Instant;
+import java.util.Arrays;
+import java.util.Collections;
+import java.util.HashMap;
+import java.util.HashSet;
+import java.util.List;
+import java.util.Map;
+import java.util.Objects;
+import java.util.Set;
+import java.util.stream.Collectors;
+import org.apache.gravitino.MetadataObject;
+import org.apache.gravitino.authorization.Group;
+import org.apache.gravitino.authorization.Owner;
+import org.apache.gravitino.authorization.Privilege;
+import org.apache.gravitino.authorization.Role;
+import org.apache.gravitino.authorization.RoleChange;
+import org.apache.gravitino.authorization.SecurableObject;
+import org.apache.gravitino.authorization.SecurableObjects;
+import org.apache.gravitino.authorization.User;
+import org.apache.gravitino.authorization.ranger.defines.VXGroup;
+import org.apache.gravitino.authorization.ranger.defines.VXGroupList;
+import org.apache.gravitino.authorization.ranger.defines.VXUser;
+import org.apache.gravitino.authorization.ranger.defines.VXUserList;
+import org.apache.gravitino.connector.AuthorizationPropertiesMeta;
+import org.apache.gravitino.connector.authorization.AuthorizationPlugin;
+import org.apache.gravitino.exceptions.AuthorizationPluginException;
+import org.apache.gravitino.meta.AuditInfo;
+import org.apache.gravitino.meta.GroupEntity;
+import org.apache.gravitino.meta.UserEntity;
+import org.apache.gravitino.utils.PrincipalUtils;
+import org.apache.ranger.RangerServiceException;
+import org.apache.ranger.plugin.model.RangerPolicy;
+import org.apache.ranger.plugin.model.RangerRole;
+import org.apache.ranger.plugin.util.GrantRevokeRoleRequest;
+import org.apache.ranger.plugin.util.SearchFilter;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+/** Ranger authorization operations plugin abstract class. */
+public abstract class RangerAuthorizationPlugin implements AuthorizationPlugin
{
+ private static final Logger LOG =
LoggerFactory.getLogger(RangerAuthorizationPlugin.class);
+
+ protected String catalogProvider;
+ protected RangerClientExtend rangerClient;
+ protected String rangerServiceName;
+ /** Mapping Gravitino privilege name to the underlying authorization system
privileges. */
+ protected Map> privilegesMapping = null;
+ // The owner privileges, the owner can do anything on the metadata object
+ protected Set ownerPrivileges = null;
+
+ /**
+ * Because Ranger doesn't support the precise filter, Ranger will return the
policy meets the
+ * wildcard(*,?) conditions, just like `*.*.*` policy will match
`db1.table1.column1` So we need
+ * to manually precisely filter the policies.
+ */
+ // Search Ranger policy filter keys
+ protected List policyFilterKeys = null;
+ // Search Ranger policy precise filter keys
+ protected List policyPreciseFilterKeys = null;
+
+ public static final String MANAGED_BY_GRAVITINO = "MANAGED_BY_GRAVITINO";
+
+ // TODO: Maybe need to move to the configuration in the future
+ public static final String RANGER_ADMIN_NAME = "admin";
+
+ public RangerAuthorizationPlugin(String catalogProvider, Map
config) {
+super();
+this.catalogProvider = catalogProvider;
+String rangerUrl =
config.get(AuthorizationPropertiesMeta.RANGER_ADMIN_URL);
+String authType = config.get(AuthorizationPropertiesMeta.RANGER_AUTH_TYPE);
+String username = config.get(AuthorizationPropertiesMeta.RANGER_USERNAME);
+// Apache Ranger Password should be minimum 8 characters with min one
alphabet and one numeric.
+String password = config.get(AuthorizationPro
Re: [PR] [#3963] feat(core): Apache Ranger Hive authorization pushdown [gravitino]
jerqi commented on code in PR #4515:
URL: https://github.com/apache/gravitino/pull/4515#discussion_r1723281849
##
authorizations/authorization-ranger/src/main/java/org/apache/gravitino/authorization/ranger/RangerAuthorizationPlugin.java:
##
@@ -0,0 +1,1006 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+package org.apache.gravitino.authorization.ranger;
+
+import com.google.common.annotations.VisibleForTesting;
+import com.google.common.collect.ImmutableMap;
+import com.google.common.collect.Lists;
+import com.google.errorprone.annotations.FormatMethod;
+import com.google.errorprone.annotations.FormatString;
+import java.io.IOException;
+import java.time.Instant;
+import java.util.Arrays;
+import java.util.Collections;
+import java.util.HashMap;
+import java.util.HashSet;
+import java.util.List;
+import java.util.Map;
+import java.util.Objects;
+import java.util.Set;
+import java.util.stream.Collectors;
+import org.apache.gravitino.MetadataObject;
+import org.apache.gravitino.authorization.Group;
+import org.apache.gravitino.authorization.Owner;
+import org.apache.gravitino.authorization.Privilege;
+import org.apache.gravitino.authorization.Role;
+import org.apache.gravitino.authorization.RoleChange;
+import org.apache.gravitino.authorization.SecurableObject;
+import org.apache.gravitino.authorization.SecurableObjects;
+import org.apache.gravitino.authorization.User;
+import org.apache.gravitino.authorization.ranger.defines.VXGroup;
+import org.apache.gravitino.authorization.ranger.defines.VXGroupList;
+import org.apache.gravitino.authorization.ranger.defines.VXUser;
+import org.apache.gravitino.authorization.ranger.defines.VXUserList;
+import org.apache.gravitino.connector.AuthorizationPropertiesMeta;
+import org.apache.gravitino.connector.authorization.AuthorizationPlugin;
+import org.apache.gravitino.exceptions.AuthorizationPluginException;
+import org.apache.gravitino.meta.AuditInfo;
+import org.apache.gravitino.meta.GroupEntity;
+import org.apache.gravitino.meta.UserEntity;
+import org.apache.gravitino.utils.PrincipalUtils;
+import org.apache.ranger.RangerServiceException;
+import org.apache.ranger.plugin.model.RangerPolicy;
+import org.apache.ranger.plugin.model.RangerRole;
+import org.apache.ranger.plugin.util.GrantRevokeRoleRequest;
+import org.apache.ranger.plugin.util.SearchFilter;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+/** Ranger authorization operations plugin abstract class. */
+public abstract class RangerAuthorizationPlugin implements AuthorizationPlugin
{
+ private static final Logger LOG =
LoggerFactory.getLogger(RangerAuthorizationPlugin.class);
+
+ protected String catalogProvider;
+ protected RangerClientExtend rangerClient;
+ protected String rangerServiceName;
+ /** Mapping Gravitino privilege name to the underlying authorization system
privileges. */
+ protected Map> privilegesMapping = null;
+ // The owner privileges, the owner can do anything on the metadata object
+ protected Set ownerPrivileges = null;
+
+ /**
+ * Because Ranger doesn't support the precise filter, Ranger will return the
policy meets the
+ * wildcard(*,?) conditions, just like `*.*.*` policy will match
`db1.table1.column1` So we need
+ * to manually precisely filter the policies.
+ */
+ // Search Ranger policy filter keys
+ protected List policyFilterKeys = null;
+ // Search Ranger policy precise filter keys
+ protected List policyPreciseFilterKeys = null;
+
+ public static final String MANAGED_BY_GRAVITINO = "MANAGED_BY_GRAVITINO";
+
+ // TODO: Maybe need to move to the configuration in the future
+ public static final String RANGER_ADMIN_NAME = "admin";
+
+ public RangerAuthorizationPlugin(String catalogProvider, Map
config) {
+super();
+this.catalogProvider = catalogProvider;
+String rangerUrl =
config.get(AuthorizationPropertiesMeta.RANGER_ADMIN_URL);
+String authType = config.get(AuthorizationPropertiesMeta.RANGER_AUTH_TYPE);
+String username = config.get(AuthorizationPropertiesMeta.RANGER_USERNAME);
+// Apache Ranger Password should be minimum 8 characters with min one
alphabet and one numeric.
+String password = config.get(AuthorizationPro
Re: [PR] [#3963] feat(core): Apache Ranger Hive authorization pushdown [gravitino]
jerqi commented on code in PR #4515:
URL: https://github.com/apache/gravitino/pull/4515#discussion_r1723281358
##
authorizations/authorization-ranger/src/main/java/org/apache/gravitino/authorization/ranger/RangerAuthorizationPlugin.java:
##
@@ -0,0 +1,1006 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+package org.apache.gravitino.authorization.ranger;
+
+import com.google.common.annotations.VisibleForTesting;
+import com.google.common.collect.ImmutableMap;
+import com.google.common.collect.Lists;
+import com.google.errorprone.annotations.FormatMethod;
+import com.google.errorprone.annotations.FormatString;
+import java.io.IOException;
+import java.time.Instant;
+import java.util.Arrays;
+import java.util.Collections;
+import java.util.HashMap;
+import java.util.HashSet;
+import java.util.List;
+import java.util.Map;
+import java.util.Objects;
+import java.util.Set;
+import java.util.stream.Collectors;
+import org.apache.gravitino.MetadataObject;
+import org.apache.gravitino.authorization.Group;
+import org.apache.gravitino.authorization.Owner;
+import org.apache.gravitino.authorization.Privilege;
+import org.apache.gravitino.authorization.Role;
+import org.apache.gravitino.authorization.RoleChange;
+import org.apache.gravitino.authorization.SecurableObject;
+import org.apache.gravitino.authorization.SecurableObjects;
+import org.apache.gravitino.authorization.User;
+import org.apache.gravitino.authorization.ranger.defines.VXGroup;
+import org.apache.gravitino.authorization.ranger.defines.VXGroupList;
+import org.apache.gravitino.authorization.ranger.defines.VXUser;
+import org.apache.gravitino.authorization.ranger.defines.VXUserList;
+import org.apache.gravitino.connector.AuthorizationPropertiesMeta;
+import org.apache.gravitino.connector.authorization.AuthorizationPlugin;
+import org.apache.gravitino.exceptions.AuthorizationPluginException;
+import org.apache.gravitino.meta.AuditInfo;
+import org.apache.gravitino.meta.GroupEntity;
+import org.apache.gravitino.meta.UserEntity;
+import org.apache.gravitino.utils.PrincipalUtils;
+import org.apache.ranger.RangerServiceException;
+import org.apache.ranger.plugin.model.RangerPolicy;
+import org.apache.ranger.plugin.model.RangerRole;
+import org.apache.ranger.plugin.util.GrantRevokeRoleRequest;
+import org.apache.ranger.plugin.util.SearchFilter;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+/** Ranger authorization operations plugin abstract class. */
+public abstract class RangerAuthorizationPlugin implements AuthorizationPlugin
{
+ private static final Logger LOG =
LoggerFactory.getLogger(RangerAuthorizationPlugin.class);
+
+ protected String catalogProvider;
+ protected RangerClientExtend rangerClient;
+ protected String rangerServiceName;
+ /** Mapping Gravitino privilege name to the underlying authorization system
privileges. */
+ protected Map> privilegesMapping = null;
+ // The owner privileges, the owner can do anything on the metadata object
+ protected Set ownerPrivileges = null;
+
+ /**
+ * Because Ranger doesn't support the precise filter, Ranger will return the
policy meets the
+ * wildcard(*,?) conditions, just like `*.*.*` policy will match
`db1.table1.column1` So we need
+ * to manually precisely filter the policies.
+ */
+ // Search Ranger policy filter keys
+ protected List policyFilterKeys = null;
+ // Search Ranger policy precise filter keys
+ protected List policyPreciseFilterKeys = null;
+
+ public static final String MANAGED_BY_GRAVITINO = "MANAGED_BY_GRAVITINO";
+
+ // TODO: Maybe need to move to the configuration in the future
+ public static final String RANGER_ADMIN_NAME = "admin";
+
+ public RangerAuthorizationPlugin(String catalogProvider, Map
config) {
+super();
+this.catalogProvider = catalogProvider;
+String rangerUrl =
config.get(AuthorizationPropertiesMeta.RANGER_ADMIN_URL);
+String authType = config.get(AuthorizationPropertiesMeta.RANGER_AUTH_TYPE);
+String username = config.get(AuthorizationPropertiesMeta.RANGER_USERNAME);
+// Apache Ranger Password should be minimum 8 characters with min one
alphabet and one numeric.
+String password = config.get(AuthorizationPro
Re: [PR] [#3963] feat(core): Apache Ranger Hive authorization pushdown [gravitino]
jerqi commented on code in PR #4515:
URL: https://github.com/apache/gravitino/pull/4515#discussion_r1723280170
##
authorizations/authorization-ranger/src/main/java/org/apache/gravitino/authorization/ranger/RangerAuthorizationPlugin.java:
##
@@ -0,0 +1,1006 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+package org.apache.gravitino.authorization.ranger;
+
+import com.google.common.annotations.VisibleForTesting;
+import com.google.common.collect.ImmutableMap;
+import com.google.common.collect.Lists;
+import com.google.errorprone.annotations.FormatMethod;
+import com.google.errorprone.annotations.FormatString;
+import java.io.IOException;
+import java.time.Instant;
+import java.util.Arrays;
+import java.util.Collections;
+import java.util.HashMap;
+import java.util.HashSet;
+import java.util.List;
+import java.util.Map;
+import java.util.Objects;
+import java.util.Set;
+import java.util.stream.Collectors;
+import org.apache.gravitino.MetadataObject;
+import org.apache.gravitino.authorization.Group;
+import org.apache.gravitino.authorization.Owner;
+import org.apache.gravitino.authorization.Privilege;
+import org.apache.gravitino.authorization.Role;
+import org.apache.gravitino.authorization.RoleChange;
+import org.apache.gravitino.authorization.SecurableObject;
+import org.apache.gravitino.authorization.SecurableObjects;
+import org.apache.gravitino.authorization.User;
+import org.apache.gravitino.authorization.ranger.defines.VXGroup;
+import org.apache.gravitino.authorization.ranger.defines.VXGroupList;
+import org.apache.gravitino.authorization.ranger.defines.VXUser;
+import org.apache.gravitino.authorization.ranger.defines.VXUserList;
+import org.apache.gravitino.connector.AuthorizationPropertiesMeta;
+import org.apache.gravitino.connector.authorization.AuthorizationPlugin;
+import org.apache.gravitino.exceptions.AuthorizationPluginException;
+import org.apache.gravitino.meta.AuditInfo;
+import org.apache.gravitino.meta.GroupEntity;
+import org.apache.gravitino.meta.UserEntity;
+import org.apache.gravitino.utils.PrincipalUtils;
+import org.apache.ranger.RangerServiceException;
+import org.apache.ranger.plugin.model.RangerPolicy;
+import org.apache.ranger.plugin.model.RangerRole;
+import org.apache.ranger.plugin.util.GrantRevokeRoleRequest;
+import org.apache.ranger.plugin.util.SearchFilter;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+/** Ranger authorization operations plugin abstract class. */
+public abstract class RangerAuthorizationPlugin implements AuthorizationPlugin
{
+ private static final Logger LOG =
LoggerFactory.getLogger(RangerAuthorizationPlugin.class);
+
+ protected String catalogProvider;
+ protected RangerClientExtend rangerClient;
+ protected String rangerServiceName;
+ /** Mapping Gravitino privilege name to the underlying authorization system
privileges. */
+ protected Map> privilegesMapping = null;
+ // The owner privileges, the owner can do anything on the metadata object
+ protected Set ownerPrivileges = null;
+
+ /**
+ * Because Ranger doesn't support the precise filter, Ranger will return the
policy meets the
+ * wildcard(*,?) conditions, just like `*.*.*` policy will match
`db1.table1.column1` So we need
+ * to manually precisely filter the policies.
+ */
+ // Search Ranger policy filter keys
+ protected List policyFilterKeys = null;
+ // Search Ranger policy precise filter keys
+ protected List policyPreciseFilterKeys = null;
+
+ public static final String MANAGED_BY_GRAVITINO = "MANAGED_BY_GRAVITINO";
+
+ // TODO: Maybe need to move to the configuration in the future
+ public static final String RANGER_ADMIN_NAME = "admin";
+
+ public RangerAuthorizationPlugin(String catalogProvider, Map
config) {
+super();
+this.catalogProvider = catalogProvider;
+String rangerUrl =
config.get(AuthorizationPropertiesMeta.RANGER_ADMIN_URL);
+String authType = config.get(AuthorizationPropertiesMeta.RANGER_AUTH_TYPE);
+String username = config.get(AuthorizationPropertiesMeta.RANGER_USERNAME);
+// Apache Ranger Password should be minimum 8 characters with min one
alphabet and one numeric.
+String password = config.get(AuthorizationPro
Re: [PR] [#3963] feat(core): Apache Ranger Hive authorization pushdown [gravitino]
jerqi commented on code in PR #4515:
URL: https://github.com/apache/gravitino/pull/4515#discussion_r1723258341
##
authorizations/authorization-ranger/src/main/java/org/apache/gravitino/authorization/ranger/RangerAuthorizationPlugin.java:
##
@@ -0,0 +1,1006 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+package org.apache.gravitino.authorization.ranger;
+
+import com.google.common.annotations.VisibleForTesting;
+import com.google.common.collect.ImmutableMap;
+import com.google.common.collect.Lists;
+import com.google.errorprone.annotations.FormatMethod;
+import com.google.errorprone.annotations.FormatString;
+import java.io.IOException;
+import java.time.Instant;
+import java.util.Arrays;
+import java.util.Collections;
+import java.util.HashMap;
+import java.util.HashSet;
+import java.util.List;
+import java.util.Map;
+import java.util.Objects;
+import java.util.Set;
+import java.util.stream.Collectors;
+import org.apache.gravitino.MetadataObject;
+import org.apache.gravitino.authorization.Group;
+import org.apache.gravitino.authorization.Owner;
+import org.apache.gravitino.authorization.Privilege;
+import org.apache.gravitino.authorization.Role;
+import org.apache.gravitino.authorization.RoleChange;
+import org.apache.gravitino.authorization.SecurableObject;
+import org.apache.gravitino.authorization.SecurableObjects;
+import org.apache.gravitino.authorization.User;
+import org.apache.gravitino.authorization.ranger.defines.VXGroup;
+import org.apache.gravitino.authorization.ranger.defines.VXGroupList;
+import org.apache.gravitino.authorization.ranger.defines.VXUser;
+import org.apache.gravitino.authorization.ranger.defines.VXUserList;
+import org.apache.gravitino.connector.AuthorizationPropertiesMeta;
+import org.apache.gravitino.connector.authorization.AuthorizationPlugin;
+import org.apache.gravitino.exceptions.AuthorizationPluginException;
+import org.apache.gravitino.meta.AuditInfo;
+import org.apache.gravitino.meta.GroupEntity;
+import org.apache.gravitino.meta.UserEntity;
+import org.apache.gravitino.utils.PrincipalUtils;
+import org.apache.ranger.RangerServiceException;
+import org.apache.ranger.plugin.model.RangerPolicy;
+import org.apache.ranger.plugin.model.RangerRole;
+import org.apache.ranger.plugin.util.GrantRevokeRoleRequest;
+import org.apache.ranger.plugin.util.SearchFilter;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+/** Ranger authorization operations plugin abstract class. */
+public abstract class RangerAuthorizationPlugin implements AuthorizationPlugin
{
+ private static final Logger LOG =
LoggerFactory.getLogger(RangerAuthorizationPlugin.class);
+
+ protected String catalogProvider;
+ protected RangerClientExtend rangerClient;
+ protected String rangerServiceName;
+ /** Mapping Gravitino privilege name to the underlying authorization system
privileges. */
+ protected Map> privilegesMapping = null;
+ // The owner privileges, the owner can do anything on the metadata object
+ protected Set ownerPrivileges = null;
+
+ /**
+ * Because Ranger doesn't support the precise filter, Ranger will return the
policy meets the
+ * wildcard(*,?) conditions, just like `*.*.*` policy will match
`db1.table1.column1` So we need
+ * to manually precisely filter the policies.
+ */
+ // Search Ranger policy filter keys
+ protected List policyFilterKeys = null;
+ // Search Ranger policy precise filter keys
+ protected List policyPreciseFilterKeys = null;
+
+ public static final String MANAGED_BY_GRAVITINO = "MANAGED_BY_GRAVITINO";
+
+ // TODO: Maybe need to move to the configuration in the future
+ public static final String RANGER_ADMIN_NAME = "admin";
+
+ public RangerAuthorizationPlugin(String catalogProvider, Map
config) {
+super();
+this.catalogProvider = catalogProvider;
+String rangerUrl =
config.get(AuthorizationPropertiesMeta.RANGER_ADMIN_URL);
+String authType = config.get(AuthorizationPropertiesMeta.RANGER_AUTH_TYPE);
+String username = config.get(AuthorizationPropertiesMeta.RANGER_USERNAME);
+// Apache Ranger Password should be minimum 8 characters with min one
alphabet and one numeric.
+String password = config.get(AuthorizationPro
Re: [PR] [#3963] feat(core): Apache Ranger Hive authorization pushdown [gravitino]
jerqi commented on code in PR #4515:
URL: https://github.com/apache/gravitino/pull/4515#discussion_r1723258341
##
authorizations/authorization-ranger/src/main/java/org/apache/gravitino/authorization/ranger/RangerAuthorizationPlugin.java:
##
@@ -0,0 +1,1006 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+package org.apache.gravitino.authorization.ranger;
+
+import com.google.common.annotations.VisibleForTesting;
+import com.google.common.collect.ImmutableMap;
+import com.google.common.collect.Lists;
+import com.google.errorprone.annotations.FormatMethod;
+import com.google.errorprone.annotations.FormatString;
+import java.io.IOException;
+import java.time.Instant;
+import java.util.Arrays;
+import java.util.Collections;
+import java.util.HashMap;
+import java.util.HashSet;
+import java.util.List;
+import java.util.Map;
+import java.util.Objects;
+import java.util.Set;
+import java.util.stream.Collectors;
+import org.apache.gravitino.MetadataObject;
+import org.apache.gravitino.authorization.Group;
+import org.apache.gravitino.authorization.Owner;
+import org.apache.gravitino.authorization.Privilege;
+import org.apache.gravitino.authorization.Role;
+import org.apache.gravitino.authorization.RoleChange;
+import org.apache.gravitino.authorization.SecurableObject;
+import org.apache.gravitino.authorization.SecurableObjects;
+import org.apache.gravitino.authorization.User;
+import org.apache.gravitino.authorization.ranger.defines.VXGroup;
+import org.apache.gravitino.authorization.ranger.defines.VXGroupList;
+import org.apache.gravitino.authorization.ranger.defines.VXUser;
+import org.apache.gravitino.authorization.ranger.defines.VXUserList;
+import org.apache.gravitino.connector.AuthorizationPropertiesMeta;
+import org.apache.gravitino.connector.authorization.AuthorizationPlugin;
+import org.apache.gravitino.exceptions.AuthorizationPluginException;
+import org.apache.gravitino.meta.AuditInfo;
+import org.apache.gravitino.meta.GroupEntity;
+import org.apache.gravitino.meta.UserEntity;
+import org.apache.gravitino.utils.PrincipalUtils;
+import org.apache.ranger.RangerServiceException;
+import org.apache.ranger.plugin.model.RangerPolicy;
+import org.apache.ranger.plugin.model.RangerRole;
+import org.apache.ranger.plugin.util.GrantRevokeRoleRequest;
+import org.apache.ranger.plugin.util.SearchFilter;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+/** Ranger authorization operations plugin abstract class. */
+public abstract class RangerAuthorizationPlugin implements AuthorizationPlugin
{
+ private static final Logger LOG =
LoggerFactory.getLogger(RangerAuthorizationPlugin.class);
+
+ protected String catalogProvider;
+ protected RangerClientExtend rangerClient;
+ protected String rangerServiceName;
+ /** Mapping Gravitino privilege name to the underlying authorization system
privileges. */
+ protected Map> privilegesMapping = null;
+ // The owner privileges, the owner can do anything on the metadata object
+ protected Set ownerPrivileges = null;
+
+ /**
+ * Because Ranger doesn't support the precise filter, Ranger will return the
policy meets the
+ * wildcard(*,?) conditions, just like `*.*.*` policy will match
`db1.table1.column1` So we need
+ * to manually precisely filter the policies.
+ */
+ // Search Ranger policy filter keys
+ protected List policyFilterKeys = null;
+ // Search Ranger policy precise filter keys
+ protected List policyPreciseFilterKeys = null;
+
+ public static final String MANAGED_BY_GRAVITINO = "MANAGED_BY_GRAVITINO";
+
+ // TODO: Maybe need to move to the configuration in the future
+ public static final String RANGER_ADMIN_NAME = "admin";
+
+ public RangerAuthorizationPlugin(String catalogProvider, Map
config) {
+super();
+this.catalogProvider = catalogProvider;
+String rangerUrl =
config.get(AuthorizationPropertiesMeta.RANGER_ADMIN_URL);
+String authType = config.get(AuthorizationPropertiesMeta.RANGER_AUTH_TYPE);
+String username = config.get(AuthorizationPropertiesMeta.RANGER_USERNAME);
+// Apache Ranger Password should be minimum 8 characters with min one
alphabet and one numeric.
+String password = config.get(AuthorizationPro
Re: [PR] [#3963] feat(core): Apache Ranger Hive authorization pushdown [gravitino]
jerqi commented on code in PR #4515:
URL: https://github.com/apache/gravitino/pull/4515#discussion_r1723250966
##
authorizations/authorization-ranger/src/main/java/org/apache/gravitino/authorization/ranger/RangerAuthorizationPlugin.java:
##
@@ -0,0 +1,1006 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+package org.apache.gravitino.authorization.ranger;
+
+import com.google.common.annotations.VisibleForTesting;
+import com.google.common.collect.ImmutableMap;
+import com.google.common.collect.Lists;
+import com.google.errorprone.annotations.FormatMethod;
+import com.google.errorprone.annotations.FormatString;
+import java.io.IOException;
+import java.time.Instant;
+import java.util.Arrays;
+import java.util.Collections;
+import java.util.HashMap;
+import java.util.HashSet;
+import java.util.List;
+import java.util.Map;
+import java.util.Objects;
+import java.util.Set;
+import java.util.stream.Collectors;
+import org.apache.gravitino.MetadataObject;
+import org.apache.gravitino.authorization.Group;
+import org.apache.gravitino.authorization.Owner;
+import org.apache.gravitino.authorization.Privilege;
+import org.apache.gravitino.authorization.Role;
+import org.apache.gravitino.authorization.RoleChange;
+import org.apache.gravitino.authorization.SecurableObject;
+import org.apache.gravitino.authorization.SecurableObjects;
+import org.apache.gravitino.authorization.User;
+import org.apache.gravitino.authorization.ranger.defines.VXGroup;
+import org.apache.gravitino.authorization.ranger.defines.VXGroupList;
+import org.apache.gravitino.authorization.ranger.defines.VXUser;
+import org.apache.gravitino.authorization.ranger.defines.VXUserList;
+import org.apache.gravitino.connector.AuthorizationPropertiesMeta;
+import org.apache.gravitino.connector.authorization.AuthorizationPlugin;
+import org.apache.gravitino.exceptions.AuthorizationPluginException;
+import org.apache.gravitino.meta.AuditInfo;
+import org.apache.gravitino.meta.GroupEntity;
+import org.apache.gravitino.meta.UserEntity;
+import org.apache.gravitino.utils.PrincipalUtils;
+import org.apache.ranger.RangerServiceException;
+import org.apache.ranger.plugin.model.RangerPolicy;
+import org.apache.ranger.plugin.model.RangerRole;
+import org.apache.ranger.plugin.util.GrantRevokeRoleRequest;
+import org.apache.ranger.plugin.util.SearchFilter;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+/** Ranger authorization operations plugin abstract class. */
+public abstract class RangerAuthorizationPlugin implements AuthorizationPlugin
{
+ private static final Logger LOG =
LoggerFactory.getLogger(RangerAuthorizationPlugin.class);
+
+ protected String catalogProvider;
+ protected RangerClientExtend rangerClient;
+ protected String rangerServiceName;
+ /** Mapping Gravitino privilege name to the underlying authorization system
privileges. */
+ protected Map> privilegesMapping = null;
+ // The owner privileges, the owner can do anything on the metadata object
+ protected Set ownerPrivileges = null;
+
+ /**
+ * Because Ranger doesn't support the precise filter, Ranger will return the
policy meets the
+ * wildcard(*,?) conditions, just like `*.*.*` policy will match
`db1.table1.column1` So we need
+ * to manually precisely filter the policies.
+ */
+ // Search Ranger policy filter keys
+ protected List policyFilterKeys = null;
+ // Search Ranger policy precise filter keys
+ protected List policyPreciseFilterKeys = null;
+
+ public static final String MANAGED_BY_GRAVITINO = "MANAGED_BY_GRAVITINO";
+
+ // TODO: Maybe need to move to the configuration in the future
+ public static final String RANGER_ADMIN_NAME = "admin";
+
+ public RangerAuthorizationPlugin(String catalogProvider, Map
config) {
+super();
+this.catalogProvider = catalogProvider;
+String rangerUrl =
config.get(AuthorizationPropertiesMeta.RANGER_ADMIN_URL);
+String authType = config.get(AuthorizationPropertiesMeta.RANGER_AUTH_TYPE);
+String username = config.get(AuthorizationPropertiesMeta.RANGER_USERNAME);
+// Apache Ranger Password should be minimum 8 characters with min one
alphabet and one numeric.
+String password = config.get(AuthorizationPro
Re: [PR] [#3963] feat(core): Apache Ranger Hive authorization pushdown [gravitino]
jerqi commented on code in PR #4515:
URL: https://github.com/apache/gravitino/pull/4515#discussion_r1723227774
##
authorizations/authorization-ranger/src/main/java/org/apache/gravitino/authorization/ranger/RangerAuthorizationPlugin.java:
##
@@ -0,0 +1,1006 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+package org.apache.gravitino.authorization.ranger;
+
+import com.google.common.annotations.VisibleForTesting;
+import com.google.common.collect.ImmutableMap;
+import com.google.common.collect.Lists;
+import com.google.errorprone.annotations.FormatMethod;
+import com.google.errorprone.annotations.FormatString;
+import java.io.IOException;
+import java.time.Instant;
+import java.util.Arrays;
+import java.util.Collections;
+import java.util.HashMap;
+import java.util.HashSet;
+import java.util.List;
+import java.util.Map;
+import java.util.Objects;
+import java.util.Set;
+import java.util.stream.Collectors;
+import org.apache.gravitino.MetadataObject;
+import org.apache.gravitino.authorization.Group;
+import org.apache.gravitino.authorization.Owner;
+import org.apache.gravitino.authorization.Privilege;
+import org.apache.gravitino.authorization.Role;
+import org.apache.gravitino.authorization.RoleChange;
+import org.apache.gravitino.authorization.SecurableObject;
+import org.apache.gravitino.authorization.SecurableObjects;
+import org.apache.gravitino.authorization.User;
+import org.apache.gravitino.authorization.ranger.defines.VXGroup;
+import org.apache.gravitino.authorization.ranger.defines.VXGroupList;
+import org.apache.gravitino.authorization.ranger.defines.VXUser;
+import org.apache.gravitino.authorization.ranger.defines.VXUserList;
+import org.apache.gravitino.connector.AuthorizationPropertiesMeta;
+import org.apache.gravitino.connector.authorization.AuthorizationPlugin;
+import org.apache.gravitino.exceptions.AuthorizationPluginException;
+import org.apache.gravitino.meta.AuditInfo;
+import org.apache.gravitino.meta.GroupEntity;
+import org.apache.gravitino.meta.UserEntity;
+import org.apache.gravitino.utils.PrincipalUtils;
+import org.apache.ranger.RangerServiceException;
+import org.apache.ranger.plugin.model.RangerPolicy;
+import org.apache.ranger.plugin.model.RangerRole;
+import org.apache.ranger.plugin.util.GrantRevokeRoleRequest;
+import org.apache.ranger.plugin.util.SearchFilter;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+/** Ranger authorization operations plugin abstract class. */
+public abstract class RangerAuthorizationPlugin implements AuthorizationPlugin
{
+ private static final Logger LOG =
LoggerFactory.getLogger(RangerAuthorizationPlugin.class);
+
+ protected String catalogProvider;
+ protected RangerClientExtend rangerClient;
+ protected String rangerServiceName;
+ /** Mapping Gravitino privilege name to the underlying authorization system
privileges. */
+ protected Map> privilegesMapping = null;
+ // The owner privileges, the owner can do anything on the metadata object
+ protected Set ownerPrivileges = null;
+
+ /**
+ * Because Ranger doesn't support the precise filter, Ranger will return the
policy meets the
+ * wildcard(*,?) conditions, just like `*.*.*` policy will match
`db1.table1.column1` So we need
+ * to manually precisely filter the policies.
+ */
+ // Search Ranger policy filter keys
+ protected List policyFilterKeys = null;
+ // Search Ranger policy precise filter keys
+ protected List policyPreciseFilterKeys = null;
+
+ public static final String MANAGED_BY_GRAVITINO = "MANAGED_BY_GRAVITINO";
+
+ // TODO: Maybe need to move to the configuration in the future
+ public static final String RANGER_ADMIN_NAME = "admin";
+
+ public RangerAuthorizationPlugin(String catalogProvider, Map
config) {
+super();
+this.catalogProvider = catalogProvider;
+String rangerUrl =
config.get(AuthorizationPropertiesMeta.RANGER_ADMIN_URL);
+String authType = config.get(AuthorizationPropertiesMeta.RANGER_AUTH_TYPE);
+String username = config.get(AuthorizationPropertiesMeta.RANGER_USERNAME);
+// Apache Ranger Password should be minimum 8 characters with min one
alphabet and one numeric.
+String password = config.get(AuthorizationPro
Re: [PR] [#3963] feat(core): Apache Ranger Hive authorization pushdown [gravitino]
jerqi commented on code in PR #4515:
URL: https://github.com/apache/gravitino/pull/4515#discussion_r1723222084
##
authorizations/authorization-ranger/src/main/java/org/apache/gravitino/authorization/ranger/RangerAuthorizationPlugin.java:
##
@@ -0,0 +1,1006 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+package org.apache.gravitino.authorization.ranger;
+
+import com.google.common.annotations.VisibleForTesting;
+import com.google.common.collect.ImmutableMap;
+import com.google.common.collect.Lists;
+import com.google.errorprone.annotations.FormatMethod;
+import com.google.errorprone.annotations.FormatString;
+import java.io.IOException;
+import java.time.Instant;
+import java.util.Arrays;
+import java.util.Collections;
+import java.util.HashMap;
+import java.util.HashSet;
+import java.util.List;
+import java.util.Map;
+import java.util.Objects;
+import java.util.Set;
+import java.util.stream.Collectors;
+import org.apache.gravitino.MetadataObject;
+import org.apache.gravitino.authorization.Group;
+import org.apache.gravitino.authorization.Owner;
+import org.apache.gravitino.authorization.Privilege;
+import org.apache.gravitino.authorization.Role;
+import org.apache.gravitino.authorization.RoleChange;
+import org.apache.gravitino.authorization.SecurableObject;
+import org.apache.gravitino.authorization.SecurableObjects;
+import org.apache.gravitino.authorization.User;
+import org.apache.gravitino.authorization.ranger.defines.VXGroup;
+import org.apache.gravitino.authorization.ranger.defines.VXGroupList;
+import org.apache.gravitino.authorization.ranger.defines.VXUser;
+import org.apache.gravitino.authorization.ranger.defines.VXUserList;
+import org.apache.gravitino.connector.AuthorizationPropertiesMeta;
+import org.apache.gravitino.connector.authorization.AuthorizationPlugin;
+import org.apache.gravitino.exceptions.AuthorizationPluginException;
+import org.apache.gravitino.meta.AuditInfo;
+import org.apache.gravitino.meta.GroupEntity;
+import org.apache.gravitino.meta.UserEntity;
+import org.apache.gravitino.utils.PrincipalUtils;
+import org.apache.ranger.RangerServiceException;
+import org.apache.ranger.plugin.model.RangerPolicy;
+import org.apache.ranger.plugin.model.RangerRole;
+import org.apache.ranger.plugin.util.GrantRevokeRoleRequest;
+import org.apache.ranger.plugin.util.SearchFilter;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+/** Ranger authorization operations plugin abstract class. */
+public abstract class RangerAuthorizationPlugin implements AuthorizationPlugin
{
+ private static final Logger LOG =
LoggerFactory.getLogger(RangerAuthorizationPlugin.class);
+
+ protected String catalogProvider;
+ protected RangerClientExtend rangerClient;
+ protected String rangerServiceName;
+ /** Mapping Gravitino privilege name to the underlying authorization system
privileges. */
+ protected Map> privilegesMapping = null;
+ // The owner privileges, the owner can do anything on the metadata object
+ protected Set ownerPrivileges = null;
+
+ /**
+ * Because Ranger doesn't support the precise filter, Ranger will return the
policy meets the
+ * wildcard(*,?) conditions, just like `*.*.*` policy will match
`db1.table1.column1` So we need
+ * to manually precisely filter the policies.
+ */
+ // Search Ranger policy filter keys
+ protected List policyFilterKeys = null;
+ // Search Ranger policy precise filter keys
+ protected List policyPreciseFilterKeys = null;
+
+ public static final String MANAGED_BY_GRAVITINO = "MANAGED_BY_GRAVITINO";
+
+ // TODO: Maybe need to move to the configuration in the future
+ public static final String RANGER_ADMIN_NAME = "admin";
+
+ public RangerAuthorizationPlugin(String catalogProvider, Map
config) {
+super();
+this.catalogProvider = catalogProvider;
+String rangerUrl =
config.get(AuthorizationPropertiesMeta.RANGER_ADMIN_URL);
+String authType = config.get(AuthorizationPropertiesMeta.RANGER_AUTH_TYPE);
+String username = config.get(AuthorizationPropertiesMeta.RANGER_USERNAME);
+// Apache Ranger Password should be minimum 8 characters with min one
alphabet and one numeric.
+String password = config.get(AuthorizationPro
Re: [PR] [#3963] feat(core): Apache Ranger Hive authorization pushdown [gravitino]
jerqi commented on code in PR #4515:
URL: https://github.com/apache/gravitino/pull/4515#discussion_r1723220083
##
authorizations/authorization-ranger/src/main/java/org/apache/gravitino/authorization/ranger/RangerAuthorizationPlugin.java:
##
@@ -0,0 +1,1006 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+package org.apache.gravitino.authorization.ranger;
+
+import com.google.common.annotations.VisibleForTesting;
+import com.google.common.collect.ImmutableMap;
+import com.google.common.collect.Lists;
+import com.google.errorprone.annotations.FormatMethod;
+import com.google.errorprone.annotations.FormatString;
+import java.io.IOException;
+import java.time.Instant;
+import java.util.Arrays;
+import java.util.Collections;
+import java.util.HashMap;
+import java.util.HashSet;
+import java.util.List;
+import java.util.Map;
+import java.util.Objects;
+import java.util.Set;
+import java.util.stream.Collectors;
+import org.apache.gravitino.MetadataObject;
+import org.apache.gravitino.authorization.Group;
+import org.apache.gravitino.authorization.Owner;
+import org.apache.gravitino.authorization.Privilege;
+import org.apache.gravitino.authorization.Role;
+import org.apache.gravitino.authorization.RoleChange;
+import org.apache.gravitino.authorization.SecurableObject;
+import org.apache.gravitino.authorization.SecurableObjects;
+import org.apache.gravitino.authorization.User;
+import org.apache.gravitino.authorization.ranger.defines.VXGroup;
+import org.apache.gravitino.authorization.ranger.defines.VXGroupList;
+import org.apache.gravitino.authorization.ranger.defines.VXUser;
+import org.apache.gravitino.authorization.ranger.defines.VXUserList;
+import org.apache.gravitino.connector.AuthorizationPropertiesMeta;
+import org.apache.gravitino.connector.authorization.AuthorizationPlugin;
+import org.apache.gravitino.exceptions.AuthorizationPluginException;
+import org.apache.gravitino.meta.AuditInfo;
+import org.apache.gravitino.meta.GroupEntity;
+import org.apache.gravitino.meta.UserEntity;
+import org.apache.gravitino.utils.PrincipalUtils;
+import org.apache.ranger.RangerServiceException;
+import org.apache.ranger.plugin.model.RangerPolicy;
+import org.apache.ranger.plugin.model.RangerRole;
+import org.apache.ranger.plugin.util.GrantRevokeRoleRequest;
+import org.apache.ranger.plugin.util.SearchFilter;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+/** Ranger authorization operations plugin abstract class. */
+public abstract class RangerAuthorizationPlugin implements AuthorizationPlugin
{
+ private static final Logger LOG =
LoggerFactory.getLogger(RangerAuthorizationPlugin.class);
+
+ protected String catalogProvider;
+ protected RangerClientExtend rangerClient;
+ protected String rangerServiceName;
+ /** Mapping Gravitino privilege name to the underlying authorization system
privileges. */
+ protected Map> privilegesMapping = null;
+ // The owner privileges, the owner can do anything on the metadata object
+ protected Set ownerPrivileges = null;
+
+ /**
+ * Because Ranger doesn't support the precise filter, Ranger will return the
policy meets the
+ * wildcard(*,?) conditions, just like `*.*.*` policy will match
`db1.table1.column1` So we need
+ * to manually precisely filter the policies.
+ */
+ // Search Ranger policy filter keys
+ protected List policyFilterKeys = null;
+ // Search Ranger policy precise filter keys
+ protected List policyPreciseFilterKeys = null;
+
+ public static final String MANAGED_BY_GRAVITINO = "MANAGED_BY_GRAVITINO";
+
+ // TODO: Maybe need to move to the configuration in the future
+ public static final String RANGER_ADMIN_NAME = "admin";
+
+ public RangerAuthorizationPlugin(String catalogProvider, Map
config) {
+super();
+this.catalogProvider = catalogProvider;
+String rangerUrl =
config.get(AuthorizationPropertiesMeta.RANGER_ADMIN_URL);
+String authType = config.get(AuthorizationPropertiesMeta.RANGER_AUTH_TYPE);
+String username = config.get(AuthorizationPropertiesMeta.RANGER_USERNAME);
+// Apache Ranger Password should be minimum 8 characters with min one
alphabet and one numeric.
+String password = config.get(AuthorizationPro
Re: [PR] [#3963] feat(core): Apache Ranger Hive authorization pushdown [gravitino]
jerqi commented on code in PR #4515:
URL: https://github.com/apache/gravitino/pull/4515#discussion_r1723216181
##
authorizations/authorization-ranger/src/main/java/org/apache/gravitino/authorization/ranger/defines/RangerCommonEnums.java:
##
@@ -0,0 +1,29 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+package org.apache.gravitino.authorization.ranger.defines;
+
+//
apache/ranger/security-admin/src/main/java/org/apache/ranger/defines/RangerCommonEnums.java
+public class RangerCommonEnums {
+ /** IS_VISIBLE is an element of enum VisibilityStatus. Its value is
"IS_VISIBLE". */
+ public static final int IS_VISIBLE = 1;
Review Comment:
The value of `IS_VISBILE` is the same as the `STATUS_ENABLED`, isn't it ok?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
