flyrain commented on code in PR #808: URL: https://github.com/apache/polaris/pull/808#discussion_r1992248260
########## spec/polaris-catalog-service.yaml: ########## @@ -18,7 +18,7 @@ # --- -openapi: 3.0.3 +openapi: 3.0.3 Review Comment: nit: extra space ########## spec/polaris-catalog-apis/policy-apis.yaml: ########## @@ -0,0 +1,709 @@ +# +# Licensed to the Apache Software Foundation (ASF) under one +# or more contributor license agreements. See the NOTICE file +# distributed with this work for additional information +# regarding copyright ownership. The ASF licenses this file +# to you under the Apache License, Version 2.0 (the +# "License"); you may not use this file except in compliance +# with the License. You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, +# software distributed under the License is distributed on an +# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY +# KIND, either express or implied. See the License for the +# specific language governing permissions and limitations +# under the License. +# + +--- +paths: + + /polaris/v1/{prefix}/namespaces/{namespace}/policies: + parameters: + - $ref: '../iceberg-rest-catalog-open-api.yaml#/components/parameters/prefix' + - $ref: '../iceberg-rest-catalog-open-api.yaml#/components/parameters/namespace' + + post: + tags: + - Policy API + summary: Create a policy in the given namespace + operationId: createPolicy + description: | + Creates a policy within the specified namespace. + + A policy defines a set of rules governing actions on specified resources under predefined conditions. + In Apache Polaris, policies are created, stored, and later referenced by external engines to enforce access controls on associated resources. + + User provides the following inputs when creating a policy + - `name`(REQUIRED): The name of the policy. + - `type` (REQUIRED): The type of the policy. + - **Predefined Policies:** policies have a `system.*` prefix in their type, such as `system.data_compaction` + - `description` (OPTIONAL): Provides details about the policy's purpose and functionality + - `content` (OPTIONAL): Defines the rules that control actions and access conditions on resources. The format can be JSON, SQL, or any other format. + + The content field in the request body is validated using the policy's corresponding validator. The policy is created only if the content passes validation. + + Upon successful creation, the new policy's version will be 0. + requestBody: + required: true + content: + application/json: + schema: + $ref: '#/components/schemas/CreatePolicyRequest' + responses: + 200: + $ref: '#/components/responses/CreatePolicyResponse' + 400: + $ref: '../iceberg-rest-catalog-open-api.yaml#/components/responses/BadRequestErrorResponse' + 401: + $ref: '../iceberg-rest-catalog-open-api.yaml#/components/responses/UnauthorizedResponse' + 403: + $ref: '../iceberg-rest-catalog-open-api.yaml#/components/responses/ForbiddenResponse' + 503: + $ref: '../iceberg-rest-catalog-open-api.yaml#/components/responses/ServiceUnavailableResponse' + 5XX: + $ref: '../iceberg-rest-catalog-open-api.yaml#/components/responses/ServerErrorResponse' + get: + tags: + - Policy API + summary: List all policy identifiers underneath a given namespace + description: Return all policy identifiers under this namespace. Users can optionally filter the result by policy type + operationId: listPolicies + parameters: + - $ref: '../iceberg-rest-catalog-open-api.yaml#/components/parameters/page-token' + - $ref: '../iceberg-rest-catalog-open-api.yaml#/components/parameters/page-size' + - $ref: '#/components/parameters/policy-type' + + responses: + 200: + $ref: '#/components/responses/ListPoliciesResponse' + 400: + $ref: '../iceberg-rest-catalog-open-api.yaml#/components/responses/BadRequestErrorResponse' + 401: + $ref: '../iceberg-rest-catalog-open-api.yaml#/components/responses/UnauthorizedResponse' + 403: + $ref: '../iceberg-rest-catalog-open-api.yaml#/components/responses/ForbiddenResponse' + 404: + description: Not Found - The namespace specified does not exist + content: + application/json: + schema: + $ref: '../iceberg-rest-catalog-open-api.yaml#/components/schemas/IcebergErrorResponse' + examples: + NamespaceNotFound: + $ref: '../iceberg-rest-catalog-open-api.yaml#/components/examples/NoSuchNamespaceError' + 503: + $ref: '../iceberg-rest-catalog-open-api.yaml#/components/responses/ServiceUnavailableResponse' + 5XX: + $ref: '../iceberg-rest-catalog-open-api.yaml#/components/responses/ServerErrorResponse' + + /polaris/v1/{prefix}/namespaces/{namespace}/policies/{policyName}: + parameters: + - $ref: '../iceberg-rest-catalog-open-api.yaml#/components/parameters/prefix' + - $ref: '../iceberg-rest-catalog-open-api.yaml#/components/parameters/namespace' + - $ref: '#/components/parameters/policy-name' + + get: + tags: + - Policy API + summary: Load a policy + operationId: loadPolicy + description: | + Load a policy from the catalog + + The response contains the policy's metadata and content. For more details, refer to the definition of the `Policy` model. + responses: + 200: + $ref: '#/components/responses/LoadPolicyResponse' + 400: + $ref: '../iceberg-rest-catalog-open-api.yaml#/components/responses/BadRequestErrorResponse' + 401: + $ref: '../iceberg-rest-catalog-open-api.yaml#/components/responses/UnauthorizedResponse' + 403: + $ref: '../iceberg-rest-catalog-open-api.yaml#/components/responses/ForbiddenResponse' + 404: + description: + Not Found - NoSuchPolicyException, policy to get does not exist + content: + application/json: + schema: + $ref: '../iceberg-rest-catalog-open-api.yaml#/components/schemas/IcebergErrorResponse' + examples: + PolicyToGetDoesNotExist: + $ref: '#/components/examples/NoSuchPolicyError' + 503: + $ref: '../iceberg-rest-catalog-open-api.yaml#/components/responses/ServiceUnavailableResponse' + 5XX: + $ref: '../iceberg-rest-catalog-open-api.yaml#/components/responses/ServerErrorResponse' + + put: + tags: + - Policy API + summary: Update a policy + operationId: updatePolicy + description: | + Update a policy + + A policy's description and content can be updated. The new content is validated against the policy's corresponding validator. + Upon a successful update, the policy's version is incremented by 1. + + The update will only succeed if the current version matches the one in the catalog. + requestBody: + required: true + content: + application/json: + schema: + $ref: '#/components/schemas/UpdatePolicyRequest' + + responses: + 200: + $ref: '#/components/responses/UpdatePolicyResponse' + 400: + $ref: '../iceberg-rest-catalog-open-api.yaml#/components/responses/BadRequestErrorResponse' + 401: + $ref: '../iceberg-rest-catalog-open-api.yaml#/components/responses/UnauthorizedResponse' + 403: + $ref: '../iceberg-rest-catalog-open-api.yaml#/components/responses/ForbiddenResponse' + 404: + description: + Not Found - NoSuchPolicyException, policy to get does not exist + content: + application/json: + schema: + $ref: '../iceberg-rest-catalog-open-api.yaml#/components/schemas/IcebergErrorResponse' + examples: + PolicyToUpdateDoesNotExist: + $ref: '#/components/examples/NoSuchPolicyError' + 409: + description: "The policy version doesn't match the current-policy-version; retry after fetching latest version" + content: + application/json: + schema: + $ref: '../iceberg-rest-catalog-open-api.yaml#/components/schemas/IcebergErrorResponse' + examples: + PolicyVersionMismatch: + $ref: '#/components/examples/PolicyVersionMismatchError' + 503: + $ref: '../iceberg-rest-catalog-open-api.yaml#/components/responses/ServiceUnavailableResponse' + 5XX: + $ref: '../iceberg-rest-catalog-open-api.yaml#/components/responses/ServerErrorResponse' + + delete: + tags: + - Policy API + summary: Drop a policy from the catalog + operationId: dropPolicy + description: | + Remove a policy from the catalog. + + A policy can only be dropped if it is not attached to any resource entity. To remove the policy along with all its attachments, set detach-policy-from-all to true. + + parameters: + - name: detach-policy-from-all + in: query + required: false + description: | + When set to true, the dropPolicy operation will also delete all mappings between the policy and its attached target entities. + schema: + type: boolean + example: false + + responses: + 204: + description: Success, no content + 400: + $ref: '../iceberg-rest-catalog-open-api.yaml#/components/responses/BadRequestErrorResponse' + 401: + $ref: '../iceberg-rest-catalog-open-api.yaml#/components/responses/UnauthorizedResponse' + 403: + $ref: '../iceberg-rest-catalog-open-api.yaml#/components/responses/ForbiddenResponse' + 404: + description: + Not Found - NoSuchPolicyException, policy to get does not exist + content: + application/json: + schema: + $ref: '../iceberg-rest-catalog-open-api.yaml#/components/schemas/IcebergErrorResponse' + examples: + PolicyToDeleteDoesNotExist: + $ref: '#/components/examples/NoSuchPolicyError' + 503: + $ref: '../iceberg-rest-catalog-open-api.yaml#/components/responses/ServiceUnavailableResponse' + 5XX: + $ref: '../iceberg-rest-catalog-open-api.yaml#/components/responses/ServerErrorResponse' + + /polaris/v1/{prefix}/namespaces/{namespace}/policies/{policyName}/mappings: + parameters: + - $ref: '../iceberg-rest-catalog-open-api.yaml#/components/parameters/prefix' + - $ref: '../iceberg-rest-catalog-open-api.yaml#/components/parameters/namespace' + - $ref: '#/components/parameters/policy-name' + + put: + tags: + - Policy API + summary: Create a mapping between a policy and a resource entity + operationId: attachPolicy + description: | + Create a mapping between a policy and a resource entity + + Policy can be set at different levels: + 1. **Table-like level:** Policies specific to individual tables, views, or other table-like entities. + 2. **Namespace level:** Policies applies to a namespace. + 3. **Catalog level:** Policies that applies to a catalog + + The ability to attach a policy to a specific entity type is governed by the PolicyValidator. A policy can only be attached if the resource entity is a valid target for the specified policy type. + + In addition to the validation rules enforced by the PolicyValidator, there are additional constraints on policy attachment: + 1. For inheritable policies, only one policy of the same type can be attached to a given resource entity. + 2. For non-inheritable policies, multiple policies of the same type can be attached to the same resource entity without restriction. + + Additional parameters can be provided in `parameters` when creating a mapping to define specific behavior or constraints. + + If the policy is already attached to the target entity, the existing mapping record will be updated with the new set of parameters, replacing the previous ones. + requestBody: + required: true + content: + application/json: + schema: + $ref: '#/components/schemas/AttachPolicyRequest' + + responses: + 204: + description: Success, no content + 400: + $ref: '../iceberg-rest-catalog-open-api.yaml#/components/responses/BadRequestErrorResponse' + 401: + $ref: '../iceberg-rest-catalog-open-api.yaml#/components/responses/UnauthorizedResponse' + 403: + $ref: '../iceberg-rest-catalog-open-api.yaml#/components/responses/ForbiddenResponse' + 404: + description: + Not Found - NoSuchPolicyException, NoSuchTargetException + content: + application/json: + schema: + $ref: '../iceberg-rest-catalog-open-api.yaml#/components/schemas/IcebergErrorResponse' + examples: + PolicyToSetDoesNotExist: + $ref: '#/components/examples/NoSuchPolicyError' + TargetToSetDoesNotExist: + $ref: '#/components/examples/NoSuchTargetError' + 503: + $ref: '../iceberg-rest-catalog-open-api.yaml#/components/responses/ServiceUnavailableResponse' + 5XX: + $ref: '../iceberg-rest-catalog-open-api.yaml#/components/responses/ServerErrorResponse' + + post: + tags: + - Policy API + summary: Remove a mapping between a policy and a target entity + operationId: detachPolicy + description: | + Remove a mapping between a policy and a target entity + + A target entity can be a catalog, namespace, or any table-like entity + requestBody: + required: true + content: + application/json: + schema: + $ref: '#/components/schemas/DetachPolicyRequest' + + responses: + 204: + description: Success, no content + 400: + $ref: '../iceberg-rest-catalog-open-api.yaml#/components/responses/BadRequestErrorResponse' + 401: + $ref: '../iceberg-rest-catalog-open-api.yaml#/components/responses/UnauthorizedResponse' + 403: + $ref: '../iceberg-rest-catalog-open-api.yaml#/components/responses/ForbiddenResponse' + 404: + description: + Not Found - NoSuchPolicyException, NoSuchTargetException, NoSuchMappingException + content: + application/json: + schema: + $ref: '../iceberg-rest-catalog-open-api.yaml#/components/schemas/IcebergErrorResponse' + examples: + PolicyToUnsetDoesNotExist: + $ref: '#/components/examples/NoSuchPolicyError' + TargetToUnsetDoesNotExist: + $ref: '#/components/examples/NoSuchTargetError' + MappingToUnsetDoesNotExist: + $ref: '#/components/examples/NoSuchMappingError' + 503: + $ref: '../iceberg-rest-catalog-open-api.yaml#/components/responses/ServiceUnavailableResponse' + 5XX: + $ref: '../iceberg-rest-catalog-open-api.yaml#/components/responses/ServerErrorResponse' + + /polaris/v1/{prefix}/applicablePolicies: + parameters: + - $ref: '../iceberg-rest-catalog-open-api.yaml#/components/parameters/prefix' + + get: + tags: + - Policy API + summary: Get Applicable policies for catalog, namespace, table, or views + operationId: getApplicablePoliciesUsingParameter + description: | + Retrieves all applicable policies for a specified entity, including inherited policies from parent entities. An entity can be a table/view, namespace, or catalog. The required parameters depend on the entity type: + + - Table/View: + - The `namespace` parameter is required to specify the entity's namespace. + - The `name` parameter is required to specify the entity name. + - Namespace: + - The `namespace` parameter is required to specify the identifier. + - The `name` parameter should not be set. + - Catalog: + - Neither `namespace` nor `name` should be set. Review Comment: I guess it's fine to infer the type this way. So that we save one parameter. ########## spec/polaris-catalog-apis/policy-apis.yaml: ########## @@ -0,0 +1,709 @@ +# +# Licensed to the Apache Software Foundation (ASF) under one +# or more contributor license agreements. See the NOTICE file +# distributed with this work for additional information +# regarding copyright ownership. The ASF licenses this file +# to you under the Apache License, Version 2.0 (the +# "License"); you may not use this file except in compliance +# with the License. You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, +# software distributed under the License is distributed on an +# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY +# KIND, either express or implied. See the License for the +# specific language governing permissions and limitations +# under the License. +# + +--- +paths: + + /polaris/v1/{prefix}/namespaces/{namespace}/policies: + parameters: + - $ref: '../iceberg-rest-catalog-open-api.yaml#/components/parameters/prefix' + - $ref: '../iceberg-rest-catalog-open-api.yaml#/components/parameters/namespace' + + post: + tags: + - Policy API + summary: Create a policy in the given namespace + operationId: createPolicy + description: | + Creates a policy within the specified namespace. + + A policy defines a set of rules governing actions on specified resources under predefined conditions. + In Apache Polaris, policies are created, stored, and later referenced by external engines to enforce access controls on associated resources. + + User provides the following inputs when creating a policy + - `name`(REQUIRED): The name of the policy. + - `type` (REQUIRED): The type of the policy. + - **Predefined Policies:** policies have a `system.*` prefix in their type, such as `system.data_compaction` + - `description` (OPTIONAL): Provides details about the policy's purpose and functionality + - `content` (OPTIONAL): Defines the rules that control actions and access conditions on resources. The format can be JSON, SQL, or any other format. + + The content field in the request body is validated using the policy's corresponding validator. The policy is created only if the content passes validation. + + Upon successful creation, the new policy's version will be 0. + requestBody: + required: true + content: + application/json: + schema: + $ref: '#/components/schemas/CreatePolicyRequest' + responses: + 200: + $ref: '#/components/responses/CreatePolicyResponse' + 400: + $ref: '../iceberg-rest-catalog-open-api.yaml#/components/responses/BadRequestErrorResponse' + 401: + $ref: '../iceberg-rest-catalog-open-api.yaml#/components/responses/UnauthorizedResponse' + 403: + $ref: '../iceberg-rest-catalog-open-api.yaml#/components/responses/ForbiddenResponse' + 503: + $ref: '../iceberg-rest-catalog-open-api.yaml#/components/responses/ServiceUnavailableResponse' + 5XX: + $ref: '../iceberg-rest-catalog-open-api.yaml#/components/responses/ServerErrorResponse' + get: + tags: + - Policy API + summary: List all policy identifiers underneath a given namespace + description: Return all policy identifiers under this namespace. Users can optionally filter the result by policy type + operationId: listPolicies + parameters: + - $ref: '../iceberg-rest-catalog-open-api.yaml#/components/parameters/page-token' + - $ref: '../iceberg-rest-catalog-open-api.yaml#/components/parameters/page-size' + - $ref: '#/components/parameters/policy-type' + + responses: + 200: + $ref: '#/components/responses/ListPoliciesResponse' + 400: + $ref: '../iceberg-rest-catalog-open-api.yaml#/components/responses/BadRequestErrorResponse' + 401: + $ref: '../iceberg-rest-catalog-open-api.yaml#/components/responses/UnauthorizedResponse' + 403: + $ref: '../iceberg-rest-catalog-open-api.yaml#/components/responses/ForbiddenResponse' + 404: + description: Not Found - The namespace specified does not exist + content: + application/json: + schema: + $ref: '../iceberg-rest-catalog-open-api.yaml#/components/schemas/IcebergErrorResponse' + examples: + NamespaceNotFound: + $ref: '../iceberg-rest-catalog-open-api.yaml#/components/examples/NoSuchNamespaceError' + 503: + $ref: '../iceberg-rest-catalog-open-api.yaml#/components/responses/ServiceUnavailableResponse' + 5XX: + $ref: '../iceberg-rest-catalog-open-api.yaml#/components/responses/ServerErrorResponse' + + /polaris/v1/{prefix}/namespaces/{namespace}/policies/{policyName}: + parameters: + - $ref: '../iceberg-rest-catalog-open-api.yaml#/components/parameters/prefix' + - $ref: '../iceberg-rest-catalog-open-api.yaml#/components/parameters/namespace' + - $ref: '#/components/parameters/policy-name' + + get: + tags: + - Policy API + summary: Load a policy + operationId: loadPolicy + description: | + Load a policy from the catalog + + The response contains the policy's metadata and content. For more details, refer to the definition of the `Policy` model. + responses: + 200: + $ref: '#/components/responses/LoadPolicyResponse' + 400: + $ref: '../iceberg-rest-catalog-open-api.yaml#/components/responses/BadRequestErrorResponse' + 401: + $ref: '../iceberg-rest-catalog-open-api.yaml#/components/responses/UnauthorizedResponse' + 403: + $ref: '../iceberg-rest-catalog-open-api.yaml#/components/responses/ForbiddenResponse' + 404: + description: + Not Found - NoSuchPolicyException, policy to get does not exist + content: + application/json: + schema: + $ref: '../iceberg-rest-catalog-open-api.yaml#/components/schemas/IcebergErrorResponse' + examples: + PolicyToGetDoesNotExist: + $ref: '#/components/examples/NoSuchPolicyError' + 503: + $ref: '../iceberg-rest-catalog-open-api.yaml#/components/responses/ServiceUnavailableResponse' + 5XX: + $ref: '../iceberg-rest-catalog-open-api.yaml#/components/responses/ServerErrorResponse' + + put: + tags: + - Policy API + summary: Update a policy + operationId: updatePolicy + description: | + Update a policy + + A policy's description and content can be updated. The new content is validated against the policy's corresponding validator. + Upon a successful update, the policy's version is incremented by 1. + + The update will only succeed if the current version matches the one in the catalog. + requestBody: + required: true + content: + application/json: + schema: + $ref: '#/components/schemas/UpdatePolicyRequest' + + responses: + 200: + $ref: '#/components/responses/UpdatePolicyResponse' + 400: + $ref: '../iceberg-rest-catalog-open-api.yaml#/components/responses/BadRequestErrorResponse' + 401: + $ref: '../iceberg-rest-catalog-open-api.yaml#/components/responses/UnauthorizedResponse' + 403: + $ref: '../iceberg-rest-catalog-open-api.yaml#/components/responses/ForbiddenResponse' + 404: + description: + Not Found - NoSuchPolicyException, policy to get does not exist + content: + application/json: + schema: + $ref: '../iceberg-rest-catalog-open-api.yaml#/components/schemas/IcebergErrorResponse' + examples: + PolicyToUpdateDoesNotExist: + $ref: '#/components/examples/NoSuchPolicyError' + 409: + description: "The policy version doesn't match the current-policy-version; retry after fetching latest version" + content: + application/json: + schema: + $ref: '../iceberg-rest-catalog-open-api.yaml#/components/schemas/IcebergErrorResponse' + examples: + PolicyVersionMismatch: + $ref: '#/components/examples/PolicyVersionMismatchError' + 503: + $ref: '../iceberg-rest-catalog-open-api.yaml#/components/responses/ServiceUnavailableResponse' + 5XX: + $ref: '../iceberg-rest-catalog-open-api.yaml#/components/responses/ServerErrorResponse' + + delete: + tags: + - Policy API + summary: Drop a policy from the catalog + operationId: dropPolicy + description: | + Remove a policy from the catalog. + + A policy can only be dropped if it is not attached to any resource entity. To remove the policy along with all its attachments, set detach-policy-from-all to true. + + parameters: + - name: detach-policy-from-all + in: query + required: false + description: | + When set to true, the dropPolicy operation will also delete all mappings between the policy and its attached target entities. + schema: + type: boolean + example: false + + responses: + 204: + description: Success, no content + 400: + $ref: '../iceberg-rest-catalog-open-api.yaml#/components/responses/BadRequestErrorResponse' + 401: + $ref: '../iceberg-rest-catalog-open-api.yaml#/components/responses/UnauthorizedResponse' + 403: + $ref: '../iceberg-rest-catalog-open-api.yaml#/components/responses/ForbiddenResponse' + 404: + description: + Not Found - NoSuchPolicyException, policy to get does not exist + content: + application/json: + schema: + $ref: '../iceberg-rest-catalog-open-api.yaml#/components/schemas/IcebergErrorResponse' + examples: + PolicyToDeleteDoesNotExist: + $ref: '#/components/examples/NoSuchPolicyError' + 503: + $ref: '../iceberg-rest-catalog-open-api.yaml#/components/responses/ServiceUnavailableResponse' + 5XX: + $ref: '../iceberg-rest-catalog-open-api.yaml#/components/responses/ServerErrorResponse' + + /polaris/v1/{prefix}/namespaces/{namespace}/policies/{policyName}/mappings: + parameters: + - $ref: '../iceberg-rest-catalog-open-api.yaml#/components/parameters/prefix' + - $ref: '../iceberg-rest-catalog-open-api.yaml#/components/parameters/namespace' + - $ref: '#/components/parameters/policy-name' + + put: + tags: + - Policy API + summary: Create a mapping between a policy and a resource entity + operationId: attachPolicy + description: | + Create a mapping between a policy and a resource entity + + Policy can be set at different levels: + 1. **Table-like level:** Policies specific to individual tables, views, or other table-like entities. + 2. **Namespace level:** Policies applies to a namespace. + 3. **Catalog level:** Policies that applies to a catalog + + The ability to attach a policy to a specific entity type is governed by the PolicyValidator. A policy can only be attached if the resource entity is a valid target for the specified policy type. + + In addition to the validation rules enforced by the PolicyValidator, there are additional constraints on policy attachment: + 1. For inheritable policies, only one policy of the same type can be attached to a given resource entity. + 2. For non-inheritable policies, multiple policies of the same type can be attached to the same resource entity without restriction. + + Additional parameters can be provided in `parameters` when creating a mapping to define specific behavior or constraints. + + If the policy is already attached to the target entity, the existing mapping record will be updated with the new set of parameters, replacing the previous ones. + requestBody: + required: true + content: + application/json: + schema: + $ref: '#/components/schemas/AttachPolicyRequest' + + responses: + 204: + description: Success, no content + 400: + $ref: '../iceberg-rest-catalog-open-api.yaml#/components/responses/BadRequestErrorResponse' + 401: + $ref: '../iceberg-rest-catalog-open-api.yaml#/components/responses/UnauthorizedResponse' + 403: + $ref: '../iceberg-rest-catalog-open-api.yaml#/components/responses/ForbiddenResponse' + 404: + description: + Not Found - NoSuchPolicyException, NoSuchTargetException + content: + application/json: + schema: + $ref: '../iceberg-rest-catalog-open-api.yaml#/components/schemas/IcebergErrorResponse' + examples: + PolicyToSetDoesNotExist: + $ref: '#/components/examples/NoSuchPolicyError' + TargetToSetDoesNotExist: + $ref: '#/components/examples/NoSuchTargetError' + 503: + $ref: '../iceberg-rest-catalog-open-api.yaml#/components/responses/ServiceUnavailableResponse' + 5XX: + $ref: '../iceberg-rest-catalog-open-api.yaml#/components/responses/ServerErrorResponse' + + post: + tags: + - Policy API + summary: Remove a mapping between a policy and a target entity + operationId: detachPolicy + description: | + Remove a mapping between a policy and a target entity + + A target entity can be a catalog, namespace, or any table-like entity + requestBody: + required: true + content: + application/json: + schema: + $ref: '#/components/schemas/DetachPolicyRequest' + + responses: + 204: + description: Success, no content + 400: + $ref: '../iceberg-rest-catalog-open-api.yaml#/components/responses/BadRequestErrorResponse' + 401: + $ref: '../iceberg-rest-catalog-open-api.yaml#/components/responses/UnauthorizedResponse' + 403: + $ref: '../iceberg-rest-catalog-open-api.yaml#/components/responses/ForbiddenResponse' + 404: + description: + Not Found - NoSuchPolicyException, NoSuchTargetException, NoSuchMappingException + content: + application/json: + schema: + $ref: '../iceberg-rest-catalog-open-api.yaml#/components/schemas/IcebergErrorResponse' + examples: + PolicyToUnsetDoesNotExist: + $ref: '#/components/examples/NoSuchPolicyError' + TargetToUnsetDoesNotExist: + $ref: '#/components/examples/NoSuchTargetError' + MappingToUnsetDoesNotExist: + $ref: '#/components/examples/NoSuchMappingError' + 503: + $ref: '../iceberg-rest-catalog-open-api.yaml#/components/responses/ServiceUnavailableResponse' + 5XX: + $ref: '../iceberg-rest-catalog-open-api.yaml#/components/responses/ServerErrorResponse' + + /polaris/v1/{prefix}/applicablePolicies: + parameters: + - $ref: '../iceberg-rest-catalog-open-api.yaml#/components/parameters/prefix' + + get: + tags: + - Policy API + summary: Get Applicable policies for catalog, namespace, table, or views + operationId: getApplicablePoliciesUsingParameter + description: | + Retrieves all applicable policies for a specified entity, including inherited policies from parent entities. An entity can be a table/view, namespace, or catalog. The required parameters depend on the entity type: + + - Table/View: + - The `namespace` parameter is required to specify the entity's namespace. + - The `name` parameter is required to specify the entity name. + - Namespace: + - The `namespace` parameter is required to specify the identifier. + - The `name` parameter should not be set. + - Catalog: + - Neither `namespace` nor `name` should be set. + + An optional policyType parameter filters results to return only policies of the specified type. + + This API evaluates the entity's hierarchy and applies inheritable policies from parent entities. + + parameters: + - $ref: '../iceberg-rest-catalog-open-api.yaml#/components/parameters/page-token' + - $ref: '../iceberg-rest-catalog-open-api.yaml#/components/parameters/page-size' + - name: namespace + in: query + required: false + description: + A namespace identifier as a single string. + Multipart namespace parts should be separated by the unit separator (`0x1F`) byte. + schema: + type: string + examples: + singlepart_namespace: + value: "accounting" + multipart_namespace: + value: "accounting%1Ftax" + - $ref: '#/components/parameters/name' + - $ref: '#/components/parameters/policy-type' + + responses: + 200: + $ref: '#/components/responses/GetApplicablePoliciesResponse' + 400: + $ref: '../iceberg-rest-catalog-open-api.yaml#/components/responses/BadRequestErrorResponse' + 401: + $ref: '../iceberg-rest-catalog-open-api.yaml#/components/responses/UnauthorizedResponse' + 403: + $ref: '../iceberg-rest-catalog-open-api.yaml#/components/responses/ForbiddenResponse' + 404: + description: + Not Found + - NoSuchTableException, target table does not exist + - NoSuchViewException, target view does not exist + - NoSuchNamespaceException, target namespace does not exist + content: + application/json: + schema: + $ref: '../iceberg-rest-catalog-open-api.yaml#/components/schemas/IcebergErrorResponse' + examples: + TargetTableDoesNotExist: + $ref: '../iceberg-rest-catalog-open-api.yaml#/components/examples/NoSuchTableError' + TargetViewDoesNotExist: + $ref: '../iceberg-rest-catalog-open-api.yaml#/components/examples/NoSuchViewError' + TargetNamespaceDoesNotExist: + $ref: '../iceberg-rest-catalog-open-api.yaml#/components/examples/NoSuchNamespaceError' + 503: + $ref: '../iceberg-rest-catalog-open-api.yaml#/components/responses/ServiceUnavailableResponse' + 5XX: + $ref: '../iceberg-rest-catalog-open-api.yaml#/components/responses/ServerErrorResponse' + +components: + parameters: + policy-name: + name: policyName + in: path + required: true + schema: + $ref: '#/components/schemas/PolicyName' + + policy-type: + name: policyType + in: query + required: false + allowEmptyValue: true + schema: + $ref: '#/components/schemas/PolicyType' + + name: + name: name + in: query + required: false + description: Name of the target entity + schema: + type: string + example: "test_table" + + schemas: + + PolicyType: + description: The type of a policy + type: string + nullable: true + example: system.data-compaction + + PolicyName: + type: string + description: + A policy name. A valid policy name should only consist of uppercase and lowercase letters (A-Z, a-z), + digits (0-9), hyphens (-), underscores (_). + pattern: '^[A-Za-z0-9\-_]+$' + example: 'compaction' + + Policy: + type: object + description: | + A policy in Apache Polaris defines a set of rules for governing access, data usage, and operational consistency across various catalog resources. + Policies are stored within Polaris and can be applied to catalogs, namespaces, tables, views, and other table-like entities. + For example, they can be used for fine-grained control over who can perform specific actions on certain resources. + + The policy object includes + - **policy-type:** The type of the policy, which determines the expected format and semantics of the policy content. + - **inheritable:** A boolean flag indicating whether the policy is inheritable. + - **name:** A human-readable name for the policy, which must be unique within a given namespace. + - **description:** Detailed description of the purpose and functionalities of the policy. + - **content:** Policy content, which can be validated against predefined schemas of a policy type. + - **version:** Indicates the current version of the policy. Versions increased monotonically, the default value is 0 + - **created-at:** A timestamp indicating when the policy was created, represented as a string in ISO-8601 format + - **updated-at:** A timestamp indicating the last update time of the policy, represented as a string in ISO-8601 format + + Policies stored in Polaris serve as the persistent definition for access control and governance rules. + required: + - policy-type + - name + - version + - inheritable + properties: + policy-type: + type: string + inheritable: + type: boolean + name: + $ref: '#/components/schemas/PolicyName' + description: + type: string + content: + type: string + version: + type: integer + created-at: + type: string + example: "2024-03-03T12:00:00Z" + updated-at: + type: string + example: "2024-03-03T12:00:00Z" + + CreatePolicyRequest: + type: object + required: + - name + - type + properties: + name: + $ref: '#/components/schemas/PolicyName' + type: + type: string + description: + type: string + content: + type: string + + LoadPolicyResponse: + type: object + properties: + policy: + $ref: '#/components/schemas/Policy' + + UpdatePolicyRequest: + type: object + properties: + description: + type: string + content: + type: string + current-policy-version: + type: integer + + AttachPolicyRequest: + type: object + required: + - entity + properties: + target: + $ref: '#/components/schemas/PolicyAttachmentTarget' + parameters: + type: object + additionalProperties: + type: string + + DetachPolicyRequest: + type: object + required: + - entity + properties: + target: + $ref: '#/components/schemas/PolicyAttachmentTarget' + + PolicyAttachmentTarget: + type: object + properties: + type: + type: string + enum: + - catalog + - namespace + - table-like + example: table-like + path: + type: array + items: + type: string + description: | + A list representing the hierarchical path to the entity, ordered from the namespace level down to the entity + example: ["NS1", "NS2", "test_table_1"] Review Comment: What happens when we attach a policy to the current catalog? An empty array with type enum to be `catalog`? Should we include the catalog name in the array, even it always points to the current catalog name now? ########## spec/polaris-catalog-apis/policy-apis.yaml: ########## @@ -0,0 +1,709 @@ +# +# Licensed to the Apache Software Foundation (ASF) under one +# or more contributor license agreements. See the NOTICE file +# distributed with this work for additional information +# regarding copyright ownership. The ASF licenses this file +# to you under the Apache License, Version 2.0 (the +# "License"); you may not use this file except in compliance +# with the License. You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, +# software distributed under the License is distributed on an +# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY +# KIND, either express or implied. See the License for the +# specific language governing permissions and limitations +# under the License. +# + +--- +paths: + + /polaris/v1/{prefix}/namespaces/{namespace}/policies: + parameters: + - $ref: '../iceberg-rest-catalog-open-api.yaml#/components/parameters/prefix' + - $ref: '../iceberg-rest-catalog-open-api.yaml#/components/parameters/namespace' + + post: + tags: + - Policy API + summary: Create a policy in the given namespace + operationId: createPolicy + description: | + Creates a policy within the specified namespace. + + A policy defines a set of rules governing actions on specified resources under predefined conditions. + In Apache Polaris, policies are created, stored, and later referenced by external engines to enforce access controls on associated resources. + + User provides the following inputs when creating a policy + - `name`(REQUIRED): The name of the policy. + - `type` (REQUIRED): The type of the policy. + - **Predefined Policies:** policies have a `system.*` prefix in their type, such as `system.data_compaction` + - `description` (OPTIONAL): Provides details about the policy's purpose and functionality + - `content` (OPTIONAL): Defines the rules that control actions and access conditions on resources. The format can be JSON, SQL, or any other format. + + The content field in the request body is validated using the policy's corresponding validator. The policy is created only if the content passes validation. + + Upon successful creation, the new policy's version will be 0. + requestBody: + required: true + content: + application/json: + schema: + $ref: '#/components/schemas/CreatePolicyRequest' + responses: + 200: + $ref: '#/components/responses/CreatePolicyResponse' + 400: + $ref: '../iceberg-rest-catalog-open-api.yaml#/components/responses/BadRequestErrorResponse' + 401: + $ref: '../iceberg-rest-catalog-open-api.yaml#/components/responses/UnauthorizedResponse' + 403: + $ref: '../iceberg-rest-catalog-open-api.yaml#/components/responses/ForbiddenResponse' + 503: + $ref: '../iceberg-rest-catalog-open-api.yaml#/components/responses/ServiceUnavailableResponse' + 5XX: + $ref: '../iceberg-rest-catalog-open-api.yaml#/components/responses/ServerErrorResponse' + get: + tags: + - Policy API + summary: List all policy identifiers underneath a given namespace + description: Return all policy identifiers under this namespace. Users can optionally filter the result by policy type + operationId: listPolicies + parameters: + - $ref: '../iceberg-rest-catalog-open-api.yaml#/components/parameters/page-token' + - $ref: '../iceberg-rest-catalog-open-api.yaml#/components/parameters/page-size' + - $ref: '#/components/parameters/policy-type' + + responses: + 200: + $ref: '#/components/responses/ListPoliciesResponse' + 400: + $ref: '../iceberg-rest-catalog-open-api.yaml#/components/responses/BadRequestErrorResponse' + 401: + $ref: '../iceberg-rest-catalog-open-api.yaml#/components/responses/UnauthorizedResponse' + 403: + $ref: '../iceberg-rest-catalog-open-api.yaml#/components/responses/ForbiddenResponse' + 404: + description: Not Found - The namespace specified does not exist + content: + application/json: + schema: + $ref: '../iceberg-rest-catalog-open-api.yaml#/components/schemas/IcebergErrorResponse' + examples: + NamespaceNotFound: + $ref: '../iceberg-rest-catalog-open-api.yaml#/components/examples/NoSuchNamespaceError' + 503: + $ref: '../iceberg-rest-catalog-open-api.yaml#/components/responses/ServiceUnavailableResponse' + 5XX: + $ref: '../iceberg-rest-catalog-open-api.yaml#/components/responses/ServerErrorResponse' + + /polaris/v1/{prefix}/namespaces/{namespace}/policies/{policyName}: + parameters: + - $ref: '../iceberg-rest-catalog-open-api.yaml#/components/parameters/prefix' + - $ref: '../iceberg-rest-catalog-open-api.yaml#/components/parameters/namespace' + - $ref: '#/components/parameters/policy-name' + + get: + tags: + - Policy API + summary: Load a policy + operationId: loadPolicy + description: | + Load a policy from the catalog + + The response contains the policy's metadata and content. For more details, refer to the definition of the `Policy` model. + responses: + 200: + $ref: '#/components/responses/LoadPolicyResponse' + 400: + $ref: '../iceberg-rest-catalog-open-api.yaml#/components/responses/BadRequestErrorResponse' + 401: + $ref: '../iceberg-rest-catalog-open-api.yaml#/components/responses/UnauthorizedResponse' + 403: + $ref: '../iceberg-rest-catalog-open-api.yaml#/components/responses/ForbiddenResponse' + 404: + description: + Not Found - NoSuchPolicyException, policy to get does not exist + content: + application/json: + schema: + $ref: '../iceberg-rest-catalog-open-api.yaml#/components/schemas/IcebergErrorResponse' + examples: + PolicyToGetDoesNotExist: + $ref: '#/components/examples/NoSuchPolicyError' + 503: + $ref: '../iceberg-rest-catalog-open-api.yaml#/components/responses/ServiceUnavailableResponse' + 5XX: + $ref: '../iceberg-rest-catalog-open-api.yaml#/components/responses/ServerErrorResponse' + + put: + tags: + - Policy API + summary: Update a policy + operationId: updatePolicy + description: | + Update a policy + + A policy's description and content can be updated. The new content is validated against the policy's corresponding validator. + Upon a successful update, the policy's version is incremented by 1. + + The update will only succeed if the current version matches the one in the catalog. + requestBody: + required: true + content: + application/json: + schema: + $ref: '#/components/schemas/UpdatePolicyRequest' + + responses: + 200: + $ref: '#/components/responses/UpdatePolicyResponse' + 400: + $ref: '../iceberg-rest-catalog-open-api.yaml#/components/responses/BadRequestErrorResponse' + 401: + $ref: '../iceberg-rest-catalog-open-api.yaml#/components/responses/UnauthorizedResponse' + 403: + $ref: '../iceberg-rest-catalog-open-api.yaml#/components/responses/ForbiddenResponse' + 404: + description: + Not Found - NoSuchPolicyException, policy to get does not exist + content: + application/json: + schema: + $ref: '../iceberg-rest-catalog-open-api.yaml#/components/schemas/IcebergErrorResponse' + examples: + PolicyToUpdateDoesNotExist: + $ref: '#/components/examples/NoSuchPolicyError' + 409: + description: "The policy version doesn't match the current-policy-version; retry after fetching latest version" + content: + application/json: + schema: + $ref: '../iceberg-rest-catalog-open-api.yaml#/components/schemas/IcebergErrorResponse' + examples: + PolicyVersionMismatch: + $ref: '#/components/examples/PolicyVersionMismatchError' + 503: + $ref: '../iceberg-rest-catalog-open-api.yaml#/components/responses/ServiceUnavailableResponse' + 5XX: + $ref: '../iceberg-rest-catalog-open-api.yaml#/components/responses/ServerErrorResponse' + + delete: + tags: + - Policy API + summary: Drop a policy from the catalog + operationId: dropPolicy + description: | + Remove a policy from the catalog. + + A policy can only be dropped if it is not attached to any resource entity. To remove the policy along with all its attachments, set detach-policy-from-all to true. + + parameters: + - name: detach-policy-from-all + in: query + required: false + description: | + When set to true, the dropPolicy operation will also delete all mappings between the policy and its attached target entities. + schema: + type: boolean + example: false + + responses: + 204: + description: Success, no content + 400: + $ref: '../iceberg-rest-catalog-open-api.yaml#/components/responses/BadRequestErrorResponse' + 401: + $ref: '../iceberg-rest-catalog-open-api.yaml#/components/responses/UnauthorizedResponse' + 403: + $ref: '../iceberg-rest-catalog-open-api.yaml#/components/responses/ForbiddenResponse' + 404: + description: + Not Found - NoSuchPolicyException, policy to get does not exist + content: + application/json: + schema: + $ref: '../iceberg-rest-catalog-open-api.yaml#/components/schemas/IcebergErrorResponse' + examples: + PolicyToDeleteDoesNotExist: + $ref: '#/components/examples/NoSuchPolicyError' + 503: + $ref: '../iceberg-rest-catalog-open-api.yaml#/components/responses/ServiceUnavailableResponse' + 5XX: + $ref: '../iceberg-rest-catalog-open-api.yaml#/components/responses/ServerErrorResponse' + + /polaris/v1/{prefix}/namespaces/{namespace}/policies/{policyName}/mappings: + parameters: + - $ref: '../iceberg-rest-catalog-open-api.yaml#/components/parameters/prefix' + - $ref: '../iceberg-rest-catalog-open-api.yaml#/components/parameters/namespace' + - $ref: '#/components/parameters/policy-name' + + put: + tags: + - Policy API + summary: Create a mapping between a policy and a resource entity + operationId: attachPolicy + description: | + Create a mapping between a policy and a resource entity + + Policy can be set at different levels: + 1. **Table-like level:** Policies specific to individual tables, views, or other table-like entities. + 2. **Namespace level:** Policies applies to a namespace. + 3. **Catalog level:** Policies that applies to a catalog + + The ability to attach a policy to a specific entity type is governed by the PolicyValidator. A policy can only be attached if the resource entity is a valid target for the specified policy type. + + In addition to the validation rules enforced by the PolicyValidator, there are additional constraints on policy attachment: + 1. For inheritable policies, only one policy of the same type can be attached to a given resource entity. + 2. For non-inheritable policies, multiple policies of the same type can be attached to the same resource entity without restriction. + + Additional parameters can be provided in `parameters` when creating a mapping to define specific behavior or constraints. + + If the policy is already attached to the target entity, the existing mapping record will be updated with the new set of parameters, replacing the previous ones. + requestBody: + required: true + content: + application/json: + schema: + $ref: '#/components/schemas/AttachPolicyRequest' + + responses: + 204: + description: Success, no content + 400: + $ref: '../iceberg-rest-catalog-open-api.yaml#/components/responses/BadRequestErrorResponse' + 401: + $ref: '../iceberg-rest-catalog-open-api.yaml#/components/responses/UnauthorizedResponse' + 403: + $ref: '../iceberg-rest-catalog-open-api.yaml#/components/responses/ForbiddenResponse' + 404: + description: + Not Found - NoSuchPolicyException, NoSuchTargetException + content: + application/json: + schema: + $ref: '../iceberg-rest-catalog-open-api.yaml#/components/schemas/IcebergErrorResponse' + examples: + PolicyToSetDoesNotExist: + $ref: '#/components/examples/NoSuchPolicyError' + TargetToSetDoesNotExist: + $ref: '#/components/examples/NoSuchTargetError' + 503: + $ref: '../iceberg-rest-catalog-open-api.yaml#/components/responses/ServiceUnavailableResponse' + 5XX: + $ref: '../iceberg-rest-catalog-open-api.yaml#/components/responses/ServerErrorResponse' + + post: + tags: + - Policy API + summary: Remove a mapping between a policy and a target entity + operationId: detachPolicy + description: | + Remove a mapping between a policy and a target entity + + A target entity can be a catalog, namespace, or any table-like entity + requestBody: + required: true + content: + application/json: + schema: + $ref: '#/components/schemas/DetachPolicyRequest' + + responses: + 204: + description: Success, no content + 400: + $ref: '../iceberg-rest-catalog-open-api.yaml#/components/responses/BadRequestErrorResponse' + 401: + $ref: '../iceberg-rest-catalog-open-api.yaml#/components/responses/UnauthorizedResponse' + 403: + $ref: '../iceberg-rest-catalog-open-api.yaml#/components/responses/ForbiddenResponse' + 404: + description: + Not Found - NoSuchPolicyException, NoSuchTargetException, NoSuchMappingException + content: + application/json: + schema: + $ref: '../iceberg-rest-catalog-open-api.yaml#/components/schemas/IcebergErrorResponse' + examples: + PolicyToUnsetDoesNotExist: + $ref: '#/components/examples/NoSuchPolicyError' + TargetToUnsetDoesNotExist: + $ref: '#/components/examples/NoSuchTargetError' + MappingToUnsetDoesNotExist: + $ref: '#/components/examples/NoSuchMappingError' + 503: + $ref: '../iceberg-rest-catalog-open-api.yaml#/components/responses/ServiceUnavailableResponse' + 5XX: + $ref: '../iceberg-rest-catalog-open-api.yaml#/components/responses/ServerErrorResponse' + + /polaris/v1/{prefix}/applicablePolicies: + parameters: + - $ref: '../iceberg-rest-catalog-open-api.yaml#/components/parameters/prefix' + + get: + tags: + - Policy API + summary: Get Applicable policies for catalog, namespace, table, or views + operationId: getApplicablePoliciesUsingParameter + description: | + Retrieves all applicable policies for a specified entity, including inherited policies from parent entities. An entity can be a table/view, namespace, or catalog. The required parameters depend on the entity type: + + - Table/View: + - The `namespace` parameter is required to specify the entity's namespace. + - The `name` parameter is required to specify the entity name. + - Namespace: + - The `namespace` parameter is required to specify the identifier. + - The `name` parameter should not be set. + - Catalog: + - Neither `namespace` nor `name` should be set. + + An optional policyType parameter filters results to return only policies of the specified type. + + This API evaluates the entity's hierarchy and applies inheritable policies from parent entities. + + parameters: + - $ref: '../iceberg-rest-catalog-open-api.yaml#/components/parameters/page-token' + - $ref: '../iceberg-rest-catalog-open-api.yaml#/components/parameters/page-size' + - name: namespace + in: query + required: false + description: + A namespace identifier as a single string. + Multipart namespace parts should be separated by the unit separator (`0x1F`) byte. + schema: + type: string + examples: + singlepart_namespace: + value: "accounting" + multipart_namespace: + value: "accounting%1Ftax" + - $ref: '#/components/parameters/name' Review Comment: Can we rename this to `table-like` or `target-name` to be more explicit? The term `name` seems too broad, may cause confusion. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
