Re: [PR] Improve GCS documentation formatting and fix grammatical issues [polaris]
github-actions[bot] closed pull request #4106: Improve GCS documentation formatting and fix grammatical issues URL: https://github.com/apache/polaris/pull/4106 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
Re: [PR] Improve GCS documentation formatting and fix grammatical issues [polaris]
github-actions[bot] commented on PR #4106: URL: https://github.com/apache/polaris/pull/4106#issuecomment-4644926461 This PR is stale because it has been open 30 days with no activity. Remove stale label or comment or this will be closed in 5 days. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
Re: [PR] Improve GCS documentation formatting and fix grammatical issues [polaris]
dimas-b commented on PR #4106: URL: https://github.com/apache/polaris/pull/4106#issuecomment-4408342196 Merging based on previous reviews - all concerns have been addressed as far as I can tell and there are no extra changes in text. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
Re: [PR] Improve GCS documentation formatting and fix grammatical issues [polaris]
dimas-b commented on PR #4106: URL: https://github.com/apache/polaris/pull/4106#issuecomment-4408359153 @fivetran-caseykarst : Actually, the PR still have old commits in its history that refer to Clause... WDYT about squashing? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
Re: [PR] Improve GCS documentation formatting and fix grammatical issues [polaris]
cakarst commented on PR #4106: URL: https://github.com/apache/polaris/pull/4106#issuecomment-4408153030 PR fell through the cracks. I updated the commit message and rebased. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
Re: [PR] Improve GCS documentation formatting and fix grammatical issues [polaris]
dimas-b commented on PR #4106: URL: https://github.com/apache/polaris/pull/4106#issuecomment-4399307940 @fivetran-caseykarst : WDYT about JB message? https://github.com/apache/polaris/pull/4106#issuecomment-4192728124 Also, this PR needs a rebase for CI to pass. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
Re: [PR] Improve GCS documentation formatting and fix grammatical issues [polaris]
github-actions[bot] commented on PR #4106: URL: https://github.com/apache/polaris/pull/4106#issuecomment-4393693659 This PR is stale because it has been open 30 days with no activity. Remove stale label or comment or this will be closed in 5 days. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
Re: [PR] Improve GCS documentation formatting and fix grammatical issues [polaris]
jbonofre commented on PR #4106: URL: https://github.com/apache/polaris/pull/4106#issuecomment-4192728124 @fivetran-caseykarst that's all good for me. I just wanted to understand better the Claude Code use here. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
Re: [PR] Improve GCS documentation formatting and fix grammatical issues [polaris]
fivetran-caseykarst commented on PR #4106: URL: https://github.com/apache/polaris/pull/4106#issuecomment-4192680668 @jbonofre How do you suggest I move forward here? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
Re: [PR] Improve GCS documentation formatting and fix grammatical issues [polaris]
fivetran-caseykarst commented on PR #4106: URL: https://github.com/apache/polaris/pull/4106#issuecomment-4184704418 It was a claude code based PR. I used claude to propose an edit to the existing doc based on my experience trying to host Polaris in GCS. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
Re: [PR] Improve GCS documentation formatting and fix grammatical issues [polaris]
jbonofre commented on PR #4106: URL: https://github.com/apache/polaris/pull/4106#issuecomment-4183784112 About my previous comment: 1. If claude helped for PR description, we can keep the claude mention 2. If claude helped for PR change, the author/contributor has to review what has been generated and push "on its own". So I would remove claude co-author here. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
Re: [PR] Improve GCS documentation formatting and fix grammatical issues [polaris]
jbonofre commented on PR #4106: URL: https://github.com/apache/polaris/pull/4106#issuecomment-4183774027 What has been generated by claude ? The PR description or the PR change or both ? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
Re: [PR] Improve GCS documentation formatting and fix grammatical issues [polaris]
dimas-b commented on PR #4106: URL: https://github.com/apache/polaris/pull/4106#issuecomment-4178589850 Thanks for the update, @cakarst ! Let's keep this PR in review for another day so that other people have a chance to comment. Planning to merge tomorrow. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
Re: [PR] Improve GCS documentation formatting and fix grammatical issues [polaris]
dimas-b commented on code in PR #4106: URL: https://github.com/apache/polaris/pull/4106#discussion_r3024290492 ## site/content/in-dev/unreleased/configuration/configuring-polaris-for-production/configuring-gcs-cloud-storage-specific.md: ## @@ -23,10 +23,55 @@ type: docs weight: 600 --- -This page provides guidance for configuring GCS Cloud Storage provider for use with Polaris. It covers credential vending, IAM roles, ACL requirements, and best practices to ensure secure and reliable integration. +This guide covers how to configure Google Cloud Storage (GCS) as a storage backend for Polaris catalogs, including credential vending, IAM configuration, and access control. -All catalog operations in Polaris for Google Cloud Storage (GCS)—including listing, reading, and writing objects—are performed using credential vending, which issues scoped (vended) tokens for secure access. +## Overview -Polaris requires both IAM roles and [Hierarchical Namespace (HNS)](https://docs.cloud.google.com/storage/docs/hns-overview) ACLs (if HNS is enabled) to be properly configured. Even with the correct IAM role (e.g., `roles/storage.objectAdmin`), access to paths such as `gs:///idsp_ns/sample_table4/` may fail with 403 errors if HNS ACLs are missing for scoped tokens. The original access token may work, but scoped (vended) tokens require HNS ACLs on the base path or relevant subpath. +Polaris uses **credential vending** to securely manage access to GCS objects. When you configure a catalog with GCS storage, Polaris issues scoped (vended) tokens with limited permissions and duration for each operation, rather than using long-lived credentials. -**Note:** HNS is not mandatory when using GCS for a catalog in Polaris. If HNS is not enabled on the bucket, only IAM roles are required for access. Always verify HNS ACLs in addition to IAM roles when troubleshooting GCS access issues with credential vending and HNS enabled. +## Storage Configuration + +When creating a Polaris catalog with GCS storage, you need to specify: + +1. **Storage Type**: `GCS` +2. **Base Location**: The default GCS path for the catalog (e.g., `gs://your-bucket/catalogs/catalog-name`) +3. **Allowed Locations**: GCS paths where the catalog can read/write data + +## IAM Configuration + +### Service Account Permissions + +The service account running Polaris (e.g., on Cloud Run) needs appropriate IAM roles to access GCS: + +**Required IAM Roles:** +- `roles/storage.objectAdmin` - For read/write access to objects +- OR `roles/storage.objectViewer` + `roles/storage.objectCreator` - For more granular control + +Grant the role at the bucket level: + +```bash +gsutil iam ch serviceAccount:[email protected]:roles/storage.objectAdmin gs://your-bucket +``` + +### User Access Permissions + +In addition to GCS IAM, users need Polaris catalog roles to access tables: + +1. Create a catalog role with appropriate privileges: + - `TABLE_READ_DATA` - Read table data + - `TABLE_WRITE_DATA` - Write table data + - `NAMESPACE_FULL_METADATA` - Access namespace/table metadata +2. Assign the catalog role to a principal role (e.g., `service_admin`) + +This two-level permission model ensures both GCS access (via IAM) and Polaris access control (via catalog roles) are properly configured. + +## Google Cloud Storage Configuration +The preferred GCS configuration to have Hierarchical Namespaces disabled on the bucket and Fine-grained ACLS for access control. Review Comment: I personally do not have enough data to say what works and what does not with certainty :wink: I know of some cases with 403 errors in HNS GCS storage, but I cannot rule out mistakes :slightly_smiling_face: PR #3996 is still in review. "Verified" would assume Polaris as a project stands behind it, but we do not have CI for GCS, so anything that works now is not guaranteed to work tomorrow :shrug: Proposal: `GCS storage without hierarchical namespaces has been confirmed by the user community to work fine with Polaris. However, issues have been reported for hierarchical namespaces, so they should be considered with caution in production deployments.` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
Re: [PR] Improve GCS documentation formatting and fix grammatical issues [polaris]
dimas-b commented on code in PR #4106: URL: https://github.com/apache/polaris/pull/4106#discussion_r3024290492 ## site/content/in-dev/unreleased/configuration/configuring-polaris-for-production/configuring-gcs-cloud-storage-specific.md: ## @@ -23,10 +23,55 @@ type: docs weight: 600 --- -This page provides guidance for configuring GCS Cloud Storage provider for use with Polaris. It covers credential vending, IAM roles, ACL requirements, and best practices to ensure secure and reliable integration. +This guide covers how to configure Google Cloud Storage (GCS) as a storage backend for Polaris catalogs, including credential vending, IAM configuration, and access control. -All catalog operations in Polaris for Google Cloud Storage (GCS)—including listing, reading, and writing objects—are performed using credential vending, which issues scoped (vended) tokens for secure access. +## Overview -Polaris requires both IAM roles and [Hierarchical Namespace (HNS)](https://docs.cloud.google.com/storage/docs/hns-overview) ACLs (if HNS is enabled) to be properly configured. Even with the correct IAM role (e.g., `roles/storage.objectAdmin`), access to paths such as `gs:///idsp_ns/sample_table4/` may fail with 403 errors if HNS ACLs are missing for scoped tokens. The original access token may work, but scoped (vended) tokens require HNS ACLs on the base path or relevant subpath. +Polaris uses **credential vending** to securely manage access to GCS objects. When you configure a catalog with GCS storage, Polaris issues scoped (vended) tokens with limited permissions and duration for each operation, rather than using long-lived credentials. -**Note:** HNS is not mandatory when using GCS for a catalog in Polaris. If HNS is not enabled on the bucket, only IAM roles are required for access. Always verify HNS ACLs in addition to IAM roles when troubleshooting GCS access issues with credential vending and HNS enabled. +## Storage Configuration + +When creating a Polaris catalog with GCS storage, you need to specify: + +1. **Storage Type**: `GCS` +2. **Base Location**: The default GCS path for the catalog (e.g., `gs://your-bucket/catalogs/catalog-name`) +3. **Allowed Locations**: GCS paths where the catalog can read/write data + +## IAM Configuration + +### Service Account Permissions + +The service account running Polaris (e.g., on Cloud Run) needs appropriate IAM roles to access GCS: + +**Required IAM Roles:** +- `roles/storage.objectAdmin` - For read/write access to objects +- OR `roles/storage.objectViewer` + `roles/storage.objectCreator` - For more granular control + +Grant the role at the bucket level: + +```bash +gsutil iam ch serviceAccount:[email protected]:roles/storage.objectAdmin gs://your-bucket +``` + +### User Access Permissions + +In addition to GCS IAM, users need Polaris catalog roles to access tables: + +1. Create a catalog role with appropriate privileges: + - `TABLE_READ_DATA` - Read table data + - `TABLE_WRITE_DATA` - Write table data + - `NAMESPACE_FULL_METADATA` - Access namespace/table metadata +2. Assign the catalog role to a principal role (e.g., `service_admin`) + +This two-level permission model ensures both GCS access (via IAM) and Polaris access control (via catalog roles) are properly configured. + +## Google Cloud Storage Configuration +The preferred GCS configuration to have Hierarchical Namespaces disabled on the bucket and Fine-grained ACLS for access control. Review Comment: I personally do not have enough data to say what works and what does not with certainty :wink: I know of some cases with 403 errors in HNS GCS storage, but I cannot rule out mistakes :slightly_smiling_face: PR #3996 is still in review. "Verified" would assume Polaris as a project stands behind it, but we do not have CI for GCS, so anything that works now is not guaranteed to work tomorrow :shrug: Proposal: `GCS storage without hierarchical namespaces have been confirmed by the user community to work fine with Polaris. However, issues have been reported for hierarchical namespaces, so they should be considered with caution in production deployments.` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
Re: [PR] Improve GCS documentation formatting and fix grammatical issues [polaris]
fivetran-caseykarst commented on code in PR #4106: URL: https://github.com/apache/polaris/pull/4106#discussion_r3023904445 ## site/content/in-dev/unreleased/configuration/configuring-polaris-for-production/configuring-gcs-cloud-storage-specific.md: ## @@ -23,10 +23,55 @@ type: docs weight: 600 --- -This page provides guidance for configuring GCS Cloud Storage provider for use with Polaris. It covers credential vending, IAM roles, ACL requirements, and best practices to ensure secure and reliable integration. +This guide covers how to configure Google Cloud Storage (GCS) as a storage backend for Polaris catalogs, including credential vending, IAM configuration, and access control. -All catalog operations in Polaris for Google Cloud Storage (GCS)—including listing, reading, and writing objects—are performed using credential vending, which issues scoped (vended) tokens for secure access. +## Overview -Polaris requires both IAM roles and [Hierarchical Namespace (HNS)](https://docs.cloud.google.com/storage/docs/hns-overview) ACLs (if HNS is enabled) to be properly configured. Even with the correct IAM role (e.g., `roles/storage.objectAdmin`), access to paths such as `gs:///idsp_ns/sample_table4/` may fail with 403 errors if HNS ACLs are missing for scoped tokens. The original access token may work, but scoped (vended) tokens require HNS ACLs on the base path or relevant subpath. +Polaris uses **credential vending** to securely manage access to GCS objects. When you configure a catalog with GCS storage, Polaris issues scoped (vended) tokens with limited permissions and duration for each operation, rather than using long-lived credentials. -**Note:** HNS is not mandatory when using GCS for a catalog in Polaris. If HNS is not enabled on the bucket, only IAM roles are required for access. Always verify HNS ACLs in addition to IAM roles when troubleshooting GCS access issues with credential vending and HNS enabled. +## Storage Configuration + +When creating a Polaris catalog with GCS storage, you need to specify: + +1. **Storage Type**: `GCS` +2. **Base Location**: The default GCS path for the catalog (e.g., `gs://your-bucket/catalogs/catalog-name`) +3. **Allowed Locations**: GCS paths where the catalog can read/write data + +## IAM Configuration + +### Service Account Permissions + +The service account running Polaris (e.g., on Cloud Run) needs appropriate IAM roles to access GCS: + +**Required IAM Roles:** +- `roles/storage.objectAdmin` - For read/write access to objects +- OR `roles/storage.objectViewer` + `roles/storage.objectCreator` - For more granular control + +Grant the role at the bucket level: + +```bash +gsutil iam ch serviceAccount:[email protected]:roles/storage.objectAdmin gs://your-bucket +``` + +### User Access Permissions + +In addition to GCS IAM, users need Polaris catalog roles to access tables: + +1. Create a catalog role with appropriate privileges: + - `TABLE_READ_DATA` - Read table data + - `TABLE_WRITE_DATA` - Write table data + - `NAMESPACE_FULL_METADATA` - Access namespace/table metadata +2. Assign the catalog role to a principal role (e.g., `service_admin`) + +This two-level permission model ensures both GCS access (via IAM) and Polaris access control (via catalog roles) are properly configured. + +## Google Cloud Storage Configuration +The preferred GCS configuration to have Hierarchical Namespaces disabled on the bucket and Fine-grained ACLS for access control. Review Comment: I mean we do have data on what works. Experimental implies someone has confirmed it at least works. To date I have not seen that confirmation. What about saying "verified". -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
Re: [PR] Improve GCS documentation formatting and fix grammatical issues [polaris]
fivetran-caseykarst commented on code in PR #4106: URL: https://github.com/apache/polaris/pull/4106#discussion_r3023904445 ## site/content/in-dev/unreleased/configuration/configuring-polaris-for-production/configuring-gcs-cloud-storage-specific.md: ## @@ -23,10 +23,55 @@ type: docs weight: 600 --- -This page provides guidance for configuring GCS Cloud Storage provider for use with Polaris. It covers credential vending, IAM roles, ACL requirements, and best practices to ensure secure and reliable integration. +This guide covers how to configure Google Cloud Storage (GCS) as a storage backend for Polaris catalogs, including credential vending, IAM configuration, and access control. -All catalog operations in Polaris for Google Cloud Storage (GCS)—including listing, reading, and writing objects—are performed using credential vending, which issues scoped (vended) tokens for secure access. +## Overview -Polaris requires both IAM roles and [Hierarchical Namespace (HNS)](https://docs.cloud.google.com/storage/docs/hns-overview) ACLs (if HNS is enabled) to be properly configured. Even with the correct IAM role (e.g., `roles/storage.objectAdmin`), access to paths such as `gs:///idsp_ns/sample_table4/` may fail with 403 errors if HNS ACLs are missing for scoped tokens. The original access token may work, but scoped (vended) tokens require HNS ACLs on the base path or relevant subpath. +Polaris uses **credential vending** to securely manage access to GCS objects. When you configure a catalog with GCS storage, Polaris issues scoped (vended) tokens with limited permissions and duration for each operation, rather than using long-lived credentials. -**Note:** HNS is not mandatory when using GCS for a catalog in Polaris. If HNS is not enabled on the bucket, only IAM roles are required for access. Always verify HNS ACLs in addition to IAM roles when troubleshooting GCS access issues with credential vending and HNS enabled. +## Storage Configuration + +When creating a Polaris catalog with GCS storage, you need to specify: + +1. **Storage Type**: `GCS` +2. **Base Location**: The default GCS path for the catalog (e.g., `gs://your-bucket/catalogs/catalog-name`) +3. **Allowed Locations**: GCS paths where the catalog can read/write data + +## IAM Configuration + +### Service Account Permissions + +The service account running Polaris (e.g., on Cloud Run) needs appropriate IAM roles to access GCS: + +**Required IAM Roles:** +- `roles/storage.objectAdmin` - For read/write access to objects +- OR `roles/storage.objectViewer` + `roles/storage.objectCreator` - For more granular control + +Grant the role at the bucket level: + +```bash +gsutil iam ch serviceAccount:[email protected]:roles/storage.objectAdmin gs://your-bucket +``` + +### User Access Permissions + +In addition to GCS IAM, users need Polaris catalog roles to access tables: + +1. Create a catalog role with appropriate privileges: + - `TABLE_READ_DATA` - Read table data + - `TABLE_WRITE_DATA` - Write table data + - `NAMESPACE_FULL_METADATA` - Access namespace/table metadata +2. Assign the catalog role to a principal role (e.g., `service_admin`) + +This two-level permission model ensures both GCS access (via IAM) and Polaris access control (via catalog roles) are properly configured. + +## Google Cloud Storage Configuration +The preferred GCS configuration to have Hierarchical Namespaces disabled on the bucket and Fine-grained ACLS for access control. Review Comment: I mean we do have data on what works. Experimental implies someone has confirmed it at least works. What about saying "verified". -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
Re: [PR] Improve GCS documentation formatting and fix grammatical issues [polaris]
dimas-b commented on code in PR #4106: URL: https://github.com/apache/polaris/pull/4106#discussion_r3023088112 ## site/content/in-dev/unreleased/configuration/configuring-polaris-for-production/configuring-gcs-cloud-storage-specific.md: ## @@ -23,10 +23,55 @@ type: docs weight: 600 --- -This page provides guidance for configuring GCS Cloud Storage provider for use with Polaris. It covers credential vending, IAM roles, ACL requirements, and best practices to ensure secure and reliable integration. +This guide covers how to configure Google Cloud Storage (GCS) as a storage backend for Polaris catalogs, including credential vending, IAM configuration, and access control. -All catalog operations in Polaris for Google Cloud Storage (GCS)—including listing, reading, and writing objects—are performed using credential vending, which issues scoped (vended) tokens for secure access. +## Overview -Polaris requires both IAM roles and [Hierarchical Namespace (HNS)](https://docs.cloud.google.com/storage/docs/hns-overview) ACLs (if HNS is enabled) to be properly configured. Even with the correct IAM role (e.g., `roles/storage.objectAdmin`), access to paths such as `gs:///idsp_ns/sample_table4/` may fail with 403 errors if HNS ACLs are missing for scoped tokens. The original access token may work, but scoped (vended) tokens require HNS ACLs on the base path or relevant subpath. +Polaris uses **credential vending** to securely manage access to GCS objects. When you configure a catalog with GCS storage, Polaris issues scoped (vended) tokens with limited permissions and duration for each operation, rather than using long-lived credentials. -**Note:** HNS is not mandatory when using GCS for a catalog in Polaris. If HNS is not enabled on the bucket, only IAM roles are required for access. Always verify HNS ACLs in addition to IAM roles when troubleshooting GCS access issues with credential vending and HNS enabled. +## Storage Configuration + +When creating a Polaris catalog with GCS storage, you need to specify: + +1. **Storage Type**: `GCS` +2. **Base Location**: The default GCS path for the catalog (e.g., `gs://your-bucket/catalogs/catalog-name`) +3. **Allowed Locations**: GCS paths where the catalog can read/write data + +## IAM Configuration + +### Service Account Permissions + +The service account running Polaris (e.g., on Cloud Run) needs appropriate IAM roles to access GCS: + +**Required IAM Roles:** +- `roles/storage.objectAdmin` - For read/write access to objects +- OR `roles/storage.objectViewer` + `roles/storage.objectCreator` - For more granular control + +Grant the role at the bucket level: + +```bash +gsutil iam ch serviceAccount:[email protected]:roles/storage.objectAdmin gs://your-bucket +``` + +### User Access Permissions + +In addition to GCS IAM, users need Polaris catalog roles to access tables: + +1. Create a catalog role with appropriate privileges: + - `TABLE_READ_DATA` - Read table data + - `TABLE_WRITE_DATA` - Write table data + - `NAMESPACE_FULL_METADATA` - Access namespace/table metadata +2. Assign the catalog role to a principal role (e.g., `service_admin`) + +This two-level permission model ensures both GCS access (via IAM) and Polaris access control (via catalog roles) are properly configured. + +## Google Cloud Storage Configuration +The preferred GCS configuration to have Hierarchical Namespaces disabled on the bucket and Fine-grained ACLS for access control. Review Comment: I'm still not comfortable with "preferred ... HNS disabled". We're in uncharted territory here apparently, so I do not think we have enough data to guide the users with certainty. I'd prefer something like `Using HNS storage should be considered "experimental" ...` WDYT? Maybe also open a GH issue with specific HNS-related errors? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
Re: [PR] Improve GCS documentation formatting and fix grammatical issues [polaris]
dimas-b commented on code in PR #4106: URL: https://github.com/apache/polaris/pull/4106#discussion_r3023088112 ## site/content/in-dev/unreleased/configuration/configuring-polaris-for-production/configuring-gcs-cloud-storage-specific.md: ## @@ -23,10 +23,55 @@ type: docs weight: 600 --- -This page provides guidance for configuring GCS Cloud Storage provider for use with Polaris. It covers credential vending, IAM roles, ACL requirements, and best practices to ensure secure and reliable integration. +This guide covers how to configure Google Cloud Storage (GCS) as a storage backend for Polaris catalogs, including credential vending, IAM configuration, and access control. -All catalog operations in Polaris for Google Cloud Storage (GCS)—including listing, reading, and writing objects—are performed using credential vending, which issues scoped (vended) tokens for secure access. +## Overview -Polaris requires both IAM roles and [Hierarchical Namespace (HNS)](https://docs.cloud.google.com/storage/docs/hns-overview) ACLs (if HNS is enabled) to be properly configured. Even with the correct IAM role (e.g., `roles/storage.objectAdmin`), access to paths such as `gs:///idsp_ns/sample_table4/` may fail with 403 errors if HNS ACLs are missing for scoped tokens. The original access token may work, but scoped (vended) tokens require HNS ACLs on the base path or relevant subpath. +Polaris uses **credential vending** to securely manage access to GCS objects. When you configure a catalog with GCS storage, Polaris issues scoped (vended) tokens with limited permissions and duration for each operation, rather than using long-lived credentials. -**Note:** HNS is not mandatory when using GCS for a catalog in Polaris. If HNS is not enabled on the bucket, only IAM roles are required for access. Always verify HNS ACLs in addition to IAM roles when troubleshooting GCS access issues with credential vending and HNS enabled. +## Storage Configuration + +When creating a Polaris catalog with GCS storage, you need to specify: + +1. **Storage Type**: `GCS` +2. **Base Location**: The default GCS path for the catalog (e.g., `gs://your-bucket/catalogs/catalog-name`) +3. **Allowed Locations**: GCS paths where the catalog can read/write data + +## IAM Configuration + +### Service Account Permissions + +The service account running Polaris (e.g., on Cloud Run) needs appropriate IAM roles to access GCS: + +**Required IAM Roles:** +- `roles/storage.objectAdmin` - For read/write access to objects +- OR `roles/storage.objectViewer` + `roles/storage.objectCreator` - For more granular control + +Grant the role at the bucket level: + +```bash +gsutil iam ch serviceAccount:[email protected]:roles/storage.objectAdmin gs://your-bucket +``` + +### User Access Permissions + +In addition to GCS IAM, users need Polaris catalog roles to access tables: + +1. Create a catalog role with appropriate privileges: + - `TABLE_READ_DATA` - Read table data + - `TABLE_WRITE_DATA` - Write table data + - `NAMESPACE_FULL_METADATA` - Access namespace/table metadata +2. Assign the catalog role to a principal role (e.g., `service_admin`) + +This two-level permission model ensures both GCS access (via IAM) and Polaris access control (via catalog roles) are properly configured. + +## Google Cloud Storage Configuration +The preferred GCS configuration to have Hierarchical Namespaces disabled on the bucket and Fine-grained ACLS for access control. Review Comment: I'm still not comfortable with "preferred ... HNS disabled". We're in uncharted territory here apparently, so I do not think we have enough data to guide the users with certainty. I'd prefer something like `Using HNS storage should be considered "experimental" ...` WDYT? Maybe also open GH issue with specific HNS-related errors? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
Re: [PR] Improve GCS documentation formatting and fix grammatical issues [polaris]
dimas-b commented on code in PR #4106:
URL: https://github.com/apache/polaris/pull/4106#discussion_r3023068559
##
site/content/in-dev/unreleased/configuration/configuring-polaris-for-production/configuring-gcs-cloud-storage-specific.md:
##
@@ -1,32 +1,75 @@
-#
-# Licensed to the Apache Software Foundation (ASF) under one
-# or more contributor license agreements. See the NOTICE file
-# distributed with this work for additional information
-# regarding copyright ownership. The ASF licenses this file
-# to you under the Apache License, Version 2.0 (the
-# "License"); you may not use this file except in compliance
-# with the License. You may obtain a copy of the License at
-#
-# http://www.apache.org/licenses/LICENSE-2.0
-#
-# Unless required by applicable law or agreed to in writing,
-# software distributed under the License is distributed on an
-# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
-# KIND, either express or implied. See the License for the
-# specific language governing permissions and limitations
-# under the License.
-#
-title: Configuring GCS Cloud Storage
-linkTitle: Configuring GCS Cloud Storage
-type: docs
-weight: 600
-
-This page provides guidance for configuring GCS Cloud Storage provider for use
with Polaris. It covers credential vending, IAM roles, ACL requirements, and
best practices to ensure secure and reliable integration.
-
-All catalog operations in Polaris for Google Cloud Storage (GCS)—including
listing, reading, and writing objects—are performed using credential vending,
which issues scoped (vended) tokens for secure access.
-
-Polaris requires both IAM roles and [Hierarchical Namespace
(HNS)](https://docs.cloud.google.com/storage/docs/hns-overview) ACLs (if HNS is
enabled) to be properly configured. Even with the correct IAM role (e.g.,
`roles/storage.objectAdmin`), access to paths such as
`gs:///idsp_ns/sample_table4/` may fail with 403 errors if HNS ACLs are
missing for scoped tokens. The original access token may work, but scoped
(vended) tokens require HNS ACLs on the base path or relevant subpath.
-
-**Note:** HNS is not mandatory when using GCS for a catalog in Polaris. If HNS
is not enabled on the bucket, only IAM roles are required for access. Always
verify HNS ACLs in addition to IAM roles when troubleshooting GCS access issues
with credential vending and HNS enabled.
+# Configuring GCS with Polaris
+
+This guide covers how to configure Google Cloud Storage (GCS) as a storage
backend for Polaris catalogs, including credential vending, IAM configuration,
and access control.
+
+## Overview
+
+Polaris uses **credential vending** to securely manage access to GCS objects.
When you configure a catalog with GCS storage, Polaris issues scoped (vended)
tokens with limited permissions and duration for each operation, rather than
using long-lived credentials.
+
+## Storage Configuration
+
+When creating a Polaris catalog with GCS storage, you need to specify:
+
+1. **Storage Type**: `GCS`
+2. **Base Location**: The default GCS path for the catalog (e.g.,
`gs://your-bucket/catalogs/catalog-name`)
+3. **Allowed Locations**: GCS paths where the catalog can read/write data
+
+### Example Catalog Configuration
+
+```json
+{
+ "catalog": {
+"type": "INTERNAL",
+"name": "my_catalog",
+"properties": {
+ "default-base-location": "gs://your-bucket/catalogs/my_catalog"
+},
+"storageConfigInfo": {
+ "storageType": "GCS",
+ "allowedLocations": [
+"gs://your-bucket"
+ ]
+}
+ }
+}
+```
+
+## IAM Configuration
+
+### Service Account Permissions
+
+The service account running Polaris (e.g., on Cloud Run) needs appropriate IAM
roles to access GCS:
+
+**Required IAM Roles:**
+- `roles/storage.objectAdmin` - For read/write access to objects
+- OR `roles/storage.objectViewer` + `roles/storage.objectCreator` - For more
granular control
+
+Grant the role at the bucket level:
+
+```bash
+gsutil iam ch
serviceAccount:[email protected]:roles/storage.objectAdmin
gs://your-bucket
+```
+
+### User Access Permissions
+
+In addition to GCS IAM, users need Polaris catalog roles to access tables:
+
+1. Create a catalog role with appropriate privileges:
Review Comment:
fair enough 👍
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
Re: [PR] Improve GCS documentation formatting and fix grammatical issues [polaris]
fivetran-caseykarst commented on code in PR #4106:
URL: https://github.com/apache/polaris/pull/4106#discussion_r3022908385
##
site/content/in-dev/unreleased/configuration/configuring-polaris-for-production/configuring-gcs-cloud-storage-specific.md:
##
@@ -1,32 +1,75 @@
-#
-# Licensed to the Apache Software Foundation (ASF) under one
-# or more contributor license agreements. See the NOTICE file
-# distributed with this work for additional information
-# regarding copyright ownership. The ASF licenses this file
-# to you under the Apache License, Version 2.0 (the
-# "License"); you may not use this file except in compliance
-# with the License. You may obtain a copy of the License at
-#
-# http://www.apache.org/licenses/LICENSE-2.0
-#
-# Unless required by applicable law or agreed to in writing,
-# software distributed under the License is distributed on an
-# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
-# KIND, either express or implied. See the License for the
-# specific language governing permissions and limitations
-# under the License.
-#
-title: Configuring GCS Cloud Storage
-linkTitle: Configuring GCS Cloud Storage
-type: docs
-weight: 600
-
-This page provides guidance for configuring GCS Cloud Storage provider for use
with Polaris. It covers credential vending, IAM roles, ACL requirements, and
best practices to ensure secure and reliable integration.
-
-All catalog operations in Polaris for Google Cloud Storage (GCS)—including
listing, reading, and writing objects—are performed using credential vending,
which issues scoped (vended) tokens for secure access.
-
-Polaris requires both IAM roles and [Hierarchical Namespace
(HNS)](https://docs.cloud.google.com/storage/docs/hns-overview) ACLs (if HNS is
enabled) to be properly configured. Even with the correct IAM role (e.g.,
`roles/storage.objectAdmin`), access to paths such as
`gs:///idsp_ns/sample_table4/` may fail with 403 errors if HNS ACLs are
missing for scoped tokens. The original access token may work, but scoped
(vended) tokens require HNS ACLs on the base path or relevant subpath.
-
-**Note:** HNS is not mandatory when using GCS for a catalog in Polaris. If HNS
is not enabled on the bucket, only IAM roles are required for access. Always
verify HNS ACLs in addition to IAM roles when troubleshooting GCS access issues
with credential vending and HNS enabled.
+# Configuring GCS with Polaris
+
+This guide covers how to configure Google Cloud Storage (GCS) as a storage
backend for Polaris catalogs, including credential vending, IAM configuration,
and access control.
+
+## Overview
+
+Polaris uses **credential vending** to securely manage access to GCS objects.
When you configure a catalog with GCS storage, Polaris issues scoped (vended)
tokens with limited permissions and duration for each operation, rather than
using long-lived credentials.
+
+## Storage Configuration
+
+When creating a Polaris catalog with GCS storage, you need to specify:
+
+1. **Storage Type**: `GCS`
+2. **Base Location**: The default GCS path for the catalog (e.g.,
`gs://your-bucket/catalogs/catalog-name`)
+3. **Allowed Locations**: GCS paths where the catalog can read/write data
+
+### Example Catalog Configuration
+
+```json
+{
+ "catalog": {
+"type": "INTERNAL",
+"name": "my_catalog",
+"properties": {
+ "default-base-location": "gs://your-bucket/catalogs/my_catalog"
+},
+"storageConfigInfo": {
+ "storageType": "GCS",
+ "allowedLocations": [
+"gs://your-bucket"
+ ]
+}
+ }
+}
+```
+
+## IAM Configuration
+
+### Service Account Permissions
+
+The service account running Polaris (e.g., on Cloud Run) needs appropriate IAM
roles to access GCS:
+
+**Required IAM Roles:**
+- `roles/storage.objectAdmin` - For read/write access to objects
+- OR `roles/storage.objectViewer` + `roles/storage.objectCreator` - For more
granular control
+
+Grant the role at the bucket level:
+
+```bash
+gsutil iam ch
serviceAccount:[email protected]:roles/storage.objectAdmin
gs://your-bucket
+```
+
+### User Access Permissions
+
+In addition to GCS IAM, users need Polaris catalog roles to access tables:
+
+1. Create a catalog role with appropriate privileges:
+ - `TABLE_READ_DATA` - Read table data
+ - `TABLE_WRITE_DATA` - Write table data
+ - `NAMESPACE_FULL_METADATA` - Access namespace/table metadata
+2. Assign the catalog role to a principal role (e.g., `service_admin`)
+
+This two-level permission model ensures both GCS access (via IAM) and Polaris
access control (via catalog roles) are properly configured.
+
+## Google Cloud Storage Limitation
+
+Polaris does not support Hierarchical Namespaces (HNS) on the bucket.
Review Comment:
The previous doc said that HNS required HN acls. In the UX, if you select
HNS the finegrained access control toggle is greyed out and the user is forced
to select Uniform. The previous document stated that you woul
Re: [PR] Improve GCS documentation formatting and fix grammatical issues [polaris]
fivetran-caseykarst commented on code in PR #4106:
URL: https://github.com/apache/polaris/pull/4106#discussion_r3022870846
##
site/content/in-dev/unreleased/configuration/configuring-polaris-for-production/configuring-gcs-cloud-storage-specific.md:
##
@@ -1,32 +1,75 @@
-#
-# Licensed to the Apache Software Foundation (ASF) under one
-# or more contributor license agreements. See the NOTICE file
-# distributed with this work for additional information
-# regarding copyright ownership. The ASF licenses this file
-# to you under the Apache License, Version 2.0 (the
-# "License"); you may not use this file except in compliance
-# with the License. You may obtain a copy of the License at
-#
-# http://www.apache.org/licenses/LICENSE-2.0
-#
-# Unless required by applicable law or agreed to in writing,
-# software distributed under the License is distributed on an
-# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
-# KIND, either express or implied. See the License for the
-# specific language governing permissions and limitations
-# under the License.
-#
-title: Configuring GCS Cloud Storage
-linkTitle: Configuring GCS Cloud Storage
-type: docs
-weight: 600
-
-This page provides guidance for configuring GCS Cloud Storage provider for use
with Polaris. It covers credential vending, IAM roles, ACL requirements, and
best practices to ensure secure and reliable integration.
-
-All catalog operations in Polaris for Google Cloud Storage (GCS)—including
listing, reading, and writing objects—are performed using credential vending,
which issues scoped (vended) tokens for secure access.
-
-Polaris requires both IAM roles and [Hierarchical Namespace
(HNS)](https://docs.cloud.google.com/storage/docs/hns-overview) ACLs (if HNS is
enabled) to be properly configured. Even with the correct IAM role (e.g.,
`roles/storage.objectAdmin`), access to paths such as
`gs:///idsp_ns/sample_table4/` may fail with 403 errors if HNS ACLs are
missing for scoped tokens. The original access token may work, but scoped
(vended) tokens require HNS ACLs on the base path or relevant subpath.
-
-**Note:** HNS is not mandatory when using GCS for a catalog in Polaris. If HNS
is not enabled on the bucket, only IAM roles are required for access. Always
verify HNS ACLs in addition to IAM roles when troubleshooting GCS access issues
with credential vending and HNS enabled.
+# Configuring GCS with Polaris
+
+This guide covers how to configure Google Cloud Storage (GCS) as a storage
backend for Polaris catalogs, including credential vending, IAM configuration,
and access control.
+
+## Overview
+
+Polaris uses **credential vending** to securely manage access to GCS objects.
When you configure a catalog with GCS storage, Polaris issues scoped (vended)
tokens with limited permissions and duration for each operation, rather than
using long-lived credentials.
+
+## Storage Configuration
+
+When creating a Polaris catalog with GCS storage, you need to specify:
+
+1. **Storage Type**: `GCS`
+2. **Base Location**: The default GCS path for the catalog (e.g.,
`gs://your-bucket/catalogs/catalog-name`)
+3. **Allowed Locations**: GCS paths where the catalog can read/write data
+
+### Example Catalog Configuration
+
+```json
+{
+ "catalog": {
+"type": "INTERNAL",
+"name": "my_catalog",
+"properties": {
+ "default-base-location": "gs://your-bucket/catalogs/my_catalog"
+},
+"storageConfigInfo": {
+ "storageType": "GCS",
+ "allowedLocations": [
+"gs://your-bucket"
Review Comment:
Happy to remove.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
Re: [PR] Improve GCS documentation formatting and fix grammatical issues [polaris]
dimas-b commented on code in PR #4106:
URL: https://github.com/apache/polaris/pull/4106#discussion_r3022805268
##
site/content/in-dev/unreleased/configuration/configuring-polaris-for-production/configuring-gcs-cloud-storage-specific.md:
##
@@ -1,32 +1,75 @@
-#
-# Licensed to the Apache Software Foundation (ASF) under one
Review Comment:
why remove this header?
##
site/content/in-dev/unreleased/configuration/configuring-polaris-for-production/configuring-gcs-cloud-storage-specific.md:
##
@@ -1,32 +1,75 @@
-#
-# Licensed to the Apache Software Foundation (ASF) under one
-# or more contributor license agreements. See the NOTICE file
-# distributed with this work for additional information
-# regarding copyright ownership. The ASF licenses this file
-# to you under the Apache License, Version 2.0 (the
-# "License"); you may not use this file except in compliance
-# with the License. You may obtain a copy of the License at
-#
-# http://www.apache.org/licenses/LICENSE-2.0
-#
-# Unless required by applicable law or agreed to in writing,
-# software distributed under the License is distributed on an
-# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
-# KIND, either express or implied. See the License for the
-# specific language governing permissions and limitations
-# under the License.
-#
-title: Configuring GCS Cloud Storage
-linkTitle: Configuring GCS Cloud Storage
-type: docs
-weight: 600
-
-This page provides guidance for configuring GCS Cloud Storage provider for use
with Polaris. It covers credential vending, IAM roles, ACL requirements, and
best practices to ensure secure and reliable integration.
-
-All catalog operations in Polaris for Google Cloud Storage (GCS)—including
listing, reading, and writing objects—are performed using credential vending,
which issues scoped (vended) tokens for secure access.
-
-Polaris requires both IAM roles and [Hierarchical Namespace
(HNS)](https://docs.cloud.google.com/storage/docs/hns-overview) ACLs (if HNS is
enabled) to be properly configured. Even with the correct IAM role (e.g.,
`roles/storage.objectAdmin`), access to paths such as
`gs:///idsp_ns/sample_table4/` may fail with 403 errors if HNS ACLs are
missing for scoped tokens. The original access token may work, but scoped
(vended) tokens require HNS ACLs on the base path or relevant subpath.
-
-**Note:** HNS is not mandatory when using GCS for a catalog in Polaris. If HNS
is not enabled on the bucket, only IAM roles are required for access. Always
verify HNS ACLs in addition to IAM roles when troubleshooting GCS access issues
with credential vending and HNS enabled.
+# Configuring GCS with Polaris
+
+This guide covers how to configure Google Cloud Storage (GCS) as a storage
backend for Polaris catalogs, including credential vending, IAM configuration,
and access control.
+
+## Overview
+
+Polaris uses **credential vending** to securely manage access to GCS objects.
When you configure a catalog with GCS storage, Polaris issues scoped (vended)
tokens with limited permissions and duration for each operation, rather than
using long-lived credentials.
+
+## Storage Configuration
+
+When creating a Polaris catalog with GCS storage, you need to specify:
+
+1. **Storage Type**: `GCS`
+2. **Base Location**: The default GCS path for the catalog (e.g.,
`gs://your-bucket/catalogs/catalog-name`)
+3. **Allowed Locations**: GCS paths where the catalog can read/write data
+
+### Example Catalog Configuration
+
+```json
+{
+ "catalog": {
+"type": "INTERNAL",
+"name": "my_catalog",
+"properties": {
+ "default-base-location": "gs://your-bucket/catalogs/my_catalog"
+},
+"storageConfigInfo": {
+ "storageType": "GCS",
+ "allowedLocations": [
+"gs://your-bucket"
Review Comment:
Using `allowedLocations` that have wider scope than `default-base-location`
is valid, but it's an advanced use case, IMHO, and needs a targeted
discussion... probably not the best choice for a simple example 🤔
##
site/content/in-dev/unreleased/configuration/configuring-polaris-for-production/configuring-gcs-cloud-storage-specific.md:
##
@@ -1,32 +1,75 @@
-#
-# Licensed to the Apache Software Foundation (ASF) under one
-# or more contributor license agreements. See the NOTICE file
-# distributed with this work for additional information
-# regarding copyright ownership. The ASF licenses this file
-# to you under the Apache License, Version 2.0 (the
-# "License"); you may not use this file except in compliance
-# with the License. You may obtain a copy of the License at
-#
-# http://www.apache.org/licenses/LICENSE-2.0
-#
-# Unless required by applicable law or agreed to in writing,
-# software distributed under the License is distributed on an
-# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
-# KIND, either express or implied. See the License for the
-# specific language governing permiss
Re: [PR] Improve GCS documentation formatting and fix grammatical issues [polaris]
fivetran-caseykarst commented on code in PR #4106:
URL: https://github.com/apache/polaris/pull/4106#discussion_r3022877078
##
site/content/in-dev/unreleased/configuration/configuring-polaris-for-production/configuring-gcs-cloud-storage-specific.md:
##
@@ -1,32 +1,75 @@
-#
-# Licensed to the Apache Software Foundation (ASF) under one
-# or more contributor license agreements. See the NOTICE file
-# distributed with this work for additional information
-# regarding copyright ownership. The ASF licenses this file
-# to you under the Apache License, Version 2.0 (the
-# "License"); you may not use this file except in compliance
-# with the License. You may obtain a copy of the License at
-#
-# http://www.apache.org/licenses/LICENSE-2.0
-#
-# Unless required by applicable law or agreed to in writing,
-# software distributed under the License is distributed on an
-# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
-# KIND, either express or implied. See the License for the
-# specific language governing permissions and limitations
-# under the License.
-#
-title: Configuring GCS Cloud Storage
-linkTitle: Configuring GCS Cloud Storage
-type: docs
-weight: 600
-
-This page provides guidance for configuring GCS Cloud Storage provider for use
with Polaris. It covers credential vending, IAM roles, ACL requirements, and
best practices to ensure secure and reliable integration.
-
-All catalog operations in Polaris for Google Cloud Storage (GCS)—including
listing, reading, and writing objects—are performed using credential vending,
which issues scoped (vended) tokens for secure access.
-
-Polaris requires both IAM roles and [Hierarchical Namespace
(HNS)](https://docs.cloud.google.com/storage/docs/hns-overview) ACLs (if HNS is
enabled) to be properly configured. Even with the correct IAM role (e.g.,
`roles/storage.objectAdmin`), access to paths such as
`gs:///idsp_ns/sample_table4/` may fail with 403 errors if HNS ACLs are
missing for scoped tokens. The original access token may work, but scoped
(vended) tokens require HNS ACLs on the base path or relevant subpath.
-
-**Note:** HNS is not mandatory when using GCS for a catalog in Polaris. If HNS
is not enabled on the bucket, only IAM roles are required for access. Always
verify HNS ACLs in addition to IAM roles when troubleshooting GCS access issues
with credential vending and HNS enabled.
+# Configuring GCS with Polaris
+
+This guide covers how to configure Google Cloud Storage (GCS) as a storage
backend for Polaris catalogs, including credential vending, IAM configuration,
and access control.
+
+## Overview
+
+Polaris uses **credential vending** to securely manage access to GCS objects.
When you configure a catalog with GCS storage, Polaris issues scoped (vended)
tokens with limited permissions and duration for each operation, rather than
using long-lived credentials.
+
+## Storage Configuration
+
+When creating a Polaris catalog with GCS storage, you need to specify:
+
+1. **Storage Type**: `GCS`
+2. **Base Location**: The default GCS path for the catalog (e.g.,
`gs://your-bucket/catalogs/catalog-name`)
+3. **Allowed Locations**: GCS paths where the catalog can read/write data
+
+### Example Catalog Configuration
+
+```json
+{
+ "catalog": {
+"type": "INTERNAL",
+"name": "my_catalog",
+"properties": {
+ "default-base-location": "gs://your-bucket/catalogs/my_catalog"
+},
+"storageConfigInfo": {
+ "storageType": "GCS",
+ "allowedLocations": [
+"gs://your-bucket"
+ ]
+}
+ }
+}
+```
+
+## IAM Configuration
+
+### Service Account Permissions
+
+The service account running Polaris (e.g., on Cloud Run) needs appropriate IAM
roles to access GCS:
+
+**Required IAM Roles:**
+- `roles/storage.objectAdmin` - For read/write access to objects
+- OR `roles/storage.objectViewer` + `roles/storage.objectCreator` - For more
granular control
+
+Grant the role at the bucket level:
+
+```bash
+gsutil iam ch
serviceAccount:[email protected]:roles/storage.objectAdmin
gs://your-bucket
+```
+
+### User Access Permissions
+
+In addition to GCS IAM, users need Polaris catalog roles to access tables:
+
+1. Create a catalog role with appropriate privileges:
Review Comment:
Depends on what we want this doc to be. I personally like e2e docs that help
me/agent to set something up end to end rather than have to crawl the entire
docs. I will defer to community though on how docs need to be setup
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
Re: [PR] Improve GCS documentation formatting and fix grammatical issues [polaris]
fivetran-caseykarst commented on code in PR #4106: URL: https://github.com/apache/polaris/pull/4106#discussion_r3022868208 ## site/content/in-dev/unreleased/configuration/configuring-polaris-for-production/configuring-gcs-cloud-storage-specific.md: ## @@ -1,32 +1,75 @@ -# -# Licensed to the Apache Software Foundation (ASF) under one Review Comment: claude being stupid. Will add -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
Re: [PR] Improve GCS documentation formatting and fix grammatical issues [polaris]
adutra commented on PR #4106: URL: https://github.com/apache/polaris/pull/4106#issuecomment-4170841368 @cakarst thank you for this contribution. However, I note that you removed the license header and the front matter. Could you please re-add those? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
