This is an automated email from the ASF dual-hosted git repository. snazy pushed a commit to branch main in repository https://gitbox.apache.org/repos/asf/polaris.git
The following commit(s) were added to refs/heads/main by this push: new 8a059aa73 Revert PR 2033 (#2087) 8a059aa73 is described below commit 8a059aa735ac929c56487e19b929f7c68d537fe2 Author: Robert Stupp <sn...@snazy.de> AuthorDate: Tue Jul 15 17:55:19 2025 +0200 Revert PR 2033 (#2087) The PR #2033 was merged within less than 3 hours, late on a Friday. Since that change does not address an issue that seriously deserves a quick reaction nor is it a "nit", I'm proposing to revert the change. We do have [community best practices](https://polaris.apache.org/community/contributing-guidelines/) stating to give the whole community enough time to review, which did not happen. There are concerns that the PR #2033 will interfere with the whole effort to automate releases. Since there was no change to review and raise the concerns, I'd like to revert it to not cause any friction with that bigger effort. Revert "Fix invalid redirect from public page (#2041)", commit 493bc8eeb21632801b8f900caeb727493d67bc9f. Revert "[Site] Simplify the doc directory structure (#2033)", commit 2db2f10eb289e74fa84933542d27006906de37bb. --- README.md | 3 +- docs | 2 +- site/content/_index.adoc | 2 +- site/content/in-dev/_index.md | 168 +-------------------- ...polaris-catalog-service.md => release_index.md} | 15 +- site/content/in-dev/{ => unreleased}/_index.md | 0 .../in-dev/{ => unreleased}/access-control.md | 0 site/content/in-dev/{ => unreleased}/admin-tool.md | 0 .../{ => unreleased}/command-line-interface.md | 0 .../in-dev/{ => unreleased}/configuration.md | 0 .../configuring-polaris-for-production.md | 0 site/content/in-dev/{ => unreleased}/entities.md | 0 site/content/in-dev/{ => unreleased}/evolution.md | 0 .../in-dev/{ => unreleased}/generic-table.md | 0 .../{ => unreleased}/getting-started/_index.md | 0 .../getting-started/deploying-polaris/_index.md | 0 .../deploying-polaris/quickstart-deploy-aws.md | 0 .../deploying-polaris/quickstart-deploy-azure.md | 0 .../deploying-polaris/quickstart-deploy-gcp.md | 0 .../getting-started/install-dependencies.md | 0 .../{ => unreleased}/getting-started/quickstart.md | 0 .../getting-started/using-polaris.md | 0 site/content/in-dev/{ => unreleased}/helm.md | 0 site/content/in-dev/{ => unreleased}/metastores.md | 2 +- .../{ => unreleased}/polaris-catalog-service.md | 0 .../{ => unreleased}/polaris-management-service.md | 0 .../{ => unreleased}/polaris-spark-client.md | 0 site/content/in-dev/{ => unreleased}/policy.md | 0 site/content/in-dev/{ => unreleased}/realm.md | 0 site/content/in-dev/{ => unreleased}/telemetry.md | 0 site/hugo.yaml | 2 +- 31 files changed, 22 insertions(+), 172 deletions(-) diff --git a/README.md b/README.md index 3d3535b2f..20b660474 100644 --- a/README.md +++ b/README.md @@ -104,7 +104,8 @@ using different configurations. Check the `./getting-started` directory for more #### Configuring Polaris Polaris Servers can be configured using a variety of ways. -Please see the [Configuration Guide](site/content/in-dev/configuration.md) for more information. +Please see the [Configuration Guide](site/content/in-dev/unreleased/configuration.md) +for more information. Default configuration values can be found in `runtime/defaults/src/main/resources/application.properties`. diff --git a/docs b/docs index fd5b9fbeb..8c9a3b994 120000 --- a/docs +++ b/docs @@ -1 +1 @@ -site/content/in-dev/ \ No newline at end of file +site/content/in-dev/unreleased/ \ No newline at end of file diff --git a/site/content/_index.adoc b/site/content/_index.adoc index f3ec7b2f5..9d3586525 100644 --- a/site/content/_index.adoc +++ b/site/content/_index.adoc @@ -25,7 +25,7 @@ cascade: {{< blocks/cover title="Welcome to the Apache Polaris™ (incubating) web site!" image_anchor="center" color="primary" >}} Apache Polaris is an open-source, fully-featured catalog for Apache Iceberg™. It implements Iceberg's REST API, enabling seamless multi-engine interoperability across a wide range of platforms, including Apache Doris™, Apache Flink®, Apache Spark™, Dremio®, StarRocks, and Trino. -<a href="/in-dev/getting-started/install-dependencies/" class="btn btn-lg btn-dark mt-5">Get Started <i class="fas fa-arrow-alt-circle-right ms-2"></i></a> +<a href="/in-dev/unreleased/getting-started/install-dependencies/" class="btn btn-lg btn-dark mt-5">Get Started <i class="fas fa-arrow-alt-circle-right ms-2"></i></a> {{< /blocks/cover >}} diff --git a/site/content/in-dev/_index.md b/site/content/in-dev/_index.md index 023d3b0f7..d3682b20f 100644 --- a/site/content/in-dev/_index.md +++ b/site/content/in-dev/_index.md @@ -17,168 +17,8 @@ # specific language governing permissions and limitations # under the License. # -linkTitle: 'In Development' -title: 'Overview' -type: docs -weight: 200 -params: - top_hidden: true - show_page_toc: false -cascade: - type: docs - params: - show_page_toc: true -# This file will NOT be copied into a new release's versioned docs folder. +# Hide `/in-dev/` +toc_hide: true +hide_summary: true +exclude_search: true --- - -> [!WARNING] -> These pages refer to the current state of the main branch, which is still under active development. -> -> Functionalities can be changed, removed or added without prior notice. - -Apache Polaris (Incubating) is a catalog implementation for Apache Iceberg™ tables and is built on the open source Apache Iceberg™ REST protocol. - -With Polaris, you can provide centralized, secure read and write access to your Iceberg tables across different REST-compatible query engines. - - overview") - -## Key concepts - -This section introduces key concepts associated with using Apache Polaris (Incubating). - -In the following diagram, a sample [Apache Polaris (Incubating) structure](#catalog) with nested [namespaces](#namespace) is shown for Catalog1. No tables -or namespaces have been created yet for Catalog2 or Catalog3. - - structure") - -### Catalog - -In Polaris, you can create one or more catalog resources to organize Iceberg tables. - -Configure your catalog by setting values in the storage configuration for S3, Azure, or Google Cloud Storage. An Iceberg catalog enables a -query engine to manage and organize tables. The catalog forms the first architectural layer in the [Apache Iceberg™ table specification](https://iceberg.apache.org/spec/#overview) and must support the following tasks: - -- Storing the current metadata pointer for one or more Iceberg tables. A metadata pointer maps a table name to the location of that table's - current metadata file. - -- Performing atomic operations so that you can update the current metadata pointer for a table to the metadata pointer of a new version of - the table. - -To learn more about Iceberg catalogs, see the [Apache Iceberg™ documentation](https://iceberg.apache.org/concepts/catalog/). - -#### Catalog types - -A catalog can be one of the following two types: - -- Internal: The catalog is managed by Polaris. Tables from this catalog can be read and written in Polaris. - -- External: The catalog is externally managed by another Iceberg catalog provider (for example, Snowflake, Glue, Dremio Arctic). Tables from - this catalog are synced to Polaris. These tables are read-only in Polaris. - -A catalog is configured with a storage configuration that can point to S3, Azure storage, or GCS. - -### Namespace - -You create *namespaces* to logically group Iceberg tables within a catalog. A catalog can have multiple namespaces. You can also create -nested namespaces. Iceberg tables belong to namespaces. - -> [!Important] -> For the access privileges defined for a catalog to be enforced correctly, the following conditions must be met: -> -> - The directory only contains the data files that belong to a single table. -> - The directory hierarchy matches the namespace hierarchy for the catalog. -> -> For example, if a catalog includes the following items: -> -> - Top-level namespace namespace1 -> - Nested namespace namespace1a -> - A customers table, which is grouped under nested namespace namespace1a -> - An orders table, which is grouped under nested namespace namespace1a -> -> The directory hierarchy for the catalog must follow this structure: -> -> - /namespace1/namespace1a/customers/<files for the customers table *only*> -> - /namespace1/namespace1a/orders/<files for the orders table *only*> - -### Storage configuration - -A storage configuration stores a generated identity and access management (IAM) entity for your cloud storage and is created -when you create a catalog. The storage configuration is used to set the values to connect Polaris to your cloud storage. During the -catalog creation process, an IAM entity is generated and used to create a trust relationship between the cloud storage provider and Polaris -Catalog. - -When you create a catalog, you supply the following information about your cloud storage: - -| Cloud storage provider | Information | -| -----------------------| ----------- | -| Amazon S3 | <ul><li>Default base location for your Amazon S3 bucket</li><li>Locations for your Amazon S3 bucket</li><li>S3 role ARN</li><li>External ID (optional)</li></ul> | -| Google Cloud Storage (GCS) | <ul><li>Default base location for your GCS bucket</li><li>Locations for your GCS bucket</li></ul> | -| Azure | <ul><li>Default base location for your Microsoft Azure container</li><li>Locations for your Microsoft Azure container</li><li>Azure tenant ID</li></ul> | - -## Example workflow - -In the following example workflow, Bob creates an Apache Iceberg™ table named Table1 and Alice reads data from Table1. - -1. Bob uses Apache Spark™ to create the Table1 table under the - Namespace1 namespace in the Catalog1 catalog and insert values into - Table1. - - Bob can create Table1 and insert data into it because he is using a - service connection with a service principal that has - the privileges to perform these actions. - -2. Alice uses Snowflake to read data from Table1. - - Alice can read data from Table1 because she is using a service - connection with a service principal with a catalog integration that - has the privileges to perform this action. Alice - creates an unmanaged table in Snowflake to read data from Table1. - -") - -## Security and access control - -### Credential vending - -To secure interactions with service connections, Polaris vends temporary storage credentials to the query engine during query -execution. These credentials allow the query engine to run the query without requiring access to your cloud storage for -Iceberg tables. This process is called credential vending. - -As of now, the following limitation is known regarding Apache Iceberg support: - -- **remove_orphan_files:** Apache Spark can't use credential vending - for this due to a known issue. See [apache/iceberg#7914](https://github.com/apache/iceberg/pull/7914) for details. - -### Identity and access management (IAM) - -Polaris uses the identity and access management (IAM) entity to securely connect to your storage for accessing table data, Iceberg -metadata, and manifest files that store the table schema, partitions, and other metadata. Polaris retains the IAM entity for your -storage location. - -### Access control - -Polaris enforces the access control that you configure across all tables registered with the service and governs security for all -queries from query engines in a consistent manner. - -Polaris uses a role-based access control (RBAC) model that lets you centrally configure access for Polaris service principals to catalogs, -namespaces, and tables. - -Polaris RBAC uses two different role types to delegate privileges: - -- **Principal roles:** Granted to Polaris service principals and - analogous to roles in other access control systems that you grant to - service principals. - -- **Catalog roles:** Configured with certain privileges on Polaris - catalog resources and granted to principal roles. - -For more information, see [Access control]({{% ref "access-control" %}}). - -## Legal Notices - -Apache®, Apache Iceberg™, Apache Spark™, Apache Flink®, and Flink® are either registered trademarks or trademarks of the Apache Software Foundation in the United States and/or other countries. - - -<!-- -Testing the `releaseVersion` shortcode here: version is: {{< releaseVersion >}} ---> diff --git a/site/content/in-dev/polaris-catalog-service.md b/site/content/in-dev/release_index.md similarity index 69% copy from site/content/in-dev/polaris-catalog-service.md copy to site/content/in-dev/release_index.md index 02fed63f4..e8f38dbfa 100644 --- a/site/content/in-dev/polaris-catalog-service.md +++ b/site/content/in-dev/release_index.md @@ -17,10 +17,19 @@ # specific language governing permissions and limitations # under the License. # -linkTitle: 'Catalog API Spec' -weight: 900 +title: 'POLARIS VERSION INDEX MD TEMPLATE' +toc_hide: true +hide_summary: true params: show_page_toc: false +cascade: + # The latest release specifies 'false', all others must be 'true' + exclude_search: false + params: + show_page_toc: true +# This file will be copied as `_index.md` into a new release's versioned docs folder. --- -{{< redoc-polaris "generated/bundled-polaris-catalog-service.yaml" >}} +== Apache Polaris version {{< releaseVersion >}} + +Download from ... diff --git a/site/content/in-dev/_index.md b/site/content/in-dev/unreleased/_index.md similarity index 100% copy from site/content/in-dev/_index.md copy to site/content/in-dev/unreleased/_index.md diff --git a/site/content/in-dev/access-control.md b/site/content/in-dev/unreleased/access-control.md similarity index 100% rename from site/content/in-dev/access-control.md rename to site/content/in-dev/unreleased/access-control.md diff --git a/site/content/in-dev/admin-tool.md b/site/content/in-dev/unreleased/admin-tool.md similarity index 100% rename from site/content/in-dev/admin-tool.md rename to site/content/in-dev/unreleased/admin-tool.md diff --git a/site/content/in-dev/command-line-interface.md b/site/content/in-dev/unreleased/command-line-interface.md similarity index 100% rename from site/content/in-dev/command-line-interface.md rename to site/content/in-dev/unreleased/command-line-interface.md diff --git a/site/content/in-dev/configuration.md b/site/content/in-dev/unreleased/configuration.md similarity index 100% rename from site/content/in-dev/configuration.md rename to site/content/in-dev/unreleased/configuration.md diff --git a/site/content/in-dev/configuring-polaris-for-production.md b/site/content/in-dev/unreleased/configuring-polaris-for-production.md similarity index 100% rename from site/content/in-dev/configuring-polaris-for-production.md rename to site/content/in-dev/unreleased/configuring-polaris-for-production.md diff --git a/site/content/in-dev/entities.md b/site/content/in-dev/unreleased/entities.md similarity index 100% rename from site/content/in-dev/entities.md rename to site/content/in-dev/unreleased/entities.md diff --git a/site/content/in-dev/evolution.md b/site/content/in-dev/unreleased/evolution.md similarity index 100% rename from site/content/in-dev/evolution.md rename to site/content/in-dev/unreleased/evolution.md diff --git a/site/content/in-dev/generic-table.md b/site/content/in-dev/unreleased/generic-table.md similarity index 100% rename from site/content/in-dev/generic-table.md rename to site/content/in-dev/unreleased/generic-table.md diff --git a/site/content/in-dev/getting-started/_index.md b/site/content/in-dev/unreleased/getting-started/_index.md similarity index 100% rename from site/content/in-dev/getting-started/_index.md rename to site/content/in-dev/unreleased/getting-started/_index.md diff --git a/site/content/in-dev/getting-started/deploying-polaris/_index.md b/site/content/in-dev/unreleased/getting-started/deploying-polaris/_index.md similarity index 100% rename from site/content/in-dev/getting-started/deploying-polaris/_index.md rename to site/content/in-dev/unreleased/getting-started/deploying-polaris/_index.md diff --git a/site/content/in-dev/getting-started/deploying-polaris/quickstart-deploy-aws.md b/site/content/in-dev/unreleased/getting-started/deploying-polaris/quickstart-deploy-aws.md similarity index 100% rename from site/content/in-dev/getting-started/deploying-polaris/quickstart-deploy-aws.md rename to site/content/in-dev/unreleased/getting-started/deploying-polaris/quickstart-deploy-aws.md diff --git a/site/content/in-dev/getting-started/deploying-polaris/quickstart-deploy-azure.md b/site/content/in-dev/unreleased/getting-started/deploying-polaris/quickstart-deploy-azure.md similarity index 100% rename from site/content/in-dev/getting-started/deploying-polaris/quickstart-deploy-azure.md rename to site/content/in-dev/unreleased/getting-started/deploying-polaris/quickstart-deploy-azure.md diff --git a/site/content/in-dev/getting-started/deploying-polaris/quickstart-deploy-gcp.md b/site/content/in-dev/unreleased/getting-started/deploying-polaris/quickstart-deploy-gcp.md similarity index 100% rename from site/content/in-dev/getting-started/deploying-polaris/quickstart-deploy-gcp.md rename to site/content/in-dev/unreleased/getting-started/deploying-polaris/quickstart-deploy-gcp.md diff --git a/site/content/in-dev/getting-started/install-dependencies.md b/site/content/in-dev/unreleased/getting-started/install-dependencies.md similarity index 100% rename from site/content/in-dev/getting-started/install-dependencies.md rename to site/content/in-dev/unreleased/getting-started/install-dependencies.md diff --git a/site/content/in-dev/getting-started/quickstart.md b/site/content/in-dev/unreleased/getting-started/quickstart.md similarity index 100% rename from site/content/in-dev/getting-started/quickstart.md rename to site/content/in-dev/unreleased/getting-started/quickstart.md diff --git a/site/content/in-dev/getting-started/using-polaris.md b/site/content/in-dev/unreleased/getting-started/using-polaris.md similarity index 100% rename from site/content/in-dev/getting-started/using-polaris.md rename to site/content/in-dev/unreleased/getting-started/using-polaris.md diff --git a/site/content/in-dev/helm.md b/site/content/in-dev/unreleased/helm.md similarity index 100% rename from site/content/in-dev/helm.md rename to site/content/in-dev/unreleased/helm.md diff --git a/site/content/in-dev/metastores.md b/site/content/in-dev/unreleased/metastores.md similarity index 99% rename from site/content/in-dev/metastores.md rename to site/content/in-dev/unreleased/metastores.md index 2d7555d4f..224f35047 100644 --- a/site/content/in-dev/metastores.md +++ b/site/content/in-dev/unreleased/metastores.md @@ -42,7 +42,7 @@ The Relational JDBC metastore currently relies on a Quarkus-managed datasource a Please refer to the documentation here: [Configure data sources in Quarkus](https://quarkus.io/guides/datasource) -Additionally the retries can be configured via `polaris.persistence.relational.jdbc.*` properties please ref [configuration](configuration.md) +Additionally the retries can be configured via `polaris.persistence.relational.jdbc.*` properties please ref [configuration](./configuration.md) ## EclipseLink (Deprecated) > [!IMPORTANT] diff --git a/site/content/in-dev/polaris-catalog-service.md b/site/content/in-dev/unreleased/polaris-catalog-service.md similarity index 100% rename from site/content/in-dev/polaris-catalog-service.md rename to site/content/in-dev/unreleased/polaris-catalog-service.md diff --git a/site/content/in-dev/polaris-management-service.md b/site/content/in-dev/unreleased/polaris-management-service.md similarity index 100% rename from site/content/in-dev/polaris-management-service.md rename to site/content/in-dev/unreleased/polaris-management-service.md diff --git a/site/content/in-dev/polaris-spark-client.md b/site/content/in-dev/unreleased/polaris-spark-client.md similarity index 100% rename from site/content/in-dev/polaris-spark-client.md rename to site/content/in-dev/unreleased/polaris-spark-client.md diff --git a/site/content/in-dev/policy.md b/site/content/in-dev/unreleased/policy.md similarity index 100% rename from site/content/in-dev/policy.md rename to site/content/in-dev/unreleased/policy.md diff --git a/site/content/in-dev/realm.md b/site/content/in-dev/unreleased/realm.md similarity index 100% rename from site/content/in-dev/realm.md rename to site/content/in-dev/unreleased/realm.md diff --git a/site/content/in-dev/telemetry.md b/site/content/in-dev/unreleased/telemetry.md similarity index 100% rename from site/content/in-dev/telemetry.md rename to site/content/in-dev/unreleased/telemetry.md diff --git a/site/hugo.yaml b/site/hugo.yaml index 5662e32d1..1b373f778 100644 --- a/site/hugo.yaml +++ b/site/hugo.yaml @@ -99,7 +99,7 @@ menu: params: orderby: weight.desc - name: "In Development" - url: "/in-dev/" + url: "/in-dev/unreleased/" parent: "releases" weight: 10 - name: "1.0.0"