dennishuo opened a new pull request, #1305: URL: https://github.com/apache/polaris/pull/1305
Initial MVP for https://github.com/apache/polaris/issues/540 This initial implementation of Catalog Federation is described in this document: https://docs.google.com/document/d/1Q6eEytxb0btpOPcL8RtkULskOlYUCo_3FLvFRnHkzBY/edit?tab=t.0#bookmark=id.o00h4wkebh07 - Section "Securables Federation with Catalog Facade only" One key building block is managing the user-specified credentials to use for connecting to remote catalogs, explained in https://docs.google.com/document/d/1JPNx5vL4vM8DqwRwnBIPiQxwN4MXOdGx4Ki0j7vgwSM/edit?tab=t.0 Key elements implemented here: - Creation of ExternalCatalogs with ConnectionConfigInfo defined to point at remote Iceberg REST Catalogs - UserSecretsManager interface and UserSecretReference to define a safe control flow where secrets specified or referenced in API requests are canonicalized into non-secret "reference" objects to be persisted with relevant entities; provide a default UnsafeInMemorySecretsManager which exercises the relevant control flows - Instantiation of outbound Catalog clients in IcebergCatalogHandler for federated catalogs and re-routing all read and write requests to the remote catalog instead of using a Polaris-local IcebergCatalog - Updated Resolver logic to accommodate "partial resolution" in a way that already supports correct Catalog-level RBAC now, and will support finer-grained RBAC in the future once we add JIT-creation of entities (as described in Milestone 2 of the proposal document) This provides the functionality originally demoed as a prototype here: https://youtu.be/4iU5wXQrAn4?t=1022 All the behaviors are gated behind this FeatureConfiguration: `ENABLE_CATALOG_FEDERATION` Special thanks to @XJDKC for co-authoring this PR with me! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@polaris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
