Thanks, Dennis, for starting this discussion. As for me the PR looks ok in its current state, although I posted a few comments.
I tend to think that in order to proceed with the federated catalogs implementation, it is probably best to merge the PR now and later rework the parts that deal with external connection authentication (i.e. use Iceberg Auth Manager) and secret storage. However, that will likely lead to significant and non-backward-compatible changes in Admin API areas that deal with federated catalogs. I am personally fine with this approach. Still, I think it would be good to hear what other people think about this. Cheers, Dmitri. On Wed, Mar 19, 2025 at 6:08 PM Dennis Huo <huoi...@gmail.com> wrote: > Hi all, > > For anyone who may not have seen the PR on github, just wanted to ensure > folks have a chance to give input on the mailing list here about the latest > API spec proposal for supporting Catalog Federation: > > https://github.com/apache/polaris/pull/1026 > > This adds a ConnectionConfigInfo field to Catalog with various subtypes to > represent different possible federation sources, but just beginning with > "ICEBERG_REST" in the enum. This also defines how the authentication > parameters for connecting to the remote catalog are defined or can be > extended for other connection types. > > I know we have interest in adding other authentication modes soon for > Iceberg REST and also adding other catalog sources in the near future such > as Hive Metastore, maybe Glue/S3Tables/OneLake/etc. > > The main tracking issue for Catalog Federation can be found here for more > context: https://github.com/apache/polaris/issues/540 > > Any input much appreciated! >
