The current /catalog/v1 URI prefix encapsulates the Iceberg REST Catalog API. This API is not owned by Polaris.
If Polaris were to add new endpoints under /catalog/v1 and then Iceberg releases a v2 REST Catalog API what happens to the Polaris-specific endpoints? What if Iceberg adds an endpoint that matches the Polaris endpoint URI (unlikely but possible)? >From my POV this is the primary reason for putting the new API under a different prefix. A secondary concern is that listing methods in the Iceberg REST API will certainly not include entities created via the proposed new API, so mixing new create endpoints with old lising endpoints does not make sense to me from the REST API design perspective. As for a practical proposal, how about the following? * /catalog/v1 - remains supported for Iceberg REST API v1, but deprecated and will not be used for v2. * /catalog/iceberg/ - encapsulates the Iceberg REST API v1 (and future API versions). * /catalog/generic/v1 - the API proposed in the referenced document. * /management - current Polaris Management API. Now, "generic" is, of course, a naming concern. Other opinions are welcome regarding how to call the new API. Options that look reasonable to me (in order of preference): * /catalog/polaris/v1 - this implies that Polaris evolves this API as the primary API for all things catalogued by Polaris (not including roles, grants, etc., which fall under /management) * /catalog/api/v1 - same as above, but different path segment * /catalog/storage-object/v1 * /catalog/generic/v1 - Works fine for the current proposal, but it will look awkward if we have to add typed / specialized endpoints later (so those API extensions may need yet another prefix). * /catalog/generic-table/v1 - same as above * /catalog/storage-entity/v1 Cheers, Dmitri. On Mon, Feb 24, 2025 at 3:10 PM yun zou <yunzou.colost...@gmail.com> wrote: > Hi All, > > Thanks all for attending the review today and providing your valuable > feedback, I really appreciate it ! And apologize for the late notice for > the review meeting here. > > I think we had a good discussion, and agreed to > 1) Move on to provide a new Spark Catalog Plugin and new set of APIs with > restricted scope to table managements, but keep the doors open for future > extensions includes new fields or even new APIs. > 2) Update the field create_at_ms to catalog_register_at to be more clear > that the timestamp is for the registration time with Polaris. > > We will start working on figuring out more implementation details for the > plugins as well as refactoring needed for Apache Polaris. If it turns out > too complicated, we can have a separate review or discussion about the > implementation part also. > > As a follow up of the meeting, we would like to open this thread for > discussion for the URI prefix for the new endpoint. As Dmitri brought up on > the design doc, since this set of APIs are new and not restricted by > Iceberg anymore, shall we introduce a new prefix like /polaris/catalog/v1/ > instead of reuse the current /catalog/v1? > > I think my concern about this proposal is that it seems to diverge how the > APIs path are formalized today, the APIs in Polaris today use a path that > is prefixed by functionality, like /catalog/ or /management/. If we start > using /polaris/catalog/, shall we change the /management also to > /polaris/management ? Shall we move the new Policy Engine related APIs to > /polaris/catalog also? > > Another choice we have is to use /polaris-catalog/v1 instead /catalog/v1 as > a prefix, which aligns with the current API path better. Then we have the > same question, what do we do with the Policy Engine APIs? The Policies can > eventually be applied to both table entities created through the /catalog > APIs and /polaris/catalog APIs. > > Overall, I would prefer to stay consistent with the current APIs and > continue to use /catalog/v1, since regardless of Iceberg or Generic tables, > they are just entities managed by Polaris, and the APIs don't have > 'iceberg' in the path also. I would like to hear what people think about > this? > > Link to the doc: > > https://docs.google.com/document/d/1_R9jBIwoH3CV9G7gSoRJPQVEcOsROp4IEiGoeYeQE8A/edit?pli=1&tab=t.0 > > Best Regards, > Yun >
