> For example, if we are to expose Open Lineage or OSI endpoints in IRC config, the IRC service code will have to be aware of OL / OSI modules.
I don't agree with this statement; the IRC client is always free to disregard any of these endpoints/modules. Returning this information does not force a client to take any action based on this extra endpoint information. I do agree that if this forced the IRC client to be fundamentally altered because of these additional endpoints, then that would be a significant issue. But as Russell stated above, "what would actually break if someone interpreted the spec this way?" Best, Adnan Hemani On Mon, Jun 22, 2026 at 3:30 PM Dmitri Bourlatchkov <[email protected]> wrote: > Hi Russell, > > Your points sound reasonable to me. > > Yet, noone answered my second question from my initial email. That is: why > do we need to expose non-IRC endpoints in the IRC config response in the > first place? > > Regarding the maintainability aspect, I see this becoming a more > substantial concern if it evolves into a self-sustaining pattern. > > For example, if we are to expose Open Lineage or OSI endpoints in IRC > config, the IRC service code will have to be aware of OL / OSI modules. I > think this dependency is conceptually incorrect. > > I do not so much mind existing non-IRC endpoints in the IRC config, but > extending this to new APIs feels like an anti-pattern to me. > > Cheers, > Dmitri. > > On Mon, Jun 22, 2026 at 5:46 PM Russell Spitzer <[email protected] > > > wrote: > > > From a very pragmatic point of view, I'm not sure what we'd gain from > > separate > > configuration mechanisms. We already have one and it's worked well so > far, > > do we have a compelling reason to swap right now? > > > > As best I can tell, the worst case is that the IRC spec later tightens > the > > semantics > > of the endpoints field, making our extra entries non-compliant. In that > > situation > > we'd have to change our response, add a Polaris-specific config/discovery > > endpoint, and > > have our clients move over to it. But that's exactly the work this > proposal > > is asking us to do today > > so we would be taking same burden now instead of potentially never. I'd > > rather defer > > it until something concrete forces our hand. > > > > The one counter I can see is that decoupling gets harder as more clients > > adopt it. But > > since the Polaris endpoints are already namespaced (/polaris/...), > lifting > > them into a separate endpoint > > later stays mechanical, so I don't think we're meaningfully cheaper > doing > > it now. > > > > > > I'd like to gently point out that it doesn't feel especially important > > whether the exact wording > > or original intent of the spec supports what we're doing. The discussion > is > > starting to feel > > like a legalistic reading of the spec rather than a focus on concrete > > impact. I'd suggest we focus > > on arguments like "what would actually break if someone interpreted the > > spec this way?" > > rather than "what was the original intent of the text?" > > > > On Mon, Jun 22, 2026 at 3:23 PM Adnan Hemani via dev < > > [email protected]> > > wrote: > > > > > Hi Dmitri, > > > > > > I think you and I view the same statement very differently - I > definitely > > > don't think it's "pretty clear that the scope of the data returned > under > > > that type is limited to the API defined by the spec". I see it this > way: > > if > > > this statement truly meant ONLY IRC endpoints, the spec would > explicitly > > > use the words "IRC spec endpoints". The absence of clarification in any > > > specification rarely means you should "read-between-the-lines" to > assume > > > the writer's "intent". > > > > > > I will look into Iceberg mailing list threads and PRs for background > > > information to clarify things. > > > > > > Best, > > > Adnan Hemani > > > > > > > > > > > > On Mon, Jun 22, 2026 at 12:27 PM Dmitri Bourlatchkov <[email protected] > > > > > wrote: > > > > > > > HI Adnan, > > > > > > > > The IRC spec defines a "type" for the config response. A property of > > that > > > > "type" is a list of "endpoints supported by the server". There is no > > > > statement about generalizing those endpoints to non-IRC APIs. > > > > > > > > Therefore, I think it is pretty clear that the scope of the data > > returned > > > > under that type is limited to the API defined by the spec, which is > > IRC. > > > > > > > > Let's expand this discussion a bit. > > > > > > > > What is the rationale for returning non-IRC endpoints in the IRC > config > > > > response? > > > > > > > > Thanks, > > > > Dmitri. > > > > > > > > > > > > > > > > On Mon, Jun 22, 2026 at 3:21 PM Adnan Hemani via dev < > > > > [email protected]> wrote: > > > > > > > >> Hi Dmitri, > > > >> > > > >> I'm not sure where you are assuming that "endpoints" only refers to > > > >> Iceberg > > > >> endpoints in the IRC spec. Do you have any thing you can point us to > > > >> regarding this? > > > >> > > > >> Best, > > > >> Adnan Hemani > > > >> > > > >> On Mon, Jun 22, 2026 at 12:18 PM Dmitri Bourlatchkov < > > [email protected]> > > > >> wrote: > > > >> > > > >> > Hi Adam, > > > >> > > > > >> > > #1 - Is GET ' > > > >> > http://localhost:8181/api/catalog/v1/config?warehouse=polaris' > only > > > >> > an Iceberg Catalog concept? > > > >> > > > > >> > My interpretation is YES, because the payload of that response is > > > >> defined > > > >> > by the IRC API spec [1]. > > > >> > > > > >> > I believe one can reasonably assume that the statement "endpoints > > > >> supported > > > >> > by the server" is scoped only to the IRC API itself. There's no > > > >> provision > > > >> > in that spec about covering all possible APIs. > > > >> > > > > >> > That said, I'd be happy to discuss how clients can discover > Polaris > > > >> > features in general without overloading existing specifications. > > > >> > > > > >> > [1] > > > >> > > > > >> > > > > >> > > > > > > https://github.com/apache/iceberg/blob/apache-iceberg-1.11.0/open-api/rest-catalog-open-api.yaml#L105 > > > >> > > > > >> > On Mon, Jun 22, 2026 at 2:52 PM Adam Christian < > > > >> > [email protected]> wrote: > > > >> > > > > >> > > I agree with both Yufei & Dmitri: > > > >> > > > > > >> > > #1 - It seems like improper coupling for an Iceberg REST Catalog > > > >> config > > > >> > to > > > >> > > return a configuration unrelated to Iceberg. > > > >> > > #2 - Polaris is a superset of an Iceberg REST Catalog. > > > >> > > > > > >> > > So, in my opinion, the question is more aptly framed as two > > > questions: > > > >> > > > > > >> > > #1 - Is GET ' > > > >> > http://localhost:8181/api/catalog/v1/config?warehouse=polaris > > > >> > > ' > > > >> > > only an Iceberg Catalog concept? > > > >> > > #2 - Should GET ' > > > >> > > http://localhost:8181/api/catalog/v1/config?warehouse=polaris' > > > return > > > >> > > additional endpoints? > > > >> > > > > > >> > > I believe that the answer to #1 is no, but that requires changes > > to > > > >> the > > > >> > > codebase. I believe the answer to #2 is yes. > > > >> > > > > > >> > > Based on the codebase, we are exposing several endpoints in this > > > >> > > configuration API which are not Iceberg endpoints. For example, > in > > > the > > > >> > > Generic Tables case, it is explicitly not Iceberg. In my > opinion, > > > >> this is > > > >> > > alright because our API specification only says "Configuration > > API," > > > >> not > > > >> > > "Iceberg Configuration API." Yes, it is Iceberg-compatible, but > it > > > is > > > >> not > > > >> > > Iceberg-centric. This retrieves the configuration for the > Catalog; > > > >> > > regardless of whether it is an Iceberg Catalog. Now, if this is > > > truly > > > >> a > > > >> > > superset, it implies that IcebergCatalogHandler.java should not > > > handle > > > >> > > returning the configuration. Additionally, we need to return a > > > >> superset > > > >> > of > > > >> > > the ConfigResponse (an Iceberg-core concept) for the REST API > > > >> response. > > > >> > > > > > >> > > Now, if the answer to #1 is yes, that means we should probably > > have > > > a > > > >> > > separate API for configuration. > > > >> > > > > > >> > > What do y'all think? > > > >> > > > > > >> > > Go community, > > > >> > > > > > >> > > Adam > > > >> > > > > > >> > > On Mon, Jun 22, 2026 at 1:55 PM Yufei Gu <[email protected]> > > > >> wrote: > > > >> > > > > > >> > > > I think a Polaris client is also an IRC client, just with > > > additional > > > >> > > > capabilities. In that sense, Polaris can be viewed as a > superset > > > of > > > >> > IRC. > > > >> > > If > > > >> > > > the config endpoint is intended for capability discovery, > > > returning > > > >> > > Polaris > > > >> > > > specific endpoints seems reasonable. Standard IRC clients can > > > simply > > > >> > > ignore > > > >> > > > endpoints they don't recognize. > > > >> > > > > > > >> > > > A concrete example is the Polaris Spark client, which checks > > > server > > > >> > > > capabilities before using Polaris specific functionality: > > > >> > > > > > > >> > > > public List<TableIdentifier> listGenericTables(Namespace ns) { > > > >> > > > Endpoint.check(endpoints, > > > >> PolarisEndpoints.V1_LIST_GENERIC_TABLES); > > > >> > > > > > > >> > > > > > > >> > > > Yufei > > > >> > > > > > > >> > > > > > > >> > > > On Fri, Jun 19, 2026 at 7:43 AM Dmitri Bourlatchkov < > > > >> [email protected]> > > > >> > > > wrote: > > > >> > > > > > > >> > > > > Hi All, > > > >> > > > > > > > >> > > > > During my review of [4816] I realized [1] that Polaris > returns > > > >> some > > > >> > > > non-IRC > > > >> > > > > endpoints in the IRC config responses. > > > >> > > > > > > > >> > > > > For example: > > > >> > > > > > > > >> > > > > GET ' > > > >> http://localhost:8181/api/catalog/v1/config?warehouse=polaris' > > > >> > > > > > > > >> > > > > Result: > > > >> > > > > [...] > > > >> > > > > "endpoints": [ > > > >> > > > > "GET /v1/{prefix}/namespaces", > > > >> > > > > "GET /v1/{prefix}/namespaces/{namespace}", > > > >> > > > > "HEAD /v1/{prefix}/namespaces/{namespace}", > > > >> > > > > [...] > > > >> > > > > "DELETE > > > >> > > > > > > > >> > > > > > > >> > > > > > >> > > > > >> > > > > > > polaris/v1/{prefix}/namespaces/{namespace}/generic-tables/{generic-table}", > > > >> > > > > "GET > > > >> > > > > > > > >> > > > > > > >> > > > > > >> > > > > >> > > > > > > polaris/v1/{prefix}/namespaces/{namespace}/generic-tables/{generic-table}", > > > >> > > > > "GET > > /polaris/v1/{prefix}/namespaces/{namespace}/policies", > > > >> > > > > "POST > > /polaris/v1/{prefix}/namespaces/{namespace}/policies", > > > >> > > > > > > > >> > > > > The latter group of endpoints is not related to the Iceberg > > REST > > > >> > > Catalog > > > >> > > > > API. > > > >> > > > > > > > >> > > > > I wonder what the rationale might be for returning them in > the > > > IRC > > > >> > > config > > > >> > > > > response. > > > >> > > > > > > > >> > > > > From my POV, returning them in the IRC config response is > > > >> incorrect > > > >> > > > because > > > >> > > > > these endpoints form APIs that follow a different > > specification > > > >> and > > > >> > > > clients > > > >> > > > > using the IRC API do not need that information to properly > use > > > the > > > >> > IRC > > > >> > > > API. > > > >> > > > > > > > >> > > > > WDYT? > > > >> > > > > > > > >> > > > > [1] > > > >> > > https://github.com/apache/polaris/pull/4816#discussion_r3438945230 > > > >> > > > > > > > >> > > > > [4816] https://github.com/apache/polaris/pull/4816 > > > >> > > > > > > > >> > > > > Thanks, > > > >> > > > > Dmitri. > > > >> > > > > > > > >> > > > > > > >> > > > > > >> > > > > >> > > > > > > > > > >
