Thanks, that helps clarify the disagreement. I agree there is a use case for Polaris-aware clients discovering Polaris capabilities. I do not think that is disputed.
The part I am still missing is whether we are intentionally defining the existing IRC config response as the Polaris-wide capability discovery contract. If yes, I think we should state that explicitly and document it as a Polaris extension, including how optional modules contribute endpoint/capability metadata without coupling the IRC service code to every feature. If no, then I think new non-IRC APIs should not rely on the IRC config response for discovery by default. So for me the decision is less about whether clients can ignore unknown endpoints, and more about whether `/api/catalog/v1/config` is intentionally becoming Polaris service discovery. Robert On Wed, Jun 24, 2026 at 12:55 AM Dmitri Bourlatchkov <[email protected]> wrote: > Hi Yun, > > Thanks for sharing your perspective. It is very helpful! > > Cheers, > Dmitri. > > On Tue, Jun 23, 2026 at 4:42 PM yun zou <[email protected]> > wrote: > > > Hi, > > > > Sorry, I haven't had a chance to read through the entire thread yet. From > > what I can tell, the main question is whether we should return non-IRC > > endpoints in the IRC configuration. > > > > From my perspective, Polaris is evolving beyond IRC and becoming a > superset > > of Iceberg while still remaining Iceberg-native. For concepts that are > > shared between IRC and non-IRC functionality, it's generally better to > > reuse the same abstractions rather than introduce separate ones. > Namespaces > > are a good example of this. Reusing common concepts helps reduce > > maintenance overhead and lowers the onboarding burden for users. > > > > Regarding the configuration, I view it as a generic mechanism for > > describing the server's overall configuration and capabilities. In > > particular, the endpoints section should simply indicate all endpoints > > currently supported by Polaris. Clients can then determine which > operations > > are available based on the returned configuration and advertised > > capabilities. > > > > For example, the Polaris Spark client may recognize from the > configuration > > that generic tables are supported by the server and choose to interact > with > > the corresponding endpoints. In contrast, the Iceberg Spark client may > > choose to interact only with Iceberg endpoints. In both cases, the > > configuration serves as a capability discovery mechanism, while > individual > > clients decide which capabilities they want to consume. > > > > As Russell mentioned, I don't see a strong reason to introduce a separate > > getConfig API solely to distinguish between IRC and non-IRC endpoints. > > > > Unless there is a specific compatibility or security concern, exposing > the > > full set of supported endpoints through the existing configuration > > mechanism seems like the simpler, more extensible approach. > > > > One question I have is: if we choose not to expose these capabilities > > through the existing getConfig response, what is the proposed mechanism > for > > future clients to discover the capabilities of a deployed Polaris > service? > > > > Best Regards, > > Yun > > > > > > > > On Tue, Jun 23, 2026 at 12:57 PM Dmitri Bourlatchkov <[email protected]> > > wrote: > > > > > Hi Adnan, > > > > > > Does Polaris gain anything from separating these two getConfig APIs > > > > > > > > > I'm not exactly sure what you mean by "separating". My proposal is > simply > > > to remove non-IRC endpoints from IRC config. > > > > > > Nonetheless, in either case, the benefit will be that IRC API services > in > > > Polaris (server-side) will not have a dependency on other, non-IRC REST > > > APIs. > > > > > > More concretely, I do not mind existing non-IRC endpoints in the IRC > > > config. However, I think Polaris should maintain this distinction for > new > > > APIs. This will simplify the modular design for new features. > > > > > > Cheers, > > > Dmitri. > > > > > > > > > On Tue, Jun 23, 2026 at 2:37 PM Adnan Hemani via dev < > > > [email protected]> > > > wrote: > > > > > > > Robert, > > > > > > > > I think Yufei and Russell explained above that we can either: > > > > > > > > * Continue doing what we are doing today - IRC clients see (and > ignore) > > > > endpoints they will not action upon. OR > > > > * Separate it out into two different getConfig APIs - one for > Polaris, > > > one > > > > for IRC. In this case, the Polaris getConfig would be a strict > superset > > > of > > > > the IRC API - and the Polaris-specific clients would use the Polaris > > > > getConfig API. > > > > > > > > My understanding of why Russell brought up the point regarding "Can > > > non-IRC > > > > endpoints be added to the IRC config response without breaking > > clients?" > > > > is: There is clearly a use case for Polaris clients using the > getConfig > > > API > > > > - so the real question, in my opinion, is what is the benefit of > > creating > > > > two `getConfig` APIs versus continuing our current approach which > > reduces > > > > code paths and complexity? Does Polaris gain anything from separating > > > these > > > > two getConfig APIs other than establishing a strict, principled > stance > > on > > > > our interpretation of the IRC spec text? > > > > > > > > Best, > > > > Adnan Hemani > > > > > > > > On Tue, Jun 23, 2026 at 9:50 AM Robert Stupp <[email protected]> wrote: > > > > > > > > > Hi all, > > > > > > > > > > I think we should separate two questions: > > > > > > > > > > 1. Can non-IRC endpoints be added to the IRC config response > without > > > > > breaking clients? > > > > > 2. What can a normal Iceberg REST client actually do with that > > > > information? > > > > > > > > > > The first question only tells us whether the behavior is tolerated. > > > > > It does not establish that the IRC config response is the right > > > contract > > > > > for Polaris-wide capability discovery. > > > > > > > > > > For a normal IRC client, I do not see a concrete use case for > > > > OpenLineage, > > > > > OSI, generic-table, or policy endpoints in that response. > > > > > A Polaris-aware client may need discovery, but that sounds like > > Polaris > > > > > capability discovery, not IRC capability discovery. > > > > > > > > > > So what is the concrete use case for exposing those endpoints to > > normal > > > > IRC > > > > > clients? > > > > > > > > > > Best, > > > > > Robert > > > > > > > > > > On Tue, Jun 23, 2026 at 3:26 PM Dmitri Bourlatchkov < > > > > > [email protected]> wrote: > > > > > > > > > > > Hi Yufei, > > > > > > > > > > > > > The community has already decided to reuse the IRC config > > endpoint > > > > for > > > > > > capability discovery > > > > > > > > > > > > Could you give a link to that decision? > > > > > > > > > > > > Thanks, > > > > > > Dmitri. > > > > > > > > > > > > On Mon, Jun 22, 2026 at 9:45 PM Yufei Gu <[email protected]> > > > wrote: > > > > > > > > > > > > > > Yet, no one answered my second question from the initial > email. > > > > That > > > > > > is: > > > > > > > why do we need to expose non-IRC endpoints in the IRC config > > > response > > > > > in > > > > > > > the first place? > > > > > > > > > > > > > > Apologies if I didn't explain this clearly in my earlier email. > > > > > > > > > > > > > > With Polaris, you can use a pure IRC client, or you can use a > > > Polaris > > > > > > > client that leverages Polaris specific capabilities such as > > generic > > > > > > tables > > > > > > > and policies in addition to IRC functionality. The community > has > > > > > already > > > > > > > decided to reuse the IRC config endpoint for capability > > discovery. > > > I > > > > > > shared > > > > > > > an example from the Polaris Spark client previously, but I'll > > > repeat > > > > it > > > > > > > here because it illustrates the point well. Note that the > client > > > > > supports > > > > > > > both Iceberg tables and generic tables: > > > > > > > > > > > > > > public List<TableIdentifier> listGenericTables(Namespace ns) { > > > > > > > Endpoint.check(endpoints, > > > > PolarisEndpoints.V1_LIST_GENERIC_TABLES); > > > > > > > } > > > > > > > > > > > > > > In this model, the config endpoint acts as a capability > discovery > > > > > > > mechanism. Standard IRC clients can simply ignore endpoints > they > > do > > > > not > > > > > > > recognize, while Polaris aware clients can use the additional > > > > > information > > > > > > > to enable Polaris specific functionality. > > > > > > > > > > > > > > I also think Adam made a good point. The /api/catalog/v1/config > > > > > endpoint > > > > > > > returns both IRC and Polaris capabilities. Otherwise, we need > to > > > > > > introduce > > > > > > > a separate Polaris-specific discovery endpoint, requiring > clients > > > to > > > > > > query > > > > > > > and manage two different capability sources. > > > > > > > > > > > > > > Yufei > > > > > > > > > > > > > > > > > > > > > On Mon, Jun 22, 2026 at 5:28 PM Adnan Hemani via dev < > > > > > > > [email protected]> > > > > > > > wrote: > > > > > > > > > > > > > > > Ah yes, my mistake. But I think a similar point exists: Does > > the > > > > > > service > > > > > > > > need to know about all new endpoint information? Can we not > > make > > > it > > > > > > dumb > > > > > > > > and gate on a server-wide feature flag or something similar? > > > > > > > > > > > > > > > > -Adnan > > > > > > > > > > > > > > > > On Mon, Jun 22, 2026 at 5:02 PM Dmitri Bourlatchkov < > > > > > [email protected]> > > > > > > > > wrote: > > > > > > > > > > > > > > > > > Hi Adnan, > > > > > > > > > > > > > > > > > > You might have misinterpreted my message. My point was > about > > > the > > > > > > server > > > > > > > > > (service) side, not client side. > > > > > > > > > > > > > > > > > > Cheers, > > > > > > > > > Dmitri. > > > > > > > > > > > > > > > > > > On Mon, Jun 22, 2026 at 7:58 PM Adnan Hemani via dev < > > > > > > > > > [email protected]> > > > > > > > > > wrote: > > > > > > > > > > > > > > > > > > > > For example, if we are to expose Open Lineage or OSI > > > > endpoints > > > > > in > > > > > > > IRC > > > > > > > > > > config, the IRC service code will have to be aware of OL > / > > > OSI > > > > > > > modules. > > > > > > > > > > > > > > > > > > > > I don't agree with this statement; the IRC client is > always > > > > free > > > > > to > > > > > > > > > > disregard any of these endpoints/modules. Returning this > > > > > > information > > > > > > > > does > > > > > > > > > > not force a client to take any action based on this extra > > > > > endpoint > > > > > > > > > > information. > > > > > > > > > > > > > > > > > > > > I do agree that if this forced the IRC client to be > > > > fundamentally > > > > > > > > altered > > > > > > > > > > because of these additional endpoints, then that would > be a > > > > > > > significant > > > > > > > > > > issue. But as Russell stated above, "what would actually > > > break > > > > if > > > > > > > > someone > > > > > > > > > > interpreted the spec this way?" > > > > > > > > > > > > > > > > > > > > Best, > > > > > > > > > > Adnan Hemani > > > > > > > > > > > > > > > > > > > > On Mon, Jun 22, 2026 at 3:30 PM Dmitri Bourlatchkov < > > > > > > > [email protected]> > > > > > > > > > > wrote: > > > > > > > > > > > > > > > > > > > > > Hi Russell, > > > > > > > > > > > > > > > > > > > > > > Your points sound reasonable to me. > > > > > > > > > > > > > > > > > > > > > > Yet, noone answered my second question from my initial > > > email. > > > > > > That > > > > > > > > is: > > > > > > > > > > why > > > > > > > > > > > do we need to expose non-IRC endpoints in the IRC > config > > > > > response > > > > > > > in > > > > > > > > > the > > > > > > > > > > > first place? > > > > > > > > > > > > > > > > > > > > > > Regarding the maintainability aspect, I see this > > becoming a > > > > > more > > > > > > > > > > > substantial concern if it evolves into a > self-sustaining > > > > > pattern. > > > > > > > > > > > > > > > > > > > > > > For example, if we are to expose Open Lineage or OSI > > > > endpoints > > > > > in > > > > > > > IRC > > > > > > > > > > > config, the IRC service code will have to be aware of > OL > > / > > > > OSI > > > > > > > > > modules. I > > > > > > > > > > > think this dependency is conceptually incorrect. > > > > > > > > > > > > > > > > > > > > > > I do not so much mind existing non-IRC endpoints in the > > IRC > > > > > > config, > > > > > > > > but > > > > > > > > > > > extending this to new APIs feels like an anti-pattern > to > > > me. > > > > > > > > > > > > > > > > > > > > > > Cheers, > > > > > > > > > > > Dmitri. > > > > > > > > > > > > > > > > > > > > > > On Mon, Jun 22, 2026 at 5:46 PM Russell Spitzer < > > > > > > > > > > [email protected] > > > > > > > > > > > > > > > > > > > > > > > wrote: > > > > > > > > > > > > > > > > > > > > > > > From a very pragmatic point of view, I'm not sure > what > > > we'd > > > > > > gain > > > > > > > > from > > > > > > > > > > > > separate > > > > > > > > > > > > configuration mechanisms. We already have one and > it's > > > > worked > > > > > > > well > > > > > > > > so > > > > > > > > > > > far, > > > > > > > > > > > > do we have a compelling reason to swap right now? > > > > > > > > > > > > > > > > > > > > > > > > As best I can tell, the worst case is that the IRC > spec > > > > later > > > > > > > > > tightens > > > > > > > > > > > the > > > > > > > > > > > > semantics > > > > > > > > > > > > of the endpoints field, making our extra entries > > > > > non-compliant. > > > > > > > In > > > > > > > > > that > > > > > > > > > > > > situation > > > > > > > > > > > > we'd have to change our response, add a > > Polaris-specific > > > > > > > > > > config/discovery > > > > > > > > > > > > endpoint, and > > > > > > > > > > > > have our clients move over to it. But that's exactly > > the > > > > work > > > > > > > this > > > > > > > > > > > proposal > > > > > > > > > > > > is asking us to do today > > > > > > > > > > > > so we would be taking same burden now instead of > > > > potentially > > > > > > > never. > > > > > > > > > I'd > > > > > > > > > > > > rather defer > > > > > > > > > > > > it until something concrete forces our hand. > > > > > > > > > > > > > > > > > > > > > > > > The one counter I can see is that decoupling gets > > harder > > > as > > > > > > more > > > > > > > > > > clients > > > > > > > > > > > > adopt it. But > > > > > > > > > > > > since the Polaris endpoints are already namespaced > > > > > > > (/polaris/...), > > > > > > > > > > > lifting > > > > > > > > > > > > them into a separate endpoint > > > > > > > > > > > > later stays mechanical, so I don't think we're > > > > meaningfully > > > > > > > > cheaper > > > > > > > > > > > doing > > > > > > > > > > > > it now. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > I'd like to gently point out that it doesn't feel > > > > especially > > > > > > > > > important > > > > > > > > > > > > whether the exact wording > > > > > > > > > > > > or original intent of the spec supports what we're > > doing. > > > > The > > > > > > > > > > discussion > > > > > > > > > > > is > > > > > > > > > > > > starting to feel > > > > > > > > > > > > like a legalistic reading of the spec rather than a > > focus > > > > on > > > > > > > > concrete > > > > > > > > > > > > impact. I'd suggest we focus > > > > > > > > > > > > on arguments like "what would actually break if > someone > > > > > > > interpreted > > > > > > > > > the > > > > > > > > > > > > spec this way?" > > > > > > > > > > > > rather than "what was the original intent of the > text?" > > > > > > > > > > > > > > > > > > > > > > > > On Mon, Jun 22, 2026 at 3:23 PM Adnan Hemani via dev > < > > > > > > > > > > > > [email protected]> > > > > > > > > > > > > wrote: > > > > > > > > > > > > > > > > > > > > > > > > > Hi Dmitri, > > > > > > > > > > > > > > > > > > > > > > > > > > I think you and I view the same statement very > > > > differently > > > > > - > > > > > > I > > > > > > > > > > > definitely > > > > > > > > > > > > > don't think it's "pretty clear that the scope of > the > > > data > > > > > > > > returned > > > > > > > > > > > under > > > > > > > > > > > > > that type is limited to the API defined by the > > spec". I > > > > see > > > > > > it > > > > > > > > this > > > > > > > > > > > way: > > > > > > > > > > > > if > > > > > > > > > > > > > this statement truly meant ONLY IRC endpoints, the > > spec > > > > > would > > > > > > > > > > > explicitly > > > > > > > > > > > > > use the words "IRC spec endpoints". The absence of > > > > > > > clarification > > > > > > > > in > > > > > > > > > > any > > > > > > > > > > > > > specification rarely means you should > > > > > > "read-between-the-lines" > > > > > > > to > > > > > > > > > > > assume > > > > > > > > > > > > > the writer's "intent". > > > > > > > > > > > > > > > > > > > > > > > > > > I will look into Iceberg mailing list threads and > PRs > > > for > > > > > > > > > background > > > > > > > > > > > > > information to clarify things. > > > > > > > > > > > > > > > > > > > > > > > > > > Best, > > > > > > > > > > > > > Adnan Hemani > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > On Mon, Jun 22, 2026 at 12:27 PM Dmitri > Bourlatchkov > > < > > > > > > > > > > [email protected] > > > > > > > > > > > > > > > > > > > > > > > > > wrote: > > > > > > > > > > > > > > > > > > > > > > > > > > > HI Adnan, > > > > > > > > > > > > > > > > > > > > > > > > > > > > The IRC spec defines a "type" for the config > > > response. > > > > A > > > > > > > > property > > > > > > > > > > of > > > > > > > > > > > > that > > > > > > > > > > > > > > "type" is a list of "endpoints supported by the > > > > server". > > > > > > > There > > > > > > > > is > > > > > > > > > > no > > > > > > > > > > > > > > statement about generalizing those endpoints to > > > non-IRC > > > > > > APIs. > > > > > > > > > > > > > > > > > > > > > > > > > > > > Therefore, I think it is pretty clear that the > > scope > > > of > > > > > the > > > > > > > > data > > > > > > > > > > > > returned > > > > > > > > > > > > > > under that type is limited to the API defined by > > the > > > > > spec, > > > > > > > > which > > > > > > > > > is > > > > > > > > > > > > IRC. > > > > > > > > > > > > > > > > > > > > > > > > > > > > Let's expand this discussion a bit. > > > > > > > > > > > > > > > > > > > > > > > > > > > > What is the rationale for returning non-IRC > > endpoints > > > > in > > > > > > the > > > > > > > > IRC > > > > > > > > > > > config > > > > > > > > > > > > > > response? > > > > > > > > > > > > > > > > > > > > > > > > > > > > Thanks, > > > > > > > > > > > > > > Dmitri. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > On Mon, Jun 22, 2026 at 3:21 PM Adnan Hemani via > > dev > > > < > > > > > > > > > > > > > > [email protected]> wrote: > > > > > > > > > > > > > > > > > > > > > > > > > > > >> Hi Dmitri, > > > > > > > > > > > > > >> > > > > > > > > > > > > > >> I'm not sure where you are assuming that > > "endpoints" > > > > > only > > > > > > > > refers > > > > > > > > > > to > > > > > > > > > > > > > >> Iceberg > > > > > > > > > > > > > >> endpoints in the IRC spec. Do you have any thing > > you > > > > can > > > > > > > point > > > > > > > > > us > > > > > > > > > > to > > > > > > > > > > > > > >> regarding this? > > > > > > > > > > > > > >> > > > > > > > > > > > > > >> Best, > > > > > > > > > > > > > >> Adnan Hemani > > > > > > > > > > > > > >> > > > > > > > > > > > > > >> On Mon, Jun 22, 2026 at 12:18 PM Dmitri > > > Bourlatchkov < > > > > > > > > > > > > [email protected]> > > > > > > > > > > > > > >> wrote: > > > > > > > > > > > > > >> > > > > > > > > > > > > > >> > Hi Adam, > > > > > > > > > > > > > >> > > > > > > > > > > > > > > >> > > #1 - Is GET ' > > > > > > > > > > > > > >> > > > > > > > > > > http://localhost:8181/api/catalog/v1/config?warehouse=polaris > > > > > > > > > ' > > > > > > > > > > > only > > > > > > > > > > > > > >> > an Iceberg Catalog concept? > > > > > > > > > > > > > >> > > > > > > > > > > > > > > >> > My interpretation is YES, because the payload > of > > > > that > > > > > > > > response > > > > > > > > > > is > > > > > > > > > > > > > >> defined > > > > > > > > > > > > > >> > by the IRC API spec [1]. > > > > > > > > > > > > > >> > > > > > > > > > > > > > > >> > I believe one can reasonably assume that the > > > > statement > > > > > > > > > > "endpoints > > > > > > > > > > > > > >> supported > > > > > > > > > > > > > >> > by the server" is scoped only to the IRC API > > > itself. > > > > > > > There's > > > > > > > > > no > > > > > > > > > > > > > >> provision > > > > > > > > > > > > > >> > in that spec about covering all possible APIs. > > > > > > > > > > > > > >> > > > > > > > > > > > > > > >> > That said, I'd be happy to discuss how clients > > can > > > > > > > discover > > > > > > > > > > > Polaris > > > > > > > > > > > > > >> > features in general without overloading > existing > > > > > > > > > specifications. > > > > > > > > > > > > > >> > > > > > > > > > > > > > > >> > [1] > > > > > > > > > > > > > >> > > > > > > > > > > > > > > >> > > > > > > > > > > > > > > >> > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > https://github.com/apache/iceberg/blob/apache-iceberg-1.11.0/open-api/rest-catalog-open-api.yaml#L105 > > > > > > > > > > > > > >> > > > > > > > > > > > > > > >> > On Mon, Jun 22, 2026 at 2:52 PM Adam > Christian < > > > > > > > > > > > > > >> > [email protected]> wrote: > > > > > > > > > > > > > >> > > > > > > > > > > > > > > >> > > I agree with both Yufei & Dmitri: > > > > > > > > > > > > > >> > > > > > > > > > > > > > > > >> > > #1 - It seems like improper coupling for an > > > > Iceberg > > > > > > REST > > > > > > > > > > Catalog > > > > > > > > > > > > > >> config > > > > > > > > > > > > > >> > to > > > > > > > > > > > > > >> > > return a configuration unrelated to Iceberg. > > > > > > > > > > > > > >> > > #2 - Polaris is a superset of an Iceberg > REST > > > > > Catalog. > > > > > > > > > > > > > >> > > > > > > > > > > > > > > > >> > > So, in my opinion, the question is more > aptly > > > > framed > > > > > > as > > > > > > > > two > > > > > > > > > > > > > questions: > > > > > > > > > > > > > >> > > > > > > > > > > > > > > > >> > > #1 - Is GET ' > > > > > > > > > > > > > >> > > > > > > > > > > http://localhost:8181/api/catalog/v1/config?warehouse=polaris > > > > > > > > > > > > > >> > > ' > > > > > > > > > > > > > >> > > only an Iceberg Catalog concept? > > > > > > > > > > > > > >> > > #2 - Should GET ' > > > > > > > > > > > > > >> > > > > > > > > > > > > > http://localhost:8181/api/catalog/v1/config?warehouse=polaris > > > > > > > > > > ' > > > > > > > > > > > > > return > > > > > > > > > > > > > >> > > additional endpoints? > > > > > > > > > > > > > >> > > > > > > > > > > > > > > > >> > > I believe that the answer to #1 is no, but > > that > > > > > > requires > > > > > > > > > > changes > > > > > > > > > > > > to > > > > > > > > > > > > > >> the > > > > > > > > > > > > > >> > > codebase. I believe the answer to #2 is yes. > > > > > > > > > > > > > >> > > > > > > > > > > > > > > > >> > > Based on the codebase, we are exposing > several > > > > > > endpoints > > > > > > > > in > > > > > > > > > > this > > > > > > > > > > > > > >> > > configuration API which are not Iceberg > > > endpoints. > > > > > For > > > > > > > > > > example, > > > > > > > > > > > in > > > > > > > > > > > > > the > > > > > > > > > > > > > >> > > Generic Tables case, it is explicitly not > > > Iceberg. > > > > > In > > > > > > my > > > > > > > > > > > opinion, > > > > > > > > > > > > > >> this is > > > > > > > > > > > > > >> > > alright because our API specification only > > says > > > > > > > > > "Configuration > > > > > > > > > > > > API," > > > > > > > > > > > > > >> not > > > > > > > > > > > > > >> > > "Iceberg Configuration API." Yes, it is > > > > > > > > Iceberg-compatible, > > > > > > > > > > but > > > > > > > > > > > it > > > > > > > > > > > > > is > > > > > > > > > > > > > >> not > > > > > > > > > > > > > >> > > Iceberg-centric. This retrieves the > > > configuration > > > > > for > > > > > > > the > > > > > > > > > > > Catalog; > > > > > > > > > > > > > >> > > regardless of whether it is an Iceberg > > Catalog. > > > > Now, > > > > > > if > > > > > > > > this > > > > > > > > > > is > > > > > > > > > > > > > truly > > > > > > > > > > > > > >> a > > > > > > > > > > > > > >> > > superset, it implies that > > > > IcebergCatalogHandler.java > > > > > > > > should > > > > > > > > > > not > > > > > > > > > > > > > handle > > > > > > > > > > > > > >> > > returning the configuration. Additionally, > we > > > need > > > > > to > > > > > > > > > return a > > > > > > > > > > > > > >> superset > > > > > > > > > > > > > >> > of > > > > > > > > > > > > > >> > > the ConfigResponse (an Iceberg-core concept) > > for > > > > the > > > > > > > REST > > > > > > > > > API > > > > > > > > > > > > > >> response. > > > > > > > > > > > > > >> > > > > > > > > > > > > > > > >> > > Now, if the answer to #1 is yes, that means > we > > > > > should > > > > > > > > > probably > > > > > > > > > > > > have > > > > > > > > > > > > > a > > > > > > > > > > > > > >> > > separate API for configuration. > > > > > > > > > > > > > >> > > > > > > > > > > > > > > > >> > > What do y'all think? > > > > > > > > > > > > > >> > > > > > > > > > > > > > > > >> > > Go community, > > > > > > > > > > > > > >> > > > > > > > > > > > > > > > >> > > Adam > > > > > > > > > > > > > >> > > > > > > > > > > > > > > > >> > > On Mon, Jun 22, 2026 at 1:55 PM Yufei Gu < > > > > > > > > > > [email protected]> > > > > > > > > > > > > > >> wrote: > > > > > > > > > > > > > >> > > > > > > > > > > > > > > > >> > > > I think a Polaris client is also an IRC > > > client, > > > > > just > > > > > > > > with > > > > > > > > > > > > > additional > > > > > > > > > > > > > >> > > > capabilities. In that sense, Polaris can > be > > > > viewed > > > > > > as > > > > > > > a > > > > > > > > > > > superset > > > > > > > > > > > > > of > > > > > > > > > > > > > >> > IRC. > > > > > > > > > > > > > >> > > If > > > > > > > > > > > > > >> > > > the config endpoint is intended for > > capability > > > > > > > > discovery, > > > > > > > > > > > > > returning > > > > > > > > > > > > > >> > > Polaris > > > > > > > > > > > > > >> > > > specific endpoints seems reasonable. > > Standard > > > > IRC > > > > > > > > clients > > > > > > > > > > can > > > > > > > > > > > > > simply > > > > > > > > > > > > > >> > > ignore > > > > > > > > > > > > > >> > > > endpoints they don't recognize. > > > > > > > > > > > > > >> > > > > > > > > > > > > > > > > >> > > > A concrete example is the Polaris Spark > > > client, > > > > > > which > > > > > > > > > checks > > > > > > > > > > > > > server > > > > > > > > > > > > > >> > > > capabilities before using Polaris specific > > > > > > > > functionality: > > > > > > > > > > > > > >> > > > > > > > > > > > > > > > > >> > > > public List<TableIdentifier> > > > > > > > listGenericTables(Namespace > > > > > > > > > > ns) { > > > > > > > > > > > > > >> > > > Endpoint.check(endpoints, > > > > > > > > > > > > > >> PolarisEndpoints.V1_LIST_GENERIC_TABLES); > > > > > > > > > > > > > >> > > > > > > > > > > > > > > > > >> > > > > > > > > > > > > > > > > >> > > > Yufei > > > > > > > > > > > > > >> > > > > > > > > > > > > > > > > >> > > > > > > > > > > > > > > > > >> > > > On Fri, Jun 19, 2026 at 7:43 AM Dmitri > > > > > Bourlatchkov > > > > > > < > > > > > > > > > > > > > >> [email protected]> > > > > > > > > > > > > > >> > > > wrote: > > > > > > > > > > > > > >> > > > > > > > > > > > > > > > > >> > > > > Hi All, > > > > > > > > > > > > > >> > > > > > > > > > > > > > > > > > >> > > > > During my review of [4816] I realized > [1] > > > that > > > > > > > Polaris > > > > > > > > > > > returns > > > > > > > > > > > > > >> some > > > > > > > > > > > > > >> > > > non-IRC > > > > > > > > > > > > > >> > > > > endpoints in the IRC config responses. > > > > > > > > > > > > > >> > > > > > > > > > > > > > > > > > >> > > > > For example: > > > > > > > > > > > > > >> > > > > > > > > > > > > > > > > > >> > > > > GET ' > > > > > > > > > > > > > >> > > > > > > > http://localhost:8181/api/catalog/v1/config?warehouse=polaris > > > > > > > > ' > > > > > > > > > > > > > >> > > > > > > > > > > > > > > > > > >> > > > > Result: > > > > > > > > > > > > > >> > > > > [...] > > > > > > > > > > > > > >> > > > > "endpoints": [ > > > > > > > > > > > > > >> > > > > "GET /v1/{prefix}/namespaces", > > > > > > > > > > > > > >> > > > > "GET > > > /v1/{prefix}/namespaces/{namespace}", > > > > > > > > > > > > > >> > > > > "HEAD > > > > /v1/{prefix}/namespaces/{namespace}", > > > > > > > > > > > > > >> > > > > [...] > > > > > > > > > > > > > >> > > > > "DELETE > > > > > > > > > > > > > >> > > > > > > > > > > > > > > > > > >> > > > > > > > > > > > > > > > > >> > > > > > > > > > > > > > > > >> > > > > > > > > > > > > > > >> > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > polaris/v1/{prefix}/namespaces/{namespace}/generic-tables/{generic-table}", > > > > > > > > > > > > > >> > > > > "GET > > > > > > > > > > > > > >> > > > > > > > > > > > > > > > > > >> > > > > > > > > > > > > > > > > >> > > > > > > > > > > > > > > > >> > > > > > > > > > > > > > > >> > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > polaris/v1/{prefix}/namespaces/{namespace}/generic-tables/{generic-table}", > > > > > > > > > > > > > >> > > > > "GET > > > > > > > > > > > > > /polaris/v1/{prefix}/namespaces/{namespace}/policies", > > > > > > > > > > > > > >> > > > > "POST > > > > > > > > > > > > > /polaris/v1/{prefix}/namespaces/{namespace}/policies", > > > > > > > > > > > > > >> > > > > > > > > > > > > > > > > > >> > > > > The latter group of endpoints is not > > related > > > > to > > > > > > the > > > > > > > > > > Iceberg > > > > > > > > > > > > REST > > > > > > > > > > > > > >> > > Catalog > > > > > > > > > > > > > >> > > > > API. > > > > > > > > > > > > > >> > > > > > > > > > > > > > > > > > >> > > > > I wonder what the rationale might be for > > > > > returning > > > > > > > > them > > > > > > > > > in > > > > > > > > > > > the > > > > > > > > > > > > > IRC > > > > > > > > > > > > > >> > > config > > > > > > > > > > > > > >> > > > > response. > > > > > > > > > > > > > >> > > > > > > > > > > > > > > > > > >> > > > > From my POV, returning them in the IRC > > > config > > > > > > > response > > > > > > > > > is > > > > > > > > > > > > > >> incorrect > > > > > > > > > > > > > >> > > > because > > > > > > > > > > > > > >> > > > > these endpoints form APIs that follow a > > > > > different > > > > > > > > > > > > specification > > > > > > > > > > > > > >> and > > > > > > > > > > > > > >> > > > clients > > > > > > > > > > > > > >> > > > > using the IRC API do not need that > > > information > > > > > to > > > > > > > > > properly > > > > > > > > > > > use > > > > > > > > > > > > > the > > > > > > > > > > > > > >> > IRC > > > > > > > > > > > > > >> > > > API. > > > > > > > > > > > > > >> > > > > > > > > > > > > > > > > > >> > > > > WDYT? > > > > > > > > > > > > > >> > > > > > > > > > > > > > > > > > >> > > > > [1] > > > > > > > > > > > > > >> > > > > > > > > > > > > > > > > > > > https://github.com/apache/polaris/pull/4816#discussion_r3438945230 > > > > > > > > > > > > > >> > > > > > > > > > > > > > > > > > >> > > > > [4816] > > > > > > https://github.com/apache/polaris/pull/4816 > > > > > > > > > > > > > >> > > > > > > > > > > > > > > > > > >> > > > > Thanks, > > > > > > > > > > > > > >> > > > > Dmitri. > > > > > > > > > > > > > >> > > > > > > > > > > > > > > > > > >> > > > > > > > > > > > > > > > > >> > > > > > > > > > > > > > > > >> > > > > > > > > > > > > > > >> > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > -- > > > > > > Dmitri Bourlatchkov > > > > > > Senior Staff Software Engineer, Dremio > > > > > > Dremio.com > > > > > > < > > > > > > > > > > > > > > > > > > > > > https://www.dremio.com/?utm_medium=email&utm_source=signature&utm_term=na&utm_content=email-signature&utm_campaign=email-signature > > > > > > > > > > > > > / > > > > > > Follow Us on LinkedIn <https://www.linkedin.com/company/dremio> > / > > > Get > > > > > > Started <https://www.dremio.com/get-started/> > > > > > > > > > > > > > > > > > > The Agentic Lakehouse > > > > > > The only lakehouse built for agents, managed by agents > > > > > > > > > > > > > > > > > > > > >
