Hi Dmitri, I see the value of keeping Polaris modular, but I have a slightly different view on this particular case.
To me, semantic models are closer to tables, views, and policies than to metrics or events. The proposal introduces a new Polaris entity type with its own lifecycle, authorization model, and metadata management. In that sense, semantic model support is part of the core Polaris metadata model rather than an optional auxiliary capability like metrics. For that reason, I would lean toward treating semantic models similarly to other Polaris entities and keeping the API as part of the core Polaris service. We already provide a feature flag to disable the functionality, which gives operators and downstream distributions the flexibility to turn it off when it is not needed. Thanks, Yufei On Mon, Jun 22, 2026 at 2:28 PM Dmitri Bourlatchkov < [email protected]> wrote: > Hi Yufei, > > Persisting OSI data as Polaris entities sounds reasonable to me. > > However, I believe the REST API layer for OSI should be structured as a > module with opt in/out opportunities for downstream builds (similar to the > Metric query API). This is not a feature flag concern, but a point about > the composition of the Polaris code. A modular approach promotes code > clarity and allows both including the new API into default Polaris > images as well as flexibility downstream projects. I do not see any > downside to the modular approach. > > Feature flags can certainly be supported in the new API modules. > > Cheers, > Dmitri. > > On Mon, Jun 22, 2026 at 5:00 PM Yufei Gu <[email protected]> wrote: > > > Anand, thanks for chiming in. Looking forward to work together on it. > > > > Dmitri, Adam, Adnan, thanks for the clarification. I think we can > separate > > a few concerns here. > > > > Apache Ossie specifies the OSI model spec itself, but not the CRUD REST > > endpoints for managing OSI documents in Polaris. Polaris has the > > opportunity to define those APIs. As Adam mentioned, the validator is > > intended for Ossie schema validation. That should definitely be version > > based, so Polaris can validate the submitted document against the > > corresponding OSI spec version while keeping the REST API contract under > > Polaris control. > > > > On the "first class" point, I think Adnan's interpretation is correct. > The > > intent is that a semantic model is a Polaris entity in the same sense as > an > > Iceberg table, view, generic table, or policy. It participates in the > > Polaris metadata model, authorization model, and lifecycle as a managed > > entity. In that sense, it is different from metrics or events, which are > > auxiliary data associated with entities rather than entities themselves. > > > > On the "always active" point, providing a feature flag makes sense, this > is > > already included in PR 4816. We can run the OSI API by default in the > > Apache Polaris build, but allow downstream admins to turn it off if they > do > > not need it in their deployment. > > > > Thanks, > > Yufei > > > > > > On Mon, Jun 22, 2026 at 1:37 PM Anand Kumar Sankaran via dev < > > [email protected]> wrote: > > > > > JB and Yufei, > > > > > > Thanks for doing this. We have customers asking for this as well. Happy > > to > > > help in any way possible. > > > > > > - > > > Anand > > > > > > From: Adnan Hemani via dev <[email protected]> > > > Date: Monday, June 22, 2026 at 12:18 PM > > > To: [email protected] <[email protected]> > > > Cc: Adnan Hemani <[email protected]> > > > Subject: Re: [DISCUSS] Semantic Layer Support in Apache Polaris > > > > > > This Message Is From an External Sender > > > This message came from outside your organization. > > > Report Suspicious< > > > > > > https://us-phishalarm-ewt.proofpoint.com/EWT/v1/Iz9xO38YGHZK!YhNDZAGomgiHL51L-6FL3QPZjxHXwiq6JCAQHbb6PAE7K6Eqwb--zyy23NolE2-B94Vu6rTO00mQ6c0S3xLY-wGl3G8wkj5qTIJjWF_iK7wIvcJej0eX1hsbj7Uhl7_c$ > > > > > > > > > > > > > Hi Adam, Dmitri, Yufei, > > > > > > Adding in a clarification: I believe "first class" in the context of > OSI > > > would mean that it is given the same level of importance as a Polaris > > > entity as a Table or View would. Is that generally correct? > > > > > > Best, > > > Adnan Hemani > > > > > > On Mon, Jun 22, 2026 at 10:50 AM Adam Christian < > > > [email protected]> wrote: > > > > > > > Hi Dmitri, > > > > > > > > This proposal [1] includes a second tab with the detailed design. It > > > shows > > > > the REST APIs that handle the CRUD operations for OSI Semantic > Models. > > > The > > > > Semantic Model will be validated in the OsiDocumentValidator which I > > > assume > > > > will validate against the Apache Ossie version. In my reading, > Polaris > > > does > > > > not control it; we will leverage the upstream spec. > > > > > > > > Regarding OSI functionality if this feature is always active, I > assume > > > > users would benefit from it being active. If an admin user does not > > want > > > to > > > > leverage OSI inside their Polaris instance, they simply won't grant > the > > > > privileges to the consuming users. > > > > > > > > [1] - > > > > > > > > > > > > > > https://urldefense.com/v3/__https://docs.google.com/document/d/1ZdI-1w_5LbyCMhvUhLCtOt-N1Z89L2P-oiGLaYayCZg/edit?usp=sharing__;!!Iz9xO38YGHZK!5zRt5PLr106Rj8WbH_RftJ4SqCWP119n37Z77kzoNL-_JhobudorMvD0UdqyXJTi1PCMu0vGL3KGPBIq6oELUw$ > > > > > > > > On Thu, Jun 18, 2026 at 6:13 PM Dmitri Bourlatchkov < > [email protected]> > > > > wrote: > > > > > > > > > Hi Yufei, > > > > > > > > > > Sorry for getting to this proposal late. I postred some comments on > > PR > > > > > 4816, recounting the key points here in more detail. > > > > > > > > > > * From the proposal doc: Goal G1: "Store OSI 0.1.x documents as > > > > first-class > > > > > Polaris entities, scoped under a Namespace" > > > > > > > > > > I believe this needs a bit more discussion before we proceed to > > > concrete > > > > > code changes. The idea of persisting OSI data is totally valid. > > > However, > > > > > I'm not sure what "first class" means in this context? Does it mean > > > that > > > > > OSI functionality has to be active all the time? > > > > > > > > > > My initial perception of this proposal is that as a use case it is > > > > similar > > > > > to persisting Metrics (or Events) in Polaris. That is, the feature > is > > > > > valuable, but downstream projects may want to have the flexibility > of > > > > > deciding whether to include it or not. > > > > > > > > > > * Another point I'd like to clarity is about the REST API > definition. > > > Are > > > > > API endpoints going be defined and controlled by the Polaris > project? > > > > > > > > > > * Are REST API payload types defined and controlled by Polaris or > by > > > > Apache > > > > > Ossie [1]? > > > > > > > > > > [1] > > > > > > > > > > > https://urldefense.com/v3/__https://www.mail-archive.com/[email protected]/msg86564.html__;!!Iz9xO38YGHZK!5zRt5PLr106Rj8WbH_RftJ4SqCWP119n37Z77kzoNL-_JhobudorMvD0UdqyXJTi1PCMu0vGL3KGPBLfgVHpkQ$ > > > > > > > > > > Thanks, > > > > > Dmitri. > > > > > > > > > > On Fri, May 29, 2026 at 6:34 PM Yufei Gu <[email protected]> > > wrote: > > > > > > > > > > > Hi folks, > > > > > > > > > > > > As AI agents, BI tools, notebooks, and query engines increasingly > > > > consume > > > > > > the same data, semantic definitions such as metrics and > dimensions > > > are > > > > > > often duplicated across multiple systems. This leads to > > inconsistent > > > > > > definitions, duplicated effort, and governance challenges. The > rise > > > of > > > > AI > > > > > > agents further amplifies this problem, as agents rely on semantic > > > > context > > > > > > to understand data and reason about business concepts. Without a > > > shared > > > > > > semantic layer, organizations often end up maintaining multiple > > > > versions > > > > > of > > > > > > the same business definitions across tools and applications. > > > > > > > > > > > > JB and I would like to start a discussion on adding semantic > layer > > > > > support > > > > > > to Apache Polaris so semantic models can be defined once, > governed > > > > > > centrally, and consumed consistently across tools. The > proposal[1] > > > > > > introduces semantic models as a first class Polaris entity using > > the > > > > Open > > > > > > Semantic Interchange (OSI)[2] specification[3]. At a high level, > > the > > > > > > proposal adds: > > > > > > > > > > > > - A new SEMANTIC_MODEL entity type > > > > > > - CRUD APIs for semantic models > > > > > > - Schema validation and authorization > > > > > > > > > > > > Polaris remains a metadata service and does not execute metrics > or > > > > > semantic > > > > > > queries. > > > > > > Feedback on the overall direction, design, and OSI adoption would > > be > > > > > > greatly appreciated. > > > > > > > > > > > > 1. > > > > > > > > > > > > > > > > > > > > > > > > > > > https://urldefense.com/v3/__https://docs.google.com/document/d/1ZdI-1w_5LbyCMhvUhLCtOt-N1Z89L2P-oiGLaYayCZg/edit?usp=sharing__;!!Iz9xO38YGHZK!5zRt5PLr106Rj8WbH_RftJ4SqCWP119n37Z77kzoNL-_JhobudorMvD0UdqyXJTi1PCMu0vGL3KGPBIq6oELUw$ > > > > > > 2. > > > > > > https://urldefense.com/v3/__https://open-semantic-interchange.org__;!!Iz9xO38YGHZK!5zRt5PLr106Rj8WbH_RftJ4SqCWP119n37Z77kzoNL-_JhobudorMvD0UdqyXJTi1PCMu0vGL3KGPBKnqDA0QQ$ > > > > > > 3. > > > > > > > > > > > > > > > > > > > > > > > > > > > https://urldefense.com/v3/__https://github.com/open-semantic-interchange/OSI/blob/main/core-spec/spec.md__;!!Iz9xO38YGHZK!5zRt5PLr106Rj8WbH_RftJ4SqCWP119n37Z77kzoNL-_JhobudorMvD0UdqyXJTi1PCMu0vGL3KGPBLfoGUc7Q$ > > > > > > > > > > > > > > > > > > Yufei > > > > > > > > > > > > > > > > > > > > > > > > > > -- > Dmitri Bourlatchkov > Senior Staff Software Engineer, Dremio > Dremio.com > < > https://www.dremio.com/?utm_medium=email&utm_source=signature&utm_term=na&utm_content=email-signature&utm_campaign=email-signature > > > / > Follow Us on LinkedIn <https://www.linkedin.com/company/dremio> / Get > Started <https://www.dremio.com/get-started/> > > > The Agentic Lakehouse > The only lakehouse built for agents, managed by agents >
