think there are two levels: * code itself -> don't think there is a debate about modularity there, it is easier to integrate, refactor, drop potentially etc * docker image -> while I agree it is better to have an adjusted bundle it is also true end users will want supported runtime so default is the real question and being forced to build a custom distro defeats the default build and increases support work. Also note it is true for jdbc driver so h2 must not come in the default image following the "minimal surface" logic. So my 2cts would be to get something in between with a promotion logic of feature once mature enough in the default build.
hope it makes sense Romain Manni-Bucau @rmannibucau <https://x.com/rmannibucau> | .NET Blog <https://dotnetbirdie.github.io/> | Blog <https://rmannibucau.github.io/> | Old Blog <http://rmannibucau.wordpress.com> | Github <https://github.com/rmannibucau> | LinkedIn <https://www.linkedin.com/in/rmannibucau> | Book <https://www.packtpub.com/en-us/product/java-ee-8-high-performance-9781788473064> Javaccino founder (Java/.NET service - contact via linkedin) Le ven. 26 juin 2026 à 13:11, Alexandre Dutra <[email protected]> a écrit : > Hi all, > > I am not a fan of gating an entire API behind a feature flag. > > Another reason not mentioned yet is: if a security vulnerability is > detected in the new code, and that code is shipped unconditionally in > polaris-runtime-service, then all deployments of that artifact will be > flagged by security scans, regardless of whether they opted out of it via > the feature flag. If the CVE targets a separate module instead, only users > of that module would be affected. > > That is another reason why I think isolating the API in its module is a > better design choice. > > Thanks, > Alex > > Le ven. 26 juin 2026 à 01:02, Yufei Gu <[email protected]> a écrit : > > > Hi Dmitri, > > > > Thanks for the clarification. > > > > Could you elaborate on why having an empty HTTP layer is a concern for > > downstream systems? If the feature is disabled, couldn't we simply > return a > > 404 or 501, similar to how Quarkus behaves when an endpoint is not > > registered? > > > > Thanks, > > > > Yufei > > > > > > On Wed, Jun 24, 2026 at 12:38 PM Dmitri Bourlatchkov <[email protected]> > > wrote: > > > > > Hi Yufei, > > > > > > As I commented in this thread earlier, storing OSI data as Polaris > > entities > > > is a reasonable approach. > > > > > > However, adding hard dependencies from `runtime/service` to the new OSI > > > RESP API impl. is not acceptable from my POV, as it forces > > > downstream projects into exposing the OSI API without explicit opt-in. > > > Feature flags are not relevant here because they work only after REST > > > requests are accepted at the HTTP layer. > > > > > > This is discussed from a more general perspective in [1] > > > > > > All in all, I do not see any disadvantage to using separate modules for > > new > > > REST API implementations, but disadvantages in bundling them into > > > runtime/serice do exist. > > > > > > [1] https://lists.apache.org/thread/d9dj3w8ktwdn6w27z7tvvgkljgw3n43b > > > > > > Cheers, > > > Dmitri. > > > > > > On Tue, Jun 23, 2026 at 8:58 PM Yufei Gu <[email protected]> wrote: > > > > > > > Hi Dmitri, > > > > > > > > I see the value of keeping Polaris modular, but I have a slightly > > > different > > > > view on this particular case. > > > > > > > > To me, semantic models are closer to tables, views, and policies than > > to > > > > metrics or events. The proposal introduces a new Polaris entity type > > with > > > > its own lifecycle, authorization model, and metadata management. In > > that > > > > sense, semantic model support is part of the core Polaris metadata > > model > > > > rather than an optional auxiliary capability like metrics. > > > > > > > > For that reason, I would lean toward treating semantic models > similarly > > > to > > > > other Polaris entities and keeping the API as part of the core > Polaris > > > > service. We already provide a feature flag to disable the > > functionality, > > > > which gives operators and downstream distributions the flexibility to > > > turn > > > > it off when it is not needed. > > > > > > > > Thanks, > > > > Yufei > > > > > > > > > > > > On Mon, Jun 22, 2026 at 2:28 PM Dmitri Bourlatchkov < > > > > [email protected]> wrote: > > > > > > > > > Hi Yufei, > > > > > > > > > > Persisting OSI data as Polaris entities sounds reasonable to me. > > > > > > > > > > However, I believe the REST API layer for OSI should be structured > > as a > > > > > module with opt in/out opportunities for downstream builds (similar > > to > > > > the > > > > > Metric query API). This is not a feature flag concern, but a point > > > about > > > > > the composition of the Polaris code. A modular approach promotes > code > > > > > clarity and allows both including the new API into default Polaris > > > > > images as well as flexibility downstream projects. I do not see any > > > > > downside to the modular approach. > > > > > > > > > > Feature flags can certainly be supported in the new API modules. > > > > > > > > > > Cheers, > > > > > Dmitri. > > > > > > > > > > On Mon, Jun 22, 2026 at 5:00 PM Yufei Gu <[email protected]> > > wrote: > > > > > > > > > > > Anand, thanks for chiming in. Looking forward to work together on > > it. > > > > > > > > > > > > Dmitri, Adam, Adnan, thanks for the clarification. I think we can > > > > > separate > > > > > > a few concerns here. > > > > > > > > > > > > Apache Ossie specifies the OSI model spec itself, but not the > CRUD > > > REST > > > > > > endpoints for managing OSI documents in Polaris. Polaris has the > > > > > > opportunity to define those APIs. As Adam mentioned, the > validator > > is > > > > > > intended for Ossie schema validation. That should definitely be > > > version > > > > > > based, so Polaris can validate the submitted document against the > > > > > > corresponding OSI spec version while keeping the REST API > contract > > > > under > > > > > > Polaris control. > > > > > > > > > > > > On the "first class" point, I think Adnan's interpretation is > > > correct. > > > > > The > > > > > > intent is that a semantic model is a Polaris entity in the same > > sense > > > > as > > > > > an > > > > > > Iceberg table, view, generic table, or policy. It participates in > > the > > > > > > Polaris metadata model, authorization model, and lifecycle as a > > > managed > > > > > > entity. In that sense, it is different from metrics or events, > > which > > > > are > > > > > > auxiliary data associated with entities rather than entities > > > > themselves. > > > > > > > > > > > > On the "always active" point, providing a feature flag makes > sense, > > > > this > > > > > is > > > > > > already included in PR 4816. We can run the OSI API by default in > > the > > > > > > Apache Polaris build, but allow downstream admins to turn it off > if > > > > they > > > > > do > > > > > > not need it in their deployment. > > > > > > > > > > > > Thanks, > > > > > > Yufei > > > > > > > > > > > > > > > > > > On Mon, Jun 22, 2026 at 1:37 PM Anand Kumar Sankaran via dev < > > > > > > [email protected]> wrote: > > > > > > > > > > > > > JB and Yufei, > > > > > > > > > > > > > > Thanks for doing this. We have customers asking for this as > well. > > > > Happy > > > > > > to > > > > > > > help in any way possible. > > > > > > > > > > > > > > - > > > > > > > Anand > > > > > > > > > > > > > > From: Adnan Hemani via dev <[email protected]> > > > > > > > Date: Monday, June 22, 2026 at 12:18 PM > > > > > > > To: [email protected] <[email protected]> > > > > > > > Cc: Adnan Hemani <[email protected]> > > > > > > > Subject: Re: [DISCUSS] Semantic Layer Support in Apache Polaris > > > > > > > > > > > > > > This Message Is From an External Sender > > > > > > > This message came from outside your organization. > > > > > > > Report Suspicious< > > > > > > > > > > > > > > > > > > > > > > > > > > > > https://us-phishalarm-ewt.proofpoint.com/EWT/v1/Iz9xO38YGHZK!YhNDZAGomgiHL51L-6FL3QPZjxHXwiq6JCAQHbb6PAE7K6Eqwb--zyy23NolE2-B94Vu6rTO00mQ6c0S3xLY-wGl3G8wkj5qTIJjWF_iK7wIvcJej0eX1hsbj7Uhl7_c$ > > > > > > > > > > > > > > > > > > > > > > > > > > > > > Hi Adam, Dmitri, Yufei, > > > > > > > > > > > > > > Adding in a clarification: I believe "first class" in the > context > > > of > > > > > OSI > > > > > > > would mean that it is given the same level of importance as a > > > Polaris > > > > > > > entity as a Table or View would. Is that generally correct? > > > > > > > > > > > > > > Best, > > > > > > > Adnan Hemani > > > > > > > > > > > > > > On Mon, Jun 22, 2026 at 10:50 AM Adam Christian < > > > > > > > [email protected]> wrote: > > > > > > > > > > > > > > > Hi Dmitri, > > > > > > > > > > > > > > > > This proposal [1] includes a second tab with the detailed > > design. > > > > It > > > > > > > shows > > > > > > > > the REST APIs that handle the CRUD operations for OSI > Semantic > > > > > Models. > > > > > > > The > > > > > > > > Semantic Model will be validated in the OsiDocumentValidator > > > which > > > > I > > > > > > > assume > > > > > > > > will validate against the Apache Ossie version. In my > reading, > > > > > Polaris > > > > > > > does > > > > > > > > not control it; we will leverage the upstream spec. > > > > > > > > > > > > > > > > Regarding OSI functionality if this feature is always > active, I > > > > > assume > > > > > > > > users would benefit from it being active. If an admin user > does > > > not > > > > > > want > > > > > > > to > > > > > > > > leverage OSI inside their Polaris instance, they simply won't > > > grant > > > > > the > > > > > > > > privileges to the consuming users. > > > > > > > > > > > > > > > > [1] - > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > https://urldefense.com/v3/__https://docs.google.com/document/d/1ZdI-1w_5LbyCMhvUhLCtOt-N1Z89L2P-oiGLaYayCZg/edit?usp=sharing__;!!Iz9xO38YGHZK!5zRt5PLr106Rj8WbH_RftJ4SqCWP119n37Z77kzoNL-_JhobudorMvD0UdqyXJTi1PCMu0vGL3KGPBIq6oELUw$ > > > > > > > > > > > > > > > > On Thu, Jun 18, 2026 at 6:13 PM Dmitri Bourlatchkov < > > > > > [email protected]> > > > > > > > > wrote: > > > > > > > > > > > > > > > > > Hi Yufei, > > > > > > > > > > > > > > > > > > Sorry for getting to this proposal late. I postred some > > > comments > > > > on > > > > > > PR > > > > > > > > > 4816, recounting the key points here in more detail. > > > > > > > > > > > > > > > > > > * From the proposal doc: Goal G1: "Store OSI 0.1.x > documents > > as > > > > > > > > first-class > > > > > > > > > Polaris entities, scoped under a Namespace" > > > > > > > > > > > > > > > > > > I believe this needs a bit more discussion before we > proceed > > to > > > > > > > concrete > > > > > > > > > code changes. The idea of persisting OSI data is totally > > valid. > > > > > > > However, > > > > > > > > > I'm not sure what "first class" means in this context? Does > > it > > > > mean > > > > > > > that > > > > > > > > > OSI functionality has to be active all the time? > > > > > > > > > > > > > > > > > > My initial perception of this proposal is that as a use > case > > it > > > > is > > > > > > > > similar > > > > > > > > > to persisting Metrics (or Events) in Polaris. That is, the > > > > feature > > > > > is > > > > > > > > > valuable, but downstream projects may want to have the > > > > flexibility > > > > > of > > > > > > > > > deciding whether to include it or not. > > > > > > > > > > > > > > > > > > * Another point I'd like to clarity is about the REST API > > > > > definition. > > > > > > > Are > > > > > > > > > API endpoints going be defined and controlled by the > Polaris > > > > > project? > > > > > > > > > > > > > > > > > > * Are REST API payload types defined and controlled by > > Polaris > > > or > > > > > by > > > > > > > > Apache > > > > > > > > > Ossie [1]? > > > > > > > > > > > > > > > > > > [1] > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > https://urldefense.com/v3/__https://www.mail-archive.com/[email protected]/msg86564.html__;!!Iz9xO38YGHZK!5zRt5PLr106Rj8WbH_RftJ4SqCWP119n37Z77kzoNL-_JhobudorMvD0UdqyXJTi1PCMu0vGL3KGPBLfgVHpkQ$ > > > > > > > > > > > > > > > > > > Thanks, > > > > > > > > > Dmitri. > > > > > > > > > > > > > > > > > > On Fri, May 29, 2026 at 6:34 PM Yufei Gu < > > [email protected] > > > > > > > > > > wrote: > > > > > > > > > > > > > > > > > > > Hi folks, > > > > > > > > > > > > > > > > > > > > As AI agents, BI tools, notebooks, and query engines > > > > increasingly > > > > > > > > consume > > > > > > > > > > the same data, semantic definitions such as metrics and > > > > > dimensions > > > > > > > are > > > > > > > > > > often duplicated across multiple systems. This leads to > > > > > > inconsistent > > > > > > > > > > definitions, duplicated effort, and governance > challenges. > > > The > > > > > rise > > > > > > > of > > > > > > > > AI > > > > > > > > > > agents further amplifies this problem, as agents rely on > > > > semantic > > > > > > > > context > > > > > > > > > > to understand data and reason about business concepts. > > > Without > > > > a > > > > > > > shared > > > > > > > > > > semantic layer, organizations often end up maintaining > > > multiple > > > > > > > > versions > > > > > > > > > of > > > > > > > > > > the same business definitions across tools and > > applications. > > > > > > > > > > > > > > > > > > > > JB and I would like to start a discussion on adding > > semantic > > > > > layer > > > > > > > > > support > > > > > > > > > > to Apache Polaris so semantic models can be defined once, > > > > > governed > > > > > > > > > > centrally, and consumed consistently across tools. The > > > > > proposal[1] > > > > > > > > > > introduces semantic models as a first class Polaris > entity > > > > using > > > > > > the > > > > > > > > Open > > > > > > > > > > Semantic Interchange (OSI)[2] specification[3]. At a high > > > > level, > > > > > > the > > > > > > > > > > proposal adds: > > > > > > > > > > > > > > > > > > > > - A new SEMANTIC_MODEL entity type > > > > > > > > > > - CRUD APIs for semantic models > > > > > > > > > > - Schema validation and authorization > > > > > > > > > > > > > > > > > > > > Polaris remains a metadata service and does not execute > > > metrics > > > > > or > > > > > > > > > semantic > > > > > > > > > > queries. > > > > > > > > > > Feedback on the overall direction, design, and OSI > adoption > > > > would > > > > > > be > > > > > > > > > > greatly appreciated. > > > > > > > > > > > > > > > > > > > > 1. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > https://urldefense.com/v3/__https://docs.google.com/document/d/1ZdI-1w_5LbyCMhvUhLCtOt-N1Z89L2P-oiGLaYayCZg/edit?usp=sharing__;!!Iz9xO38YGHZK!5zRt5PLr106Rj8WbH_RftJ4SqCWP119n37Z77kzoNL-_JhobudorMvD0UdqyXJTi1PCMu0vGL3KGPBIq6oELUw$ > > > > > > > > > > 2. > > > > > > > > > > > > > > > > > > > > > > > > > > > > https://urldefense.com/v3/__https://open-semantic-interchange.org__;!!Iz9xO38YGHZK!5zRt5PLr106Rj8WbH_RftJ4SqCWP119n37Z77kzoNL-_JhobudorMvD0UdqyXJTi1PCMu0vGL3KGPBKnqDA0QQ$ > > > > > > > > > > 3. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > https://urldefense.com/v3/__https://github.com/open-semantic-interchange/OSI/blob/main/core-spec/spec.md__;!!Iz9xO38YGHZK!5zRt5PLr106Rj8WbH_RftJ4SqCWP119n37Z77kzoNL-_JhobudorMvD0UdqyXJTi1PCMu0vGL3KGPBLfoGUc7Q$ > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > Yufei > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > -- > > > > > Dmitri Bourlatchkov > > > > > Senior Staff Software Engineer, Dremio > > > > > Dremio.com > > > > > < > > > > > > > > > > > > > > > https://www.dremio.com/?utm_medium=email&utm_source=signature&utm_term=na&utm_content=email-signature&utm_campaign=email-signature > > > > > > > > > > > / > > > > > Follow Us on LinkedIn <https://www.linkedin.com/company/dremio> / > > Get > > > > > Started <https://www.dremio.com/get-started/> > > > > > > > > > > > > > > > The Agentic Lakehouse > > > > > The only lakehouse built for agents, managed by agents > > > > > > > > > > > > > > >
