Hi everyone,
I prefer to store and expose the semantic API using a direct JSON
payload format ("raw").
I also suggest that we focus exclusively on the Apache Ossie format
for now, without any wrapping.
Regarding the evolution of the Ossie specification, I believe it is
better to store raw JSON and handle the plumbing on our end. This
approach allows any tool, including AI tools, to directly consume the
Ossie definition.
Regards,
JB
On Mon, Jul 13, 2026 at 8:13 PM Yufei Gu <[email protected]> wrote:
>
> Hi Dmitri, Thanks! I think option 2 is a reasonable choice. The advantage I
> see with option 1 is future extensibility. If we later decide to support
> another semantic model format that isn't JSON, such as OKF, option 2 would
> require a REST spec change, while option 1 would not. That's why I slightly
> prefer option 1, although I think either option is much better than
> modeling the semantic model structure directly in the REST API. What do you
> think?
>
> Thanks,
> Yufei
>
>
> On Mon, Jul 13, 2026 at 11:10 AM Yufei Gu <[email protected]> wrote:
>
> > Hi Robert,
> >
> > Thanks for the thoughtful feedback.
> >
> > I agree that the current API is narrower than a complete semantic layer.
> > Today it's primarily about managing the lifecycle of semantic model
> > documents. Discovery, search, and client consumption are all important
> > topics, but I think they're broader than the Ossie semantic model itself.
> > We touched on some of these in the last community sync[1], and I'm happy to
> > continue that discussion in a separate thread.
> >
> > I also agree that we should be careful not to overstate what this initial
> > API provides. Calling it a beta semantic model registry or document hosting
> > API seems reasonable.
> >
> > That said, I think those questions are orthogonal to the payload
> > representation discussion. My original topic was simply whether the REST
> > API should treat the semantic model as a raw document, an opaque JSON
> > payload, or model the schema directly. Since the consumption story is still
> > evolving, I think keeping the REST contract loosely coupled to the
> > underlying semantic model specification gives us the most flexibility.
> >
> > [1]
> > https://drive.google.com/file/d/1hxYkk2t-BcnFOk8eJG9NYCHfjkOXg3Iz/view?usp=sharing
> >
> > Thanks,
> >
> > Yufei
> >
> > On Mon, Jul 13, 2026 at 6:16 AM Robert Stupp <[email protected]> wrote:
> >
> >> Hi,
> >>
> >> I think the payload representation question depends on the client model we
> >> expect this API to support.
> >>
> >> I support Polaris hosting Apache Ossie semantic-model documents as a beta
> >> foundation. That seems like a useful first step while Ossie itself is
> >> still
> >> evolving.
> >>
> >> But I do not think we should describe the current API as enabling AI
> >> tools,
> >> BI tools, or human semantic-model discovery yet.
> >>
> >> The merged API is primarily namespace/name CRUD for a semantic-model
> >> document.
> >> That is opaque document storage and exact point-retrieval.
> >>
> >> It does not enable clients to discover the right semantic model from a
> >> query,
> >> nor to find semantic models by table, metric, domain, user, or capability.
> >>
> >> It also does not define a standard consuming API or tool contract,
> >> search/indexing contract, freshness model, or current/trusted/certified
> >> model
> >> semantics.
> >>
> >> That distinction matters because the REST API is the user-facing contract.
> >>
> >> If the beta API is intended only as opaque document hosting, I think the
> >> spec
> >> and docs should say that clearly, and users/clients should not infer
> >> broader
> >> discovery or interoperability semantics from the CRUD API.
> >>
> >> I am not asking to solve the full semantic-layer story immediately.
> >>
> >> I am asking that durable implementation work does not get ahead of the
> >> client-consumption story.
> >>
> >> The client model should drive the persistent data model, not the other way
> >> around.
> >>
> >> Once semantic models are stored as durable Polaris entities, choices
> >> around
> >> identity, versioning, validation, indexing, size limits, source-table
> >> references, and freshness become much harder to change.
> >>
> >> For now, I would be comfortable describing this as beta Apache Ossie
> >> document
> >> hosting / semantic-model registry work.
> >>
> >> I would not yet be comfortable describing it as enabling AI, BI, or human
> >> semantic workflows until the discovery and client-consumption story
> >> exists.
> >>
> >> Robert
> >>
> >>
> >> On Fri, Jul 10, 2026 at 3:10 PM Dmitri Bourlatchkov <[email protected]>
> >> wrote:
> >>
> >> > Hi Yufei,
> >> >
> >> > Thanks for starting this thread!
> >> >
> >> > I tend to think it is best to represent Ossie data as direct JSON
> >> without
> >> > defining its structure in the Polaris OpenAPI spec (Ossie schemas are
> >> > controlled by Ossie, not Polaris). I believe this corresponds to Option
> >> 2
> >> > from your email.
> >> >
> >> > Polaris code that implements the new API will then interpret the Ossie
> >> > parts according to the declared version of the Ossie spec.
> >> >
> >> > With that in mind, the API should clearly state the format (Ossie or
> >> OKF)
> >> > and the specification version of the semantic data sub-object
> >> (apologies if
> >> > it has that already, I'm behind on the related PR updates). Obviously
> >> OKF
> >> > will have a different representation in the payload, but this should not
> >> > prevent Ossie from leveraging JSON synergies.
> >> >
> >> > Anand's work on solving a similar problem in the Metrics API [4115] may
> >> be
> >> > reusable here.
> >> >
> >> > [4115] https://github.com/apache/polaris/pull/4115
> >> >
> >> > Cheers,
> >> > Dmitri.
> >> >
> >> >
> >> > On Wed, Jul 8, 2026 at 2:20 PM Yufei Gu <[email protected]> wrote:
> >> >
> >> > > Hi folks,
> >> > >
> >> > > Following JB's suggestion, I'd like to start a dedicated discussion on
> >> > the
> >> > > REST API payload representation for semantic models.
> >> > >
> >> > > I think there are three possible approaches:
> >> > >
> >> > > 1.
> >> > >
> >> > > Represent the semantic model as a raw string.
> >> > > 2.
> >> > >
> >> > > Represent the semantic model as an opaque JSON document.
> >> > > 3.
> >> > >
> >> > > Model the semantic model structure directly in the REST
> >> specification.
> >> > >
> >> > > I think it's helpful to separate the REST API from Polaris' internal
> >> > > representation. The REST API is the long-term contract with clients,
> >> > while
> >> > > the internal representation can evolve independently.
> >> > >
> >> > > I'm comfortable with either option 1 or option 2. Both avoid coupling
> >> the
> >> > > REST API to the Ossie schema and allow Polaris to validate the payload
> >> > > based on the semantic model type and version while preserving the
> >> > document
> >> > > through write and read operations.
> >> > >
> >> > > My concern is with option 3. Since the Ossie schema is versioned and
> >> > > expected to evolve, modeling the full semantic model structure
> >> directly
> >> > in
> >> > > the REST specification would tightly couple the Polaris REST API to
> >> Ossie
> >> > > versions. Every Ossie schema evolution could require changes to the
> >> REST
> >> > > specification, generated clients, and potentially client applications.
> >> > >
> >> > > Between options 1 and 2, I think there is an additional tradeoff.
> >> > >
> >> > > An opaque JSON document assumes that semantic models are always
> >> > represented
> >> > > as JSON. While that works well for Ossie today, Polaris may support
> >> other
> >> > > semantic model formats in the future. For example, OKF[1] is defined
> >> as
> >> > > Markdown rather than JSON. Using a raw string keeps the REST API
> >> > > independent of any particular document format, allowing Polaris to
> >> > support
> >> > > JSON, Markdown, or other representations without changing the API
> >> > contract.
> >> > >
> >> > > So my current view is:
> >> > >
> >> > > -
> >> > >
> >> > > Option 1 provides the greatest flexibility and is format agnostic.
> >> > > -
> >> > >
> >> > > Option 2 is a natural choice if we want to optimize specifically
> >> for
> >> > > JSON based semantic models.
> >> > > -
> >> > >
> >> > > Option 3 provides strong typing, but at the cost of coupling the
> >> REST
> >> > > API to Ossie schema evolution.
> >> > >
> >> > > I'm happy with either option 1 or option 2, but I'd avoid option 3 for
> >> > the
> >> > > reasons above.
> >> > >
> >> > > Thoughts?
> >> > >
> >> > > 1.
> >> > >
> >> > >
> >> >
> >> https://cloud.google.com/blog/products/data-analytics/how-the-open-knowledge-format-can-improve-data-sharing
> >> > >
> >> > > Thanks,
> >> > >
> >> > > Yufei
> >> > >
> >> >
> >>
> >
