Hi, I think the payload representation question depends on the client model we expect this API to support.
I support Polaris hosting Apache Ossie semantic-model documents as a beta foundation. That seems like a useful first step while Ossie itself is still evolving. But I do not think we should describe the current API as enabling AI tools, BI tools, or human semantic-model discovery yet. The merged API is primarily namespace/name CRUD for a semantic-model document. That is opaque document storage and exact point-retrieval. It does not enable clients to discover the right semantic model from a query, nor to find semantic models by table, metric, domain, user, or capability. It also does not define a standard consuming API or tool contract, search/indexing contract, freshness model, or current/trusted/certified model semantics. That distinction matters because the REST API is the user-facing contract. If the beta API is intended only as opaque document hosting, I think the spec and docs should say that clearly, and users/clients should not infer broader discovery or interoperability semantics from the CRUD API. I am not asking to solve the full semantic-layer story immediately. I am asking that durable implementation work does not get ahead of the client-consumption story. The client model should drive the persistent data model, not the other way around. Once semantic models are stored as durable Polaris entities, choices around identity, versioning, validation, indexing, size limits, source-table references, and freshness become much harder to change. For now, I would be comfortable describing this as beta Apache Ossie document hosting / semantic-model registry work. I would not yet be comfortable describing it as enabling AI, BI, or human semantic workflows until the discovery and client-consumption story exists. Robert On Fri, Jul 10, 2026 at 3:10 PM Dmitri Bourlatchkov <[email protected]> wrote: > Hi Yufei, > > Thanks for starting this thread! > > I tend to think it is best to represent Ossie data as direct JSON without > defining its structure in the Polaris OpenAPI spec (Ossie schemas are > controlled by Ossie, not Polaris). I believe this corresponds to Option 2 > from your email. > > Polaris code that implements the new API will then interpret the Ossie > parts according to the declared version of the Ossie spec. > > With that in mind, the API should clearly state the format (Ossie or OKF) > and the specification version of the semantic data sub-object (apologies if > it has that already, I'm behind on the related PR updates). Obviously OKF > will have a different representation in the payload, but this should not > prevent Ossie from leveraging JSON synergies. > > Anand's work on solving a similar problem in the Metrics API [4115] may be > reusable here. > > [4115] https://github.com/apache/polaris/pull/4115 > > Cheers, > Dmitri. > > > On Wed, Jul 8, 2026 at 2:20 PM Yufei Gu <[email protected]> wrote: > > > Hi folks, > > > > Following JB's suggestion, I'd like to start a dedicated discussion on > the > > REST API payload representation for semantic models. > > > > I think there are three possible approaches: > > > > 1. > > > > Represent the semantic model as a raw string. > > 2. > > > > Represent the semantic model as an opaque JSON document. > > 3. > > > > Model the semantic model structure directly in the REST specification. > > > > I think it's helpful to separate the REST API from Polaris' internal > > representation. The REST API is the long-term contract with clients, > while > > the internal representation can evolve independently. > > > > I'm comfortable with either option 1 or option 2. Both avoid coupling the > > REST API to the Ossie schema and allow Polaris to validate the payload > > based on the semantic model type and version while preserving the > document > > through write and read operations. > > > > My concern is with option 3. Since the Ossie schema is versioned and > > expected to evolve, modeling the full semantic model structure directly > in > > the REST specification would tightly couple the Polaris REST API to Ossie > > versions. Every Ossie schema evolution could require changes to the REST > > specification, generated clients, and potentially client applications. > > > > Between options 1 and 2, I think there is an additional tradeoff. > > > > An opaque JSON document assumes that semantic models are always > represented > > as JSON. While that works well for Ossie today, Polaris may support other > > semantic model formats in the future. For example, OKF[1] is defined as > > Markdown rather than JSON. Using a raw string keeps the REST API > > independent of any particular document format, allowing Polaris to > support > > JSON, Markdown, or other representations without changing the API > contract. > > > > So my current view is: > > > > - > > > > Option 1 provides the greatest flexibility and is format agnostic. > > - > > > > Option 2 is a natural choice if we want to optimize specifically for > > JSON based semantic models. > > - > > > > Option 3 provides strong typing, but at the cost of coupling the REST > > API to Ossie schema evolution. > > > > I'm happy with either option 1 or option 2, but I'd avoid option 3 for > the > > reasons above. > > > > Thoughts? > > > > 1. > > > > > https://cloud.google.com/blog/products/data-analytics/how-the-open-knowledge-format-can-improve-data-sharing > > > > Thanks, > > > > Yufei > > >
