Hi Robert, Thanks for the thoughtful feedback.
I agree that the current API is narrower than a complete semantic layer. Today it's primarily about managing the lifecycle of semantic model documents. Discovery, search, and client consumption are all important topics, but I think they're broader than the Ossie semantic model itself. We touched on some of these in the last community sync[1], and I'm happy to continue that discussion in a separate thread. I also agree that we should be careful not to overstate what this initial API provides. Calling it a beta semantic model registry or document hosting API seems reasonable. That said, I think those questions are orthogonal to the payload representation discussion. My original topic was simply whether the REST API should treat the semantic model as a raw document, an opaque JSON payload, or model the schema directly. Since the consumption story is still evolving, I think keeping the REST contract loosely coupled to the underlying semantic model specification gives us the most flexibility. [1] https://drive.google.com/file/d/1hxYkk2t-BcnFOk8eJG9NYCHfjkOXg3Iz/view?usp=sharing Thanks, Yufei On Mon, Jul 13, 2026 at 6:16 AM Robert Stupp <[email protected]> wrote: > Hi, > > I think the payload representation question depends on the client model we > expect this API to support. > > I support Polaris hosting Apache Ossie semantic-model documents as a beta > foundation. That seems like a useful first step while Ossie itself is still > evolving. > > But I do not think we should describe the current API as enabling AI tools, > BI tools, or human semantic-model discovery yet. > > The merged API is primarily namespace/name CRUD for a semantic-model > document. > That is opaque document storage and exact point-retrieval. > > It does not enable clients to discover the right semantic model from a > query, > nor to find semantic models by table, metric, domain, user, or capability. > > It also does not define a standard consuming API or tool contract, > search/indexing contract, freshness model, or current/trusted/certified > model > semantics. > > That distinction matters because the REST API is the user-facing contract. > > If the beta API is intended only as opaque document hosting, I think the > spec > and docs should say that clearly, and users/clients should not infer > broader > discovery or interoperability semantics from the CRUD API. > > I am not asking to solve the full semantic-layer story immediately. > > I am asking that durable implementation work does not get ahead of the > client-consumption story. > > The client model should drive the persistent data model, not the other way > around. > > Once semantic models are stored as durable Polaris entities, choices around > identity, versioning, validation, indexing, size limits, source-table > references, and freshness become much harder to change. > > For now, I would be comfortable describing this as beta Apache Ossie > document > hosting / semantic-model registry work. > > I would not yet be comfortable describing it as enabling AI, BI, or human > semantic workflows until the discovery and client-consumption story exists. > > Robert > > > On Fri, Jul 10, 2026 at 3:10 PM Dmitri Bourlatchkov <[email protected]> > wrote: > > > Hi Yufei, > > > > Thanks for starting this thread! > > > > I tend to think it is best to represent Ossie data as direct JSON without > > defining its structure in the Polaris OpenAPI spec (Ossie schemas are > > controlled by Ossie, not Polaris). I believe this corresponds to Option 2 > > from your email. > > > > Polaris code that implements the new API will then interpret the Ossie > > parts according to the declared version of the Ossie spec. > > > > With that in mind, the API should clearly state the format (Ossie or OKF) > > and the specification version of the semantic data sub-object (apologies > if > > it has that already, I'm behind on the related PR updates). Obviously OKF > > will have a different representation in the payload, but this should not > > prevent Ossie from leveraging JSON synergies. > > > > Anand's work on solving a similar problem in the Metrics API [4115] may > be > > reusable here. > > > > [4115] https://github.com/apache/polaris/pull/4115 > > > > Cheers, > > Dmitri. > > > > > > On Wed, Jul 8, 2026 at 2:20 PM Yufei Gu <[email protected]> wrote: > > > > > Hi folks, > > > > > > Following JB's suggestion, I'd like to start a dedicated discussion on > > the > > > REST API payload representation for semantic models. > > > > > > I think there are three possible approaches: > > > > > > 1. > > > > > > Represent the semantic model as a raw string. > > > 2. > > > > > > Represent the semantic model as an opaque JSON document. > > > 3. > > > > > > Model the semantic model structure directly in the REST > specification. > > > > > > I think it's helpful to separate the REST API from Polaris' internal > > > representation. The REST API is the long-term contract with clients, > > while > > > the internal representation can evolve independently. > > > > > > I'm comfortable with either option 1 or option 2. Both avoid coupling > the > > > REST API to the Ossie schema and allow Polaris to validate the payload > > > based on the semantic model type and version while preserving the > > document > > > through write and read operations. > > > > > > My concern is with option 3. Since the Ossie schema is versioned and > > > expected to evolve, modeling the full semantic model structure directly > > in > > > the REST specification would tightly couple the Polaris REST API to > Ossie > > > versions. Every Ossie schema evolution could require changes to the > REST > > > specification, generated clients, and potentially client applications. > > > > > > Between options 1 and 2, I think there is an additional tradeoff. > > > > > > An opaque JSON document assumes that semantic models are always > > represented > > > as JSON. While that works well for Ossie today, Polaris may support > other > > > semantic model formats in the future. For example, OKF[1] is defined as > > > Markdown rather than JSON. Using a raw string keeps the REST API > > > independent of any particular document format, allowing Polaris to > > support > > > JSON, Markdown, or other representations without changing the API > > contract. > > > > > > So my current view is: > > > > > > - > > > > > > Option 1 provides the greatest flexibility and is format agnostic. > > > - > > > > > > Option 2 is a natural choice if we want to optimize specifically for > > > JSON based semantic models. > > > - > > > > > > Option 3 provides strong typing, but at the cost of coupling the > REST > > > API to Ossie schema evolution. > > > > > > I'm happy with either option 1 or option 2, but I'd avoid option 3 for > > the > > > reasons above. > > > > > > Thoughts? > > > > > > 1. > > > > > > > > > https://cloud.google.com/blog/products/data-analytics/how-the-open-knowledge-format-can-improve-data-sharing > > > > > > Thanks, > > > > > > Yufei > > > > > >
