> I'm not up-to-date with Ossie though... Is it explicit about the spec version in its JSON data representation?
Yes. Check the Ossie spec in the repo https://github.com/apache/ossie for more details. And Polaris side, both my design doc and and REST spec have covered that. > If not, then Polaris will have to indicate the version of the Ossie spec > somehow. This will probably require an envelope in the response anyway. Could you clarify the envelope approach here? I explicitly asked about this in my last emails and provided examples to facilitate discussion. Would you mind taking a look and provide more details of what's in your mind? Yufei On Fri, Aug 14, 2026 at 2:12 PM Dmitri Bourlatchkov <[email protected]> wrote: > Hi Yufei, > > I'm fine with JSON response payloads following the Apache Ossie JSON > structure in v1 API and deferring other payload types to future API > revisions. > > I'm not up-to-date with Ossie though... Is it explicit about the spec > version in its JSON data representation? > > If not, then Polaris will have to indicate the version of the Ossie spec > somehow. This will probably require an envelope in the response anyway. > > The alternative is to revision the Polaris API (v1, v2, v3, etc.) for every > Ossie spec change, which seems impractical to me. > > WDYT? > > Thanks, > Dmitri. > > On Tue, Jul 21, 2026 at 9:37 PM Yufei Gu <[email protected]> wrote: > > > Thanks for the discussion. > > > > Dmitri, could you clarify what the envelope would look like? > > > > My current understanding is that it might look roughly like this: > > > > { > > "format": "ossie", > > "version": "1.0", > > "payload": { > > "...": "..." > > } > > } > > > > For a non JSON format, the payload would instead need to be > > represented as an encoded string: > > > > { > > "format": "okf", > > "version": "1.0", > > "payload": "encoded document content" > > } > > > > If that is the intended design, I am not sure the envelope gives us > > much beyond option 1, as clients still need format specific handling. > > > > So the clearest choices seem to be either: > > 1. Direct JSON for Ossie, with no envelope, that seems to be what JB > > proposed. In that case, we'd have to change the spec or introduce a v2 > > endpoint when we introduce a new format like OKF. > > 2. a format-agnostic envelope with an opaque string payload. > > > > Am I missing another benefit of the envelope approach? > > > > Thanks, > > Yufei > > > > Yufei > > > > > > On Thu, Jul 16, 2026 at 4:22 AM Jean-Baptiste Onofré <[email protected]> > > wrote: > > > > > > Hi everyone, > > > > > > I prefer to store and expose the semantic API using a direct JSON > > > payload format ("raw"). > > > > > > I also suggest that we focus exclusively on the Apache Ossie format > > > for now, without any wrapping. > > > > > > Regarding the evolution of the Ossie specification, I believe it is > > > better to store raw JSON and handle the plumbing on our end. This > > > approach allows any tool, including AI tools, to directly consume the > > > Ossie definition. > > > > > > Regards, > > > JB > > > > > > On Mon, Jul 13, 2026 at 8:13 PM Yufei Gu <[email protected]> wrote: > > > > > > > > Hi Dmitri, Thanks! I think option 2 is a reasonable choice. The > > advantage I > > > > see with option 1 is future extensibility. If we later decide to > > support > > > > another semantic model format that isn't JSON, such as OKF, option 2 > > would > > > > require a REST spec change, while option 1 would not. That's why I > > slightly > > > > prefer option 1, although I think either option is much better than > > > > modeling the semantic model structure directly in the REST API. What > > do you > > > > think? > > > > > > > > Thanks, > > > > Yufei > > > > > > > > > > > > On Mon, Jul 13, 2026 at 11:10 AM Yufei Gu <[email protected]> > > wrote: > > > > > > > > > Hi Robert, > > > > > > > > > > Thanks for the thoughtful feedback. > > > > > > > > > > I agree that the current API is narrower than a complete semantic > > layer. > > > > > Today it's primarily about managing the lifecycle of semantic model > > > > > documents. Discovery, search, and client consumption are all > > important > > > > > topics, but I think they're broader than the Ossie semantic model > > itself. > > > > > We touched on some of these in the last community sync[1], and I'm > > happy to > > > > > continue that discussion in a separate thread. > > > > > > > > > > I also agree that we should be careful not to overstate what this > > initial > > > > > API provides. Calling it a beta semantic model registry or document > > hosting > > > > > API seems reasonable. > > > > > > > > > > That said, I think those questions are orthogonal to the payload > > > > > representation discussion. My original topic was simply whether the > > REST > > > > > API should treat the semantic model as a raw document, an opaque > JSON > > > > > payload, or model the schema directly. Since the consumption story > > is still > > > > > evolving, I think keeping the REST contract loosely coupled to the > > > > > underlying semantic model specification gives us the most > > flexibility. > > > > > > > > > > [1] > > > > > > > > https://drive.google.com/file/d/1hxYkk2t-BcnFOk8eJG9NYCHfjkOXg3Iz/view?usp=sharing > > > > > > > > > > Thanks, > > > > > > > > > > Yufei > > > > > > > > > > On Mon, Jul 13, 2026 at 6:16 AM Robert Stupp <[email protected]> > wrote: > > > > > > > > > >> Hi, > > > > >> > > > > >> I think the payload representation question depends on the client > > model we > > > > >> expect this API to support. > > > > >> > > > > >> I support Polaris hosting Apache Ossie semantic-model documents as > > a beta > > > > >> foundation. That seems like a useful first step while Ossie itself > > is > > > > >> still > > > > >> evolving. > > > > >> > > > > >> But I do not think we should describe the current API as enabling > AI > > > > >> tools, > > > > >> BI tools, or human semantic-model discovery yet. > > > > >> > > > > >> The merged API is primarily namespace/name CRUD for a > semantic-model > > > > >> document. > > > > >> That is opaque document storage and exact point-retrieval. > > > > >> > > > > >> It does not enable clients to discover the right semantic model > > from a > > > > >> query, > > > > >> nor to find semantic models by table, metric, domain, user, or > > capability. > > > > >> > > > > >> It also does not define a standard consuming API or tool contract, > > > > >> search/indexing contract, freshness model, or > > current/trusted/certified > > > > >> model > > > > >> semantics. > > > > >> > > > > >> That distinction matters because the REST API is the user-facing > > contract. > > > > >> > > > > >> If the beta API is intended only as opaque document hosting, I > > think the > > > > >> spec > > > > >> and docs should say that clearly, and users/clients should not > infer > > > > >> broader > > > > >> discovery or interoperability semantics from the CRUD API. > > > > >> > > > > >> I am not asking to solve the full semantic-layer story > immediately. > > > > >> > > > > >> I am asking that durable implementation work does not get ahead of > > the > > > > >> client-consumption story. > > > > >> > > > > >> The client model should drive the persistent data model, not the > > other way > > > > >> around. > > > > >> > > > > >> Once semantic models are stored as durable Polaris entities, > choices > > > > >> around > > > > >> identity, versioning, validation, indexing, size limits, > > source-table > > > > >> references, and freshness become much harder to change. > > > > >> > > > > >> For now, I would be comfortable describing this as beta Apache > Ossie > > > > >> document > > > > >> hosting / semantic-model registry work. > > > > >> > > > > >> I would not yet be comfortable describing it as enabling AI, BI, > or > > human > > > > >> semantic workflows until the discovery and client-consumption > story > > > > >> exists. > > > > >> > > > > >> Robert > > > > >> > > > > >> > > > > >> On Fri, Jul 10, 2026 at 3:10 PM Dmitri Bourlatchkov < > > [email protected]> > > > > >> wrote: > > > > >> > > > > >> > Hi Yufei, > > > > >> > > > > > >> > Thanks for starting this thread! > > > > >> > > > > > >> > I tend to think it is best to represent Ossie data as direct > JSON > > > > >> without > > > > >> > defining its structure in the Polaris OpenAPI spec (Ossie > schemas > > are > > > > >> > controlled by Ossie, not Polaris). I believe this corresponds to > > Option > > > > >> 2 > > > > >> > from your email. > > > > >> > > > > > >> > Polaris code that implements the new API will then interpret the > > Ossie > > > > >> > parts according to the declared version of the Ossie spec. > > > > >> > > > > > >> > With that in mind, the API should clearly state the format > (Ossie > > or > > > > >> OKF) > > > > >> > and the specification version of the semantic data sub-object > > > > >> (apologies if > > > > >> > it has that already, I'm behind on the related PR updates). > > Obviously > > > > >> OKF > > > > >> > will have a different representation in the payload, but this > > should not > > > > >> > prevent Ossie from leveraging JSON synergies. > > > > >> > > > > > >> > Anand's work on solving a similar problem in the Metrics API > > [4115] may > > > > >> be > > > > >> > reusable here. > > > > >> > > > > > >> > [4115] https://github.com/apache/polaris/pull/4115 > > > > >> > > > > > >> > Cheers, > > > > >> > Dmitri. > > > > >> > > > > > >> > > > > > >> > On Wed, Jul 8, 2026 at 2:20 PM Yufei Gu <[email protected]> > > wrote: > > > > >> > > > > > >> > > Hi folks, > > > > >> > > > > > > >> > > Following JB's suggestion, I'd like to start a dedicated > > discussion on > > > > >> > the > > > > >> > > REST API payload representation for semantic models. > > > > >> > > > > > > >> > > I think there are three possible approaches: > > > > >> > > > > > > >> > > 1. > > > > >> > > > > > > >> > > Represent the semantic model as a raw string. > > > > >> > > 2. > > > > >> > > > > > > >> > > Represent the semantic model as an opaque JSON document. > > > > >> > > 3. > > > > >> > > > > > > >> > > Model the semantic model structure directly in the REST > > > > >> specification. > > > > >> > > > > > > >> > > I think it's helpful to separate the REST API from Polaris' > > internal > > > > >> > > representation. The REST API is the long-term contract with > > clients, > > > > >> > while > > > > >> > > the internal representation can evolve independently. > > > > >> > > > > > > >> > > I'm comfortable with either option 1 or option 2. Both avoid > > coupling > > > > >> the > > > > >> > > REST API to the Ossie schema and allow Polaris to validate the > > payload > > > > >> > > based on the semantic model type and version while preserving > > the > > > > >> > document > > > > >> > > through write and read operations. > > > > >> > > > > > > >> > > My concern is with option 3. Since the Ossie schema is > > versioned and > > > > >> > > expected to evolve, modeling the full semantic model structure > > > > >> directly > > > > >> > in > > > > >> > > the REST specification would tightly couple the Polaris REST > > API to > > > > >> Ossie > > > > >> > > versions. Every Ossie schema evolution could require changes > to > > the > > > > >> REST > > > > >> > > specification, generated clients, and potentially client > > applications. > > > > >> > > > > > > >> > > Between options 1 and 2, I think there is an additional > > tradeoff. > > > > >> > > > > > > >> > > An opaque JSON document assumes that semantic models are > always > > > > >> > represented > > > > >> > > as JSON. While that works well for Ossie today, Polaris may > > support > > > > >> other > > > > >> > > semantic model formats in the future. For example, OKF[1] is > > defined > > > > >> as > > > > >> > > Markdown rather than JSON. Using a raw string keeps the REST > API > > > > >> > > independent of any particular document format, allowing > Polaris > > to > > > > >> > support > > > > >> > > JSON, Markdown, or other representations without changing the > > API > > > > >> > contract. > > > > >> > > > > > > >> > > So my current view is: > > > > >> > > > > > > >> > > - > > > > >> > > > > > > >> > > Option 1 provides the greatest flexibility and is format > > agnostic. > > > > >> > > - > > > > >> > > > > > > >> > > Option 2 is a natural choice if we want to optimize > > specifically > > > > >> for > > > > >> > > JSON based semantic models. > > > > >> > > - > > > > >> > > > > > > >> > > Option 3 provides strong typing, but at the cost of > coupling > > the > > > > >> REST > > > > >> > > API to Ossie schema evolution. > > > > >> > > > > > > >> > > I'm happy with either option 1 or option 2, but I'd avoid > > option 3 for > > > > >> > the > > > > >> > > reasons above. > > > > >> > > > > > > >> > > Thoughts? > > > > >> > > > > > > >> > > 1. > > > > >> > > > > > > >> > > > > > > >> > > > > > >> > > > https://cloud.google.com/blog/products/data-analytics/how-the-open-knowledge-format-can-improve-data-sharing > > > > >> > > > > > > >> > > Thanks, > > > > >> > > > > > > >> > > Yufei > > > > >> > > > > > > >> > > > > > >> > > > > > > > >
