Hi Yufei, Thanks for the pointer.
The existing spec version field in the Ossie JSON schema seems to be sufficient for identifying the format. Polaris does not have to duplicate it. At this point, I do not see a strong reason for Polaris to use envelopes in Ossie JSON payloads. Cheers, Dmitri. On Fri, Aug 14, 2026 at 5:42 PM Yufei Gu <[email protected]> wrote: > > I'm not up-to-date with Ossie though... Is it explicit about the spec > version > in its JSON data representation? > > Yes. Check the Ossie spec in the repo https://github.com/apache/ossie for > more details. And Polaris side, both my design doc and and REST spec have > covered that. > > > If not, then Polaris will have to indicate the version of the Ossie spec > somehow. > This will probably require an envelope in the response anyway. > > Could you clarify the envelope approach here? I explicitly asked about this > in my last emails and provided examples to facilitate discussion. Would you > mind taking a look and provide more details of what's in your mind? > > Yufei > > > On Fri, Aug 14, 2026 at 2:12 PM Dmitri Bourlatchkov <[email protected]> > wrote: > > > Hi Yufei, > > > > I'm fine with JSON response payloads following the Apache Ossie JSON > > structure in v1 API and deferring other payload types to future API > > revisions. > > > > I'm not up-to-date with Ossie though... Is it explicit about the spec > > version in its JSON data representation? > > > > If not, then Polaris will have to indicate the version of the Ossie spec > > somehow. This will probably require an envelope in the response anyway. > > > > The alternative is to revision the Polaris API (v1, v2, v3, etc.) for > every > > Ossie spec change, which seems impractical to me. > > > > WDYT? > > > > Thanks, > > Dmitri. > > > > On Tue, Jul 21, 2026 at 9:37 PM Yufei Gu <[email protected]> wrote: > > > > > Thanks for the discussion. > > > > > > Dmitri, could you clarify what the envelope would look like? > > > > > > My current understanding is that it might look roughly like this: > > > > > > { > > > "format": "ossie", > > > "version": "1.0", > > > "payload": { > > > "...": "..." > > > } > > > } > > > > > > For a non JSON format, the payload would instead need to be > > > represented as an encoded string: > > > > > > { > > > "format": "okf", > > > "version": "1.0", > > > "payload": "encoded document content" > > > } > > > > > > If that is the intended design, I am not sure the envelope gives us > > > much beyond option 1, as clients still need format specific handling. > > > > > > So the clearest choices seem to be either: > > > 1. Direct JSON for Ossie, with no envelope, that seems to be what JB > > > proposed. In that case, we'd have to change the spec or introduce a v2 > > > endpoint when we introduce a new format like OKF. > > > 2. a format-agnostic envelope with an opaque string payload. > > > > > > Am I missing another benefit of the envelope approach? > > > > > > Thanks, > > > Yufei > > > > > > Yufei > > > > > > > > > On Thu, Jul 16, 2026 at 4:22 AM Jean-Baptiste Onofré <[email protected]> > > > wrote: > > > > > > > > Hi everyone, > > > > > > > > I prefer to store and expose the semantic API using a direct JSON > > > > payload format ("raw"). > > > > > > > > I also suggest that we focus exclusively on the Apache Ossie format > > > > for now, without any wrapping. > > > > > > > > Regarding the evolution of the Ossie specification, I believe it is > > > > better to store raw JSON and handle the plumbing on our end. This > > > > approach allows any tool, including AI tools, to directly consume the > > > > Ossie definition. > > > > > > > > Regards, > > > > JB > > > > > > > > On Mon, Jul 13, 2026 at 8:13 PM Yufei Gu <[email protected]> > wrote: > > > > > > > > > > Hi Dmitri, Thanks! I think option 2 is a reasonable choice. The > > > advantage I > > > > > see with option 1 is future extensibility. If we later decide to > > > support > > > > > another semantic model format that isn't JSON, such as OKF, option > 2 > > > would > > > > > require a REST spec change, while option 1 would not. That's why I > > > slightly > > > > > prefer option 1, although I think either option is much better than > > > > > modeling the semantic model structure directly in the REST API. > What > > > do you > > > > > think? > > > > > > > > > > Thanks, > > > > > Yufei > > > > > > > > > > > > > > > On Mon, Jul 13, 2026 at 11:10 AM Yufei Gu <[email protected]> > > > wrote: > > > > > > > > > > > Hi Robert, > > > > > > > > > > > > Thanks for the thoughtful feedback. > > > > > > > > > > > > I agree that the current API is narrower than a complete semantic > > > layer. > > > > > > Today it's primarily about managing the lifecycle of semantic > model > > > > > > documents. Discovery, search, and client consumption are all > > > important > > > > > > topics, but I think they're broader than the Ossie semantic model > > > itself. > > > > > > We touched on some of these in the last community sync[1], and > I'm > > > happy to > > > > > > continue that discussion in a separate thread. > > > > > > > > > > > > I also agree that we should be careful not to overstate what this > > > initial > > > > > > API provides. Calling it a beta semantic model registry or > document > > > hosting > > > > > > API seems reasonable. > > > > > > > > > > > > That said, I think those questions are orthogonal to the payload > > > > > > representation discussion. My original topic was simply whether > the > > > REST > > > > > > API should treat the semantic model as a raw document, an opaque > > JSON > > > > > > payload, or model the schema directly. Since the consumption > story > > > is still > > > > > > evolving, I think keeping the REST contract loosely coupled to > the > > > > > > underlying semantic model specification gives us the most > > > flexibility. > > > > > > > > > > > > [1] > > > > > > > > > > > > https://drive.google.com/file/d/1hxYkk2t-BcnFOk8eJG9NYCHfjkOXg3Iz/view?usp=sharing > > > > > > > > > > > > Thanks, > > > > > > > > > > > > Yufei > > > > > > > > > > > > On Mon, Jul 13, 2026 at 6:16 AM Robert Stupp <[email protected]> > > wrote: > > > > > > > > > > > >> Hi, > > > > > >> > > > > > >> I think the payload representation question depends on the > client > > > model we > > > > > >> expect this API to support. > > > > > >> > > > > > >> I support Polaris hosting Apache Ossie semantic-model documents > as > > > a beta > > > > > >> foundation. That seems like a useful first step while Ossie > itself > > > is > > > > > >> still > > > > > >> evolving. > > > > > >> > > > > > >> But I do not think we should describe the current API as > enabling > > AI > > > > > >> tools, > > > > > >> BI tools, or human semantic-model discovery yet. > > > > > >> > > > > > >> The merged API is primarily namespace/name CRUD for a > > semantic-model > > > > > >> document. > > > > > >> That is opaque document storage and exact point-retrieval. > > > > > >> > > > > > >> It does not enable clients to discover the right semantic model > > > from a > > > > > >> query, > > > > > >> nor to find semantic models by table, metric, domain, user, or > > > capability. > > > > > >> > > > > > >> It also does not define a standard consuming API or tool > contract, > > > > > >> search/indexing contract, freshness model, or > > > current/trusted/certified > > > > > >> model > > > > > >> semantics. > > > > > >> > > > > > >> That distinction matters because the REST API is the user-facing > > > contract. > > > > > >> > > > > > >> If the beta API is intended only as opaque document hosting, I > > > think the > > > > > >> spec > > > > > >> and docs should say that clearly, and users/clients should not > > infer > > > > > >> broader > > > > > >> discovery or interoperability semantics from the CRUD API. > > > > > >> > > > > > >> I am not asking to solve the full semantic-layer story > > immediately. > > > > > >> > > > > > >> I am asking that durable implementation work does not get ahead > of > > > the > > > > > >> client-consumption story. > > > > > >> > > > > > >> The client model should drive the persistent data model, not the > > > other way > > > > > >> around. > > > > > >> > > > > > >> Once semantic models are stored as durable Polaris entities, > > choices > > > > > >> around > > > > > >> identity, versioning, validation, indexing, size limits, > > > source-table > > > > > >> references, and freshness become much harder to change. > > > > > >> > > > > > >> For now, I would be comfortable describing this as beta Apache > > Ossie > > > > > >> document > > > > > >> hosting / semantic-model registry work. > > > > > >> > > > > > >> I would not yet be comfortable describing it as enabling AI, BI, > > or > > > human > > > > > >> semantic workflows until the discovery and client-consumption > > story > > > > > >> exists. > > > > > >> > > > > > >> Robert > > > > > >> > > > > > >> > > > > > >> On Fri, Jul 10, 2026 at 3:10 PM Dmitri Bourlatchkov < > > > [email protected]> > > > > > >> wrote: > > > > > >> > > > > > >> > Hi Yufei, > > > > > >> > > > > > > >> > Thanks for starting this thread! > > > > > >> > > > > > > >> > I tend to think it is best to represent Ossie data as direct > > JSON > > > > > >> without > > > > > >> > defining its structure in the Polaris OpenAPI spec (Ossie > > schemas > > > are > > > > > >> > controlled by Ossie, not Polaris). I believe this corresponds > to > > > Option > > > > > >> 2 > > > > > >> > from your email. > > > > > >> > > > > > > >> > Polaris code that implements the new API will then interpret > the > > > Ossie > > > > > >> > parts according to the declared version of the Ossie spec. > > > > > >> > > > > > > >> > With that in mind, the API should clearly state the format > > (Ossie > > > or > > > > > >> OKF) > > > > > >> > and the specification version of the semantic data sub-object > > > > > >> (apologies if > > > > > >> > it has that already, I'm behind on the related PR updates). > > > Obviously > > > > > >> OKF > > > > > >> > will have a different representation in the payload, but this > > > should not > > > > > >> > prevent Ossie from leveraging JSON synergies. > > > > > >> > > > > > > >> > Anand's work on solving a similar problem in the Metrics API > > > [4115] may > > > > > >> be > > > > > >> > reusable here. > > > > > >> > > > > > > >> > [4115] https://github.com/apache/polaris/pull/4115 > > > > > >> > > > > > > >> > Cheers, > > > > > >> > Dmitri. > > > > > >> > > > > > > >> > > > > > > >> > On Wed, Jul 8, 2026 at 2:20 PM Yufei Gu <[email protected] > > > > > wrote: > > > > > >> > > > > > > >> > > Hi folks, > > > > > >> > > > > > > > >> > > Following JB's suggestion, I'd like to start a dedicated > > > discussion on > > > > > >> > the > > > > > >> > > REST API payload representation for semantic models. > > > > > >> > > > > > > > >> > > I think there are three possible approaches: > > > > > >> > > > > > > > >> > > 1. > > > > > >> > > > > > > > >> > > Represent the semantic model as a raw string. > > > > > >> > > 2. > > > > > >> > > > > > > > >> > > Represent the semantic model as an opaque JSON document. > > > > > >> > > 3. > > > > > >> > > > > > > > >> > > Model the semantic model structure directly in the REST > > > > > >> specification. > > > > > >> > > > > > > > >> > > I think it's helpful to separate the REST API from Polaris' > > > internal > > > > > >> > > representation. The REST API is the long-term contract with > > > clients, > > > > > >> > while > > > > > >> > > the internal representation can evolve independently. > > > > > >> > > > > > > > >> > > I'm comfortable with either option 1 or option 2. Both avoid > > > coupling > > > > > >> the > > > > > >> > > REST API to the Ossie schema and allow Polaris to validate > the > > > payload > > > > > >> > > based on the semantic model type and version while > preserving > > > the > > > > > >> > document > > > > > >> > > through write and read operations. > > > > > >> > > > > > > > >> > > My concern is with option 3. Since the Ossie schema is > > > versioned and > > > > > >> > > expected to evolve, modeling the full semantic model > structure > > > > > >> directly > > > > > >> > in > > > > > >> > > the REST specification would tightly couple the Polaris REST > > > API to > > > > > >> Ossie > > > > > >> > > versions. Every Ossie schema evolution could require changes > > to > > > the > > > > > >> REST > > > > > >> > > specification, generated clients, and potentially client > > > applications. > > > > > >> > > > > > > > >> > > Between options 1 and 2, I think there is an additional > > > tradeoff. > > > > > >> > > > > > > > >> > > An opaque JSON document assumes that semantic models are > > always > > > > > >> > represented > > > > > >> > > as JSON. While that works well for Ossie today, Polaris may > > > support > > > > > >> other > > > > > >> > > semantic model formats in the future. For example, OKF[1] is > > > defined > > > > > >> as > > > > > >> > > Markdown rather than JSON. Using a raw string keeps the REST > > API > > > > > >> > > independent of any particular document format, allowing > > Polaris > > > to > > > > > >> > support > > > > > >> > > JSON, Markdown, or other representations without changing > the > > > API > > > > > >> > contract. > > > > > >> > > > > > > > >> > > So my current view is: > > > > > >> > > > > > > > >> > > - > > > > > >> > > > > > > > >> > > Option 1 provides the greatest flexibility and is format > > > agnostic. > > > > > >> > > - > > > > > >> > > > > > > > >> > > Option 2 is a natural choice if we want to optimize > > > specifically > > > > > >> for > > > > > >> > > JSON based semantic models. > > > > > >> > > - > > > > > >> > > > > > > > >> > > Option 3 provides strong typing, but at the cost of > > coupling > > > the > > > > > >> REST > > > > > >> > > API to Ossie schema evolution. > > > > > >> > > > > > > > >> > > I'm happy with either option 1 or option 2, but I'd avoid > > > option 3 for > > > > > >> > the > > > > > >> > > reasons above. > > > > > >> > > > > > > > >> > > Thoughts? > > > > > >> > > > > > > > >> > > 1. > > > > > >> > > > > > > > >> > > > > > > > >> > > > > > > >> > > > > > > https://cloud.google.com/blog/products/data-analytics/how-the-open-knowledge-format-can-improve-data-sharing > > > > > >> > > > > > > > >> > > Thanks, > > > > > >> > > > > > > > >> > > Yufei > > > > > >> > > > > > > > >> > > > > > > >> > > > > > > > > > > > >
