Hi folks, A quick update on the Tag work. *Current status*: - PR1: API contract (https://github.com/apache/polaris/pull/5366): ready for review - *NEW! *PR2: Definition CRUD (https://github.com/apache/polaris/pull/5391): open as Draft, ready for review if PR1 LGTY - PR3: Assignment writes and storage: planned - PR4: Reads, inheritance, and reverse lookup: planned
*More on PR2:* - PR2 makes Tag definitions usable through create, list, load, update, rename, and delete. It intentionally stops before assignments, so their persistence model remains open for the next slice. - The PR is stacked on #5366 and will be rebased once that PR merges. *Asks:* - For #5366, please call out any remaining API contract concerns. For #5391, I would especially appreciate feedback on the slice boundary and the decision to reuse the existing entity persistence model. - The design doc remains here: https://docs.google.com/document/d/1rIJGzcsmGhfrBiRXPac51hr-jeJuuKQQBYjgBdOb9-k/edit?usp=sharing I’ll keep using this thread for new delivery slices, material status changes, and specific community asks. Thanks, -ej On Mon, Aug 24, 2026 at 5:04 PM EJ Wang <[email protected]> wrote: > Hi folks, > > Following up on this thread, I have opened a PR to land the public API > contract for Tags: https://github.com/apache/polaris/pull/5366 > > The PR defines Tag management, assignment and unassignment, direct and > inherited reads, and reverse lookup. V1 covers catalogs, namespaces, > Iceberg and generic tables as whole objects, and top-level Iceberg table > columns. Views, generic-table columns, nested fields, multi-value > assignments, and tag-based authorization are deferred. > > I plan to deliver the capability through four PRs that merge in order: the > API contract in this PR, Tag definition CRUD, assignment writes and > storage, then reads and reverse lookup. A separate follow-up will add > grants on Tag resources to the management APIs. That grant surface is > distinct from using Tags to control access to tagged objects, which remains > outside v1. > > The updated design doc is here: > https://docs.google.com/document/d/1rIJGzcsmGhfrBiRXPac51hr-jeJuuKQQBYjgBdOb9-k/edit?usp=sharing > > The PR is currently Draft while we finish aligning on the public contract. > It is intended to merge as the first delivery slice, not remain as a > design-only artifact. Please call out any remaining scope or contract > concerns. If the list is aligned, I will mark it ready for review. > > Thanks, > -ej > > On Wed, Aug 12, 2026 at 2:05 PM EJ Wang <[email protected]> > wrote: > >> Thanks Dmitri, these comments were very useful. >> >> I went through the three areas you called out and updated the proposal >> accordingly. >> >> On the permission/policy direction, *I agree the Tag model should leave >> room for permissions or policies to consume tags later*, including the >> direction JB proposed. I am keeping that outside the v1 Tag contract, >> though. In v1, tags classify resources; they do not themselves grant or >> deny access. Polaris Policy looks like the closest existing foundation if >> we later want a portable tag-aware policy model, but I think that deserves >> a separate proposal rather than baking policy semantics into the Tag >> storage model now. >> >> I also made the authorizer path more explicit. *A future OPA, Ranger, or >> other authorizer could receive the target's complete effective tags as >> resource attributes*. The authorization path would resolve those tags >> internally, applying target-types, inheritance, closest-wins, grandfathered >> values, and the same coherent-read guarantees as the Tag API. At minimum, >> the portable input can include the tag definition ID, current name, and >> selected value; provenance can be additional context. If Polaris cannot >> resolve the complete effective state, authorization should fail closed >> rather than treat the resource as untagged. >> >> That also makes the persistence expectation on the read path clearer: an >> implementation needs to resolve the target and relevant ancestors, obtain >> the applicable tag definitions and assignments, and produce one coherent >> effective result. *Those observable semantics are the backend contract; >> the physical lookup/indexing strategy is not.* >> >> On the Java interface suggestion, I added Java-shaped records for the >> durable logical model so the definition, target identity, and assignment >> shapes are easier to review from JDBC and NoSQL perspectives. I stopped >> short of proposing operation interfaces in pseudo-code, though. My current >> thinking is that we should first agree on the durable facts and required >> behavior, then design the actual persistence SPI around the needs of the >> implementations. I did not want an illustrative interface in this design to >> accidentally become the persistence contract. >> >> So Part 2 now separates the two intentionally: >> >> *logical data + behavior/conformance requirements are specified; >> transaction, CAS, atomic batch, provider-native operations, and the >> eventual Java SPI remain implementation/design choices.* >> >> Thanks again for the review, and definitely keep the comments coming :) >> >> I've updated the doc, please check it out the latest and the greatest: >> >> https://docs.google.com/document/d/1rIJGzcsmGhfrBiRXPac51hr-jeJuuKQQBYjgBdOb9-k/edit?pli=1&tab=t.0 >> >> -ej >> >> On Fri, Aug 7, 2026 at 3:39 PM Dmitri Bourlatchkov <[email protected]> >> wrote: >> >>> Hi EJ, JB, >>> >>> I left some comments on EJ's doc. I actually have a lot of comments on >>> the >>> REST API design, I only posted some of them to start a discussion >>> without overloading the doc. >>> >>> Overall, I believe EJ's proposal should also allow permission assignments >>> on tags that JB proposed (eventually). We just need to clearly define the >>> persistence expectations for looking up related tags on the read path. >>> >>> We should probably specify whether and how tags are exposed to >>> authorizers >>> (OPA, Ranger). I imagine people will want to use them in external policy >>> engines the moment the feature is available. >>> >>> On the persistence side, I believe it would be nice to define actual java >>> interfaces (perhaps in pseudo code) to allow easier review from the NoSQL >>> persistence perspective (also commented in the doc). >>> >>> Cheers, >>> Dmitri. >>> >>> On Thu, Jul 30, 2026 at 12:39 AM Jean-Baptiste Onofré <[email protected]> >>> wrote: >>> >>> > Hi EJ >>> > >>> > Thanks for starting this discussion. >>> > >>> > For the record, here's my initial proposal about tagging: >>> > https://lists.apache.org/thread/nmqmmjfmocfllb71fcmyp9syc9gyn820 >>> > >>> > At that time, only Dmitri replied :) >>> > So, I would be happy to work with you on this, as I still have the PoC >>> > I created for my initial proposal. >>> > >>> > I will try to join the scheduled meeting (no guarantee). >>> > >>> > Regards >>> > JB >>> > >>> > On Fri, Jul 17, 2026 at 6:53 AM EJ Wang < >>> [email protected]> >>> > wrote: >>> > > >>> > > Hi folks, >>> > > >>> > > I have prepared a Google Doc >>> > > < >>> > >>> https://docs.google.com/document/d/1rIJGzcsmGhfrBiRXPac51hr-jeJuuKQQBYjgBdOb9-k/edit?usp=sharing >>> > > >>> > > for the Polaris tag spec proposal. >>> > > >>> > > The goal is simple: add a native tag model to Polaris so users can >>> > classify >>> > > catalog objects, read those classifications back, and find objects by >>> > tag. >>> > > >>> > > The proposal covers: >>> > > * tag definitions as catalog-scoped Polaris entities >>> > > * tag assignments on catalogs, namespaces, table-like objects, and >>> > columns >>> > > * allowed values on tag definitions >>> > > * direct and inherited tag reads >>> > > * direct by-tag lookup >>> > > * the durable model behind the API >>> > > * how this compares with the existing Polaris Policy API (tag design >>> > > referenced policy heavily, given their pattern similarity) >>> > > >>> > > Please take a look and leave comments in the doc. Let me know WDYT! >>> > > >>> > > I would also like to discuss this in the July 23 community sync. A >>> > separate >>> > > dedicated review meeting will be scheduled separately, likely within >>> the >>> > > next two weeks. >>> > > >>> > > Thanks, >>> > > -ej >>> > >>> >>
