Hi all! Coming back from vacation, I want to re-iterate and close the loop over the catalog community sync [1], where Labels in IRC [2] were the main topic of discussion.
Thanks to everyone who joined and participated, this was a good discussion. I want to bring focus back to the proposal and SPEC change [3] that's actually on the table: a single optional `labels` field on the load response, flat k/v the catalog already owns, read-only, backward compatible, with no interpretation or enforcement defined by the spec. This is a deliberate choice to keep it minimal. Iceberg always leans toward standardizing the least common thing to enable value, while keeping the rest as innovation space for others (in this case, catalogs). This proposal leans into openness and exposes catalog metadata that drives many capabilities like policy enforcement, lifecycle management, data governance, attribution, etc. The mechanics of these capabilities are well outside of Iceberg's main function, but critical for systems managing Iceberg data. There's a lot of confusion between implementations and what the spec enables, but where the room converged was: - Labels are catalog-owned metadata exposed at load time. The catalog presents what it knows about an object when the object is loaded, nothing more. Labels are similar to table properties, but expose data owned by the catalog rather than by table metadata. - It is not a coordination protocol. Renames / x-system sync / identity and lineage / semantics of a label across catalogs are questions that live outside this protocol. The protocol's goal is only to present the current view at the current moment in time. - Labels are not competing with Read Restrictions, they are complementary tools. Read Restrictions is server-side enforcement (the catalog evaluates policy), while Labels are client-side (the catalog exposes context, but consumers/engines decide what to do with it). Read Restrictions are especially good when you are fully a client of another catalog, while Labels can be useful when you consume another catalog and apply your own policies without a trust relationship. We need both of these approaches in IRC. - Governance is one use case, not the whole point. Most of the discussion was centered around ABAC/TBAC, but the value is broader: ownership and point of contact, cost attribution, retention / TTL / GDPR cleanup, attribute-based maintenance, discovery and AI context. None of these need enforcement, identity, or a tag-entity model to be useful. - The structured entity for tags (stable identity, rename semantics, inheritance, x-catalog lineage) is an important follow-up, but the goal here is to unblock value while we discuss the full entity model. This is a similar pattern to what we already have in IRC: a dedicated entity API is richer than exposure through another API. I will keep driving the structured tag entity further. I would like to move to a VOTE on the read path. If there are concrete concerns about the spec as proposed, the field, the shape, or compatibility, let's discuss them in this thread. Use-case-specific coordination (x-system tag sync, rename semantics, identity binding) are the main topics for the structured follow-up and, per the discussion, are out of scope for the read path. [1] Catalog Sync Recording: https://www.youtube.com/watch?v=lCTV-Rhiwyc [2] Spec Proposal doc: https://docs.google.com/document/d/1aj-6JlfBiMYEEVtNuh5WLMOrRQiMCcyYUGbouPM4hXI/edit?tab=t.0#heading=h.2w0kmp1v1gwv [3] Spec PR: https://github.com/apache/iceberg/pull/15750 Best, Andrei
