Thanks Prashant! I think we're actually pretty close on the logging / AI context point.
Agree that statement "meant to be surfaced, logged, joined against" was a bit too broad. I don't think the spec should say that labels are always safe to log, but I also don't think we should prohibit logging them or using them as AI context. This seems ultimately like a client-side decision. Labels carry a similar kind of context as table / column names and descriptions: depending on the catalog and the particular metadata, sometimes it is perfectly fine to expose or log them, and sometimes it isn't. "config" is somewhat different here because it carries FileIO/client configuration and can contain credentials, so we have a much stronger reason to treat it as sensitive by default. I think we tighter this by adding an explicit expectation in the label description: «Clients should treat label contents as opaque unless they understand the semantics of a particular label.» On the structured metadata question / multiple values: I think this is exactly where I would prefer not to make the current field more specific. The current definition already deliberately says that labels are catalog-provided metadata and doesn't prescribe how they are produced, stored, persisted, or versioned. I don't mind expanding this where there are concrete ambiguities, but I'd like to preserve that degree of freedom. And to clarify what I meant by "designing in a vacuum": I don't mean that tags themselves are novel or that there aren't established tag models we can learn from. The vacuum I'm concerned about is how this abstraction will actually be used through Iceberg REST Catalog. Tags and catalog metadata are also a major area of differentiation and innovation between catalogs. I think we need to be extra careful when standardizing a structured entity here: if we define the abstraction too tightly before we understand the usage patterns, we may unintentionally constrain that innovation space or force catalog-specific models into an abstraction that doesn't fit them well. That's the rationale for starting with the minimal common denominator on the read path. Let catalogs expose the metadata they already have without requiring Iceberg to standardize its full semantics. Once we see patterns of usage across implementations, we'll be in a much better position to decide which parts of the tag model are actually common and should be standardized, and which parts should deliberately remain vendor-specific. For example, whether "point of contact" should be one value, multiple values, have identity of its own, inheritance semantics, etc. is exactly the kind of thing I'd rather learn from real usage before making it part of the Iceberg contract. A flat K/V representation doesn't prevent catalogs from having richer models behind it, and a structured entity can be added later once we have a clearer interoperability boundary to standardize. So I'd propose adding the client guidance above, keeping the read path intentionally minimal, and continuing the structured entity discussion as the next step. And thanks for clarifying that this isn't a VOTE blocker <3. Best, Andrei вт, 1 сент. 2026 г., 23:03 Prashant Singh <[email protected]>: > Thanks for the response, Ryan! > > > This is needlessly dangerous. Config is used for a different purpose and > we do not want to mix user-controlled properties into catalog-controlled > configuration! > > I understand that *config* is currently used for catalog-controlled > configuration. However, looking at the config spec [1], it is essentially a > bag of K/V pairs with some special defined keys. The spec doesn't > explicitly state who configures them or exactly what the content must be. > > As I understand the current labels proposal, they are also just bags of > K/V pairs that the catalog tracks, with no attached semantic meaning or > requirement for the client to act on them. If labels are strictly > informational K/V pairs, the client can always filter out keys they don't > want to show from config. I raised this question because I couldn't find > a prior conclusion on this specific point. We might be approaching this > from different angles regarding the separation of concerns, but ultimately, > both approaches result in writing a bag of K/V pairs in the spec that are > not part of table metadata. > > > For instance, we voted to add the function list and load endpoints > first. What is the rationale for waiting on other endpoints in this case, > when there are clear use cases for exchanging labels, but the need to > modify them is considerably less clear? > > To clarify my earlier point, I am not asking to hold the spec until > update/create endpoints are added. My suggestion is to define the > structured entity first. We did this for the load endpoint for function, > even before the REST spec, we discussed how function metadata should handle > overload conflicts, renames, and whether it should be part of a namespace > or a catalog-level API.My ask is simply that we conclude on what the tags > metadata will look like before closing this out. For example, we should > consider: Can there be multiple values for labels (e.g., multiple points of > contact)? If so, do we use a List<String> or just comma-separated strings > ? > > > Thanks for the response, Andrei! > > > while labels are meant to be surfaced, logged, joined against > > Since AI context is one of the proposed use cases for labels, ai-summary > field might say, "The table contains data from US / China / Mexico." > Logging this exposes which country's data belongs to the table. Similarly, > logging which columns are sensitive exposes an attack surface. Because of > this, I don't think catalog metadata can always be safely logged; we should > leave it to the client's better judgment to decide what is appropriate to > log. > > > designing the entity model before the simple surface exists and has > adoption means designing it in a vacuum > > Because TAGs are a well-established industry standard, I respectfully > disagree that we are designing in a vacuum. > > *Last question:* > > > Are these things you want on record, or are you opposing the addition? > > I wanted to have a detailed community discussion on how people plan to use > this and to look at reference implementations for catalog-to-catalog sync. > I am not convinced the current K/V approach is sufficient for governance > use cases, though I agree it works fine for others. > > If the question is whether I will veto this proposal during a VOTE, I will > not. I respect the effort you have all put into this. Although my > preference remains to define clear consumption guidelines before > introducing another K/V protocol, if the general consensus is to proceed, I > am happy to let the community vote decide the path forward > > [1] > https://github.com/apache/iceberg/blob/main/open-api/rest-catalog-open-api.yaml#L3861 > > Best, > > Prashant Singh > > On Mon, Aug 31, 2026 at 4:06 PM Ryan Blue <[email protected]> wrote: > >> > Why do we even need labels when *config* exists then? >> >> This is needlessly dangerous. Config is used for a different purpose and >> we do not want to mix user-controlled properties into catalog-controlled >> configuration! >> >> > Why not do the structured entity first? >> >> Catalogs already maintain this metadata and there are useful reasons to >> expose it without shoving it into table properties (or config). I like the >> idea of standardizing a way to modify this. But, I see no compelling reason >> to hold back a read-only implementation that can immediately address the >> cases where these are sent through table properties. >> >> We've used similar approaches for other new features. For instance, we >> voted to add the function list and load endpoints first. What is the >> rationale for waiting on other endpoints in this case, when there are clear >> use cases for exchanging labels, but the need to modify them is >> considerably less clear? >> >> Ryan >> >> On Mon, Aug 31, 2026 at 10:01 AM Andrei Tserakhau via dev < >> [email protected]> wrote: >> >>> Hi Prashant, >>> >>> Thanks for the questions. Let me take them in order. >>> >>> On governance: I hear the concerns, but governance isn't the scope of >>> this proposal, and the proposal doesn't require any particular >>> implementation. Labels are a mechanism to expose catalog metadata; what a >>> consumer does with it, governance included, is out of scope here. If >>> someone wants to propose a formal >>> governance spec built on labels for Iceberg, that's the right place to >>> work out attachment, inheritance, and cross-system reconciliation. This >>> proposal mandates none of it. >>> >>> On config: IRC already separates k/v by scope rather than using one bag, >>> e.g. table properties, namespace properties, the /v1/config endpoint, >>> table-level config on load, storage credentials, and remote-signing >>> settings. Each is its own carrier because it has its own scope, ownership, >>> and lifecycle. Labels are catalog-object metadata, a distinct scope, so >>> they follow the same pattern: their own field. >>> >>> Config specifically is client configuration, not metadata. Most Iceberg >>> clients merge the load-table config into the properties used to build the >>> session and FileIO, they don't read it as metadata: Java [1][2], Rust [3], >>> Go [4]. It also carries secrets (session tokens, s3.secret-access-key) that >>> must be hidden from the user, while labels are meant to be surfaced, >>> logged, joined against. Opposite exposure requirements, you don't want >>> classification metadata and credentials in the same map with the same >>> handling. And catalogs already shipping non-standard keys through config is >>> the fragmentation this fixes. >>> >>> On "why not structured first": there are many different tag/label >>> implementations across catalogs, each with its own model. The point of the >>> read path is to keep that complexity out of the REST spec while still >>> letting >>> catalogs differentiate above it. The read path is the substrate the >>> structured entity projects onto, so it comes first by construction; >>> designing the entity model before the simple surface exists and has >>> adoption means designing it in a vacuum, and it's a much larger >>> cross-vendor effort. Same sequencing IRC already uses, a read surface >>> before the write path: functions expose only list and load today, no >>> create/update/drop [5]. The structured entity lands additively later, not >>> as a change to the read path. >>> >>> Overall, these read as the same concerns raised in the sync discussions. >>> Are these things you want on record, or are you opposing the addition? If >>> it's not a blocker, I'd like to move forward with a vote. >>> >>> Regards, >>> Andrei >>> >>> [1] RESTSessionCatalog: config fed into the table auth session >>> >>> https://github.com/apache/iceberg/blob/7f7c0023b1faff6162cc864d3819393ce024d66a/core/src/main/java/org/apache/iceberg/rest/RESTSessionCatalog.java#L531-L534 >>> [2] RESTCatalogProperties: config used for table-level client overrides >>> (scan-planning-mode) >>> >>> https://github.com/apache/iceberg/blob/7f7c0023b1faff6162cc864d3819393ce024d66a/core/src/main/java/org/apache/iceberg/rest/RESTCatalogProperties.java#L50 >>> [3] iceberg-rust: load-table config chained with client props into >>> load_file_io >>> >>> https://github.com/apache/iceberg-rust/blob/d6d06f3c4a60e5f3a6d3d789a43513a919d9eacb/crates/catalog/rest/src/catalog.rs#L1181-L1187 >>> [4] iceberg-go: response Config merged into props, then used to build >>> FileIO >>> >>> https://github.com/apache/iceberg-go/blob/e9a0d3bfd5499b1f089d156bfc6bb4b3f9c1b311/catalog/rest/rest.go#L1272 >>> [5] IRC functions: read-only surface (list + load), no create/update/drop >>> >>> https://github.com/apache/iceberg/blob/7f7c0023b1faff6162cc864d3819393ce024d66a/open-api/rest-catalog-open-api.yaml#L625-L664 >>> >>> On Mon, Aug 31, 2026 at 4:40 AM Prashant Singh <[email protected]> >>> wrote: >>> >>>> Hi Andrei, >>>> >>>> Thanks for bringing this back to the dev list and the detailed writeup. >>>> I understand the motivating case is catalog-to-catalog / federation. >>>> >>>> My concerns are mostly when labels are used for governance. I have >>>> raised these in the spec PR already linking them as well : >>>> >>>> - Attachment / inheritance semantics: >>>> https://github.com/apache/iceberg/pull/15750#discussion_r3647710626 >>>> - Nested-field label conflicts: >>>> https://github.com/apache/iceberg/pull/15750#discussion_r3648060761 >>>> - Stable id for cross-catalog sync: >>>> https://github.com/apache/iceberg/pull/15750#discussion_r3648112804 >>>> In a nutshell, these boil down to: >>>> >>>> 1. Why do we even need labels when *config* exists then ? >>>> >>>> *LoadTableResult#config* is already a string map that catalogs ship >>>> non-standard keys through. The use cases listed - >>>> ownership and point of contact, cost attribution, retention / TTL / >>>> GDPR cleanup, discovery - can all be carried there today. >>>> >>>> 2. Why not do the structured entity first? >>>> >>>> "Flat now, structured later" why not do structured entities first then >>>> ? Federation / Catalog Sync is where I expect it to matter >>>> most: I cannot tell inherited from direct, and a renamed label leaves >>>> me nothing stable to bind policy to. >>>> >>>> I could be convinced otherwise, but I would rather we work these out >>>> here than defer them to the structured follow-up, since the read path is >>>> what >>>> implementers will build against. Looking forward to hearing from >>>> other community members as well on their take on this scenario. >>>> >>>> Thanks, >>>> Prashant >>>> >>>> On Wed, Aug 26, 2026 at 2:04 PM Andrei Tserakhau via dev < >>>> [email protected]> wrote: >>>> >>>>> Hi all! >>>>> >>>>> Coming back from vacation, I want to re-iterate and close the loop >>>>> over the >>>>> catalog community sync [1], where Labels in IRC [2] were the main >>>>> topic of >>>>> discussion. >>>>> >>>>> Thanks to everyone who joined and participated, this was a good >>>>> discussion. >>>>> >>>>> I want to bring focus back to the proposal and SPEC change [3] that's >>>>> actually on the table: a single optional `labels` field on the load >>>>> response, flat k/v the catalog already owns, read-only, backward >>>>> compatible, >>>>> with no interpretation or enforcement defined by the spec. >>>>> >>>>> This is a deliberate choice to keep it minimal. Iceberg always leans >>>>> toward >>>>> standardizing the least common thing to enable value, while keeping >>>>> the rest >>>>> as innovation space for others (in this case, catalogs). >>>>> >>>>> This proposal leans into openness and exposes catalog metadata that >>>>> drives >>>>> many capabilities like policy enforcement, lifecycle management, data >>>>> governance, attribution, etc. The mechanics of these capabilities are >>>>> well >>>>> outside of Iceberg's main function, but critical for systems managing >>>>> Iceberg data. >>>>> >>>>> There's a lot of confusion between implementations and what the spec >>>>> enables, but where the room converged was: >>>>> >>>>> - Labels are catalog-owned metadata exposed at load time. The catalog >>>>> presents what it knows about an object when the object is loaded, >>>>> nothing >>>>> more. Labels are similar to table properties, but expose data owned >>>>> by the >>>>> catalog rather than by table metadata. >>>>> >>>>> - It is not a coordination protocol. Renames / x-system sync / >>>>> identity and >>>>> lineage / semantics of a label across catalogs are questions that >>>>> live >>>>> outside this protocol. The protocol's goal is only to present the >>>>> current >>>>> view at the current moment in time. >>>>> >>>>> - Labels are not competing with Read Restrictions, they are >>>>> complementary >>>>> tools. Read Restrictions is server-side enforcement (the catalog >>>>> evaluates >>>>> policy), while Labels are client-side (the catalog exposes context, >>>>> but >>>>> consumers/engines decide what to do with it). Read Restrictions are >>>>> especially good when you are fully a client of another catalog, while >>>>> Labels can be useful when you consume another catalog and apply your >>>>> own >>>>> policies without a trust relationship. We need both of these >>>>> approaches in >>>>> IRC. >>>>> >>>>> - Governance is one use case, not the whole point. Most of the >>>>> discussion >>>>> was centered around ABAC/TBAC, but the value is broader: ownership >>>>> and >>>>> point of contact, cost attribution, retention / TTL / GDPR cleanup, >>>>> attribute-based maintenance, discovery and AI context. None of these >>>>> need >>>>> enforcement, identity, or a tag-entity model to be useful. >>>>> >>>>> - The structured entity for tags (stable identity, rename semantics, >>>>> inheritance, x-catalog lineage) is an important follow-up, but the >>>>> goal >>>>> here is to unblock value while we discuss the full entity model. >>>>> This is a >>>>> similar pattern to what we already have in IRC: a dedicated entity >>>>> API is >>>>> richer than exposure through another API. I will keep driving the >>>>> structured tag entity further. >>>>> >>>>> I would like to move to a VOTE on the read path. >>>>> >>>>> If there are concrete concerns about the spec as proposed, the field, >>>>> the >>>>> shape, or compatibility, let's discuss them in this thread. >>>>> >>>>> Use-case-specific coordination (x-system tag sync, rename semantics, >>>>> identity binding) are the main topics for the structured follow-up >>>>> and, per >>>>> the discussion, are out of scope for the read path. >>>>> >>>>> [1] Catalog Sync Recording: >>>>> https://www.youtube.com/watch?v=lCTV-Rhiwyc >>>>> [2] Spec Proposal doc: >>>>> https://docs.google.com/document/d/1aj-6JlfBiMYEEVtNuh5WLMOrRQiMCcyYUGbouPM4hXI/edit?tab=t.0#heading=h.2w0kmp1v1gwv >>>>> [3] Spec PR: https://github.com/apache/iceberg/pull/15750 >>>>> >>>>> Best, >>>>> Andrei >>>>> >>>>
