Hi Ryan,
Agree here: the engine-facing consumption (cost attribution, policy
attachment) goes through SupportsLabels, no table needed. The table is for
the other consumer - SQL/people or AI Agent :). Some cases that i see here:
1) Exploration. "which columns are classified as X here" is a query a
person runs, not something an engine surfaces:
SELECT field_name, key, value
FROM prod.db.orders.labels
WHERE scope = 'field' AND key = 'classification';
2) Composition. .labels joins with .files / .partitions / .snapshots in one
query - e.g. attribute bytes to an owner label, i.e. cost attribution as a
report someone runs, not a log an engine emits:
SELECT l.value AS owner, SUM(f.file_size_in_bytes) AS bytes
FROM prod.db.orders.files f, prod.db.orders.labels l
WHERE l.scope = 'object' AND l.key = 'owner'
GROUP BY l.value;
I think the key value to have a metadata table is joinability, you can’t
have it with a programmatic label API.
So there is a place for the SQL consumer, the engine path is a different
usecase and the co-live together.
Thanks,
Andrei
On Fri, Sep 25, 2026 at 10:51 PM Ryan Blue <[email protected]> wrote:
> > Are we OK with catalog-provided metadata tables as a separate category?
>
> I'm okay with providing metadata through a system table like this, as long
> as we think that people will want to access this data that way. Question 3,
> "If no, what should the SQL surface for labels be instead?" makes me think
> that a SQL surface is _assumed_ to be needed.
>
> I don't think it is necessarily the case that we need to expose these for
> SQL users. I thought that we wanted labels to expose additional context to
> engines for things like cost attribution logs or attaching an engine's
> policy to a table. That doesn't require a table-like user surface.
>
> I'm fine adding a metadata table if there's a use for it, but if we don't
> need one then it's simpler not to add and maintain it. And that avoids
> needing to answer questions like this as well.
>
>
>
> On Fri, Sep 25, 2026 at 1:13 PM Andrei Tserakhau via dev <
> [email protected]> wrote:
>
>> Hi all,
>>
>> Labels in the REST spec recently landed [1]. A catalog can now expose
>> object-level and per-field labels on load-table responses. As one of the
>> follow-ups, there is a proposal to add a .labels metadata table backed
>> by catalog data.
>>
>> During discussion of the follow-ups ([3], [4]), Peter raised a good
>> question on [3]: every metadata table today is derived from table metadata.
>> .labels would be different because the data comes from the catalog, may
>> vary by catalog, and may be absent if the catalog has nothing to return.
>>
>> I think this is less about Labels itself and more about a new *kind* of
>> data we would expose: metadata owned by the catalog rather than by storage.
>> Labels would be the first example, but later the same pattern could be used
>> to expose other catalog information as metadata tables.
>>
>> So the broader question is: do we want metadata tables to also expose
>> catalog-provided information?
>>
>> I think labels are a reasonable first case. They are structured, useful
>> to query via SQL, and fit the same access pattern as .snapshots or
>> .partitions. The table can stay read-only, limited to the spec-defined
>> shape, and empty when the catalog returns nothing.
>>
>> The tradeoff is that this breaks the current assumption that metadata
>> tables are deterministic projections of table metadata. It’s not explicitly
>> written anywhere, but that assumption exists today.
>>
>> To summarize the questions:
>>
>> 1.
>>
>> Are we OK with catalog-provided metadata tables as a separate
>> category?
>> 2.
>>
>> If yes, should we mark them somehow so they are clearly different
>> from spec-backed metadata tables?
>> 3.
>>
>> If no, what should the SQL surface for labels be instead?
>>
>> Thanks,
>>
>> Andrei
>>
>> [1] REST spec labels: https://github.com/apache/iceberg/pull/15750
>> [2] SupportsLabels: https://github.com/apache/iceberg/pull/18046
>> [3] Labels metadata table: https://github.com/apache/iceberg/pull/18048
>> [4] Spark DESCRIBE: https://github.com/apache/iceberg/pull/18049
>> [5] Design:
>> https://docs.google.com/document/d/1aj-6JlfBiMYEEVtNuh5WLMOrRQiMCcyYUGbouPM4hXI/edit?tab=t.0#heading=h.2w0kmp1v1gwv
>>
>
