Re: [DISCUSS] REST Spec: generic CatalogObjectIdentifier

2026-04-29 Thread Steven Wu
Thanks for the discussion in today's catalog meeting. We have reached a
consensus on Option A (a flat list of levels).

I have created two PRs.
1. Spec: https://github.com/apache/iceberg/pull/16144
2. Java impl: https://github.com/apache/iceberg/pull/16160

I also included the CatalogObjectType schema and enum in the PRs. For the
Java implementation, the new CatalogObjectIdentifier and CatalogObjectType
classes are added to the api module (same as the TableIdentifier class).

Please review this so we can unblock parallel work, such as events,
functions, and resolving relations.

On Tue, Apr 28, 2026 at 6:33 PM Yufei Gu  wrote:

> I think Option A is a clean solution and I’d prefer to go with it.
> However, there are backward compatibility concerns, especially for the
> events endpoint[1], where we are trying to use a unified identifier to
> represent all object types. I’m not sure there’s a clean way to apply
> Option A in the current event spec. One possible approach is to allow both
> representations for existing entities like tables and views, so that Option
> A and Option B can coexist.
>
> 1. https://github.com/apache/iceberg/pull/12584/changes#r3024014158
>
> Yufei
>
>
> On Thu, Apr 23, 2026 at 2:27 PM Steven Wu  wrote:
>
>> Thanks Christian, Ryan, Dan for the comments in the doc. There seems to
>> be no objection to the generic identifier concept. However, there is one
>> central open question regarding the identifier structure: (A) a flat list
>> of level (B) current table identifier tuple pattern of (namespace, name)
>>
>>
>> https://docs.google.com/document/d/1NTQhgNbP2dkIMuXUMA5JdwliVQKCp1TU_ux5J_AaPiw/edit?disco=AAAB33pnNqQ
>>
>>
>> On Sat, Apr 11, 2026 at 12:24 PM Christian Thiel <
>> [email protected]> wrote:
>>
>>> Thanks for the nice write-up, Steven — with all the use cases converging
>>> on this, it's a good moment to settle on a notation.
>>> I've added my thoughts in the doc. Happy to discuss in one of the
>>> upcoming syncs!
>>>
>>> Best,
>>> Christian
>>>
>>> On Wed, 1 Apr 2026 at 14:47, Steven Wu  wrote:
>>>
 Hi,

 I like to discuss a proposal to introduce a generic
 CatalogObjectIdentifier
 
 .

 The Iceberg REST catalog spec currently defines a single
 TableIdentifier schema as a (namespace, name) tuple. This identifier is
 used for both tables and views throughout the spec, which is semantically
 inaccurate for view objects and does not generalize to other catalog object
 types.

 Several concurrent efforts have independently surfaced the need for a
 generic catalog object identifier:

-

Events endpoint (

 PR
#12584
):
The spec introduced a CatalogObjectIdentifier to reference any catalog
object in event payloads, including namespaces, tables, and views.
-

Function endpoints (PR #15180
): The list/load
function spec needs identifiers for function objects. A comment suggests
using a generic CatalogObjectIdentifier instead of a function-specific 
 one.
-

Universal relation load (PR #15830
): The batch load
endpoint for tables and views introduced CatalogObjectType as a
discriminator, which pairs naturally with a generic identifier.


 As Iceberg evolves to support more object types — functions, indices —
 each will need catalog identifiers. Defining a single reusable schema now
 avoids type proliferation and naming confusion.

 Thanks,
 Steven

>>>


Re: [DISCUSS] REST Spec: generic CatalogObjectIdentifier

2026-04-28 Thread Yufei Gu
I think Option A is a clean solution and I’d prefer to go with it. However,
there are backward compatibility concerns, especially for the events
endpoint[1], where we are trying to use a unified identifier to represent
all object types. I’m not sure there’s a clean way to apply Option A in the
current event spec. One possible approach is to allow both representations
for existing entities like tables and views, so that Option A and Option B
can coexist.

1. https://github.com/apache/iceberg/pull/12584/changes#r3024014158

Yufei


On Thu, Apr 23, 2026 at 2:27 PM Steven Wu  wrote:

> Thanks Christian, Ryan, Dan for the comments in the doc. There seems to be
> no objection to the generic identifier concept. However, there is one
> central open question regarding the identifier structure: (A) a flat list
> of level (B) current table identifier tuple pattern of (namespace, name)
>
>
> https://docs.google.com/document/d/1NTQhgNbP2dkIMuXUMA5JdwliVQKCp1TU_ux5J_AaPiw/edit?disco=AAAB33pnNqQ
>
>
> On Sat, Apr 11, 2026 at 12:24 PM Christian Thiel <
> [email protected]> wrote:
>
>> Thanks for the nice write-up, Steven — with all the use cases converging
>> on this, it's a good moment to settle on a notation.
>> I've added my thoughts in the doc. Happy to discuss in one of the
>> upcoming syncs!
>>
>> Best,
>> Christian
>>
>> On Wed, 1 Apr 2026 at 14:47, Steven Wu  wrote:
>>
>>> Hi,
>>>
>>> I like to discuss a proposal to introduce a generic
>>> CatalogObjectIdentifier
>>> 
>>> .
>>>
>>> The Iceberg REST catalog spec currently defines a single TableIdentifier
>>> schema as a (namespace, name) tuple. This identifier is used for both
>>> tables and views throughout the spec, which is semantically inaccurate for
>>> view objects and does not generalize to other catalog object types.
>>>
>>> Several concurrent efforts have independently surfaced the need for a
>>> generic catalog object identifier:
>>>
>>>-
>>>
>>>Events endpoint (
>>>
>>> PR
>>>#12584
>>>):
>>>The spec introduced a CatalogObjectIdentifier to reference any catalog
>>>object in event payloads, including namespaces, tables, and views.
>>>-
>>>
>>>Function endpoints (PR #15180
>>>): The list/load
>>>function spec needs identifiers for function objects. A comment suggests
>>>using a generic CatalogObjectIdentifier instead of a function-specific 
>>> one.
>>>-
>>>
>>>Universal relation load (PR #15830
>>>): The batch load
>>>endpoint for tables and views introduced CatalogObjectType as a
>>>discriminator, which pairs naturally with a generic identifier.
>>>
>>>
>>> As Iceberg evolves to support more object types — functions, indices —
>>> each will need catalog identifiers. Defining a single reusable schema now
>>> avoids type proliferation and naming confusion.
>>>
>>> Thanks,
>>> Steven
>>>
>>


Re: [DISCUSS] REST Spec: generic CatalogObjectIdentifier

2026-04-23 Thread Steven Wu
Thanks Christian, Ryan, Dan for the comments in the doc. There seems to be
no objection to the generic identifier concept. However, there is one
central open question regarding the identifier structure: (A) a flat list
of level (B) current table identifier tuple pattern of (namespace, name)

https://docs.google.com/document/d/1NTQhgNbP2dkIMuXUMA5JdwliVQKCp1TU_ux5J_AaPiw/edit?disco=AAAB33pnNqQ


On Sat, Apr 11, 2026 at 12:24 PM Christian Thiel 
wrote:

> Thanks for the nice write-up, Steven — with all the use cases converging
> on this, it's a good moment to settle on a notation.
> I've added my thoughts in the doc. Happy to discuss in one of the upcoming
> syncs!
>
> Best,
> Christian
>
> On Wed, 1 Apr 2026 at 14:47, Steven Wu  wrote:
>
>> Hi,
>>
>> I like to discuss a proposal to introduce a generic
>> CatalogObjectIdentifier
>> 
>> .
>>
>> The Iceberg REST catalog spec currently defines a single TableIdentifier
>> schema as a (namespace, name) tuple. This identifier is used for both
>> tables and views throughout the spec, which is semantically inaccurate for
>> view objects and does not generalize to other catalog object types.
>>
>> Several concurrent efforts have independently surfaced the need for a
>> generic catalog object identifier:
>>
>>-
>>
>>Events endpoint (
>>
>> PR
>>#12584
>>):
>>The spec introduced a CatalogObjectIdentifier to reference any catalog
>>object in event payloads, including namespaces, tables, and views.
>>-
>>
>>Function endpoints (PR #15180
>>): The list/load
>>function spec needs identifiers for function objects. A comment suggests
>>using a generic CatalogObjectIdentifier instead of a function-specific 
>> one.
>>-
>>
>>Universal relation load (PR #15830
>>): The batch load
>>endpoint for tables and views introduced CatalogObjectType as a
>>discriminator, which pairs naturally with a generic identifier.
>>
>>
>> As Iceberg evolves to support more object types — functions, indices —
>> each will need catalog identifiers. Defining a single reusable schema now
>> avoids type proliferation and naming confusion.
>>
>> Thanks,
>> Steven
>>
>


Re: [DISCUSS] REST Spec: generic CatalogObjectIdentifier

2026-04-11 Thread Christian Thiel
Thanks for the nice write-up, Steven — with all the use cases converging on
this, it's a good moment to settle on a notation.
I've added my thoughts in the doc. Happy to discuss in one of the upcoming
syncs!

Best,
Christian

On Wed, 1 Apr 2026 at 14:47, Steven Wu  wrote:

> Hi,
>
> I like to discuss a proposal to introduce a generic
> CatalogObjectIdentifier
> 
> .
>
> The Iceberg REST catalog spec currently defines a single TableIdentifier
> schema as a (namespace, name) tuple. This identifier is used for both
> tables and views throughout the spec, which is semantically inaccurate for
> view objects and does not generalize to other catalog object types.
>
> Several concurrent efforts have independently surfaced the need for a
> generic catalog object identifier:
>
>-
>
>Events endpoint (
>
> PR
>#12584
>):
>The spec introduced a CatalogObjectIdentifier to reference any catalog
>object in event payloads, including namespaces, tables, and views.
>-
>
>Function endpoints (PR #15180
>): The list/load
>function spec needs identifiers for function objects. A comment suggests
>using a generic CatalogObjectIdentifier instead of a function-specific one.
>-
>
>Universal relation load (PR #15830
>): The batch load
>endpoint for tables and views introduced CatalogObjectType as a
>discriminator, which pairs naturally with a generic identifier.
>
>
> As Iceberg evolves to support more object types — functions, indices —
> each will need catalog identifiers. Defining a single reusable schema now
> avoids type proliferation and naming confusion.
>
> Thanks,
> Steven
>