Spec is an interesting topic we did not discuss. Robert, how do you
envision this to be used?
In my mind, if a new table format v3 is launched, there are 2 approaches we
can go with, taking CreateTable as an example:
(1) increment the related operation version, which means that POST
/v2/{prefix}/namespaces/{ns}/tables will be created and allow creating
tables in the v3 version.
(2) update the existing table metadata model to support both v2 and v3
fields, and the server enforces the payload differently based on the
TableMetadata.format-version field. If the server does not support v3, it
can return unsupported at that time.
Either way we go, the table-spec version does not need to be a capability.
(1) seems to be cleaner, but has some overhead in provisioning a new
endpoint compared to (2).
Do you see another way to do this leveraging the table-spec version?
-Jack
On Tue, Jul 2, 2024 at 6:03 AM Eduard Tudenhöfner
<eduard.tudenhoef...@databricks.com.invalid> wrote:
>
> I couldn't make it to the catalog sync meeting yesterday but I watched the
> recording today (thanks for providing that).
>
> The missing piece is how (new, capabilities-aware) clients handle the case
>> when a service does _not_ return the capabilities field (absent). My
>> proposal would be that a client should in this case assume that all
>> _currently_ existing capabilities are supported.
>>
>> - tables: [1]
>> - views: [1]
>> - remote-signing: [1]
>> - multi-table-commit: [1]
>> - register-table: [1]
>> - table-metrics: [1]
>> - table-spec: [1,2]
>> - view-spec: [1,2]
>>
>>
>> The one thing I would like to add here is that the current PR uses the
> *tables* capability (as version 1) as the default when a server doesn't
> return *capabilities *but it might be also ok to include *views *(as
> version 1) because the current client impl has *some* code to deal with
> errors in case endpoints don't exist.
>
> Unless we agree that the currently existing functionality in the REST spec
> is the *default* behavior to be assumed for older server, I'm not sure
> about including *remote-signing / multi-table-commit / register-table /
> table-metrics* as it has been indicated in earlier comments on the PR/ML
> that not every REST server supports these.
>
> That being said, we should discuss whether we want the *default* behavior
> (when an older server doesn't send back *capabilities*) to be
> a) *tables* (version 1) only
> b) the currently existing functionality as defined in the REST spec (as
> version 1)
>
>
> On another note: Including *table-spec / view-spec* seems to be more
> informative in its nature as I don't think a client would act differently
> right now when seeing these.
>
> Thanks
> Eduard
>
>
