Hi all,

PR #6028 [1] adds spec/polaris-pagination.yaml, a small set of pagination
definitions for Polaris-specific list operations: a pageToken and a pageSize
 query parameter, and a page token schema. Please give explicit sign-off
here before any operation starts using it, because the rules differ from
the Iceberg REST Catalog API. I'd like to settle that.

The file states four rules:

   1. Any response of an operation that uses these definitions may be
   paginated. A client must always check for a page token in the response and,
   when one is present, send it unchanged as the pageToken query parameter
   to get the next page. The server omits the page token from the last page. A
   client must treat a missing or null page token as the end of the results.
   2. A client does not need to send pageToken or pageSize to start
   paginating. The server may paginate a response to a request that has
   neither.
   3. The server decides the page size. A page may hold fewer results than
   pageSize.
   4. A server that does not paginate returns all results in one response,
   with no page token.

Scope: the rules apply only to operations that reference this file. Nothing
on main references it today. Operations that use the Iceberg REST Catalog
pagination definitions keep the Iceberg rules, including the Polaris
catalog listings that reuse them (policies, generic tables, semantic models)
.

How this relates to the September thread [2] and #5553 [3]: that thread
found that a configured LIST_PAGINATION_MAX_PAGE_SIZE truncated requests
with no pageToken, while the Iceberg spec promises such a request the
complete result. We agreed to reject those requests rather than truncate
them, and #5553 did that. The concern there was clients that rely on
the Iceberg
promise and never look for a token. These rules don't touch those
operations. For a new operation that uses this file, no such clients exist: the
contract tells every client from day one to follow the token, so a first
page is never mistaken for the whole result. It also gives the server the
bound on response size that motivated the max page size in the first place.

There is precedent: the Shares API spec on main (
spec/polaris-shares-api.yaml) already says that when pageSize is absent the
service's maximum is used, and that nextPageToken is present when more
results exist. This file writes that behavior down once for reuse.

First intended users: the two list operations in the storage configuration
spec, PR #5943 [4] (listStorageConfigurations and listStorageMappings, in
the management API).

One related question: getApplicablePolicies declares the Iceberg pagination
parameters but doesn't paginate yet (#5311 [5]). Should it keep the Iceberg
rules when that is fixed, like listPolicies next to it, or move to these
definitions? Moving it would change the contract of an existing operation, so
I'd treat it as a separate decision either way.

Please reply with +1 or any objections. If there are none in about 72 hours,
 I'll take that as agreement and ask for #6028 to be merged.

[1] https://github.com/apache/polaris/pull/6028 [2]
https://lists.apache.org/thread/k81ptyktdbdf8gynncgk3o04mqt85zyk [3]
https://github.com/apache/polaris/pull/5553 [4]
https://github.com/apache/polaris/pull/5943 [5]
https://github.com/apache/polaris/issues/5311

Thanks,
Rishi

Reply via email to