Hi JB, > I would state in the spec that an empty pageToken means the > same as no pageToken.
Good point! +1 > LIST_PAGINATION_MAX_PAGE_SIZE is unlimited by default [...] Also a very good point! In practice I think LIST_PAGINATION_MAX_PAGE_SIZE applies to the IRC API and acts as a safety valve to be configured by Polaris users. I think new APIs that use Rishi's (and compatible) spec wording probably need a _different_ config for limiting page sizes, and it has to be ON by default. I think this can be resolved in implementation PRs. The spec seems to be unaffected by this. I believe Anand already took this approach in [4115]. > Regarding getApplicablePolicies [...] Let's have a separate discussion for that. I believe we cannot break this API contract now that we have released it without any "beta" or "experimental" annotations... that is per our evolutions guidelines [1] Cheers, Dmitri. [1] https://polaris.apache.org/releases/1.9.0/evolution/ [4115] https://github.com/apache/polaris/pull/4115 On Sat, Nov 14, 2026 at 2:33 AM Jean-Baptiste Onofré <[email protected]> wrote: > Hi Rishi, > > I'm +1 on the approach. I don't think it conflicts with what we agreed in > September. Fail-closed protects clients written against the Iceberg promise > that a request with no pageToken gets the complete result. A new Polaris > native operation makes no such promise, so "always follow the token" is the > cleaner contract, and it bounds response size from day one. One shared > definition is also better than a copy per operation. > > We had a similar discussion about pagination in the Directory Spec PR. > > I have a few comments (nothing blocking PR #6028): > - As Prithvi noted in the September thread, Java Iceberg clients send an > empty pageToken to start paginating. These operations always allow > pagination, so I would state in the spec that an empty pageToken means the > same as no pageToken. Then a client with Iceberg habits doesn't get a 400. > - The storage configuration listings have filters (targetExists, > referenced). It would be good to say that a client must send the same > parameters on every page, apart from pageToken and pageSize. The spec could > also say the server may reject a token that comes with different > parameters. > - LIST_PAGINATION_MAX_PAGE_SIZE is unlimited by default. If the new > operations paginate only when an operator sets a maximum, a client that > ignores the token works in every default deployment. Then it silently loses > results the day someone sets a limit :) That's the September problem again, > just moved :) I would suggest that these operations use a reasonable > default page size even when no maximum is set, so clients use the token > from the start. This belongs to the impl PRs after #5943, not in #6028. > - #5533 showed that the generic tables CLI followed next-page-token only > when --page-size was set. When the storage configuration operations land, > the Python CLI should always follow the token. > > Regarding getApplicablePolicies, I would keep the Iceberg rules and fix > #5311 under the current contract, the same as listPolicies. Putting one > policy operation under different rules would be confusing, and it would > change the contract for existing clients. If we ever want to move all the > Polaris catalog listings (policies, generic tables, semantic models, ...) > to these definitions, that should be its own discussion covering all of > them. > > The Shares and Directory APIs already behave this way, so switching it to > reference the new file could be a follow-up with no behavior change. > > Regards > JB > > On Sat, Nov 14, 2026 at 2:18 AM Srinivas Rishindra <[email protected] > > > wrote: > > > Hi all, > > > > PR #6028 [1] adds spec/polaris-pagination.yaml, a small set of pagination > > definitions for Polaris-specific list operations: a pageToken and a > > pageSize > > query parameter, and a page token schema. Please give explicit sign-off > > here before any operation starts using it, because the rules differ from > > the Iceberg REST Catalog API. I'd like to settle that. > > > > The file states four rules: > > > > 1. Any response of an operation that uses these definitions may be > > paginated. A client must always check for a page token in the response > > and, > > when one is present, send it unchanged as the pageToken query > parameter > > to get the next page. The server omits the page token from the last > > page. A > > client must treat a missing or null page token as the end of the > > results. > > 2. A client does not need to send pageToken or pageSize to start > > paginating. The server may paginate a response to a request that has > > neither. > > 3. The server decides the page size. A page may hold fewer results > than > > pageSize. > > 4. A server that does not paginate returns all results in one > response, > > with no page token. > > > > Scope: the rules apply only to operations that reference this file. > Nothing > > on main references it today. Operations that use the Iceberg REST Catalog > > pagination definitions keep the Iceberg rules, including the Polaris > > catalog listings that reuse them (policies, generic tables, semantic > > models) > > . > > > > How this relates to the September thread [2] and #5553 [3]: that thread > > found that a configured LIST_PAGINATION_MAX_PAGE_SIZE truncated requests > > with no pageToken, while the Iceberg spec promises such a request the > > complete result. We agreed to reject those requests rather than truncate > > them, and #5553 did that. The concern there was clients that rely on > > the Iceberg > > promise and never look for a token. These rules don't touch those > > operations. For a new operation that uses this file, no such clients > > exist: the > > contract tells every client from day one to follow the token, so a first > > page is never mistaken for the whole result. It also gives the server the > > bound on response size that motivated the max page size in the first > place. > > > > There is precedent: the Shares API spec on main ( > > spec/polaris-shares-api.yaml) already says that when pageSize is absent > the > > service's maximum is used, and that nextPageToken is present when more > > results exist. This file writes that behavior down once for reuse. > > > > First intended users: the two list operations in the storage > configuration > > spec, PR #5943 [4] (listStorageConfigurations and listStorageMappings, in > > the management API). > > > > One related question: getApplicablePolicies declares the Iceberg > pagination > > parameters but doesn't paginate yet (#5311 [5]). Should it keep the > Iceberg > > rules when that is fixed, like listPolicies next to it, or move to these > > definitions? Moving it would change the contract of an existing > operation, > > so > > I'd treat it as a separate decision either way. > > > > Please reply with +1 or any objections. If there are none in about 72 > > hours, > > I'll take that as agreement and ask for #6028 to be merged. > > > > [1] https://github.com/apache/polaris/pull/6028 [2] > > https://lists.apache.org/thread/k81ptyktdbdf8gynncgk3o04mqt85zyk [3] > > https://github.com/apache/polaris/pull/5553 [4] > > https://github.com/apache/polaris/pull/5943 [5] > > https://github.com/apache/polaris/issues/5311 > > > > Thanks, > > Rishi > > >
