Hey Dmitri, This actually matches my interpretation of the IRC spec. It says <https://github.com/apache/iceberg/blob/c7df5200df462764ba0b3e81484243532c941caf/open-api/rest-catalog-open-api.yaml#L2024> :
> Servers that support pagination should identify the `pageToken` parameter and return a `next-page-token` in the response if there are more results available. My interpretation of the above is that next-page-token uniquely describes whether or not more results are available. Not the size of the response. In fact, the spec defines page-size as "an *upper bound* of the number of results that a client will receive". Tangentially, I would prefer if the spec described this in looser terms, such as a "requested upper bound". What the spec does *not* say is that a client can safely assume there are no more results if it receives less than page-size elements. I think that you are probably right that a client exists which makes an incorrect assumption here though :) --EM On Tue, Oct 14, 2025 at 4:37 PM Dmitri Bourlatchkov <[email protected]> wrote: > Hi Andrew and everyone, > > Adding pagination to the Management API would be very helpful. > > As to reusing the pagination parameter sepantics of the Iceberg REST > spec... I'm not so sure. > > I do believe that servers should have ultimate control over page sizes. So > any client-side "size" parameters should be suggestions or hints at most. > > As a continuation of that approach, the server should always be able to > produce a partial response (with a next page token) even if the client did > not provide any explicit pagination parameters. > > That said, given that existing clients may expect to get "full" results > from the Management API when they do not use pagination parameters, I think > it should be fine to enable that behaviour with a feature flag. > > WDYT? > > Thanks, > Dmitri. > > On Fri, Oct 10, 2025 at 8:08 PM Andrew Guterman < > [email protected]> > wrote: > > > Hey folks, > > > > I wanted to gauge sentiment on adding pagination to non-IRC APIs, such as > > the management APIs, as the number of management entities (catalogs, > > principals, etc) can grow large and become un-listable all at once. > > > > I'm not sure if this has been discussed previously but I couldn't find a > > thread nor PRs related to it. > > > > My proposal is to not reinvent the wheel and just re-use the spec and > > implementation of the IRC APIs, where requests contain a "page-token" and > > "page-size" param, and responses return a "next-page-token". > > > > Let me know what you think. > > > > Best, > > Andrew > > >
