Hello Dmitri, I do agree that pagination is probably not a hard requirement for the typical one by one catalog management use case. My main concern is around avoiding a very large API response. Taking one example, for a shared tenants environment where we are creating two principals (e.g. RW and RO) per tenants, as the number of tenant increases (e.g. lets say 20k tenants), a single list principal would return 40k entries in a single response. I think streaming the results to STDOUT may be a diff question whether the CLI should use pagination or not internally for listing requests. We could use pagination to avoid a single large API response without changing the existing CLI output behavior.
Thanks, Yong On 2026/08/08 00:13:47 Dmitri Bourlatchkov wrote: > Hi Yong, > > It would be nice to support pagination in the CLI from the completeness POV. > > However, I think this is largely a concern of practical CLI users. If the > amount of data people deal with is manageable in a single request, > pagination is probably not a hard requirement. > > I personally use the CLI only for one-by-one catalog management where the > amount of data is fairly small (create/update). > > If we add pagination, I think we will have to support proper unix-style > STDOUT streams to allow piping large amounts of data to other CLI tools. > This may have an impact on the modern colourful CLI output, but TBH, I'm > not very familiar with how it works ATM :) > > Cheers, > Dmitri. > > On Fri, Aug 7, 2026 at 2:14 AM Yong Zheng <[email protected]> wrote: > > > Hello, > > > > We have pageSize where "For servers that support pagination, this signals > > an upper bound of the number of results that a client will receive. For > > servers that do not support pagination, clients may receive results larger > > than the indicated pageSize", however, we are not take advantage of this. > > Currently all of the API used in CLI are straight full fetch which can be > > very expensive for large scale systems where the number of polaris entities > > are high (e.g. use unique principal for single tenant ingestions where the > > number of unique tenant is high). Should we consider adopting this setting > > in CLI? > > > > Thanks, > > Yong Zheng > > >
