+1, I think supporting pagination internally in the CLI is reasonable. I agree with Yong that pagination and streaming output are separate concerns. We can use pagination for listing requests to avoid a potentially very large API response, while preserving the existing CLI output behavior. This also gives us better scalability for large deployments without introducing a user visible behavior change.
Yufei On Sat, Aug 8, 2026 at 12:55 PM Yong Zheng <[email protected]> wrote: > Hello Dmitri, > > I do agree that pagination is probably not a hard requirement for the > typical one by one catalog management use case. My main concern is around > avoiding a very large API response. Taking one example, for a shared > tenants environment where we are creating two principals (e.g. RW and RO) > per tenants, as the number of tenant increases (e.g. lets say 20k tenants), > a single list principal would return 40k entries in a single response. I > think streaming the results to STDOUT may be a diff question whether the > CLI should use pagination or not internally for listing requests. We could > use pagination to avoid a single large API response without changing the > existing CLI output behavior. > > Thanks, > Yong > > On 2026/08/08 00:13:47 Dmitri Bourlatchkov wrote: > > Hi Yong, > > > > It would be nice to support pagination in the CLI from the completeness > POV. > > > > However, I think this is largely a concern of practical CLI users. If the > > amount of data people deal with is manageable in a single request, > > pagination is probably not a hard requirement. > > > > I personally use the CLI only for one-by-one catalog management where the > > amount of data is fairly small (create/update). > > > > If we add pagination, I think we will have to support proper unix-style > > STDOUT streams to allow piping large amounts of data to other CLI tools. > > This may have an impact on the modern colourful CLI output, but TBH, I'm > > not very familiar with how it works ATM :) > > > > Cheers, > > Dmitri. > > > > On Fri, Aug 7, 2026 at 2:14 AM Yong Zheng <[email protected]> wrote: > > > > > Hello, > > > > > > We have pageSize where "For servers that support pagination, this > signals > > > an upper bound of the number of results that a client will receive. For > > > servers that do not support pagination, clients may receive results > larger > > > than the indicated pageSize", however, we are not take advantage of > this. > > > Currently all of the API used in CLI are straight full fetch which can > be > > > very expensive for large scale systems where the number of polaris > entities > > > are high (e.g. use unique principal for single tenant ingestions where > the > > > number of unique tenant is high). Should we consider adopting this > setting > > > in CLI? > > > > > > Thanks, > > > Yong Zheng > > > > > >
