Thanks for the review and feedback. Here is the PR for adding support of pagination in generic table API: https://github.com/apache/polaris/pull/5533
On 2026/09/14 16:23:03 Yufei Gu wrote: > Thanks for raising this, Yong! I agree with Prithvi and JB. A new overload > method looks good to me. > > Yufei > > > On Mon, Sep 14, 2026 at 8:47 AM Dmitri Bourlatchkov <[email protected]> > wrote: > > > HI Prithvi, Yong, > > > > I believe breaking SPI changes are quite alright. They are in-line with our > > evolution policy [1]. > > > > My personal view on SPI backward-compatibilioty is that altering method > > signatures is a constructive signal to SPI implementors (outside of Polaris > > repo) that they need to consider that Polaris Core now has _different_ > > expectations of the SPI. > > > > Trying to make the implementor's ride "smooth" (e.g. by offering default > > methods that do not break compilation) is actually likely to produce > > hidden issues, because Polaris Core will expect _new_ behaviour from the > > SPI implementation, but this expectation may be missed without hard CI > > failures. > > > > Previous dev. experiences contain some positive feedback on that approach > > too [2]. > > > > [1] > > > > https://polaris.apache.org/releases/1.7.0/evolution/#using-polaris-as-a-build-time-dependency > > > > [2] https://lists.apache.org/thread/s26pj6vp95gq13ltmcvj72s9f1870foj > > > > That said, I'm +1 to implement pagination for Generic Tables following > > Option 2 from Yong's email. > > > > Cheers, > > Dmitri. > > > > On Sun, Sep 13, 2026 at 8:06 AM Prithvi S <[email protected]> > > wrote: > > > > > Hi Yong, > > > > > > Thanks again for splitting this from the policy thread :) Same bug, > > > slightly different question because GenericTableCatalog is a polaris-core > > > SPI with federation impls (Hive / Hadoop / BigQuery), whereas > > > listSemanticModels was not. > > > > > > I would not replace listGenericTables(Namespace) the way semantic models > > > did. That is a breaking SPI change for every federated catalog, and it is > > > more than we need to honor the REST API. I’d add a PageToken overload on > > > the SPI with a default that keeps today’s behavior: > > > > > > default Page<TableIdentifier> listGenericTables(Namespace ns, PageToken > > > > token) { > > > > return Page.fromItems(listGenericTables(ns)); > > > > } > > > > > > > > > PolarisGenericTableCatalog can then thread the token into listEntities > > the > > > same way LocalIcebergCatalog does. Federation impls keep compiling and > > > still return everything, which matches “server does not support > > > pagination". We can give them real paging later if we want. That is > > option > > > 1, just on the interface so the handler does not have to downcast. > > > > > > WDYT? > > > > > > Thanks, > > > Prithvi S > > > > > > On Sun, Sep 13, 2026 at 6:25 AM Yong Zheng <[email protected]> wrote: > > > > > > > Hello, > > > > > > > > I would like to start this ML for discussing if we should implement the > > > > pagination support for generic table API (in this case, > > > > /polaris/v1/{prefix}/namespaces/{namespace}/generic-tables). Here is a > > > > reference report issue for this matter: > > > > https://github.com/apache/polaris/issues/5311. > > > > > > > > Looking at the what we currently have as of today, the parameters reach > > > > the adapter then got dropped. We are calling PageToken.readEverything() > > > > regardless ( > > > > > > > > > https://github.com/apache/polaris/blob/main/runtime/service/src/main/java/org/apache/polaris/service/catalog/generic/PolarisGenericTableCatalog.java#L196 > > > > ). > > > > > > > > Now to honor this, I think we have two approaches that we can consider: > > > > > > > > option 1: No change to SPI and use method overload instead > > > > > > > > This would be similar to what we are doing in LocalIcebergCatalog where > > > we > > > > will be adding Page listTables(namespace, pageToken) via method > > overload. > > > > Here is the reference: > > > > > > > > > https://github.com/apache/polaris/blame/main/runtime/service/src/main/java/org/apache/polaris/service/catalog/iceberg/LocalIcebergCatalog.java#L632 > > > > > > > > option 2: Change to SPI with listGenericTables(namespace, pageToken) > > > > > > > > This would be similar to listSemanticModels which got implemented > > > > recently. Here is the reference: > > > > > > > > > https://github.com/apache/polaris/blob/48126190377cfb2d0a54b4b9905a7375fde457ec/extensions/semantic-models/src/main/java/org/apache/polaris/service/catalog/semanticmodel/SemanticModelCatalogAdapter.java#L92 > > > > > > > > Thanks, > > > > Yong > > > > > > > > > >
