HI Prithvi, Yong, I believe breaking SPI changes are quite alright. They are in-line with our evolution policy [1].
My personal view on SPI backward-compatibilioty is that altering method signatures is a constructive signal to SPI implementors (outside of Polaris repo) that they need to consider that Polaris Core now has _different_ expectations of the SPI. Trying to make the implementor's ride "smooth" (e.g. by offering default methods that do not break compilation) is actually likely to produce hidden issues, because Polaris Core will expect _new_ behaviour from the SPI implementation, but this expectation may be missed without hard CI failures. Previous dev. experiences contain some positive feedback on that approach too [2]. [1] https://polaris.apache.org/releases/1.7.0/evolution/#using-polaris-as-a-build-time-dependency [2] https://lists.apache.org/thread/s26pj6vp95gq13ltmcvj72s9f1870foj That said, I'm +1 to implement pagination for Generic Tables following Option 2 from Yong's email. Cheers, Dmitri. On Sun, Sep 13, 2026 at 8:06 AM Prithvi S <[email protected]> wrote: > Hi Yong, > > Thanks again for splitting this from the policy thread :) Same bug, > slightly different question because GenericTableCatalog is a polaris-core > SPI with federation impls (Hive / Hadoop / BigQuery), whereas > listSemanticModels was not. > > I would not replace listGenericTables(Namespace) the way semantic models > did. That is a breaking SPI change for every federated catalog, and it is > more than we need to honor the REST API. I’d add a PageToken overload on > the SPI with a default that keeps today’s behavior: > > default Page<TableIdentifier> listGenericTables(Namespace ns, PageToken > > token) { > > return Page.fromItems(listGenericTables(ns)); > > } > > > PolarisGenericTableCatalog can then thread the token into listEntities the > same way LocalIcebergCatalog does. Federation impls keep compiling and > still return everything, which matches “server does not support > pagination". We can give them real paging later if we want. That is option > 1, just on the interface so the handler does not have to downcast. > > WDYT? > > Thanks, > Prithvi S > > On Sun, Sep 13, 2026 at 6:25 AM Yong Zheng <[email protected]> wrote: > > > Hello, > > > > I would like to start this ML for discussing if we should implement the > > pagination support for generic table API (in this case, > > /polaris/v1/{prefix}/namespaces/{namespace}/generic-tables). Here is a > > reference report issue for this matter: > > https://github.com/apache/polaris/issues/5311. > > > > Looking at the what we currently have as of today, the parameters reach > > the adapter then got dropped. We are calling PageToken.readEverything() > > regardless ( > > > https://github.com/apache/polaris/blob/main/runtime/service/src/main/java/org/apache/polaris/service/catalog/generic/PolarisGenericTableCatalog.java#L196 > > ). > > > > Now to honor this, I think we have two approaches that we can consider: > > > > option 1: No change to SPI and use method overload instead > > > > This would be similar to what we are doing in LocalIcebergCatalog where > we > > will be adding Page listTables(namespace, pageToken) via method overload. > > Here is the reference: > > > https://github.com/apache/polaris/blame/main/runtime/service/src/main/java/org/apache/polaris/service/catalog/iceberg/LocalIcebergCatalog.java#L632 > > > > option 2: Change to SPI with listGenericTables(namespace, pageToken) > > > > This would be similar to listSemanticModels which got implemented > > recently. Here is the reference: > > > https://github.com/apache/polaris/blob/48126190377cfb2d0a54b4b9905a7375fde457ec/extensions/semantic-models/src/main/java/org/apache/polaris/service/catalog/semanticmodel/SemanticModelCatalogAdapter.java#L92 > > > > Thanks, > > Yong > > >
