laserninja opened a new issue, #10669:
URL: https://github.com/apache/gravitino/issues/10669
### What would you like to be improved?
### What would you like to be improved?
The Iceberg REST spec defines `pageToken` and `pageSize` query parameters on
the `listNamespaces`, `listTables`, and `listViews` endpoints, but Gravitino's
Iceberg REST server does not accept or handle these parameters. Clients with
large catalogs cannot paginate results — everything is returned in a single
response.
**Affected endpoints and code:**
| Endpoint | File | Issue |
|---|---|---|
| `GET /v1/{prefix}/namespaces` | IcebergNamespaceOperations.java |
`listNamespaces()` only accepts `parent` `@QueryParam`; no
`pageToken`/`pageSize` |
| `GET /v1/{prefix}/namespaces/{namespace}/tables` |
IcebergTableOperations.java | `listTable()` has no pagination query params at
all |
| `GET /v1/{prefix}/namespaces/{namespace}/views` |
IcebergViewOperations.java | `listView()` has no pagination query params at all
|
The response builders do pass through `nextPageToken` from the underlying
response (e.g.,
`ListTablesResponse.builder().nextPageToken(listTablesResponse.nextPageToken())`),
but this value is always `null` since pagination parameters are never received
and never forwarded to the catalog backend.
The dispatcher interfaces (`IcebergTableOperationDispatcher`,
`IcebergNamespaceOperationDispatcher`, `IcebergViewOperationDispatcher`) also
lack pagination parameters in their method signatures, so the entire call chain
from REST handler → dispatcher → executor → `IcebergCatalogWrapper` has no
pagination support.
**Note:** The Lance REST server already supports pagination on its list
endpoints (`page_token` and `limit` query params in
`LanceNamespaceOperations.java`), which can serve as a reference implementation
pattern.
### How should we improve?
1. Add `@QueryParam("pageToken") String pageToken` and
`@QueryParam("pageSize") Integer pageSize` to the REST endpoint methods in
`IcebergTableOperations`, `IcebergNamespaceOperations`, and
`IcebergViewOperations`.
2. Update the dispatcher interfaces and their implementations to accept and
forward pagination parameters.
3. Pass pagination parameters through to the underlying Iceberg catalog
operations (e.g., `CatalogHandlers` / `SupportsNamespaces.listNamespaces()`),
implementing server-side pagination where the backing catalog supports it, or
applying in-memory pagination as a fallback.
4. Ensure `nextPageToken` in the response is correctly populated when there
are more results.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
