danhuawang opened a new issue, #11284:
URL: https://github.com/apache/gravitino/issues/11284
### What would you like to be improved?
Gravitino Iceberg REST server supports the `POST
/v1/{prefix}/namespaces/{namespace}/tables/{table}/plan` endpoint for
initiating server-side scan planning, but does not support the `POST
/v1/{prefix}/namespaces/{namespace}/tables/{table}/tasks` endpoint for fetching
scan task results.
The Iceberg REST scan planning protocol is a two-step process:
1. **Submit scan plan**: `POST
/v1/{prefix}/namespaces/{namespace}/tables/{table}/plan` — returns a plan task
token
2. **Fetch scan tasks**: `POST
/v1/{prefix}/namespaces/{namespace}/tables/{table}/tasks` — uses the plan task
token to retrieve the actual scan tasks
Gravitino currently advertises the `/plan` endpoint in its `/v1/config`
response:
```json
"endpoints": [
...
"POST /v1/{prefix}/namespaces/{namespace}/tables/{table}/plan",
...
]
```
But it does **not** advertise or implement the `/tasks` endpoint. When a
client (e.g., pyiceberg with `scan-planning-mode=server`) successfully submits
a scan plan and then attempts to fetch the tasks, it fails with:
```
NotImplementedError: Server does not support endpoint: POST
/v1/{prefix}/namespaces/{namespace}/tables/{table}/tasks
```
This makes the server-side scan planning feature incomplete and unusable for
clients that follow the two-step protocol.
### Steps to reproduce
1. Configure Gravitino Iceberg REST server with JDBC backend and S3 storage
2. Configure pyiceberg client with `scan-planning-mode=server`
3. Create a table and append data
4. Attempt to scan the table using server-side planning:
```python
from pyiceberg.catalog import load_catalog
catalog = load_catalog(
"local",
**{
"type": "rest",
"uri": "http://localhost:8181/iceberg/";,
"s3.endpoint": "http://localhost:9000";,
"s3.access-key-id": "admin",
"s3.secret-access-key": "password",
"scan-planning-mode": "server",
"auth": {"type": "noop"},
},
)
table = catalog.load_table("default.my_table")
tasks = list(table.scan().plan_files())
# Raises: NotImplementedError: Server does not support endpoint:
# POST /v1/{prefix}/namespaces/{namespace}/tables/{table}/tasks
```
### Environment
- Gravitino version: main branch
(`datastratosandbox/gravitino:1.3.0-SNAPSHOT-20260527024943`)
- Client: pyiceberg (latest)
- Configuration: JDBC catalog backend with S3 storage
### How should we improve?
Implement the `POST
/v1/{prefix}/namespaces/{namespace}/tables/{table}/tasks` endpoint in the
Gravitino Iceberg REST server and advertise it in the `/v1/config` response.
The endpoint should accept a plan task token (returned by the `/plan`
endpoint) and return the corresponding scan tasks (file scan tasks with data
file paths, partition info, etc.).
Alternatively, if Gravitino's scan planning implementation returns all tasks
in a single response from the `/plan` endpoint (without requiring a separate
`/tasks` call), the `/plan` endpoint response should indicate completion so
that clients do not attempt to call `/tasks`.
Reference: [Apache Iceberg REST Catalog Spec - Scan
Planning](https://github.com/apache/iceberg/blob/main/open-api/rest-catalog-open-api.yaml)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
