laserninja commented on code in PR #12194:
URL: https://github.com/apache/gravitino/pull/12194#discussion_r3763539049
##########
docs/iceberg-rest-service.md:
##########
@@ -697,13 +697,20 @@ Gravitino provides the build-in
`org.apache.gravitino.iceberg.common.cache.Local
Gravitino caches scan plan results to speed up repeated queries with identical
parameters. The cache uses snapshot ID as part of the cache key, so queries
against different snapshots will not use stale cached data.
-Plan scan responses follow the Iceberg 1.11 REST API: completed plans return
structured `file-scan-tasks` only. Legacy `plan-tasks` JSON strings (used by
some Iceberg 1.9.x–1.10.x clients) are not emitted.
+Plan scan responses follow the Iceberg 1.11 REST API: `file-scan-tasks` are
returned as structured tasks rather than as the JSON strings used by some
Iceberg 1.9.x–1.10.x clients.
-| Configuration item | Description
| Default value | Required | Since
Version |
-|------------------------------------------------------------|----------------------------------------------------------|---------------|----------|---------------|
-| `gravitino.iceberg-rest.scan-plan-cache-impl` | The
implementation of the scan plan cache. | (none) | No
| 1.2.0 |
-| `gravitino.iceberg-rest.scan-plan-cache-capacity` | The capacity of
the scan plan cache. | 200 | No | 1.2.0
|
-| `gravitino.iceberg-rest.scan-plan-cache-expire-minutes` | The expiration
time (in minutes) of the scan plan cache. | 60 | No | 1.2.0
|
+Scan planning is synchronous: `POST
/v1/{prefix}/namespaces/{namespace}/tables/{table}/plan` always returns status
`COMPLETED`, never `SUBMITTED`, so a plan is never left running in the
background.
+
+A plan is handed to the client in batches of at most
`scan-plan-task-batch-size` file scan tasks. The first batch is returned inline
in the plan response; each remaining batch is offered as a `plan-task` that the
client exchanges for its tasks through `POST
/v1/{prefix}/namespaces/{namespace}/tables/{table}/tasks`, the second step of
the Iceberg REST scan planning protocol. A plan that fits in one batch carries
no `plan-tasks`, so most scans complete in a single round trip.
+
+A `plan-task` is an opaque string to clients, and describes its own unit of
work to the server: the scan it was planned from, with the snapshot pinned at
planning time, plus the range of tasks it covers. Nothing is stored server side
between the two calls, so a `plan-task` stays redeemable after a server restart
and on any Gravitino instance serving the same catalog. Redeeming one replans
the pinned snapshot unless the plan is still in the scan plan cache, so
enabling the cache is recommended when planning large tables. A `plan-task`
that this server did not issue, was issued for another table, or refers to a
plan that can no longer be reproduced (for example because its snapshot
expired) returns `404` with a `NoSuchPlanTaskException` error.
+
+| Configuration item | Description
| Default value | Required | Since Version |
+|------------------------------------------------------------|--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|---------------|----------|---------------|
+| `gravitino.iceberg-rest.scan-plan-cache-impl` | The
implementation of the scan plan cache.
| (none) | No | 1.2.0 |
+| `gravitino.iceberg-rest.scan-plan-cache-capacity` | The capacity of
the scan plan cache.
| 200 | No | 1.2.0 |
+| `gravitino.iceberg-rest.scan-plan-cache-expire-minutes` | The expiration
time (in minutes) of the scan plan cache.
| 60 | No | 1.2.0 |
+| `gravitino.iceberg-rest.scan-plan-task-batch-size` | Maximum number
of file scan tasks returned inline by one scan planning response. Tasks beyond
this limit are offered as `plan-tasks`. Set to 0 to always return every task
inline. | 100 | No | 1.3.0 |
Review Comment:
You are right, fixed in both places - `VERSION_2_0_0` in `IcebergConfig` and
`2.0.0` in the docs table.
I had copied 1.3.0 from the neighbouring entries in `IcebergConfig`, but all
of those were added before `gradle.properties` was bumped to `2.0.0-SNAPSHOT`
in ee6b17409 (2026-06-29). Every config added since then uses `VERSION_2_0_0`,
so this one should too.
##########
iceberg/iceberg-common/src/main/java/org/apache/gravitino/iceberg/common/IcebergConfig.java:
##########
@@ -341,6 +341,18 @@ public class IcebergConfig extends Config implements
OverwriteDefaultConfig {
.checkValue(value -> value > 0,
ConfigConstants.POSITIVE_NUMBER_ERROR_MSG)
.createWithDefault(60);
+ public static final ConfigEntry<Integer> SCAN_PLAN_TASK_BATCH_SIZE =
+ new ConfigBuilder(IcebergConstants.SCAN_PLAN_TASK_BATCH_SIZE)
+ .doc(
+ "Maximum number of file scan tasks returned inline by one scan
planning response. "
+ + "Tasks beyond this limit are handed out as plan-tasks that
clients "
+ + "exchange for the remaining tasks. Set to 0 to disable
batching and always "
+ + "return every task inline.")
+ .version(ConfigConstants.VERSION_1_3_0)
+ .intConf()
Review Comment:
You are right, fixed in both places - `VERSION_2_0_0` in `IcebergConfig` and
`2.0.0` in the docs table.
I had copied 1.3.0 from the neighbouring entries in `IcebergConfig`, but all
of those were added before `gradle.properties` was bumped to `2.0.0-SNAPSHOT`
in ee6b17409 (2026-06-29). Every config added since then uses `VERSION_2_0_0`,
so this one should too.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
