lukeFalsina opened a new pull request, #3724:
URL: https://github.com/apache/iceberg-python/pull/3724
## Summary
- Implement async REST `fetchPlanningResult` / `cancelPlanning` polling when
`planTableScan` returns `status=submitted`
- Apply completed-plan `storage-credentials` to the scan-scoped FileIO
(layered on existing IO properties)
- Keep public `RestCatalog.plan_scan(...) -> list[FileScanTask]` unchanged;
credentials flow through internal `_plan_scan_result` / `_file_io_from_plan`
- Honor per-table `scan-planning-mode` from `LoadTableResponse.config`,
which takes precedence over the catalog-level / `GET /v1/config` setting (same
precedence as Java)
Related: #2775, #3495
Java reference: apache/iceberg#13400 (async planning), apache/iceberg#15572
(table-level scan planning override)
## Rationale
Unblocks REST catalogs that return async plans (for example policy-protected
tables). Finishes the unchecked async items from #2775 and the plan-credential
gap from #3495.
Per-table `loadTable` overrides let a server request Scan Plan API only
where needed (e.g. policy-protected tables) while other tables keep client-side
planning, without forcing a catalog-wide `scan-planning-mode=server`.
## User-facing
- `table.scan()` with `scan-planning-mode=server` now handles async plans
automatically
- New optional `rest-scan-planning.poll-timeout-ms` (default 300000)
- No breaking change to `RestCatalog.plan_scan` return type
- Effective scan planning mode resolution:
1. `scan-planning-mode` in the table's `loadTable` response `config` (wins
when present)
2. Catalog-level client / `GET /v1/config` property
3. Default `client`
- Plan endpoint must still be advertised for server-side planning
- Invalid catalog-level `scan-planning-mode` values are ignored with a
warning (they cannot block a valid `loadTable` override or the default);
invalid `loadTable` values still raise
## Test plan
- [x] `make lint`
- [x] `make test` (3812+ passed)
- [x] Unit tests in `tests/catalog/test_scan_planning_models.py` for poll
success / timeout / failed / cancelled and IO property retention
- [x] Unit tests for `loadTable` override precedence, client default without
override, and invalid catalog mode surviving a valid table override
- [x] Existing REST scan integration suite (CI)
- [x] Manual validation against an async-capable catalog (Snowflake Horizon
Iceberg REST Catalog Scan Plan API), including mixed protected / unprotected
tables in one session without a catalog-wide `scan-planning-mode`
Made with [Cursor](https://cursor.com)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
