gripleaf opened a new pull request, #329:
URL: https://github.com/apache/paimon-cpp/pull/329
### Purpose
Linked issue: N/A
Repeated scans currently recreate schema managers and schema-derived
metadata. Add `TableScanResources` to reuse these resources across scans of the
same managed table and branch.
The container shares SchemaManager, SnapshotManager, Arrow and partition
schemas, primary-key field metadata, and statistics evolution caches. Each scan
still discovers the latest schema and maintains independent filters, snapshot
selection, limits, and streaming progress.
Support resource forwarding through `$ro` and `$audit_log`, validate
conflicting paths, branches, and file systems, and preserve existing
`SetTableSchema()` behavior.
### Tests
Added 12 regression tests covering:
- Schema and statistics metadata reuse across append-only and primary-key
scans.
- Schema updates and caller-supplied schema isolation.
- Concurrent scans and retries after schema read failures.
- Independent filters, limits, snapshot selection, and streaming progress.
- Branch handling, system-table forwarding, and format-table rejection.
- Resource lifetime, builder reset, and configuration validation.
Validation passed:
- All 2,103 core unit tests.
- All 87 Scan integration tests.
- Pre-commit checks for all changed files.
- `git diff --check`.
### API and Format
Public API additions:
- `TableScanResources::Create()`.
- `ScanContextBuilder::WithTableResources()`.
- `ScanContext::GetTableResources()`.
- An optional resource parameter on the `ScanContext` constructor.
Existing builder-based scan creation remains supported without changes. No
storage format or protocol changes.
### Documentation
Updated `docs/source/api/scan.rst` and public API comments with usage
examples, configuration validation, concurrency requirements, and resource
lifetime semantics.
Cached schema versions remain available for the container's lifetime.
Recreate the container after deleting and recreating a table or branch.
### Generative AI tooling
Generated-by: OpenAI Codex (GPT-6)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
