lucasfang opened a new pull request, #533: URL: https://github.com/apache/paimon-cpp/pull/533
### Purpose Related to #459. **Depends on #527 (PR2); this is PR3 of the Table refactoring series.** A Context currently combines table-loading identity, storage configuration, dynamic options and execution settings. Operators created from a loaded table therefore still receive another set of table-loading parameters, leaving two places to supply its path, schema, filesystem and catalog identity. Separate those responsibilities: - Add public `TableLoadOptions` for path, explicit filesystem, URI scheme routing, serialized Paimon table schema, catalog, identifier and expected catalog UUID. - Make Scan/Read/Write/Commit Contexts carry execution settings only. Remove table-loading fields, embedded option maps, `WithBranch()` and FormatTable-based builder constructors. - Change static factories to `Create(load_options, context, dynamic_options)`; loaded tables take `NewXxx(context, dynamic_options)`. The static factories share one table-loading implementation. - Bind managed tables to their catalog services, per-table filesystem, UUID and routing map. Keep catalog services alive when a table outlives a stack-allocated catalog; preserve operation-specific catalog write/commit validation. - Keep serialized Paimon table metadata separate from a reader's Arrow projection. Context construction performs no table metadata I/O. - Preserve the catalog-scoped filesystem for explicit-path/UUID loads, including tables with inherited URI routing. Preserve external FormatTable directory semantics so data partitions named `schema` or `branch` remain readable; keep native missing-schema errors unchanged. - Migrate internal callers, examples, benchmarks, documentation and tests to the separated arguments. Most of the file count comes from these call-site migrations. The resource binding follows the local Java Paimon `CatalogUtils.loadTable()` / `toFormatTable()` model: catalog metadata determines the table and its FileIO. PR4 will cover general `Table::Copy()`, branch switching and historical-schema time travel; those APIs are outside this PR. **Review scope:** #527 is still open, so the default Files changed tab also includes PR2. [Review only the PR3 increment](https://github.com/lucasfang/paimon-cpp/compare/7f6bc02bd8046ecb19bb5a4cd0fb72d170325d4c...2de582d505ef4a3fbdb688be0406f8717966ad92). PR3 is based on PR2's latest CI-warning fixes and should be rebased onto main after #527 merges. Suggested review order: `table_load_options.h` and the four public Context headers; `table_factory.cpp`; managed/format/system `NewXxx()` implementations; catalog binding and regression tests. Call-site migrations can then be reviewed against those signatures. ### Tests Publication commit: `2de582d505ef4a3fbdb688be0406f8717966ad92`. On the publication commit: ```sh cmake --build build --target paimon-core-test paimon-read-inte-test -j 40 ./build/debug/paimon-core-test --gtest_filter='TableTest.*:FormatTableTest.*:SystemTableTest.*:ReadContextTest.*:ScanContextTest.*:WriteContextTest.*:CommitContextTest.*:InternalReadContextTest.*:BranchManagerTest.*' ./build/debug/paimon-read-inte-test --gtest_filter='SystemTableReadInteTest.*' git diff origin/codex/table-hierarchy HEAD --check ``` - Build passed; selected Core tests: **184/184 passed**; system-table read integration tests: **32/32 passed**. - `pre-commit run --files <all 113 changed files>` passed, including clang-format, C++ lint, codespell and Sphinx lint. The whitespace check passed. - The complete CTest suite was not repeated after the CI-warning-only synchronization; its earlier results and the final fix reruns are listed below. Earlier PR3 validation before synchronizing PR2's CI-warning-only follow-up: - Full configured build succeeded. - The full 36-target CTest run passed 35 targets and exposed a changed missing-schema error in the Blob suite. After preserving that error behavior, both complete Core and Blob suites passed (2/2 targets, 278.90 seconds). The other 34 targets were not rerun after that final error-preservation adjustment. - Four focused factory/filesystem/external-format regressions passed. Regression coverage includes Context creation without table I/O, invalid load combinations, catalog UUID/path checks, catalog lifetime and routing, explicit filesystem precedence, main-branch identifier aliases, native missing-schema errors, and actual scan/read/write/commit of external format partitions named `schema` and `branch` (including a data filename resembling a schema version). Java was inspected as a semantic reference; Java tests were not run. ### API and Format Breaking C++ API changes: - Add exported `paimon::TableLoadOptions` in `include/paimon/catalog/table_load_options.h`. - Scan/Read Context builders no longer take a table path or FormatTable; Write/Commit builders take the commit user without table-loading arguments. - Remove loading-related Context accessors/setters, embedded options and `WithBranch()`. Specify a branch through the table identifier or dynamic options. - Replace the four static `Create(context)` signatures with `Create(load_options, context, dynamic_options = {})`. - Add the separate optional dynamic options argument to `Table::NewScan/NewRead/NewWrite/NewCommit` and their implementations. - Existing callers must migrate to the new signatures; no compatibility overloads are retained. No storage-format or protocol changes. ### Documentation Update catalog and format-table API references, read/write/commit/realtime and cache/index guides, examples and benchmark callers. The catalog guide includes both loaded-table and static-factory usage and explains catalog identity, filesystem selection and branch inputs. ### Generative AI tooling Generated-by: OpenAI Codex (GPT-6) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
