lucasfang opened a new pull request, #533:
URL: https://github.com/apache/paimon-cpp/pull/533

   ### Purpose
   
   Related to #459. **Depends on #527 (PR2); this is PR3 of the Table 
refactoring series.**
   
   A Context currently combines table-loading identity, storage configuration, 
dynamic options and execution settings. Operators created from a loaded table 
therefore still receive another set of table-loading parameters, leaving two 
places to supply its path, schema, filesystem and catalog identity.
   
   Separate those responsibilities:
   
   - Add public `TableLoadOptions` for path, explicit filesystem, URI scheme 
routing, serialized Paimon table schema, catalog, identifier and expected 
catalog UUID.
   - Make Scan/Read/Write/Commit Contexts carry execution settings only. Remove 
table-loading fields, embedded option maps, `WithBranch()` and 
FormatTable-based builder constructors.
   - Change static factories to `Create(load_options, context, 
dynamic_options)`; loaded tables take `NewXxx(context, dynamic_options)`. The 
static factories share one table-loading implementation.
   - Bind managed tables to their catalog services, per-table filesystem, UUID 
and routing map. Keep catalog services alive when a table outlives a 
stack-allocated catalog; preserve operation-specific catalog write/commit 
validation.
   - Keep serialized Paimon table metadata separate from a reader's Arrow 
projection. Context construction performs no table metadata I/O.
   - Preserve the catalog-scoped filesystem for explicit-path/UUID loads, 
including tables with inherited URI routing. Preserve external FormatTable 
directory semantics so data partitions named `schema` or `branch` remain 
readable; keep native missing-schema errors unchanged.
   - Migrate internal callers, examples, benchmarks, documentation and tests to 
the separated arguments. Most of the file count comes from these call-site 
migrations.
   
   The resource binding follows the local Java Paimon 
`CatalogUtils.loadTable()` / `toFormatTable()` model: catalog metadata 
determines the table and its FileIO. PR4 will cover general `Table::Copy()`, 
branch switching and historical-schema time travel; those APIs are outside this 
PR.
   
   **Review scope:** #527 is still open, so the default Files changed tab also 
includes PR2. [Review only the PR3 
increment](https://github.com/lucasfang/paimon-cpp/compare/7f6bc02bd8046ecb19bb5a4cd0fb72d170325d4c...2de582d505ef4a3fbdb688be0406f8717966ad92).
 PR3 is based on PR2's latest CI-warning fixes and should be rebased onto main 
after #527 merges.
   
   Suggested review order: `table_load_options.h` and the four public Context 
headers; `table_factory.cpp`; managed/format/system `NewXxx()` implementations; 
catalog binding and regression tests. Call-site migrations can then be reviewed 
against those signatures.
   
   ### Tests
   
   Publication commit: `2de582d505ef4a3fbdb688be0406f8717966ad92`.
   
   On the publication commit:
   
   ```sh
   cmake --build build --target paimon-core-test paimon-read-inte-test -j 40
   ./build/debug/paimon-core-test 
--gtest_filter='TableTest.*:FormatTableTest.*:SystemTableTest.*:ReadContextTest.*:ScanContextTest.*:WriteContextTest.*:CommitContextTest.*:InternalReadContextTest.*:BranchManagerTest.*'
   ./build/debug/paimon-read-inte-test 
--gtest_filter='SystemTableReadInteTest.*'
   git diff origin/codex/table-hierarchy HEAD --check
   ```
   
   - Build passed; selected Core tests: **184/184 passed**; system-table read 
integration tests: **32/32 passed**.
   - `pre-commit run --files <all 113 changed files>` passed, including 
clang-format, C++ lint, codespell and Sphinx lint. The whitespace check passed.
   - The complete CTest suite was not repeated after the CI-warning-only 
synchronization; its earlier results and the final fix reruns are listed below.
   
   Earlier PR3 validation before synchronizing PR2's CI-warning-only follow-up:
   
   - Full configured build succeeded.
   - The full 36-target CTest run passed 35 targets and exposed a changed 
missing-schema error in the Blob suite. After preserving that error behavior, 
both complete Core and Blob suites passed (2/2 targets, 278.90 seconds). The 
other 34 targets were not rerun after that final error-preservation adjustment.
   - Four focused factory/filesystem/external-format regressions passed.
   
   Regression coverage includes Context creation without table I/O, invalid 
load combinations, catalog UUID/path checks, catalog lifetime and routing, 
explicit filesystem precedence, main-branch identifier aliases, native 
missing-schema errors, and actual scan/read/write/commit of external format 
partitions named `schema` and `branch` (including a data filename resembling a 
schema version).
   
   Java was inspected as a semantic reference; Java tests were not run.
   
   ### API and Format
   
   Breaking C++ API changes:
   
   - Add exported `paimon::TableLoadOptions` in 
`include/paimon/catalog/table_load_options.h`.
   - Scan/Read Context builders no longer take a table path or FormatTable; 
Write/Commit builders take the commit user without table-loading arguments.
   - Remove loading-related Context accessors/setters, embedded options and 
`WithBranch()`. Specify a branch through the table identifier or dynamic 
options.
   - Replace the four static `Create(context)` signatures with 
`Create(load_options, context, dynamic_options = {})`.
   - Add the separate optional dynamic options argument to 
`Table::NewScan/NewRead/NewWrite/NewCommit` and their implementations.
   - Existing callers must migrate to the new signatures; no compatibility 
overloads are retained.
   
   No storage-format or protocol changes.
   
   ### Documentation
   
   Update catalog and format-table API references, read/write/commit/realtime 
and cache/index guides, examples and benchmark callers. The catalog guide 
includes both loaded-table and static-factory usage and explains catalog 
identity, filesystem selection and branch inputs.
   
   ### Generative AI tooling
   
   Generated-by: OpenAI Codex (GPT-6)
   
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]

Reply via email to