SteNicholas commented on code in PR #467:
URL: https://github.com/apache/paimon-cpp/pull/467#discussion_r4384867758
##########
docs/source/user_guide/write.rst:
##########
@@ -231,7 +231,9 @@ produce a correct ``Snapshot``, which commonly includes
(but is not limited to):
The C++ writer supports Append and PK tables and can produce
``CommitMessage`` objects for both. ``FileStoreCommit`` supports direct
file-system commits for both table types on non-object-store paths.
- Object-store paths require REST catalog commit mode. Changelog is out of
+ Object-store paths require a catalog-mediated commit, configured with either
+ ``CommitContextBuilder::WithCatalog()`` or ``UseRESTCatalogCommit()``;
+ neither needs an atomic rename on the object store. Changelog is out of
Review Comment:
[P2] Require version management for object-store commits
`WithCatalog()` alone does not enable object-store commits.
`IsCatalogCommit()` requires a catalog implementing `VersionManagedCatalog` and
advertising `SupportsVersionManagement()`; `FileSystemCatalog` falls back to
rename-based commits and remains rejected on object-store paths. Please qualify
this guidance to require a version-managed catalog, such as REST, so users do
not follow a configuration that still fails.
##########
docs/source/user_guide/system_tables.rst:
##########
@@ -26,6 +26,47 @@ Branch-qualified paths are also supported. For example,
``/warehouse/db.db/orders$branch_audit$files`` reads the files system table
from
the ``audit`` branch.
+Table-Level System Tables
+-------------------------
+
+These suffixes are recognized on a data table path. Matching is
+case-insensitive, and an unrecognized suffix fails with
+``unsupported system table: <name>`` instead of falling back to the base table.
+
+* ``$options``: the latest base table options as key/value rows.
+* ``$snapshots``: snapshot commit history.
+* ``$schemas``: schema evolution history.
+* ``$tags``: tags and the snapshots they reference.
+* ``$branches``: table branches, including ``main``.
+* ``$consumers``: persisted streaming consumer offsets.
+* ``$manifests``: data manifest metadata in the latest snapshot.
+* ``$files``: data file metadata in the latest snapshot.
+* ``$audit_log``: row-level changelog records with rowkind.
+* ``$binlog``: changelog records with list-wrapped data columns.
+* ``$ro``: read-optimized data, described in the next section.
+
+Global System Tables
+--------------------
+
+Four system tables are not bound to a single data table. They live under the
+reserved ``sys`` database and enumerate across all databases in the warehouse:
+
+* ``sys.catalog_options``: catalog-level configuration as key/value rows.
+* ``sys.all_table_options``: all table options as
+ ``(database_name, table_name, key, value)`` rows.
+* ``sys.tables``: metadata for all tables, including record counts and file
+ statistics.
+* ``sys.partitions``: partition-level file statistics for all tables.
+
+These need a catalog to enumerate databases and tables, so load them through a
+catalog as the identifier ``sys.<name>`` rather than from a bare
+``<warehouse>/sys/<name>`` path; a path load without catalog context fails with
+``global system table requires catalog context: <name>``.
+``sys.all_table_options``, ``sys.tables`` and ``sys.partitions`` are always
+available. ``sys.catalog_options`` is disabled by default and has to be turned
Review Comment:
[P2] Document the public API limitation for global tables
Loading through a catalog does not currently provide the advertised read
path. `FileSystemCatalog::GetTable(Identifier("sys", "tables"))` reaches
`TableSchemaExists()`, which rejects system tables. `TableScan::Create()` and
`TableRead::Create()` also call `LoadFromPath()`, which leaves the catalog null
even when `WithCatalog()` was supplied. Listing and schema lookup work, but
reading `sys.all_table_options`, `sys.tables`, and `sys.partitions` through the
public API does not. Please document that limitation instead of presenting
catalog loading as a working alternative.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
