SteNicholas opened a new pull request, #534:
URL: https://github.com/apache/paimon-cpp/pull/534

   ### Purpose
   
   Linked issue: close #520
   
   Enforce the row filters and column masks a REST catalog defines for tables 
with `query-auth.enabled=true`.
   
   - `RestCatalog` implements `QueryAuthCatalog::AuthTableQuery` against `POST 
/v1/{prefix}/databases/{database}/tables/{table}/auth`, with new 
`AuthTableQueryRequest` and `AuthTableQueryResponse` messages. External tables 
get no rules.
   - `TableQueryAuthResult` parses the predicate and transform JSON into a row 
filter and `NULL`, `FIELD_REF`, `CONCAT`, `CONCAT_WS` and `SUBSTRING` masks, 
refuses malformed or unsupported rules, and validates them against a schema by 
column name, type and field id.
   - `QueryAuthTableScan` fetches and validates the rules on every plan, pushes 
down only the row filter and the predicate conjuncts on unmasked columns, and 
wraps each split in a `QueryAuthSplit`.
   - `QueryAuthTableRead` applies the row filter, then the masks, then the read 
predicate, and refuses a split whose authorized columns no longer match the 
schema read.
   - `Split` serializes `QueryAuthSplit` with the field ids its rules were 
authorized for.
   - `TableScan::Create` / `TableRead::Create` enable query auth from the 
catalog's branch schema, and refuse paths that would bypass the rules: search, 
explicit read schemas, real-time and fallback reads, global index scans and the 
`$files`, `$binlog`, `$audit_log` and `$ro` system tables.
   - `ScanContextBuilder::SetReadFieldNames` is added, and the scan and read 
context builders carry the `branch` option into the catalog identifier.
   - `LIKE` lets `_` match any character, including a line terminator.
   
   ### Tests
   
   Unit tests:
   
   - `TableQueryAuthResultTest.*`: rule parsing, literal types and boundaries, 
constant folding, masks, schema and field-id validation, malformed and 
unsupported rules.
   - `QueryAuthSplitTest.*`: deserializing checked-in fixtures under 
`test/test_data/query_auth/`, round trips and malformed bytes.
   - `QueryAuthTableReadTest.*`: dictionary and large-string decoding, closing 
readers on failure.
   - `TableTest.TestQueryAuth*`: end-to-end plan and read with a mock 
query-auth catalog, covering branches, schema changes, pruning and limit 
pushdown, primary-key, deletion-vector and data-evolution tables, and refused 
read paths.
   - `RestCatalogTest.*QueryAuth*`, `RestCatalogTest.AuthTableQuery*`, 
`RestApiTest.AuthTableQuery`, `RestMessagesTest.AuthTableQuery*`: REST 
endpoint, request/response messages, retries and error mapping.
   - `ScanContextTest.TestWithCatalogResolvesTheBranch`, 
`ReadContextTest.TestWithCatalogResolvesTheBranch`, 
`SchemaValidationTest.TestQueryAuth`, 
`FormatTableTest.TestQueryAuthInTheSchemaCannotBeDisabledAtTheCall`, 
`PredicateTest.TestLikeWildcardsMatchLineTerminators`, 
`RapidJsonUtilTest.TestRejectUnescapedNul`, 
`RapidJsonUtilTest.TestSerializeStringWithEmbeddedNul`, 
`RapidJsonUtilTest.TestCheckNoDuplicateMembers`, 
`DataTypeJsonParserTest.ParseTypeRejectsEmbeddedNul`, 
`CoreOptionsTest.TestDefaultValue`, `CoreOptionsTest.TestFromMap`, 
`ResourcePathsTest.*`.
   
   Integration tests: `BlobTableInteTest` cases on a missing upstream table now 
expect `NotExist`.
   
   The latest revision was not built or run locally; only `clang-format` and 
`git diff --check` were run on it.
   
   ### API and Format
   
   - Public API: adds `Options::QUERY_AUTH_ENABLED` (`query-auth.enabled`), 
`ScanContextBuilder::SetReadFieldNames()`, `ScanContext::GetReadFieldNames()` 
and a `ScanContext` constructor taking the read field names; updates the docs 
of `WithCatalog` on both context builders.
   - Storage format: a split of a query-auth table serializes as a 
`QueryAuthSplit` with the field ids its rules were authorized for. Other splits 
are unchanged.
   - Protocol: the REST catalog calls `POST .../tables/{table}/auth`.
   
   ### Documentation
   
   Yes. `docs/source/user_guide/catalog.rst` documents the option, the 
supported rules, planning and reading a query-auth table, the refused read 
paths and the cross-runtime limits of query-auth splits.
   
   ### Generative AI tooling
   
   Generated-by: Claude Code (2.1.311, Claude Opus 5.5)
   
   🤖 Generated with [Claude Code](https://claude.com/claude-code)
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]

Reply via email to