SteNicholas opened a new pull request, #534:
URL: https://github.com/apache/paimon-cpp/pull/534
### Purpose
Linked issue: close #520
Enforce the row filters and column masks a REST catalog defines for tables
with `query-auth.enabled=true`.
- `RestCatalog` implements `QueryAuthCatalog::AuthTableQuery` against `POST
/v1/{prefix}/databases/{database}/tables/{table}/auth`, with new
`AuthTableQueryRequest` and `AuthTableQueryResponse` messages. External tables
get no rules.
- `TableQueryAuthResult` parses the predicate and transform JSON into a row
filter and `NULL`, `FIELD_REF`, `CONCAT`, `CONCAT_WS` and `SUBSTRING` masks,
refuses malformed or unsupported rules, and validates them against a schema by
column name, type and field id.
- `QueryAuthTableScan` fetches and validates the rules on every plan, pushes
down only the row filter and the predicate conjuncts on unmasked columns, and
wraps each split in a `QueryAuthSplit`.
- `QueryAuthTableRead` applies the row filter, then the masks, then the read
predicate, and refuses a split whose authorized columns no longer match the
schema read.
- `Split` serializes `QueryAuthSplit` with the field ids its rules were
authorized for.
- `TableScan::Create` / `TableRead::Create` enable query auth from the
catalog's branch schema, and refuse paths that would bypass the rules: search,
explicit read schemas, real-time and fallback reads, global index scans and the
`$files`, `$binlog`, `$audit_log` and `$ro` system tables.
- `ScanContextBuilder::SetReadFieldNames` is added, and the scan and read
context builders carry the `branch` option into the catalog identifier.
- `LIKE` lets `_` match any character, including a line terminator.
### Tests
Unit tests:
- `TableQueryAuthResultTest.*`: rule parsing, literal types and boundaries,
constant folding, masks, schema and field-id validation, malformed and
unsupported rules.
- `QueryAuthSplitTest.*`: deserializing checked-in fixtures under
`test/test_data/query_auth/`, round trips and malformed bytes.
- `QueryAuthTableReadTest.*`: dictionary and large-string decoding, closing
readers on failure.
- `TableTest.TestQueryAuth*`: end-to-end plan and read with a mock
query-auth catalog, covering branches, schema changes, pruning and limit
pushdown, primary-key, deletion-vector and data-evolution tables, and refused
read paths.
- `RestCatalogTest.*QueryAuth*`, `RestCatalogTest.AuthTableQuery*`,
`RestApiTest.AuthTableQuery`, `RestMessagesTest.AuthTableQuery*`: REST
endpoint, request/response messages, retries and error mapping.
- `ScanContextTest.TestWithCatalogResolvesTheBranch`,
`ReadContextTest.TestWithCatalogResolvesTheBranch`,
`SchemaValidationTest.TestQueryAuth`,
`FormatTableTest.TestQueryAuthInTheSchemaCannotBeDisabledAtTheCall`,
`PredicateTest.TestLikeWildcardsMatchLineTerminators`,
`RapidJsonUtilTest.TestRejectUnescapedNul`,
`RapidJsonUtilTest.TestSerializeStringWithEmbeddedNul`,
`RapidJsonUtilTest.TestCheckNoDuplicateMembers`,
`DataTypeJsonParserTest.ParseTypeRejectsEmbeddedNul`,
`CoreOptionsTest.TestDefaultValue`, `CoreOptionsTest.TestFromMap`,
`ResourcePathsTest.*`.
Integration tests: `BlobTableInteTest` cases on a missing upstream table now
expect `NotExist`.
The latest revision was not built or run locally; only `clang-format` and
`git diff --check` were run on it.
### API and Format
- Public API: adds `Options::QUERY_AUTH_ENABLED` (`query-auth.enabled`),
`ScanContextBuilder::SetReadFieldNames()`, `ScanContext::GetReadFieldNames()`
and a `ScanContext` constructor taking the read field names; updates the docs
of `WithCatalog` on both context builders.
- Storage format: a split of a query-auth table serializes as a
`QueryAuthSplit` with the field ids its rules were authorized for. Other splits
are unchanged.
- Protocol: the REST catalog calls `POST .../tables/{table}/auth`.
### Documentation
Yes. `docs/source/user_guide/catalog.rst` documents the option, the
supported rules, planning and reading a query-auth table, the refused read
paths and the cross-runtime limits of query-auth splits.
### Generative AI tooling
Generated-by: Claude Code (2.1.311, Claude Opus 5.5)
🤖 Generated with [Claude Code](https://claude.com/claude-code)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]