plusplusjiajia opened a new pull request, #10904: URL: https://github.com/apache/paimon/pull/10904
### Purpose With `table-read.sequence-number.enabled`, `$audit_log` reads primary-key data through `KeyValueTableRead`, whose merge reads prepend `_SEQUENCE_NUMBER` to each row (`KeyValueTableRead.unwrap`). `ReadTransform` applied the query auth row filter, the column masks and the query filter by the read type without that column, so every field was one position off. For example, on a primary-key table: - a row filter `a <= 1` returned all 10 rows instead of 2, and `a >= 8` returned none; - any column mask failed with a `ClassCastException`. The option can be set per query, so a restricted user could use it to read rows the row filter hides. When the reader that `KeyValueTableRead` selects for a split emits the sequence number (merge and incremental reads, which go through `unwrap`), the read transform now includes the column in its read and output types and remaps the query filter, so the rules apply to the right columns. Raw-file and indexed reads of primary-key tables, and append and Data Evolution tables, never emit the column and are unchanged, even if the internal `key-value.sequence_number.enabled` option is passed dynamically. ### Tests - `RESTCatalogTest#testAuditLogQueryAuthWithSequenceNumber`: on a primary-key table, `$audit_log` with and without sequence numbers applies the row filter, the masks and a query filter on a masked column, for full and projected reads, and keeps the sequence numbers. - `RESTCatalogTest#testPrimaryKeyRawQueryAuthWithSequenceNumberOption`: a raw-convertible primary-key split read with the internal option still applies the row filter and masks. - `RESTCatalogTest#testAppendQueryAuthWithSequenceNumberOption`: append and Data Evolution tables read with the internal option still apply the row filter and masks. - The `$audit_log` cases with sequence numbers fail on master; the raw-file and append cases guard against shifting reads that do not emit the column. The related tests pass locally: `MockRESTCatalogTest`, `AbstractDataTableReadTest`, `AuditLogTableTest`, `BinlogTableTest`, `TableQueryAuthResultTest`, `QueryAuthSplitTest`, `ValueContentRowDataRecordIteratorTest` and `NestedFieldQueryAuthTest`, 258 tests. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
