nevzheng commented on code in PR #13879:
URL: https://github.com/apache/iceberg/pull/13879#discussion_r3539544517
##########
open-api/rest-catalog-open-api.yaml:
##########
@@ -3663,6 +3672,313 @@ components:
additionalProperties:
type: string
+ ReadRestrictions:
+ type: object
+ description: >
+ Read restrictions for a table.
+
+ A reader evaluates the row filter against original, untransformed
column
+ values, then applies required-column-projections to the surviving
rows.
+ Each action must produce a value of the same type as the input
column.
+ If a reader cannot apply any returned restriction (a filter
expression
+ or an action), it must fail the query and must not silently return
raw,
+ partial, or empty results.
+
+ If a projection targets a nested-typed field (struct, list, or map),
+ other projections in the same ReadRestrictions must not target any
+ nested field-id (struct subfields, list elements, or map keys/values)
+ at any depth. This avoids ambiguity about which action governs a
given
+ leaf value.
+
+ An empty ReadRestrictions object (no required-column-projections and
no
+ required-row-filter) imposes no restrictions and is equivalent to the
+ field being absent from the response.
+ example:
+ required-column-projections:
+ - field-id: 4
+ action: show-last-4
+ - field-id: 6
+ action: replace-with-null
+ - field-id: 8
+ action: truncate-to-year
+ - field-id: 10
+ action: sha-256-global
+ - field-id: 12
+ action: mask-alphanum
+ required-row-filter:
+ type: eq
+ term: region
+ value: US
+ properties:
+ required-column-projections:
+ description: >
+ A list of columns that require specific actions to be applied when
reading.
+ A server must not return an action for a column whose type is not
listed in
+ that action's "Applicable to" set. If absent or empty, no required
actions
+ apply; columns not listed are not subject to any required action.
+
+ 1. For each column listed, the reader must apply the specified
action before
+ returning values for that column.
+
+ 2. The reader must replace all output references to the column
with the result
+ of the action, presenting the result under the original
field-id. For
+ example, if the action for field-id `9` is mask-alphanum, the
reader must
+ return the masked value as field-id `9` in the query output.
+
+ 3. A server must not return more than one projection for the same
field-id
+ in required-column-projections. If a duplicate field-id appears,
the reader
+ must fail the query.
+
+ 4. A projection must not target a map's key field-id. Applying an
action
+ to keys can produce duplicate or null keys, which readers
silently
+ coalesce or reject, causing data loss.
+
+ 5. The reader must fail the query if a projection references a
field-id
+ that is not present in the read schema.
Review Comment:
We discussed this in sync. My understanding is that we're concerned about
what happens when projection schema and read schema differ. Here's a table to
illustrate.
| # | Field-id in projection schema? | Field-id in read schema? | example
When this happens | Recommendation |
|---|-------------------------------|--------------------------|-------------------|----------------|
| 1 | ✓ Yes | ✓ Yes | Normal read | Enforce policy |
| 2 | ✓ Yes | ✗ No | Time travel to a snapshot predating the column | Fail
the query — a policy that favors reads would need to define how to fill in
missing columns. Recommend deferring; [BigQuery takes the same
position](https://docs.cloud.google.com/bigquery/docs/column-level-security-intro#time-travel)
and fails this case. |
| 3 | ✗ No | ✓ Yes | Column dropped via `ALTER TABLE`, reading a historical
snapshot where the column still existed | Allow read — not selecting data that
no longer exists is itself a valid form of protection (@rdblue) |
| 4 | ✗ No | ✗ No | n/a included for completeness | n/a |
I'd like to recommend the following spec text.
```txt
(5) The reader MUST fail the query if a projection references a field-id
that is not present in the snapshot schema.
// Option for additional clarity. To allow reading past snapshots.
(6) The reader MUST skip enforcement if a field-id is present in the
snapshot schema but not referenced by the projection.
```
cc: @singhpk234.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
