flyrain commented on code in PR #13879:
URL: https://github.com/apache/iceberg/pull/13879#discussion_r3532770766
##########
open-api/rest-catalog-open-api.py:
##########
@@ -589,6 +589,154 @@ class StorageCredential(BaseModel):
config: dict[str, str]
+class Action(BaseModel):
+ action: str
+ field_id: int = Field(
+ ..., alias='field-id', description='Field ID of the column being
projected.'
+ )
+
+
+class MaskAlphanum(Action):
+ """
+ Redacts the column value using the following rules to transform Unicode
code points:
+ - Digits (U+0030–U+0039, 0-9) are replaced with 'n' - The following
punctuation characters are kept as-is:
+ U+0028 '(' LEFT PARENTHESIS
+ U+0029 ')' RIGHT PARENTHESIS
+ U+002C ',' COMMA
+ U+002E '.' FULL STOP
+ U+002D '-' HYPHEN-MINUS
+ U+0040 '@' COMMERCIAL AT
+ - All other Unicode characters (including letters, whitespace, and any
punctuation
+ not listed above) are replaced with 'x'
+
+ For example: "[email protected]" -> "[email protected]"
+ NULL input is preserved (NULL -> NULL).
+ Applicable to: string
+
+ """
+
+ action: Literal['mask-alphanum'] | None = None
+
+
+class MaskToFixedValue(Action):
+ """
+ Replaces the column value with a type-specific fixed value. Readers must
use exactly the values listed below to ensure consistency across
implementations.
+ Fixed values by type: - boolean: false - int: 0 - long: 0 - float: 0.0 -
double: 0.0 - decimal(p, s): 0 (the unscaled value is 0) - string: "XXXXXXXX" -
date: 1970-01-01 - time: 00:00:00 - timestamp: 1970-01-01T00:00:00 -
timestamptz: 1970-01-01T00:00:00+00:00 - timestamp_ns:
1970-01-01T00:00:00.000000000 - timestamptz_ns:
1970-01-01T00:00:00.000000000+00:00 - uuid:
00000000-0000-0000-0000-000000000000 - fixed(n): n zero bytes - binary: empty
byte sequence - variant: {} - list: empty list [] - map: empty map {} - struct:
struct with each field set to its type-specific default (applied recursively)
+ NULL input is also replaced with the type-specific fixed value; NULL is
not preserved.
+ Applicable to: all data types except unknown, geometry, and geography.
Because those three types have no fixed value defined above, a catalog server
must not return mask-to-fixed-value for a struct, list, or map that contains a
field of type unknown, geometry, or geography at any nesting depth.
+
+ """
+
+ action: Literal['mask-to-fixed-value'] | None = None
+
+
+class ReplaceWithNull(Action):
+ """
+ Replaces the entire column value with NULL. NULL input is preserved (NULL
-> NULL). A server must not return this action for a required (non-nullable)
column.
+ Applicable to: all optional types
Review Comment:
Nit: this wording may be a bit more coherent by talking about the scope it
applies.
```suggestion
Replaces the entire column value with NULL. NULL input is preserved
(NULL -> NULL).
Applicable to: all optional types. Applying to a required (non-nullable)
column is considered invalid.
```
##########
open-api/rest-catalog-open-api.yaml:
##########
@@ -3663,6 +3672,313 @@ components:
additionalProperties:
type: string
+ ReadRestrictions:
+ type: object
+ description: >
+ Read restrictions for a table.
+
+ A reader evaluates the row filter against original, untransformed
column
+ values, then applies required-column-projections to the surviving
rows.
+ Each action must produce a value of the same type as the input
column.
+ If a reader cannot apply any returned restriction (a filter
expression
+ or an action), it must fail the query and must not silently return
raw,
+ partial, or empty results.
+
+ If a projection targets a nested-typed field (struct, list, or map),
+ other projections in the same ReadRestrictions must not target any
+ nested field-id (struct subfields, list elements, or map keys/values)
+ at any depth. This avoids ambiguity about which action governs a
given
+ leaf value.
+
+ An empty ReadRestrictions object (no required-column-projections and
no
+ required-row-filter) imposes no restrictions and is equivalent to the
+ field being absent from the response.
Review Comment:
Nit: the word "field" seems a bit overloaded here. "field" could mean
"column" or "sub-column". How about this:
```suggestion
A missing or empty ReadRestrictions object (no
required-column-projections and no
required-row-filter) imposes no restrictions.
```
##########
open-api/rest-catalog-open-api.yaml:
##########
@@ -3663,6 +3672,313 @@ components:
additionalProperties:
type: string
+ ReadRestrictions:
+ type: object
+ description: >
+ Read restrictions for a table.
+
+ A reader evaluates the row filter against original, untransformed
column
+ values, then applies required-column-projections to the surviving
rows.
+ Each action must produce a value of the same type as the input
column.
+ If a reader cannot apply any returned restriction (a filter
expression
+ or an action), it must fail the query and must not silently return
raw,
+ partial, or empty results.
+
+ If a projection targets a nested-typed field (struct, list, or map),
+ other projections in the same ReadRestrictions must not target any
+ nested field-id (struct subfields, list elements, or map keys/values)
+ at any depth. This avoids ambiguity about which action governs a
given
+ leaf value.
+
+ An empty ReadRestrictions object (no required-column-projections and
no
+ required-row-filter) imposes no restrictions and is equivalent to the
+ field being absent from the response.
+ example:
+ required-column-projections:
+ - field-id: 4
+ action: show-last-4
+ - field-id: 6
+ action: replace-with-null
+ - field-id: 8
+ action: truncate-to-year
+ - field-id: 10
+ action: sha-256-global
+ - field-id: 12
+ action: mask-alphanum
+ required-row-filter:
+ type: eq
+ term: region
+ value: US
+ properties:
+ required-column-projections:
+ description: >
+ A list of columns that require specific actions to be applied when
reading.
+ A server must not return an action for a column whose type is not
listed in
+ that action's "Applicable to" set. If absent or empty, no required
actions
+ apply; columns not listed are not subject to any required action.
+
+ 1. For each column listed, the reader must apply the specified
action before
+ returning values for that column.
+
+ 2. The reader must replace all output references to the column
with the result
+ of the action, presenting the result under the original
field-id. For
+ example, if the action for field-id `9` is mask-alphanum, the
reader must
+ return the masked value as field-id `9` in the query output.
+
+ 3. A server must not return more than one projection for the same
field-id
+ in required-column-projections. If a duplicate field-id appears,
the reader
+ must fail the query.
+
+ 4. A projection must not target a map's key field-id. Applying an
action
+ to keys can produce duplicate or null keys, which readers
silently
+ coalesce or reject, causing data loss.
+
+ 5. The reader must fail the query if a projection references a
field-id
+ that is not present in the read schema.
+ type: array
+ items:
+ $ref: '#/components/schemas/Action'
+ required-row-filter:
+ description: >
+ An expression that filters rows in the table that the
authenticated principal does not have access to.
+
+ 1. The expression must evaluate to a boolean (TRUE or FALSE;
Iceberg predicates
+ never produce NULL). A reader must discard any row for which the
filter
+ evaluates to FALSE, and no information derived from discarded
rows may be
+ included in the query result.
+
+ 2. If this property is absent, null, or always true then no
mandatory filtering is required.
+ allOf:
+ - $ref: '#/components/schemas/Expression'
+
+ Action:
+ discriminator:
+ propertyName: action
+ mapping:
Review Comment:
Just to confirm: do we plan to add UDF as a new type of action?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
