j1wonpark opened a new pull request, #7577:
URL: https://github.com/apache/kyuubi/pull/7577
### Why are the changes needed?
Close #7576.
Reading a permanent view that UNION-ALLs a masked permanent view fails with
`MISSING_ATTRIBUTES`, while reading the masked view directly works (#3581).
Both branches expand the masked view with identical output exprIds. After
`RuleApplyDataMaskingStage0` marks both branches, `DeduplicateRelations`
re-instances one branch, so that branch's `exprToMaskers()` ends up keyed on
ids now owned by the other branch. `RuleApplyDataMaskingStage1` merges all
marker maps into one `Map` (duplicate keys silently last-wins) and substitutes
unconditionally, rewiring the outer view's schema compensation Project — above
the Union — to attributes the Union never outputs. See #7576 for the full
analysis.
This change scopes the Stage1 substitution: rewrite the children first, keep
only the maskers the rewritten children actually expose (matched by exprId,
since `SubqueryAlias` may requalify outputs), and merge the per-marker maps
after that filtering. When a child is not resolved yet (a type-changing masker
leaves Union branches incompatible until type coercion runs, and `Union.output`
fails on them), the previous unfiltered substitution is kept. Branch-local
substitution keeps working; the cross-branch leak is blocked at the Union
boundary.
### How was this patch tested?
Two tests added to `DataMaskingTestBase` (run by all data masking suites): a
reproduction that failed with `MISSING_ATTRIBUTES` before this fix, and a
control proving the same view shape resolves fine without a masking policy. A
view-level type-preserving MASK policy is added to the fixture, since a
type-changing mask dies earlier with `CANNOT_UP_CAST` and would hide the bug.
```
build/mvn test -pl extensions/spark/kyuubi-spark-authz -Dtest=none \
-DwildcardSuites=org.apache.kyuubi.plugin.spark.authz.ranger.datamasking.DataMaskingForHiveParquetSuite
```
All five data masking suites pass, and the full kyuubi-spark-authz module is
green (662 succeeded / 0 failed / 42 pre-existing skips).
### Was this patch authored or co-authored using generative AI tooling?
Assisted-by: Claude:claude-fable-5
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
