j1wonpark opened a new issue, #7576: URL: https://github.com/apache/kyuubi/issues/7576
### Code of Conduct - [X] I agree to follow this project's [Code of Conduct](https://www.apache.org/foundation/policies/conduct) ### Search before asking - [X] I have searched in the [issues](https://github.com/apache/kyuubi/issues?q=is%3Aissue) and found no similar issues. ### Describe the bug Reading a permanent view that UNION-ALLs a masked permanent view fails analysis, while reading the masked view directly works (#3581): ``` [MISSING_ATTRIBUTES.RESOLVED_ATTRIBUTE_APPEAR_IN_OPERATION] Resolved attribute(s) "value2" missing from "value2" in operator !Project [cast(value2#613 as string) AS value2#611]. ``` With a view-level, type-preserving MASK policy on `perm_view_masked.value2`: ```sql CREATE OR REPLACE VIEW default.perm_view_masked AS SELECT key, value2 FROM default.src; CREATE OR REPLACE VIEW default.perm_view_union AS SELECT key, value2 FROM default.perm_view_masked WHERE key = 1 UNION ALL SELECT key, value2 FROM default.perm_view_masked WHERE key = 10; -- as the masked user SELECT key, value2 FROM default.perm_view_union WHERE key = 1; -- FAIL SELECT key, value2 FROM default.perm_view_masked WHERE key = 1; -- OK ``` The same view structure without a masking policy resolves fine, so the union/view shape itself is handled by Spark — only the masking rewrite breaks it. Root cause (confirmed with `spark.sql.planChangeLog`): both UNION branches expand the masked view with identical output exprIds. After `RuleApplyDataMaskingStage0` marks both branches, `DeduplicateRelations` re-instances the second branch's `PermanentViewMarker`, but `DataMaskingStage0Marker.scan` is not a child and keeps the pre-dedup ids, so the second branch's `exprToMaskers()` becomes keyed on ids now owned by the *first* branch. `RuleApplyDataMaskingStage1` then merges all marker maps into one `toMap` (duplicate keys silently last-wins) and substitutes unconditionally, rewiring the outer view's schema compensation Project — above the Union — to attributes the Union never outputs. Fix: scope the Stage1 substitution — heal children first, then only substitute maskers the healed children actually expose, merging the per-marker maps after that filtering. Branch-local healing keeps working; the cross-branch leak is blocked at the Union boundary. All five data masking suites and the full kyuubi-spark-authz module pass with the fix. ### Affects Version(s) master ### Kyuubi Server Log Output _No response_ ### Kyuubi Engine Log Output _No response_ ### Kyuubi Server Configurations _No response_ ### Kyuubi Engine Configurations _No response_ ### Additional context _No response_ ### Are you willing to submit PR? - [X] Yes. I would be willing to submit a PR with guidance from the Kyuubi community to fix. - [ ] No. I cannot submit a PR at this time. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
