Hello Csaba Ringhofer, Impala Public Jenkins,
I'd like you to reexamine a change. Please visit
http://gerrit.cloudera.org:8080/24530
to look at the new patch set (#2).
Change subject: IMPALA-15130: Reject column masking on complex-typed columns
......................................................................
IMPALA-15130: Reject column masking on complex-typed columns
A Ranger column-masking policy on a top-level complex-typed column
(ARRAY/MAP/STRUCT) cannot be applied: the mask transforms are scalar
and there is no way to express a masked complex value.
TableMask.createColumnMaskStmt() previously ignored such masks and
returned the column unmasked, leaking data the policy was meant to hide.
The InlineViewRef guard that was supposed to reject this ("Column
masking is not supported for complex types") was dead code as
TableMask.hasComplexColumnMask_ was never assigned.
Fail closed instead: a mask on a top-level complex column now throws an
AnalysisException. Masks on nested struct fields (e.g. nested_struct.a)
are still ignored, matching Hive, which only considers top-level columns
and so does not recognize such policies either.
Also removes the dead hasComplexColumnMask_ field/getter and the
unreachable InlineViewRef check.
Testing:
- New FE test AuthorizationStmtTest.testColumnMaskingOnComplexColumn:
a mask on a top-level ARRAY, MAP and STRUCT column is rejected, while a
mask on a nested struct field is still ignored.
- test_column_masking_with_structs_in_select_list keeps the masks on the
complex columns (tiny_struct, int_array_array, int_map_array) and the
ranger_column_masking_struct_in_select_list.test goldens now assert
that selecting those columns fails with the new error. This shows the
end-to-end behaviour next to the related masking tests in that file.
- test_column_masking: dropped only the mask on the top-level int_array
column (that column is exercised as a nested collection reference all
over ranger_column_masking_complex_types.test and would now error). The
mask on the nested field nested_struct.a is kept, since nested-field
masks are still ignored (Hive parity), so those goldens are unchanged.
Change-Id: Idddacf0b40abcaa18a5764ff617179fe898886c9
Assisted-by: Claude Opus 4.8 (1M context) <[email protected]>
---
M fe/src/main/java/org/apache/impala/analysis/InlineViewRef.java
M fe/src/main/java/org/apache/impala/authorization/TableMask.java
M fe/src/test/java/org/apache/impala/authorization/AuthorizationStmtTest.java
M
testdata/workloads/functional-query/queries/QueryTest/ranger_column_masking_struct_in_select_list.test
M tests/authorization/test_ranger.py
5 files changed, 99 insertions(+), 42 deletions(-)
git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/30/24530/2
--
To view, visit http://gerrit.cloudera.org:8080/24530
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: Idddacf0b40abcaa18a5764ff617179fe898886c9
Gerrit-Change-Number: 24530
Gerrit-PatchSet: 2
Gerrit-Owner: Zoltan Borok-Nagy <[email protected]>
Gerrit-Reviewer: Csaba Ringhofer <[email protected]>
Gerrit-Reviewer: Impala Public Jenkins <[email protected]>
