[
https://issues.apache.org/jira/browse/IMPALA-15130?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=18093410#comment-18093410
]
ASF subversion and git services commented on IMPALA-15130:
----------------------------------------------------------
Commit af1928236e53eef7e372168e47403bc67e05549a in impala's branch
refs/heads/master from Zoltan Borok-Nagy
[ https://gitbox.apache.org/repos/asf?p=impala.git;h=af1928236 ]
IMPALA-15130: Reject column masking on complex-typed columns
A Ranger column-masking policy on a top-level complex-typed column
(ARRAY/MAP/STRUCT) cannot be applied: the mask transforms are scalar
and there is no way to express a masked complex value.
TableMask.createColumnMaskStmt() previously ignored such masks and
returned the column unmasked, leaking data the policy was meant to hide.
The InlineViewRef guard that was supposed to reject this ("Column
masking is not supported for complex types") was dead code as
TableMask.hasComplexColumnMask_ was never assigned.
Fail closed instead: a mask on a top-level complex column now throws an
AnalysisException. Masks on nested struct fields (e.g. nested_struct.a)
are still ignored, matching Hive, which only considers top-level columns
and so does not recognize such policies either.
Also removes the dead hasComplexColumnMask_ field/getter and the
unreachable InlineViewRef check.
Testing:
- New FE test AuthorizationStmtTest.testColumnMaskingOnComplexColumn:
a mask on a top-level ARRAY, MAP and STRUCT column is rejected, while a
mask on a nested struct field is still ignored.
- test_column_masking_with_structs_in_select_list keeps the masks on the
complex columns (tiny_struct, int_array_array, int_map_array) and the
ranger_column_masking_struct_in_select_list.test goldens now assert
that selecting those columns fails with the new error. This shows the
end-to-end behaviour next to the related masking tests in that file.
- test_column_masking: dropped only the mask on the top-level int_array
column (that column is exercised as a nested collection reference all
over ranger_column_masking_complex_types.test and would now error). The
mask on the nested field nested_struct.a is kept, since nested-field
masks are still ignored (Hive parity), so those goldens are unchanged.
Change-Id: Idddacf0b40abcaa18a5764ff617179fe898886c9
Assisted-by: Claude Opus 4.8 (1M context) <[email protected]>
Reviewed-on: http://gerrit.cloudera.org:8080/24530
Reviewed-by: Csaba Ringhofer <[email protected]>
Tested-by: Impala Public Jenkins <[email protected]>
> Raise error when top-level complex column requires masking
> ----------------------------------------------------------
>
> Key: IMPALA-15130
> URL: https://issues.apache.org/jira/browse/IMPALA-15130
> Project: IMPALA
> Issue Type: Bug
> Components: Frontend
> Reporter: Zoltán Borók-Nagy
> Assignee: Zoltán Borók-Nagy
> Priority: Major
>
> When a Ranger column-masking policy targets a top-level complex-typed column
> (ARRAY/MAP/STRUCT), Impala silently ignores the mask.
> This is deliberate today but unsafe: TableMask.createColumnMaskStmt()
> (fe/.../authorization/TableMask.java, ~lines 90-99) drops the mask for complex
> types (and for nested children) and just logs a warning:
> if (maskedValue == null || maskedValue.equals(colName)) { // Don't need
> masking.
> return null;
> }
> if (colType.isComplexType() || colName.contains(".")) {
> // Ignore column masks on complex types or their children.
> // TODO: RANGER-3525: Clarify handling of column masks on nested types
> LOG.warn("Ignoring column mask on complex type {}.{}: {} => {}", ...);
> return null; // -> column is passed through UNMASKED
> }
> Hive allows NULL-masking on top-level columns.
> We should at least raise an error in such cases instead of ignoring the
> masking policy.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
