Impala Public Jenkins has submitted this change and it was merged. (
http://gerrit.cloudera.org:8080/19279 )
Change subject: IMPALA-11744: Table mask view should preserve the original
column order in Hive
......................................................................
IMPALA-11744: Table mask view should preserve the original column order in Hive
Ranger provides column masking and row filtering policies to mask
sensitive data for specific users/groups. When a table should be masked
in a query, Impala replaces it with a table mask view that exposes the
columns with masked expressions.
After IMPALA-9661, only selected columns are exposed in the table mask
view. However, the columns of the view are exposed in the order that
they are registered. If the registering order differs from the column
order in the table, STAR expansions will mismatch the columns.
To be specific, let's say table 'tbl' with 3 columns a, b, c should be
masked in the following query:
select b, * from tbl;
Ideally Impala should replace the TableRef of 'tbl' with a table mask
view as:
select b, * from (
select mask(a) a, mask(b) b, mask(c) c from tbl
) t;
Currently, the rewritten query is
select b, * from (
select mask(b) b, mask(a) a, mask(c) c from tbl
) t;
This incorrectly expands the STAR as "b, a, c" in the re-analyze phase.
The cause is that column 'b' is registered earlier than all other
columns. This patch fixes it by sorting the selected columns based on
their original order in the table.
Tests:
- Add tests for selecting STAR with normal columns on table and view.
Change-Id: Ic83d78312b19fa2c5ab88ac4f359bfabaeaabce6
Reviewed-on: http://gerrit.cloudera.org:8080/19279
Reviewed-by: Impala Public Jenkins <[email protected]>
Tested-by: Impala Public Jenkins <[email protected]>
---
M fe/src/main/java/org/apache/impala/analysis/Analyzer.java
M fe/src/main/java/org/apache/impala/analysis/InlineViewRef.java
M fe/src/main/java/org/apache/impala/analysis/TableRef.java
M
testdata/workloads/functional-query/queries/QueryTest/ranger_column_masking.test
M
testdata/workloads/functional-query/queries/QueryTest/ranger_column_masking_and_row_filtering.test
M
testdata/workloads/functional-query/queries/QueryTest/ranger_row_filtering.test
6 files changed, 474 insertions(+), 6 deletions(-)
Approvals:
Impala Public Jenkins: Looks good to me, approved; Verified
--
To view, visit http://gerrit.cloudera.org:8080/19279
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: merged
Gerrit-Change-Id: Ic83d78312b19fa2c5ab88ac4f359bfabaeaabce6
Gerrit-Change-Number: 19279
Gerrit-PatchSet: 6
Gerrit-Owner: Quanlong Huang <[email protected]>
Gerrit-Reviewer: Daniel Becker <[email protected]>
Gerrit-Reviewer: Fang-Yu Rao <[email protected]>
Gerrit-Reviewer: Impala Public Jenkins <[email protected]>
Gerrit-Reviewer: Qifan Chen <[email protected]>
Gerrit-Reviewer: Quanlong Huang <[email protected]>
