Quanlong Huang has uploaded this change for review. ( http://gerrit.cloudera.org:8080/17199
Change subject: IMPALA-9661: Avoid introducing unused columns in table masking view ...................................................................... IMPALA-9661: Avoid introducing unused columns in table masking view Previously, if a table has column masking policies, we replace its unanalyzed TableRef with an analyzed InlineViewRef (table masking view) in FromClause.analyze(). However, we can't detect which columns are actually used in the original query at this point. In fact, analyze() for SelectList, WhereClause, GroupByClause and other clauses containing SlotRefs happen after FromClause.analyze(). After the whole query block is analyzed, we can get the exact set of required columns. This patch refactor the codes to do table masking after analyze() to avoid introducing unused columns. Referenced columns of a TableRef are registered in analyze(), which helps to figure out what columns are actually needed. Tests: - Run column masking and row filtering tests in test_ranger.py - Run FE audit tests Change-Id: Ib015a8ab528065907b27fbdceb8e2818deb814e1 --- M fe/src/main/java/org/apache/impala/analysis/AlterViewStmt.java M fe/src/main/java/org/apache/impala/analysis/AnalysisContext.java M fe/src/main/java/org/apache/impala/analysis/Analyzer.java M fe/src/main/java/org/apache/impala/analysis/CreateTableAsSelectStmt.java M fe/src/main/java/org/apache/impala/analysis/CreateViewStmt.java M fe/src/main/java/org/apache/impala/analysis/Expr.java M fe/src/main/java/org/apache/impala/analysis/FromClause.java M fe/src/main/java/org/apache/impala/analysis/InlineViewRef.java M fe/src/main/java/org/apache/impala/analysis/InsertStmt.java M fe/src/main/java/org/apache/impala/analysis/Path.java M fe/src/main/java/org/apache/impala/analysis/QueryStmt.java M fe/src/main/java/org/apache/impala/analysis/SelectStmt.java M fe/src/main/java/org/apache/impala/analysis/SetOperationStmt.java M fe/src/main/java/org/apache/impala/analysis/SlotRef.java M fe/src/main/java/org/apache/impala/analysis/StmtNode.java M fe/src/main/java/org/apache/impala/analysis/Subquery.java M fe/src/main/java/org/apache/impala/analysis/TableRef.java M fe/src/main/java/org/apache/impala/analysis/WithClause.java M fe/src/main/java/org/apache/impala/authorization/TableMask.java M fe/src/test/java/org/apache/impala/authorization/ranger/RangerAuditLogTest.java 20 files changed, 267 insertions(+), 226 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/99/17199/1 -- To view, visit http://gerrit.cloudera.org:8080/17199 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newchange Gerrit-Change-Id: Ib015a8ab528065907b27fbdceb8e2818deb814e1 Gerrit-Change-Number: 17199 Gerrit-PatchSet: 1 Gerrit-Owner: Quanlong Huang <[email protected]>
