[
https://issues.apache.org/jira/browse/IMPALA-10387?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17266411#comment-17266411
]
ASF subversion and git services commented on IMPALA-10387:
----------------------------------------------------------
Commit 9bb7157bf014282c95ab3e233b80d77e00c95b52 in impala's branch
refs/heads/master from stiga-huang
[ https://gitbox.apache.org/repos/asf?p=impala.git;h=9bb7157 ]
IMPALA-10387: Add missing overloads of mask functions used in Ranger default
masking policies
The mask functions in Hive are implemented through GenericUDFs which can
accept an infinite number of function signatures. Impala currently don't
support GenericUDFs. So we provide builtin mask functions with limited
overloads.
This patch adds some missing overloads that could be used by Ranger
default masking policies, e.g. MASK_HASH, MASK_SHOW_LAST_4,
MASK_DATE_SHOW_YEAR, etc.
Tests:
- Add test coverage on all default masking policies applied on all
supported types.
Change-Id: Icf3e70fd7aa9f3b6d6b508b776696e61ec1fcc2e
Reviewed-on: http://gerrit.cloudera.org:8080/16930
Reviewed-by: Quanlong Huang <[email protected]>
Tested-by: Impala Public Jenkins <[email protected]>
> Implement missing overloads of mask functions used in Ranger default masking
> policies
> -------------------------------------------------------------------------------------
>
> Key: IMPALA-10387
> URL: https://issues.apache.org/jira/browse/IMPALA-10387
> Project: IMPALA
> Issue Type: New Feature
> Components: Frontend
> Reporter: Quanlong Huang
> Assignee: Quanlong Huang
> Priority: Major
>
> Some meaningless overloads of the mask functions are not implemented in
> Impala since Hive always returns NULL on them. It's expected to use the
> mask_null function (Nullify policy) directly.
> However, it's common for users to configure Redact policy (mask) on all
> columns. If there are float/double/decimal columns, they will hit errors like
> this in Impala:
> {code:java}
> I1208 22:38:03.880369 8354 jni-util.cc:288]
> b8461c997916681b:83957ca300000000]
> org.apache.impala.common.AnalysisException: No matching function with
> signature: mask(FLOAT).
> at
> org.apache.impala.analysis.FunctionCallExpr.analyzeImpl(FunctionCallExpr.java:555)
> at org.apache.impala.analysis.Expr.analyze(Expr.java:438)
> at org.apache.impala.analysis.Expr.analyze(Expr.java:432)
> at
> org.apache.impala.analysis.SelectStmt$SelectAnalyzer.analyzeSelectClause(SelectStmt.java:276)
> at
> org.apache.impala.analysis.SelectStmt$SelectAnalyzer.analyze(SelectStmt.java:215)
> at
> org.apache.impala.analysis.SelectStmt$SelectAnalyzer.access$100(SelectStmt.java:199)
> at org.apache.impala.analysis.SelectStmt.analyze(SelectStmt.java:192)
> at
> org.apache.impala.analysis.InlineViewRef.analyze(InlineViewRef.java:218)
> at org.apache.impala.analysis.FromClause.analyze(FromClause.java:78)
> at
> org.apache.impala.analysis.SelectStmt$SelectAnalyzer.analyze(SelectStmt.java:213)
> at
> org.apache.impala.analysis.SelectStmt$SelectAnalyzer.access$100(SelectStmt.java:199)
> at org.apache.impala.analysis.SelectStmt.analyze(SelectStmt.java:192)
> at
> org.apache.impala.analysis.AnalysisContext.analyze(AnalysisContext.java:473)
> at
> org.apache.impala.analysis.AnalysisContext.analyzeAndAuthorize(AnalysisContext.java:437)
> at
> org.apache.impala.service.Frontend.doCreateExecRequest(Frontend.java:1530)
> at
> org.apache.impala.service.Frontend.getTExecRequest(Frontend.java:1497)
> at
> org.apache.impala.service.Frontend.createExecRequest(Frontend.java:1467)
> at
> org.apache.impala.service.JniFrontend.createExecRequest(JniFrontend.java:154)
> {code}
> We'd better add these overloads to avoid hitting such errors in simple setup.
> These patterns are used by Ranger default masking policies:
> * mask(col)
> * mask_show_last_n(col, 4, 'x', 'x', 'x', -1, '1')
> * mask_show_first_n(col, 4, 'x', 'x', 'x', -1, '1')
> * mask_hash(col)
> * mask(col, 'x', 'x', 'x', -1, '1', 1, 0, -1)
> We should have overloads for {{col}} to be any types.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
