Fang-Yu Rao created IMPALA-9148:
-----------------------------------
Summary: AuthorizationStmtTest.testColumnMaskEnabled seems flaky
Key: IMPALA-9148
URL: https://issues.apache.org/jira/browse/IMPALA-9148
Project: IMPALA
Issue Type: Bug
Components: Frontend
Reporter: Fang-Yu Rao
Assignee: Fang-Yu Rao
Recently we have seen failed {{AuthorizationStmtTest.testColumnMaskEnabled}}
occasionally with the following error messages (e.g.,
[https://jenkins.impala.io/job/gerrit-verify-dryrun/5194/consoleFull]).
{code:java}
Impala does not support row filtering yet. Row filtering is enabled on table:
functional.alltypes_view
expected:
Impala does not support column masking yet. Column masking is enabled on
column: functional.alltypes_view.string_col
{code}
Taking a look at the {{testColumnMaskEnabled()}}, we can see the related SQL
statement is
{code:java}
select string_col from functional.alltypes_view;
{code}
I found that for this SQL statement, {{authorizeRowFilterAndColumnMask()}} in
{{RangerAuthorizationCheker.java}} will be called first
([https://github.com/apache/impala/blob/master/fe/src/main/java/org/apache/impala/authorization/ranger/RangerAuthorizationChecker.java#L183-L200]).
There will be two privilege requests, one request for column, and the other
for table. The function {{authorizeRowFilter()}} is the only function that
could produce the error message above
([https://github.com/apache/impala/blob/master/fe/src/main/java/org/apache/impala/authorization/ranger/RangerAuthorizationChecker.java#L295-L308]).
Specifically, this error would be generated if
{{plugin_.evalRowFilterPolicies(req, null).isRowFilterEnabled()}} returns true
([https://github.com/apache/impala/blob/master/fe/src/main/java/org/apache/impala/authorization/ranger/RangerAuthorizationChecker.java#L303]).
I have taken a brief look at {{isRowFilterEnabled()}}, and found that it will
return true only if there is some policy on the Ranger server that specifies
the policy of row filtering (according to my current understanding). However,
in {{testColumnMaskEnabled()}}
([https://github.com/apache/impala/blob/master/fe/src/test/java/org/apache/impala/authorization/AuthorizationStmtTest.java#L2836]),
we only add a policy for column masking. Therefore, I suspect it may be
possible that some other tests added to the Ranger server some policy for row
filtering but did not properly do the cleanup of this row filtering policy
afterwards.
To address this issue, we should add some logic to clean up the policies stored
on the Ranger server before running this JUnit test. This JUnit test assumes
that the Ranger server does not store any policies related to column masking
and row filtering before the testing.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
