Fang-Yu Rao has uploaded a new patch set (#5). ( http://gerrit.cloudera.org:8080/14798 )
Change subject: IMPALA-9149: part 1: Re-enabe Ranger-related FE tests ...................................................................... IMPALA-9149: part 1: Re-enabe Ranger-related FE tests In IMPALA-9047, we disabled some Ranger-related FE and BE tests due to changes in Ranger's behavior after upgrading Ranger from 1.2 to 2.0. This patch aims to re-enable those disabled FE tests in AuthorizationStmtTest.java and RangerAuditLogTest.java to increase Impala's test coverage of authorization via Ranger. There are at least two major changes in Ranger's behavior in the newer versions. 1. The first is that the owner of the requested resource no longer have to be explicitly granted privileges in order to access the resource. 2. The second is that a user not explicitly granted the privilege of creating a database is able to do so. Due to these changes, some of previous Ranger authorization requests that were expected to be rejected are now granted after the upgrade. To re-enable the tests affected by the first change described above, we modify AuthorizationTestBase.java to allow our FE Ranger authorization tests to specify the requesting user in an authorization test. Those tests failed after the upgrade because the default requesting user in Impala's AuthorizationTestBase.java happens to be the owner of the resources involved in our FE authorization tests. After this patch, a requesting user can be either a non-owner user or an owner user in a Ranger authorization test and the requesting user would correspond to a non-owner user if it is not explicitly specified. Note that in a Sentry authorization test, we do not use the non-owner user as the requesting user by default as we do in the Ranger authorization tests. Instead, we set the name of the requesting user to the name of the owner user in Ranger authorization tests to avoid the complexity of having to provide a customized group mapping service when instantiating a Sentry ResourceAuthorizationProvider as we do in AuthorizationTest.java, our FE tests specifically for testing authorization via Sentry. On the other hand, for those tests affected by the second change in AuthorizationStmtTest.java, in this patch we will only run them when the authorization provider is Sentry. For the affected test in RangerAuditLogTest.java, we now expect the test query to be successfully authorized. Apart from the affected tests mentioned above, we have also found several test queries that are expected to fail but result in different authorization error from Sentry and Ranger, which had not been seen in the previous version of Ranger, i.e., each of those test queries resulted in the same error message from both Sentry and Ranger before the upgrade. For now we modify the tests accordingly to make the expected error message match the actual error message, but we should take a closer look at those queries and see if there is any potential bug. Testing: - Passed AuthorizationStmtTest.java - Passed RangerAuditLogTest.java Change-Id: I228533aae34b9ac03bdbbcd51a380770ff17c7f2 --- M fe/src/main/java/org/apache/impala/authorization/ranger/RangerAuthorizationChecker.java M fe/src/main/java/org/apache/impala/authorization/sentry/SentryAuthProvider.java M fe/src/test/java/org/apache/impala/authorization/AuthorizationStmtTest.java M fe/src/test/java/org/apache/impala/authorization/AuthorizationTestBase.java M fe/src/test/java/org/apache/impala/authorization/ranger/RangerAuditLogTest.java M fe/src/test/java/org/apache/impala/common/FrontendFixture.java M fe/src/test/java/org/apache/impala/common/FrontendTestBase.java M testdata/bin/create-load-data.sh A testdata/cluster/ranger/setup/impala_group_non_owner.json R testdata/cluster/ranger/setup/impala_group_owner.json.template A testdata/cluster/ranger/setup/impala_user_non_owner.json.template R testdata/cluster/ranger/setup/impala_user_owner.json.template 12 files changed, 342 insertions(+), 244 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/98/14798/5 -- To view, visit http://gerrit.cloudera.org:8080/14798 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: I228533aae34b9ac03bdbbcd51a380770ff17c7f2 Gerrit-Change-Number: 14798 Gerrit-PatchSet: 5 Gerrit-Owner: Fang-Yu Rao <[email protected]> Gerrit-Reviewer: Fang-Yu Rao <[email protected]> Gerrit-Reviewer: Impala Public Jenkins <[email protected]> Gerrit-Reviewer: Kurt Deschler <[email protected]> Gerrit-Reviewer: Quanlong Huang <[email protected]>
