Quanlong Huang has submitted this change and it was merged. ( http://gerrit.cloudera.org:8080/19373 )
Change subject: IMPALA-10399, IMPALA-11060, IMPALA-11788: Reset Ranger policy repository in an E2E test ...................................................................... IMPALA-10399, IMPALA-11060, IMPALA-11788: Reset Ranger policy repository in an E2E test test_show_grant_hive_privilege() uses Ranger's REST API to get all the existing policies from the Ranger server after creating a policy that grants the LOCK and SELECT privileges on all the tables and columns in the unique database in order to verify the granted privileges indeed exist in Ranger's policy repository. The way we download all the policies from the Ranger server in test_show_grant_hive_privilege(), however, did not always work. Specifically, when there were already a lot of existing policies in Ranger, the policy that granted the LOCK and SELECT privileges would not be included in the result returned via one single GET request. We found that to reproduce the issue it suffices to add 300 Ranger policies before adding the policy granting those 2 privileges. Moreover, we found that even we set the argument 'stream' of requests.get() to True and used iter_content() to read the response in chunks, we still could not retrieve the policy added in test_show_grant_hive_privilege(). As a workaround, instead of changing how we download all the policies from the Ranger server, this patch resets Ranger's policy repository for Impala before we create the policy granting those 2 privileges so that this test will be more resilient to the number of existing policies in the repository. Change-Id: Iff56ec03ceeb2912039241ea302f4bb8948d61f8 Reviewed-on: http://gerrit.cloudera.org:8080/19373 Tested-by: Impala Public Jenkins <[email protected]> Reviewed-by: Quanlong Huang <[email protected]> --- M testdata/bin/create-load-data.sh A testdata/bin/setup-ranger.sh M tests/authorization/test_ranger.py M tests/common/custom_cluster_test_suite.py 4 files changed, 115 insertions(+), 73 deletions(-) Approvals: Impala Public Jenkins: Verified Quanlong Huang: Looks good to me, approved -- To view, visit http://gerrit.cloudera.org:8080/19373 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: merged Gerrit-Change-Id: Iff56ec03ceeb2912039241ea302f4bb8948d61f8 Gerrit-Change-Number: 19373 Gerrit-PatchSet: 6 Gerrit-Owner: Fang-Yu Rao <[email protected]> Gerrit-Reviewer: Andrew Sherman <[email protected]> Gerrit-Reviewer: Csaba Ringhofer <[email protected]> Gerrit-Reviewer: Fang-Yu Rao <[email protected]> Gerrit-Reviewer: Impala Public Jenkins <[email protected]> Gerrit-Reviewer: Michael Smith <[email protected]> Gerrit-Reviewer: Quanlong Huang <[email protected]>
