Internal Jenkins has submitted this change and it was merged. Change subject: IMPALA-3459: Add test for DROP TABLE PURGE for S3 ......................................................................
IMPALA-3459: Add test for DROP TABLE PURGE for S3 It was previously thought that PURGE had no effect on S3. However, the Hive Metastore actually created a .Trash directory and copied the files there when a DROP TABLE was conducted from Impala. This patch just enables the existing PURGE tests for S3. There were a few reasons this wasn't working before. The paths given to the S3 client (boto3) should not have a leading "/". This has been fixed as it doesn't make a difference for HDFS if that exists or not. Also, PURGE is a pure delete whereas a regular DROP is a copy. A copy is consistent whereas a delete is only eventually consistent, so when we PURGE a table or partition, the files will still be visible for sometime after the query has completed. The tests have been modified to accomodate for this case as well. Change-Id: I52d2451e090b00ae2fd9a879c28defa6c940047c Reviewed-on: http://gerrit.cloudera.org:8080/3036 Reviewed-by: Sailesh Mukil <[email protected]> Tested-by: Internal Jenkins --- M tests/common/skip.py M tests/metadata/test_ddl.py 2 files changed, 52 insertions(+), 44 deletions(-) Approvals: Internal Jenkins: Verified Sailesh Mukil: Looks good to me, approved -- To view, visit http://gerrit.cloudera.org:8080/3036 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-MessageType: merged Gerrit-Change-Id: I52d2451e090b00ae2fd9a879c28defa6c940047c Gerrit-PatchSet: 4 Gerrit-Project: Impala Gerrit-Branch: cdh5-trunk Gerrit-Owner: Sailesh Mukil <[email protected]> Gerrit-Reviewer: Alex Behm <[email protected]> Gerrit-Reviewer: Dan Hecht <[email protected]> Gerrit-Reviewer: Internal Jenkins Gerrit-Reviewer: Sailesh Mukil <[email protected]>
