[
https://issues.apache.org/jira/browse/HADOOP-13760?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16024130#comment-16024130
]
Aaron Fabbri commented on HADOOP-13760:
---------------------------------------
+1 I think the latest patch looks good.
I've gone over everything--some codepaths twice. I tested on us-west-2 with
local and dynamo: all clear except with -Dauth -Dlocal, but these failures also
happened before your patch. We should file JIRAs on them separately:
{noformat}
ITestS3AContractRename>AbstractContractRenameTest.testRenamePopulatesFileAncestors:261->AbstractContractRenameTest.validateAncestorsMoved:274->AbstractFSContractTestBase.assertPathDoesNotExist:305->Assert.fail:88
src path should not exist: unexpectedly found
s3a://fabbri-dev/test/testRenamePopulatesFileAncestors/source/dir1/dir2/dir3/file4
as
S3AFileStatus{path=s3a://fabbri-dev/test/testRenamePopulatesFileAncestors/source/dir1/dir2/dir3/file4;
isDirectory=false; length=256; replication=1; blocksize=33554432;
modification_time=1495676948133; access_time=0; owner=fabbri; group=fabbri;
permission=rw-rw-rw-; isSymlink=false; hasAcl=false; isEncrypted=false;
isErasureCoded=false} isEmptyDirectory=FALSE
Tests in error:
ITestS3AContractGetFileStatus>AbstractContractGetFileStatusTest.testListLocatedStatusFiltering:517->AbstractContractGetFileStatusTest.verifyListLocatedStatusNextCalls:578
» IndexOutOfBounds
ITestS3AContractGetFileStatus>AbstractContractGetFileStatusTest.testComplexDirActions:144->AbstractContractGetFileStatusTest.checkListLocatedStatusStatusComplexDir:186
» IndexOutOfBounds
{noformat}
> S3Guard: add delete tracking
> ----------------------------
>
> Key: HADOOP-13760
> URL: https://issues.apache.org/jira/browse/HADOOP-13760
> Project: Hadoop Common
> Issue Type: Sub-task
> Components: fs/s3
> Reporter: Aaron Fabbri
> Assignee: Sean Mackrory
> Attachments: HADOOP-13760-HADOOP-13345.001.patch,
> HADOOP-13760-HADOOP-13345.002.patch, HADOOP-13760-HADOOP-13345.003.patch,
> HADOOP-13760-HADOOP-13345.004.patch, HADOOP-13760-HADOOP-13345.005.patch,
> HADOOP-13760-HADOOP-13345.006.patch, HADOOP-13760-HADOOP-13345.007.patch,
> HADOOP-13760-HADOOP-13345.008.patch, HADOOP-13760-HADOOP-13345.009.patch,
> HADOOP-13760-HADOOP-13345.010.patch, HADOOP-13760-HADOOP-13345.011.patch,
> HADOOP-13760-HADOOP-13345.012.patch, HADOOP-13760-HADOOP-13345.013.patch
>
>
> Following the S3AFileSystem integration patch in HADOOP-13651, we need to add
> delete tracking.
> Current behavior on delete is to remove the metadata from the MetadataStore.
> To make deletes consistent, we need to add a {{isDeleted}} flag to
> {{PathMetadata}} and check it when returning results from functions like
> {{getFileStatus()}} and {{listStatus()}}. In HADOOP-13651, I added TODO
> comments in most of the places these new conditions are needed. The work
> does not look too bad.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
