[ https://issues.apache.org/jira/browse/HADOOP-13914?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15900523#comment-15900523 ]
Aaron Fabbri edited comment on HADOOP-13914 at 3/8/17 1:26 AM: --------------------------------------------------------------- Attaching v6 patch. Changes from v5: - Use wantEmptyDirectoryFlag to avoid extra DDB lookup if the caller doesn't care. Changed two spots mentioned by [~liuml07] and [~rajesh.balamohan]. - Make empty dir test assert messages more descriptive - Add helper function for creating FileStatus in DynamoDBMetadataStore Testing: Ran all integration and unit tests with and without {{-Ds3guard -Ddynamo}} in us-west-2 was (Author: fabbri): Attaching v6 patch. Changes from v5: - Use wantEmptyDirectoryFlag to avoid extra DDB lookup if the caller doesn't care. Changed two spots mentioned by [~liuml07] and [~rajesh.balamohan]. - Make empty dir test assert messages more descriptive - Add helper function for creating FileStatus in DynamoDBMetadataStore > s3guard: improve S3AFileStatus#isEmptyDirectory handling > -------------------------------------------------------- > > Key: HADOOP-13914 > URL: https://issues.apache.org/jira/browse/HADOOP-13914 > Project: Hadoop Common > Issue Type: Sub-task > Components: fs/s3 > Affects Versions: HADOOP-13345 > Reporter: Aaron Fabbri > Assignee: Aaron Fabbri > Attachments: HADOOP-13914-HADOOP-13345.000.patch, > HADOOP-13914-HADOOP-13345.002.patch, HADOOP-13914-HADOOP-13345.003.patch, > HADOOP-13914-HADOOP-13345.004.patch, HADOOP-13914-HADOOP-13345.005.patch, > HADOOP-13914-HADOOP-13345.006.patch, s3guard-empty-dirs.md, > test-only-HADOOP-13914.patch > > > As discussed in HADOOP-13449, proper support for the isEmptyDirectory() flag > stored in S3AFileStatus is missing from DynamoDBMetadataStore. > The approach taken by LocalMetadataStore is not suitable for the DynamoDB > implementation, and also sacrifices good code separation to minimize > S3AFileSystem changes pre-merge to trunk. > I will attach a design doc that attempts to clearly explain the problem and > preferred solution. I suggest we do this work after merging the HADOOP-13345 > branch to trunk, but am open to suggestions. > I can also attach a patch of a integration test that exercises the missing > case and demonstrates a failure with DynamoDBMetadataStore. -- This message was sent by Atlassian JIRA (v6.3.15#6346) --------------------------------------------------------------------- To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org