[
https://issues.apache.org/jira/browse/HADOOP-14226?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15954432#comment-15954432
]
Aaron Fabbri edited comment on HADOOP-14226 at 4/4/17 1:12 AM:
---------------------------------------------------------------
This patch looks good [~liuml07]. I changed the test code some as I was
working through the code. I wanted a test case with a bit more directory depth,
and fails before, but succeeds after, the change to {{DynamoDBMetadataStore}}.
Attaching v2 patch which just reworks the test case some. I had to avoid doing
a put() of the dest path before the move, since put() would (correctly) create
the ancestor directory entries. This test depends on move() to do it, so it
fails without your change and succeeds with it.
If you are ok with my test changes I can commit the patch after a complete test
run.
was (Author: fabbri):
This patch looks good [~liuml07]. I wanted a test case that failed before, but
succeeded after, the change to {{DynamoDBMetadataStore}}. Attaching v2 patch
which just reworks the test case some. I had to avoid doing a put() of the
dest path before the move, since put() would (correctly) create the ancestor
directory entries. This test depends on move() to do it, so it fails without
your change and succeeds with it.
If you are ok with my test changes I can commit the patch after a complete test
run.
> S3Guard: DynamoDBMetadataStore::move() should populate ancestor directories
> of destination paths
> ------------------------------------------------------------------------------------------------
>
> Key: HADOOP-14226
> URL: https://issues.apache.org/jira/browse/HADOOP-14226
> Project: Hadoop Common
> Issue Type: Sub-task
> Components: fs/s3
> Affects Versions: HADOOP-13345
> Reporter: Mingliang Liu
> Assignee: Mingliang Liu
> Attachments: HADOOP-14226-HADOOP-13345.000.patch,
> HADOOP-14226-HADOOP-13345.001.patch, HADOOP-14226-HADOOP-13345.002.patch
>
>
> UPDATE: Instead of changing the test, in this JIRA we make sure for each path
> to put, DDBMS::move() has records of each directory up to the root. See
> [~fabbri]'s comment in this JIRA for more detail.
> After running {{ITestDynamoDBMetadataStoreScale}}, the test data is not
> cleaned up. There is a call to {{clearMetadataStore(ms, count);}} in the
> finally clause though. The reason is that, the internally called method
> {{DynamoDBMetadataStore::deleteSubtree()}} is assuming there should be an
> item for the parent dest path:
> {code}
> parent=/fake-bucket, child=moved-here, is_dir=true
> {code}
> In DynamoDBMetadataStore implementation, we assume that _if a path exists,
> all its ancestors will also exist in the table_. We need to pre-create dest
> path to maintain this invariant so that test data can be cleaned up
> successfully.
> I think there may be other tests with the same problem. Let's
> identify/address them separately.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
