[ https://issues.apache.org/jira/browse/HADOOP-13449?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15717021#comment-15717021 ]
Aaron Fabbri commented on HADOOP-13449: --------------------------------------- I did a little research on #3. It looks like you cannot do a prefix scan on a partition key for DynamoDB. This seems to imply that, considering an operation {{deleteSubtree(delete_path)}}, a simple search by prefix to find all entries with paths that begin with {{delete_path}} would actually be a full table scan. If I'm right, that is unfortunate. The problem with the existing deleteSubtree(delete_path) implementation is that all the children under delete_path might not be reachable from delete_path by doing a simple tree walk over the state in the MetadataStore. The algorithm would work, however, if, when we created a file, we also created all its ancestor directories up to the root. This would establish an invariant that {quote} For any path p in DDB MetadataStore For each ancestor a_i from p to the root a_i is in DDB MetadataStore {quote} This actually sounds reasonable. Can we do it without changing the {{MetadataStore}} interface? I think we can: when we create(path), we always have the full absolute 'path', so we know the names of the ancestors all the way to the root. Thoughts? > S3Guard: Implement DynamoDBMetadataStore. > ----------------------------------------- > > Key: HADOOP-13449 > URL: https://issues.apache.org/jira/browse/HADOOP-13449 > Project: Hadoop Common > Issue Type: Sub-task > Components: fs/s3 > Reporter: Chris Nauroth > Assignee: Mingliang Liu > Attachments: HADOOP-13449-HADOOP-13345.000.patch, > HADOOP-13449-HADOOP-13345.001.patch, HADOOP-13449-HADOOP-13345.002.patch, > HADOOP-13449-HADOOP-13345.003.patch, HADOOP-13449-HADOOP-13345.004.patch, > HADOOP-13449-HADOOP-13345.005.patch, HADOOP-13449-HADOOP-13345.006.patch, > HADOOP-13449-HADOOP-13345.007.patch, HADOOP-13449-HADOOP-13345.008.patch, > HADOOP-13449-HADOOP-13345.009.patch, HADOOP-13449-HADOOP-13345.010.patch > > > Provide an implementation of the metadata store backed by DynamoDB. -- This message was sent by Atlassian JIRA (v6.3.4#6332) --------------------------------------------------------------------- To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org