[
https://issues.apache.org/jira/browse/HADOOP-14154?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16579034#comment-16579034
]
Aaron Fabbri commented on HADOOP-14154:
---------------------------------------
Interesting results. The fact that you had to make that code change suggests
something is not working properly. The change you added to the
perf-eval-v2.diff should make things slower, not faster, if it is implemented
correctly. The idea of that logic is that is only writes the dir listing back
to the MetadataStore if it is different than what we already got from
MetadataStore#listChildren(). Your change should mean that it is *always*
written back which we would expect to be slower (time) and more expensive ($).
Probably need to do more debugging to figure out what is happening.
> Persist isAuthoritative bit in DynamoDBMetaStore (authoritative mode support)
> -----------------------------------------------------------------------------
>
> Key: HADOOP-14154
> URL: https://issues.apache.org/jira/browse/HADOOP-14154
> Project: Hadoop Common
> Issue Type: Sub-task
> Components: fs/s3
> Affects Versions: 3.0.0-beta1
> Reporter: Rajesh Balamohan
> Assignee: Gabor Bota
> Priority: Minor
> Attachments: HADOOP-14154-HADOOP-13345.001.patch,
> HADOOP-14154-HADOOP-13345.002.patch, HADOOP-14154-spec-001.pdf,
> HADOOP-14154-spec-002.pdf, HADOOP-14154.001.patch, HADOOP-14154.002.patch,
> HADOOP-14154.003.patch, HADOOP-14154.004.patch, all-logs-v2.txt,
> perf-eval-v1.diff, perf-eval-v2.diff, run-dir-perf-itest-v2.sh,
> run-dir-perf-itest.sh
>
>
> Add support for "authoritative mode" for DynamoDBMetadataStore.
> The missing feature is to persist the bit set in
> {{DirListingMetadata.isAuthoritative}}.
> This topic has been super confusing for folks so I will also file a
> documentation Jira to explain the design better.
> We may want to also rename the DirListingMetadata.isAuthoritative field to
> .isFullListing to eliminate the multiple uses and meanings of the word
> "authoritative".
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
