[
https://issues.apache.org/jira/browse/HADOOP-14323?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15983764#comment-15983764
]
Aaron Fabbri commented on HADOOP-14323:
---------------------------------------
Thanks [~liuml07]. I reproduced the failure. Not sure why I didn't see it
before.
Actually took me some time to figure it out. This is due to LocalMetadataStore
returning mutable DirListingMetadata, which get modified in
{{S3Guard#dirListingUnion}}. Since LocalMetadataStore stores a reference to
the same DirListingMetadata in its hashtable, it ends up modifying the contents
of the LocalMetadataStore.
Attaching v3 patch:
- Moves test FS creation into helper function.
- Make test helper function static.
- Change {{LocalMetadataStore#listChildren()}} to return a copy of
DirListingMetadata, instead of its internal reference.
> ITestS3GuardListConsistency failure w/ Local, authoritative metadata store
> --------------------------------------------------------------------------
>
> Key: HADOOP-14323
> URL: https://issues.apache.org/jira/browse/HADOOP-14323
> Project: Hadoop Common
> Issue Type: Sub-task
> Components: s3
> Affects Versions: HADOOP-13345
> Reporter: Aaron Fabbri
> Assignee: Aaron Fabbri
> Priority: Minor
> Fix For: HADOOP-13345
>
> Attachments: HADOOP-14323-HADOOP-13345.001.patch,
> HADOOP-14323-HADOOP-13345.002.patch
>
>
> When doing some testing for HADOOP-14266 I noticed this test failure:
> {noformat}
> java.lang.NullPointerException: null
> at
> org.apache.hadoop.fs.s3a.ITestS3GuardListConsistency.testListStatusWriteBack(ITestS3GuardListConsistency.java:317)
> {noformat}
> I was running with LocalMetadataStore and
> {{fs.s3a.metadatastore.authoritative}} set to true. I haven't been testing
> this mode recently so not sure if this case ever worked. Lower priority but
> we should fix it.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
