[
https://issues.apache.org/jira/browse/HADOOP-13345?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15876991#comment-15876991
]
Mingliang Liu commented on HADOOP-13345:
----------------------------------------
Hi Aaron, I updated from feature branch but still have the following error.
This is the same for w/ and w/o merging from trunk and I assume it's not
related to merge (not committed yet). I'll have a look at the test. Thanks,
{code}
mvn -Dit.test='ITestS3GuardListConsistency' -Dtest=none -Dscale -Ds3guard
-Ddynamo -q clean verify
-------------------------------------------------------
T E S T S
-------------------------------------------------------
Running org.apache.hadoop.fs.s3a.ITestS3GuardListConsistency
Tests run: 2, Failures: 1, Errors: 0, Skipped: 0, Time elapsed: 4.544 sec <<<
FAILURE! - in org.apache.hadoop.fs.s3a.ITestS3GuardListConsistency
testListStatusWriteBack(org.apache.hadoop.fs.s3a.ITestS3GuardListConsistency)
Time elapsed: 3.147 sec <<< FAILURE!
java.lang.AssertionError: Unexpected number of results from metastore.
Metastore should only know about /XYZ:
DirListingMetadata{path=s3a://mliu-s3guard/test/ListStatusWriteBack,
listMap={s3a://mliu-s3guard/test/ListStatusWriteBack/XYZ=PathMetadata{fileStatus=S3AFileStatus{path=s3a://mliu-s3guard/test/ListStatusWriteBack/XYZ;
isDirectory=true; modification_time=0; access_time=0; owner=mliu; group=mliu;
permission=rwxrwxrwx; isSymlink=false} isEmptyDirectory=true},
s3a://mliu-s3guard/test/ListStatusWriteBack/123=PathMetadata{fileStatus=S3AFileStatus{path=s3a://mliu-s3guard/test/ListStatusWriteBack/123;
isDirectory=true; modification_time=0; access_time=0; owner=mliu; group=mliu;
permission=rwxrwxrwx; isSymlink=false} isEmptyDirectory=true}},
isAuthoritative=false}
at org.junit.Assert.fail(Assert.java:88)
at org.junit.Assert.assertTrue(Assert.java:41)
at
org.apache.hadoop.fs.s3a.ITestS3GuardListConsistency.testListStatusWriteBack(ITestS3GuardListConsistency.java:127)
{code}
{quote}
I'm wondering if that fix should be a separate commit instead of modifying the
merge commit?
{quote}
That makes sense. I'll file a separate JIRA for tracking this and submit a
patch for fixing it (unless Steve objects). Merging commit should be
simple/small/clear if possible.
> S3Guard: Improved Consistency for S3A
> -------------------------------------
>
> Key: HADOOP-13345
> URL: https://issues.apache.org/jira/browse/HADOOP-13345
> Project: Hadoop Common
> Issue Type: New Feature
> Components: fs/s3
> Reporter: Chris Nauroth
> Assignee: Chris Nauroth
> Attachments: HADOOP-13345.prototype1.patch, s3c.001.patch,
> S3C-ConsistentListingonS3-Design.pdf, S3GuardImprovedConsistencyforS3A.pdf,
> S3GuardImprovedConsistencyforS3AV2.pdf
>
>
> This issue proposes S3Guard, a new feature of S3A, to provide an option for a
> stronger consistency model than what is currently offered. The solution
> coordinates with a strongly consistent external store to resolve
> inconsistencies caused by the S3 eventual consistency model.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
