[
https://issues.apache.org/jira/browse/HADOOP-15480?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16488762#comment-16488762
]
Steve Loughran edited comment on HADOOP-15480 at 5/24/18 10:25 AM:
-------------------------------------------------------------------
some of the tests just use FileSystem.newInstance,
e.g.{{ITestS3AInputStreamPerformance}}. We don't do it everywhere as it makes
for slower tests.
Looking at {{AbstractITCommitProtocol}}, it is patching in the created FS into
the cache to make sure the instance with config we want gets in...though some
of the tests have asserts to verify the FS returned is what they want
the code would be changed to something like (with the relevant config, uri)..
{code}
S3AFileSystem fs = (S3AFileSystem) FileSystem.newInstance(uri, conf);
S3ATestUtils.setMetadataStore(fs, new NullMetadataStore());
{code}
of course, if we are creating a new FS instance, why not just change the config
to say "null MD store", rather than dynamically patch it?
was (Author: [email protected]):
some of the tests just use FileSystem.newInstance,
e.g.{{ITestS3AInputStreamPerformance}}. We don't do it everywhere as it makes
for slower tests.
Looking at {{AbstractITCommitProtocol}}, it is patching in the created FS into
the cache to make sure the instance with config we want gets in...though some
of the tests have asserts to verify the FS returned is what they want
> AbstractS3GuardToolTestBase.testDiffCommand fails when using dynamo
> -------------------------------------------------------------------
>
> Key: HADOOP-15480
> URL: https://issues.apache.org/jira/browse/HADOOP-15480
> Project: Hadoop Common
> Issue Type: Sub-task
> Components: fs/s3
> Affects Versions: 3.1.0
> Reporter: Gabor Bota
> Assignee: Gabor Bota
> Priority: Major
> Attachments: HADOOP-15480.001.patch
>
>
> When running org.apache.hadoop.fs.s3a.s3guard.ITestS3GuardToolDynamoDB, the
> testDiffCommand test fails with the following:
> {noformat}
> testDiffCommand(org.apache.hadoop.fs.s3a.s3guard.ITestS3GuardToolDynamoDB)
> Time elapsed: 8.059 s <<< FAILURE!
> java.lang.AssertionError:
> Mismatched metadata store outputs: MS D 0
> s3a://cloudera-dev-gabor-ireland/test/test-diff/ms_only
> MS F 100
> s3a://cloudera-dev-gabor-ireland/test/test-diff/ms_only/file-0
> MS F 100
> s3a://cloudera-dev-gabor-ireland/test/test-diff/ms_only/file-1
> MS F 100
> s3a://cloudera-dev-gabor-ireland/test/test-diff/ms_only/file-3
> MS F 100
> s3a://cloudera-dev-gabor-ireland/test/test-diff/ms_only/file-2
> MS F 100
> s3a://cloudera-dev-gabor-ireland/test/test-diff/ms_only/file-4
> S3 F 0
> s3a://cloudera-dev-gabor-ireland/test/test-diff/s3_only/file-1
> MS F 0
> s3a://cloudera-dev-gabor-ireland/test/test-diff/s3_only/file-1
> S3 F 0
> s3a://cloudera-dev-gabor-ireland/test/test-diff/s3_only/file-0
> MS F 0
> s3a://cloudera-dev-gabor-ireland/test/test-diff/s3_only/file-0
> S3 F 0
> s3a://cloudera-dev-gabor-ireland/test/test-diff/s3_only/file-2
> MS F 0
> s3a://cloudera-dev-gabor-ireland/test/test-diff/s3_only/file-2
> S3 F 0
> s3a://cloudera-dev-gabor-ireland/test/test-diff/s3_only/file-3
> MS F 0
> s3a://cloudera-dev-gabor-ireland/test/test-diff/s3_only/file-3
> S3 F 0
> s3a://cloudera-dev-gabor-ireland/test/test-diff/s3_only/file-4
> MS F 0
> s3a://cloudera-dev-gabor-ireland/test/test-diff/s3_only/file-4
> expected:<[
> s3a://cloudera-dev-gabor-ireland/test/test-diff/ms_only,
> s3a://cloudera-dev-gabor-ireland/test/test-diff/ms_only/file-0,
> s3a://cloudera-dev-gabor-ireland/test/test-diff/ms_only/file-1,
> s3a://cloudera-dev-gabor-ireland/test/test-diff/ms_only/file-3,
> s3a://cloudera-dev-gabor-ireland/test/test-diff/ms_only/file-2,
> s3a://cloudera-dev-gabor-ireland/test/test-diff/ms_only/file-4]>
> but was:<[
> s3a://cloudera-dev-gabor-ireland/test/test-diff/ms_only,
> s3a://cloudera-dev-gabor-ireland/test/test-diff/s3_only/file-1,
> s3a://cloudera-dev-gabor-ireland/test/test-diff/ms_only/file-0,
> s3a://cloudera-dev-gabor-ireland/test/test-diff/s3_only/file-0,
> s3a://cloudera-dev-gabor-ireland/test/test-diff/ms_only/file-1,
> s3a://cloudera-dev-gabor-ireland/test/test-diff/ms_only/file-3,
> s3a://cloudera-dev-gabor-ireland/test/test-diff/ms_only/file-2,
> s3a://cloudera-dev-gabor-ireland/test/test-diff/s3_only/file-2,
> s3a://cloudera-dev-gabor-ireland/test/test-diff/s3_only/file-3,
> s3a://cloudera-dev-gabor-ireland/test/test-diff/ms_only/file-4,
> s3a://cloudera-dev-gabor-ireland/test/test-diff/s3_only/file-4]>
> at org.junit.Assert.fail(Assert.java:88)
> at org.junit.Assert.failNotEquals(Assert.java:743)
> at org.junit.Assert.assertEquals(Assert.java:118)
> at
> org.apache.hadoop.fs.s3a.s3guard.AbstractS3GuardToolTestBase.testDiffCommand(AbstractS3GuardToolTestBase.java:382)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at
> org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:47)
> at
> org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
> at
> org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:44)
> at
> org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
> at
> org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26)
> at
> org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27)
> at org.junit.rules.TestWatcher$1.evaluate(TestWatcher.java:55)
> at
> org.junit.internal.runners.statements.FailOnTimeout$StatementThread.run(FailOnTimeout.java:74)
> {noformat}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
