[
https://issues.apache.org/jira/browse/HADOOP-13231?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15327520#comment-15327520
]
Steve Loughran commented on HADOOP-13231:
-----------------------------------------
The multipart upload problems are probably the same setup problem in
HADOOP-13139; creating a new test FS is force killing all in-progress
operations.
Even if I change that policy I get failures if threads >=3, failures that make
it look like the exclude list isn't being excluded. {{TestS3AContractRootDir}}
is clearly running, and breaking the distcp test
Should those named tests be .class rather than .java?
{code}
-------------------------------------------------------
T E S T S
-------------------------------------------------------
Java HotSpot(TM) 64-Bit Server VM warning: ignoring option MaxPermSize=768m;
support was removed in 8.0
Java HotSpot(TM) 64-Bit Server VM warning: ignoring option MaxPermSize=768m;
support was removed in 8.0
Java HotSpot(TM) 64-Bit Server VM warning: ignoring option MaxPermSize=768m;
support was removed in 8.0
Running org.apache.hadoop.fs.contract.s3a.TestS3AContractDelete
Running org.apache.hadoop.fs.contract.s3a.TestS3AContractDistCp
Running org.apache.hadoop.fs.contract.s3a.TestS3AContractCreate
Tests run: 6, Failures: 0, Errors: 0, Skipped: 3, Time elapsed: 14.07 sec - in
org.apache.hadoop.fs.contract.s3a.TestS3AContractCreate
Java HotSpot(TM) 64-Bit Server VM warning: ignoring option MaxPermSize=768m;
support was removed in 8.0
Running org.apache.hadoop.fs.contract.s3a.TestS3AContractGetFileStatus
Tests run: 8, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 18.313 sec - in
org.apache.hadoop.fs.contract.s3a.TestS3AContractDelete
Java HotSpot(TM) 64-Bit Server VM warning: ignoring option MaxPermSize=768m;
support was removed in 8.0
Running org.apache.hadoop.fs.contract.s3a.TestS3AContractMkdir
Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 4.619 sec - in
org.apache.hadoop.fs.contract.s3a.TestS3AContractGetFileStatus
Java HotSpot(TM) 64-Bit Server VM warning: ignoring option MaxPermSize=768m;
support was removed in 8.0
Running org.apache.hadoop.fs.contract.s3a.TestS3AContractOpen
Tests run: 5, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 12.333 sec - in
org.apache.hadoop.fs.contract.s3a.TestS3AContractMkdir
Java HotSpot(TM) 64-Bit Server VM warning: ignoring option MaxPermSize=768m;
support was removed in 8.0
Running org.apache.hadoop.fs.contract.s3a.TestS3AContractRename
Tests run: 6, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 15.982 sec - in
org.apache.hadoop.fs.contract.s3a.TestS3AContractOpen
Java HotSpot(TM) 64-Bit Server VM warning: ignoring option MaxPermSize=768m;
support was removed in 8.0
Running org.apache.hadoop.fs.contract.s3a.TestS3AContractRootDir
Tests run: 6, Failures: 1, Errors: 0, Skipped: 0, Time elapsed: 18.785 sec <<<
FAILURE! - in org.apache.hadoop.fs.contract.s3a.TestS3AContractRename
testRenameNewFileSameDir(org.apache.hadoop.fs.contract.s3a.TestS3AContractRename)
Time elapsed: 2.823 sec <<< FAILURE!
java.lang.AssertionError: Path
s3a://hwdev-steve-ireland-new/fork-3/test/rename_dest not found in directory
s3a://hwdev-steve-ireland-new/fork-3/test:
at org.junit.Assert.fail(Assert.java:88)
at org.junit.Assert.assertTrue(Assert.java:41)
at
org.apache.hadoop.fs.contract.ContractTestUtils.assertListStatusFinds(ContractTestUtils.java:740)
at
org.apache.hadoop.fs.contract.AbstractContractRenameTest.testRenameNewFileSameDir(AbstractContractRenameTest.java:48)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at
org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:47)
at
org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
at
org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:44)
at
org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
at
org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26)
at
org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27)
at
org.junit.internal.runners.statements.FailOnTimeout$StatementThread.run(FailOnTimeout.java:74)
Java HotSpot(TM) 64-Bit Server VM warning: ignoring option MaxPermSize=768m;
support was removed in 8.0
Running org.apache.hadoop.fs.contract.s3a.TestS3AContractSeek
Tests run: 6, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 19.292 sec - in
org.apache.hadoop.fs.contract.s3a.TestS3AContractRootDir
Java HotSpot(TM) 64-Bit Server VM warning: ignoring option MaxPermSize=768m;
support was removed in 8.0
Running org.apache.hadoop.fs.s3a.fileContext.TestS3AFileContext
Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.268 sec - in
org.apache.hadoop.fs.s3a.fileContext.TestS3AFileContext
Java HotSpot(TM) 64-Bit Server VM warning: ignoring option MaxPermSize=768m;
support was removed in 8.0
Running org.apache.hadoop.fs.s3a.fileContext.TestS3AFileContextCreateMkdir
Tests run: 4, Failures: 0, Errors: 1, Skipped: 0, Time elapsed: 60.495 sec <<<
FAILURE! - in org.apache.hadoop.fs.contract.s3a.TestS3AContractDistCp
largeFilesToRemote(org.apache.hadoop.fs.contract.s3a.TestS3AContractDistCp)
Time elapsed: 25.407 sec <<< ERROR!
java.io.FileNotFoundException: No such file or directory:
s3a://hwdev-steve-ireland-new/fork-1/test/TestS3AContractDistCp/largeFilesToRemote/outputDir
at
org.apache.hadoop.fs.s3a.S3AFileSystem.getFileStatus(S3AFileSystem.java:1464)
at
org.apache.hadoop.fs.s3a.S3AFileSystem.getFileStatus(S3AFileSystem.java:105)
at
org.apache.hadoop.fs.contract.ContractTestUtils.assertIsDirectory(ContractTestUtils.java:471)
at
org.apache.hadoop.tools.contract.AbstractContractDistCpTest.largeFiles(AbstractContractDistCpTest.java:173)
at
org.apache.hadoop.tools.contract.AbstractContractDistCpTest.largeFilesToRemote(AbstractContractDistCpTest.java:96)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at
org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:47)
at
org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
at
org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:44)
at
org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
at
org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26)
at
org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27)
at org.junit.rules.TestWatcher$1.evaluate(TestWatcher.java:55)
at
org.junit.internal.runners.statements.FailOnTimeout$StatementThread.run(FailOnTimeout.java:74)
Java HotSpot(TM) 64-Bit Server VM warning: ignoring option MaxPermSize=768m;
support was removed in 8.0
{code}
> Isolate test path used by a few S3A tests for more reliable parallel
> execution.
> -------------------------------------------------------------------------------
>
> Key: HADOOP-13231
> URL: https://issues.apache.org/jira/browse/HADOOP-13231
> Project: Hadoop Common
> Issue Type: Sub-task
> Components: fs/s3, test
> Reporter: Chris Nauroth
> Assignee: Chris Nauroth
> Priority: Minor
>
> I have noticed a few more spots in S3A tests that do not make use of the
> isolated test directory path when running in parallel mode. While I don't
> have any evidence that this is really causing problems for parallel test runs
> right now, it would still be good practice to clean these up.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
