steveloughran commented on pull request #1982:
URL: https://github.com/apache/hadoop/pull/1982#issuecomment-625240548
failure on a test run, which is related to stats gathering. But I don't see
this when run from the IDE. Race condition?
```
[ERROR]
ITestCommitOperations.testBulkCommitFiles:650->Assert.assertEquals:645->Assert.failNotEquals:834->Assert.fail:88
Number of records written after commit #2; first commit had 4; first commit
ancestors CommitContext{operationState=AncestorState{operation=Commitid=55;
dest=s3a://stevel-london/fork-0006/test/DELAY_LISTING_ME/testBulkCommitFiles/out;
size=6;
paths={s3a://stevel-london/fork-0006/test/DELAY_LISTING_ME/testBulkCommitFiles/out/file1
s3a://stevel-london/fork-0006
s3a://stevel-london/fork-0006/test/DELAY_LISTING_ME/testBulkCommitFiles
s3a://stevel-london/fork-0006/test/DELAY_LISTING_ME/testBulkCommitFiles/out
s3a://stevel-london/fork-0006/test
s3a://stevel-london/fork-0006/test/DELAY_LISTING_ME}}}; second commit
ancestors: CommitContext{operationState=AncestorState{operation=Commitid=55;
dest=s3a://stevel-london/fork-0006/test/DELAY_LISTING_ME/testBulkCommitFiles/out;
size=8;
paths={s3a://stevel-london/fork-0006/test/DELAY_LISTING_ME/testBulkCommitFiles/out/file1
s3a://stevel-london/fork-0006/test/DELAY_LISTING_ME/testBulkCommitFiles/out/subdir/file2
s3a://stevel-london/fork-0006
s3a://stevel-london/fork-0006/test/DELAY_LISTING_ME/testBulkCommitFiles
s3a://stevel-london/fork-0006/test/DELAY_LISTING_ME/testBulkCommitFiles/out
s3a://stevel-london/fork-0006/test
s3a://stevel-london/fork-0006/test/DELAY_LISTING_ME/testBulkCommitFiles/out/subdir
s3a://stevel-london/fork-0006/test/DELAY_LISTING_ME}}}:
s3guard_metadatastore_record_writes expected:<2> but was:<3>
[ERROR] ITestS3ACommitterMRJob.test_200_execute:304->Assert.fail:88 Job
job_1588787902323_0003 failed in state FAILED with cause Job commit failed:
java.util.concurrent.RejectedExecutionException: Task
java.util.concurrent.FutureTask@2104a338 rejected from
org.apache.hadoop.util.concurrent.HadoopThreadPoolExecutor@7272786a[Terminated,
pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 10]
at
java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2063)
at
java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:830)
at
java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1379)
at
java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:112)
at
org.apache.hadoop.fs.s3a.commit.Tasks$Builder.runParallel(Tasks.java:313)
at org.apache.hadoop.fs.s3a.commit.Tasks$Builder.run(Tasks.java:148)
at
org.apache.hadoop.fs.s3a.commit.AbstractS3ACommitter.commitPendingUploads(AbstractS3ACommitter.java:480)
at
org.apache.hadoop.fs.s3a.commit.AbstractS3ACommitter.commitJobInternal(AbstractS3ACommitter.java:620)
at
org.apache.hadoop.fs.s3a.commit.AbstractS3ACommitter.commitJob(AbstractS3ACommitter.java:722)
at
org.apache.hadoop.mapreduce.v2.app.commit.CommitterEventHandler$EventProcessor.handleJobCommit(CommitterEventHandler.java:286)
at
org.apache.hadoop.mapreduce.v2.app.commit.CommitterEventHandler$EventProcessor.run(CommitterEventHandler.java:238)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
```
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
