[
https://issues.apache.org/jira/browse/HADOOP-18416?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17585227#comment-17585227
]
ASF GitHub Bot commented on HADOOP-18416:
-----------------------------------------
mehakmeet commented on PR #4806:
URL: https://github.com/apache/hadoop/pull/4806#issuecomment-1228205573
Thoughts on how to handle Prefetch streams for this test(it doesn't have
stream capability for IOContext), Asserting in all tests to break before
checking statistics values, or skipping all tests with streams without
IOContext capability?
> ITestS3AIOStatisticsContext failure
> -----------------------------------
>
> Key: HADOOP-18416
> URL: https://issues.apache.org/jira/browse/HADOOP-18416
> Project: Hadoop Common
> Issue Type: Sub-task
> Components: fs/s3, test
> Affects Versions: 3.3.9
> Reporter: Steve Loughran
> Assignee: Mehakmeet Singh
> Priority: Major
> Labels: pull-request-available
> Attachments:
> org.apache.hadoop.fs.s3a.ITestS3AIOStatisticsContext-output.txt
>
>
> test failure running the new ITestS3AIOStatisticsContext. attaching the stack
> and log file.
> This happened on a large (12 thread) test run, but i can get it to come back
> intermittently on repeated runs of the whole suite, but never when i just run
> the single test case.
> {code}
> [ERROR]
> testThreadIOStatisticsForDifferentThreads(org.apache.hadoop.fs.s3a.ITestS3AIOStatisticsContext)
> Time elapsed: 3.616 s <<< FAILURE!
> java.lang.AssertionError:
> [Counter named stream_write_bytes]
> Expecting actual not to be null
> at
> org.apache.hadoop.fs.statistics.IOStatisticAssertions.lookupStatistic(IOStatisticAssertions.java:160)
> at
> org.apache.hadoop.fs.statistics.IOStatisticAssertions.assertThatStatisticLong(IOStatisticAssertions.java:291)
> at
> org.apache.hadoop.fs.statistics.IOStatisticAssertions.assertThatStatisticCounter(IOStatisticAssertions.java:306)
> at
> org.apache.hadoop.fs.s3a.ITestS3AIOStatisticsContext.assertThreadStatisticsForThread(ITestS3AIOStatisticsContext.java:367)
> at
> org.apache.hadoop.fs.s3a.ITestS3AIOStatisticsContext.testThreadIOStatisticsForDifferentThreads(ITestS3AIOStatisticsContext.java:260)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at
> org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:59)
> at
> org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
> at
> org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:56)
> at
> org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
> at
> org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26)
> at
> org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27)
> at org.junit.rules.TestWatcher$1.evaluate(TestWatcher.java:61)
> at
> org.junit.internal.runners.statements.FailOnTimeout$CallableStatement.call(FailOnTimeout.java:288)
> at
> org.junit.internal.runners.statements.FailOnTimeout$CallableStatement.call(FailOnTimeout.java:282)
> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> at java.lang.Thread.run(Thread.java:750)
> {code}
> I'm suspecting some race condition *or* gc pressure is releasing that
> reference in the worker thread.
> proposed test changes
> * worker thread changes its thread ID for the logs
> * stores its thread context into a field, so there's guarantee of no GC
> * logs more as it goes along.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
