[
https://issues.apache.org/jira/browse/HADOOP-18175?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17512578#comment-17512578
]
Steve Loughran commented on HADOOP-18175:
-----------------------------------------
h3. ITestS3AContractUnbuffer
fix: getPos() on a closed file to return 0;
{code}
[ERROR]
testUnbufferOnClosedFile(org.apache.hadoop.fs.contract.s3a.ITestS3AContractUnbuffer)
Time elapsed: 0.833 s <<< ERROR!
java.io.IOException: Stream is closed!
at
org.apache.hadoop.fs.s3a.read.S3PrefetchingInputStream.throwIfClosed(S3PrefetchingInputStream.java:224)
at
org.apache.hadoop.fs.s3a.read.S3PrefetchingInputStream.getPos(S3PrefetchingInputStream.java:105)
at
org.apache.hadoop.fs.FSDataInputStream.getPos(FSDataInputStream.java:81)
at
org.apache.hadoop.fs.contract.AbstractContractUnbufferTest.unbuffer(AbstractContractUnbufferTest.java:124)
at
org.apache.hadoop.fs.contract.AbstractContractUnbufferTest.testUnbufferOnClosedFile(AbstractContractUnbufferTest.java:90)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
{code}
h3. ITestS3AUnbuffer
instanceof and statistics assertions failing.
{code}
[ERROR] testUnbuffer(org.apache.hadoop.fs.s3a.ITestS3AUnbuffer) Time elapsed:
2.385 s <<< FAILURE!
java.lang.AssertionError
at org.junit.Assert.fail(Assert.java:87)
at org.junit.Assert.assertTrue(Assert.java:42)
at org.junit.Assert.assertTrue(Assert.java:53)
at
org.apache.hadoop.fs.s3a.ITestS3AUnbuffer.testUnbuffer(ITestS3AUnbuffer.java:75)
[ERROR] testUnbufferStreamStatistics(org.apache.hadoop.fs.s3a.ITestS3AUnbuffer)
Time elapsed: 1.115 s <<< FAILURE!
java.lang.AssertionError: Count of stream_read_bytes starting=0 current=0
diff=0: stream_read_bytes expected:<8> but was:<0>
at org.junit.Assert.fail(Assert.java:89)
at org.junit.Assert.failNotEquals(Assert.java:835)
at org.junit.Assert.assertEquals(Assert.java:647)
at
org.apache.hadoop.fs.s3a.S3ATestUtils$MetricDiff.assertDiffEquals(S3ATestUtils.java:998)
at
org.apache.hadoop.fs.s3a.S3ATestUtils$MetricDiff.assertDiffEquals(S3ATestUtils.java:1007)
at
org.apache.hadoop.fs.s3a.ITestS3AUnbuffer.testUnbufferStreamStatistics(ITestS3AUnbuffer.java:149)
{code}
h3. ITestS3ARequesterPays statistics assertion.
{code}
[ERROR]
testRequesterPaysOptionSuccess(org.apache.hadoop.fs.s3a.ITestS3ARequesterPays)
Time elapsed: 2.198 s <<< FAILURE!
java.lang.AssertionError:
[Counter named stream_read_opened]
Expecting:
<0L>
to be greater than:
<1L>
at
org.apache.hadoop.fs.s3a.ITestS3ARequesterPays.testRequesterPaysOptionSuccess(ITestS3ARequesterPays.java:78)
{code}
fix: wire up the statistics and/or change assertion based on stream type
h3.
testStatistics(org.apache.hadoop.fs.s3a.fileContext.ITestS3AFileContextStatistics)
T
{code}
java.io.EOFException: Attempted to seek or read past the end of the file 512
at
org.apache.hadoop.fs.s3a.read.S3InputStream.throwIfInvalidSeek(S3InputStream.java:439)
at
org.apache.hadoop.fs.s3a.read.S3InputStream.seek(S3InputStream.java:234)
at
org.apache.hadoop.fs.s3a.read.S3PrefetchingInputStream.seek(S3PrefetchingInputStream.java:161)
at org.apache.hadoop.fs.FSInputStream.read(FSInputStream.java:85)
at
org.apache.hadoop.fs.FSDataInputStream.read(FSDataInputStream.java:98)
at
org.apache.hadoop.fs.FCStatisticsBaseTest.testStatistics(FCStatisticsBaseTest.java:107)
{code}
> test failures with prefetching s3a input stream
> -----------------------------------------------
>
> Key: HADOOP-18175
> URL: https://issues.apache.org/jira/browse/HADOOP-18175
> Project: Hadoop Common
> Issue Type: Sub-task
> Components: fs/s3, test
> Affects Versions: 3.4.0
> Reporter: Steve Loughran
> Priority: Major
>
> identify and fix all test regressions from the prefetching s3a input stream
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
