[
https://issues.apache.org/jira/browse/HADOOP-12994?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15231078#comment-15231078
]
Chris Nauroth commented on HADOOP-12994:
----------------------------------------
[[email protected]], unfortunately the Azure test still failed. Sorry, I
really should have given you the full stack trace the first time. See below.
Your latest patch changed the single-byte read, but this test failure actually
happens within the bulk positional read. It's the very last assertion of
{{testReadSmallFile}}.
{{NativeAzureFsInputStream}} does not override the positional read method, so
it inherits the seek/bulk read/seek back implementation from {{FSInputStream}}.
{{NativeAzureFsInputStream#seek}} is coded to raise the EOF eagerly, so this
is where it fails. When I fixed the test locally, I did so by copy-pasting
your override of the positional read method from {{S3AInputStream}}.
Your comment about covering this logic in the base class seems apt considering
the copy-pasting to different subclasses.
{code}
testReadSmallFile(org.apache.hadoop.fs.azure.contract.TestAzureNativeContractSeek)
Time elapsed: 4.038 sec <<< ERROR!
java.io.EOFException: Attempted to seek or read past the end of the file
at
org.apache.hadoop.fs.azure.NativeAzureFileSystem$NativeAzureFsInputStream.seek(NativeAzureFileSystem.java:833)
at org.apache.hadoop.fs.FSInputStream.read(FSInputStream.java:70)
at
org.apache.hadoop.fs.BufferedFSInputStream.read(BufferedFSInputStream.java:108)
at
org.apache.hadoop.fs.FSDataInputStream.read(FSDataInputStream.java:92)
at
org.apache.hadoop.fs.contract.AbstractContractSeekTest.testReadSmallFile(AbstractContractSeekTest.java:568)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at
org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:47)
at
org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
at
org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:44)
at
org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
at
org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26)
at
org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27)
at
org.junit.internal.runners.statements.FailOnTimeout$StatementThread.run(FailOnTimeout.java:74)
{code}
> Specify PositionedReadable, add contract tests, fix problems
> ------------------------------------------------------------
>
> Key: HADOOP-12994
> URL: https://issues.apache.org/jira/browse/HADOOP-12994
> Project: Hadoop Common
> Issue Type: Improvement
> Components: fs
> Affects Versions: 2.8.0
> Reporter: Steve Loughran
> Assignee: Steve Loughran
> Attachments: HADOOP-12994-001.patch, HADOOP-12994-002.patch,
> HADOOP-12994-003.patch, HADOOP-12994-004.patch
>
>
> Some work on S3a has shown up that there aren't tests catching regressions in
> readFully, reviewing the documentation shows that its specification could be
> improved.
> # review the spec
> # review the implementations
> # add tests (proposed: to the seek contract; streams which support seek
> should support positioned readable)
> # fix code, where it differs significantly from HDFS or LocalFS
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
