[
https://issues.apache.org/jira/browse/HADOOP-15875?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17248357#comment-17248357
]
Steve Loughran commented on HADOOP-15875:
-----------------------------------------
Note that with the openFile() changes of HADOOP-16202 I'd been hoping that
passing split start/end in to a file open would be enough to fix the content
length. But it isn't, as reader code assumes its ok read past the end of a
split if the end of the split < EOF.
in HADOOP-17415 I'm wondering about whether we should be looking at the
Content-Range header of any response and use that to dynamically determine the
full length of a file. Do a full GET and the length == range, fix there. Do a
partial read and we will be able to update the length and so know what the
final EOF is
Together this should allow workers given a filename and split range to be able
to open and read data past the split end if need be, without the need for any
HEAD at open time. All store IO would be postponed until that first GET
> S3AInputStream.seek should throw EOFException if seeking past the end of file
> -----------------------------------------------------------------------------
>
> Key: HADOOP-15875
> URL: https://issues.apache.org/jira/browse/HADOOP-15875
> Project: Hadoop Common
> Issue Type: Sub-task
> Components: fs/s3
> Affects Versions: 3.2.0
> Reporter: Shixiong Zhu
> Priority: Minor
>
> I read the javadoc of `Seekable.seek` but it doesn't say what should be done
> when seeking past the end of file. Right now, DFSInputStream throws new
> EOFException, but S3AInputStream doesn't throw any error.
> I think it's better to have consistent behavior in `seek.`
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
