[
https://issues.apache.org/jira/browse/HADOOP-15606?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16653459#comment-16653459
]
Steve Loughran commented on HADOOP-15606:
-----------------------------------------
DFSInputStream already implements a {{getFileLength()}} call; this could be
propagated up, at least with its current signature (returns: long). FWIW, I'd
prefer Optional as you can do more
> FSDataInputStream/FSInputStream to implement getFileLength()
> ------------------------------------------------------------
>
> Key: HADOOP-15606
> URL: https://issues.apache.org/jira/browse/HADOOP-15606
> Project: Hadoop Common
> Issue Type: Improvement
> Components: fs, fs/s3
> Affects Versions: 3.1.0
> Reporter: Steve Loughran
> Priority: Major
>
> server-side transcoding, predicate-pushdown filtering in storage, client-side
> decryption can all result in an input stream shorter or longer than the file
> size as measured in getFileStatus/listFiles/listStatus.
> Assuming the length is known once open() returns, the FSDataInputStream can
> return the length of the data, which can then be used for accurate seeks
> within the data.
> * requires the streams to know their length; easy to check (hasCapabilities),
> or make the new method return an Optional<Long>, but caller will need to (a)
> look for this if present and (b) fall back.
> * Could be rolled out for all our own clients/connectors, would take time for
> others
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
