[
https://issues.apache.org/jira/browse/HADOOP-15606?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Steve Loughran updated HADOOP-15606:
------------------------------------
Summary: FSDataInputStream/FSInputStream to implement getFileLength()
(was: FSDataInputStream/FSInputStream to implement a length()/size() method for
amount of data)
> FSDataInputStream/FSInputStream to implement getFileLength()
> ------------------------------------------------------------
>
> Key: HADOOP-15606
> URL: https://issues.apache.org/jira/browse/HADOOP-15606
> Project: Hadoop Common
> Issue Type: Improvement
> Components: fs, fs/s3
> Affects Versions: 3.1.0
> Reporter: Steve Loughran
> Priority: Major
>
> server-side transcoding, predicate-pushdown filtering in storage, client-side
> decryption can all result in an input stream shorter or longer than the file
> size as measured in getFileStatus/listFiles/listStatus.
> Assuming the length is known once open() returns, the FSDataInputStream can
> return the length of the data, which can then be used for accurate seeks
> within the data.
> * requires the streams to know their length; easy to check (hasCapabilities),
> or make the new method return an Optional<Long>, but caller will need to (a)
> look for this if present and (b) fall back.
> * Could be rolled out for all our own clients/connectors, would take time for
> others
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
