Gopal V created HADOOP-11867:
--------------------------------
Summary: FS API: Add a high-performance vectored Read to
FSDataInputStream API
Key: HADOOP-11867
URL: https://issues.apache.org/jira/browse/HADOOP-11867
Project: Hadoop Common
Issue Type: New Feature
Affects Versions: 2.8.0
Reporter: Gopal V
The most significant way to read from a filesystem in an efficient way is to
let the FileSystem implementation handle the seek behaviour underneath the API
to be the most efficient as possible.
A better approach to the seek problem is to provide a sequence of read
locations as part of a single call, while letting the system schedule/plan the
reads ahead of time.
This is exceedingly useful for seek-heavy readers on HDFS, since this allows
for potentially optimizing away the seek-gaps within the FSDataInputStream
implementation.
For seek+read systems with even more latency than locally-attached disks,
something like a {{readFully(long[] offsets, ByteBuffer[] chunks)}} would take
of the seeks internally while reading chunk.remaining() bytes into each chunk
(which may be {{slice()}}ed off a bigger buffer).
The base implementation can stub in this as a sequence of seeks + read() into
ByteBuffers, without forcing each FS implementation to override this in any way.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
