[
https://issues.apache.org/jira/browse/HADOOP-18378?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Steve Loughran updated HADOOP-18378:
------------------------------------
Description:
Implement readFully(long position, byte[] buffer, int offset, int length) in
PrefetchingInputStream, as it currently uses FSInputStream's
[readFully|https://github.com/apache/hadoop/blob/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/FSInputStream.java#L136]
which calls read(long position, byte[] buffer, int offset, int length).
This read then seeks to the position (which is ok), but then seeks back to the
original starting position at the end (so always seeking back to 0). this is
pretty bad for the prefetching implementation as it means lots of caching to
disk and getting blocks from disk.
when backporting. follow with HADOOP-18531
was:
Implement readFully(long position, byte[] buffer, int offset, int length) in
PrefetchingInputStream, as it currently uses FSInputStream's
[readFully|https://github.com/apache/hadoop/blob/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/FSInputStream.java#L136]
which calls read(long position, byte[] buffer, int offset, int length).
This read then seeks to the position (which is ok), but then seeks back to the
original starting position at the end (so always seeking back to 0). this is
pretty bad for the prefetching implementation as it means lots of caching to
disk and getting blocks from disk.
> Implement readFully(long position, byte[] buffer, int offset, int length)
> -------------------------------------------------------------------------
>
> Key: HADOOP-18378
> URL: https://issues.apache.org/jira/browse/HADOOP-18378
> Project: Hadoop Common
> Issue Type: Sub-task
> Components: fs/s3
> Affects Versions: 3.4.0
> Reporter: Ahmar Suhail
> Assignee: Alessandro Passaro
> Priority: Minor
> Labels: pull-request-available
> Fix For: 3.4.0
>
>
> Implement readFully(long position, byte[] buffer, int offset, int length) in
> PrefetchingInputStream, as it currently uses FSInputStream's
> [readFully|https://github.com/apache/hadoop/blob/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/FSInputStream.java#L136]
> which calls read(long position, byte[] buffer, int offset, int length).
> This read then seeks to the position (which is ok), but then seeks back to
> the original starting position at the end (so always seeking back to 0). this
> is pretty bad for the prefetching implementation as it means lots of caching
> to disk and getting blocks from disk.
> when backporting. follow with HADOOP-18531
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
