[
https://issues.apache.org/jira/browse/HDFS-14345?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Gopal V updated HDFS-14345:
---------------------------
Description:
BufferedInputStream::read() has performance issues - this can be fixed by
wrapping the stream in another non-synchronized buffered inputstream, but that
incurs memory copy overheads and is sub-optimal.
https://github.com/openjdk/jdk/blob/master/src/java.base/share/classes/java/io/BufferedInputStream.java#L269
Hadoop fs streams aren't thread-safe (except for ReadFully) and are stateful
for position, so this synchronization is purely a tax without benefit.
https://github.com/apache/hadoop/blob/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/BufferedFSInputStream.java#L35
The readFully skips the BufferedInputStream super classes.
was:
BufferedInputStream::read() has performance issues - this can be fixed by
wrapping the stream in another non-synchronized buffered inputstream, but that
incurs memory copy overheads and is sub-optimal.
https://github.com/openjdk/jdk/blob/master/src/java.base/share/classes/java/io/BufferedInputStream.java#L269
Hadoop fs streams aren't thread-safe (except for ReadFully) and are stateful
for position, so this synchronization is purely a tax without benefit.
> fs.BufferedFSInputStream::read is synchronized
> ----------------------------------------------
>
> Key: HDFS-14345
> URL: https://issues.apache.org/jira/browse/HDFS-14345
> Project: Hadoop HDFS
> Issue Type: Bug
> Affects Versions: 3.1.2
> Reporter: Gopal V
> Priority: Major
>
> BufferedInputStream::read() has performance issues - this can be fixed by
> wrapping the stream in another non-synchronized buffered inputstream, but
> that incurs memory copy overheads and is sub-optimal.
> https://github.com/openjdk/jdk/blob/master/src/java.base/share/classes/java/io/BufferedInputStream.java#L269
> Hadoop fs streams aren't thread-safe (except for ReadFully) and are stateful
> for position, so this synchronization is purely a tax without benefit.
> https://github.com/apache/hadoop/blob/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/BufferedFSInputStream.java#L35
> The readFully skips the BufferedInputStream super classes.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
