Ahmed Mahran created HDFS-8416:
----------------------------------
Summary: Short circuit remote reads from shared storage
Key: HDFS-8416
URL: https://issues.apache.org/jira/browse/HDFS-8416
Project: Hadoop HDFS
Issue Type: Improvement
Components: datanode, hdfs-client, nfs, performance
Reporter: Ahmed Mahran
In a Hadoop cluster configuration that employs a shared storage system, HDFS
read and write operations are very expensive in terms of network bandwidth
consumption.
For a DFS client to read a block from a remote datanode, the block is
transmitted first from the shared storage to the datanode then from the
datanode to the DFS client. Short circuiting the shared storage to datanode hop
and allowing the client to directly access the shared storage would improve the
performance substantially.
This document describes the issue and provides a hack for the remote read.
https://docs.google.com/document/d/16wvaFDN0R10jIX1vLlEJpJh-KhJR8YNO4Pt3v9FAfvQ
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
