István Fajth created HDFS-15693:
-----------------------------------
Summary: Improve native code's performance when writing to HDFS
Key: HDFS-15693
URL: https://issues.apache.org/jira/browse/HDFS-15693
Project: Hadoop HDFS
Issue Type: Improvement
Components: fuse-dfs, native
Reporter: István Fajth
For reads, we introduced direct buffers in order to more efficiently
communicate between the JVM and the native code, and we have readDirect and
pReadDirect in hdfs.c implemented.
Writes on the other hand still use the putByteArrayRegion call, which results
in a copy of the buffer in memory.
This Jira is to explore what has to be done in order to start to use direct
buffers.
A short initial list I see at the moment:
- add a new StreamCapability for streams wanting to support writes via direct
buffer
- implement this capability in the DFSOutputStream and DFSStripedOutputStream
- implement a writeDirect method on the native side
fuse_dfs can benefit from this.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
