[jira] [Updated] (HDFS-14541) ShortCircuitReplica#unref cost about 6% cpu and 6% heap allocation because of the frequent thrown NoSuchElementException in our HBase benchmark
[
https://issues.apache.org/jira/browse/HDFS-14541?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Zheng Hu updated HDFS-14541:
Status: Open (was: Patch Available)
We use the Github workflow now, so no need to trigger the hadoop QA again.
Cancel the Patch Available status.
> ShortCircuitReplica#unref cost about 6% cpu and 6% heap allocation because of
> the frequent thrown NoSuchElementException in our HBase benchmark
>
>
> Key: HDFS-14541
> URL: https://issues.apache.org/jira/browse/HDFS-14541
> Project: Hadoop HDFS
> Issue Type: Bug
>Reporter: Zheng Hu
>Assignee: Lisheng Sun
>Priority: Major
> Attachments: HDFS-14541.000.patch, after-QPS.png,
> after-cpu-flame-graph.svg, after-heap-flame-graph.svg,
> async-prof-pid-94152-alloc-2.svg, async-prof-pid-94152-cpu-1.svg,
> before-QPS.png, before-cpu-flame-graph.svg, before-heap-flame-graph.svg
>
>
> Our XiaoMi HBase team are evaluating the performence improvement of
> HBASE-21879, and we have few CPU flame graph & heap flame graph by using
> async-profiler, and find that there're some performence issues in DFSClient
> .
> See the attached two flame graphs, we can conclude that the try catch block
> in ShortCircuitCache#trimEvictionMaps has some serious perf problem , we
> should remove the try catch from DFSClient.
> {code}
> /**
>* Trim the eviction lists.
>*/
> private void trimEvictionMaps() {
> long now = Time.monotonicNow();
> demoteOldEvictableMmaped(now);
> while (true) {
> long evictableSize = evictable.size();
> long evictableMmappedSize = evictableMmapped.size();
> if (evictableSize + evictableMmappedSize <= maxTotalSize) {
> return;
> }
> ShortCircuitReplica replica;
> try {
> if (evictableSize == 0) {
> replica = (ShortCircuitReplica)evictableMmapped.get(evictableMmapped
> .firstKey());
> } else {
> replica = (ShortCircuitReplica)evictable.get(evictable.firstKey());
> }
> } catch (NoSuchElementException e) {
> break;
> }
> if (LOG.isTraceEnabled()) {
> LOG.trace(this + ": trimEvictionMaps is purging " + replica +
> StringUtils.getStackTrace(Thread.currentThread()));
> }
> purge(replica);
> }
> }
> {code}
> Our Xiaomi HDFS Team member [~leosun08] will prepare patch for this issue.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-14541) ShortCircuitReplica#unref cost about 6% cpu and 6% heap allocation because of the frequent thrown NoSuchElementException in our HBase benchmark
[
https://issues.apache.org/jira/browse/HDFS-14541?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16866472#comment-16866472
]
Zheng Hu commented on HDFS-14541:
-
Our XiaoMi HBase Team have made a performance testing, the only difference is
whether include this patch or not. (BTW, we build a clusters with 5 nodes,
each nodes have 800GB * 12 SSD, the HBase use 50G onheap + 50GB offheap. For
addressing difference of this JIRA, we've disabled the blockCache so that all
QPS directed to HDFS client. )
|| Case || Heap FlameGraph || CPU FlameGraph || HBase QPS & Latency ||
| Before Case |
[before-heap-flame-graph.svg|https://issues.apache.org/jira/secure/attachment/12972068/before-heap-flame-graph.svg]
|
[before-cpu-flame-graph.svg|https://issues.apache.org/jira/secure/attachment/12972069/before-cpu-flame-graph.svg]
|
[before-QPS.png|https://issues.apache.org/jira/secure/attachment/12972067/before-QPS.png]
|
| After Case |
[after-heap-flame-graph.svg|https://issues.apache.org/jira/secure/attachment/12972071/after-heap-flame-graph.svg]
|
[after-cpu-flame-graph.svg|https://issues.apache.org/jira/secure/attachment/12972072/after-cpu-flame-graph.svg]
|
[after-QPS.png|https://issues.apache.org/jira/secure/attachment/12972070/after-QPS.png]
|
We can clearly see that: after the patch, the HBase throughput increased about
17.8% ( math: (33K qps - 28K qps) / 28Kqps ~ 17.8%) . Also the 6% cpu & heap
cost in flame graph disappeared, those are very good points.
[~elgoiri], I think [~leosun08] is preparing the new patch to address your
comment. Thanks.
> ShortCircuitReplica#unref cost about 6% cpu and 6% heap allocation because of
> the frequent thrown NoSuchElementException in our HBase benchmark
>
>
> Key: HDFS-14541
> URL: https://issues.apache.org/jira/browse/HDFS-14541
> Project: Hadoop HDFS
> Issue Type: Bug
>Reporter: Zheng Hu
>Assignee: Lisheng Sun
>Priority: Major
> Attachments: HDFS-14541.000.patch, after-QPS.png,
> after-cpu-flame-graph.svg, after-heap-flame-graph.svg,
> async-prof-pid-94152-alloc-2.svg, async-prof-pid-94152-cpu-1.svg,
> before-QPS.png, before-cpu-flame-graph.svg, before-heap-flame-graph.svg
>
>
> Our XiaoMi HBase team are evaluating the performence improvement of
> HBASE-21879, and we have few CPU flame graph & heap flame graph by using
> async-profiler, and find that there're some performence issues in DFSClient
> .
> See the attached two flame graphs, we can conclude that the try catch block
> in ShortCircuitCache#trimEvictionMaps has some serious perf problem , we
> should remove the try catch from DFSClient.
> {code}
> /**
>* Trim the eviction lists.
>*/
> private void trimEvictionMaps() {
> long now = Time.monotonicNow();
> demoteOldEvictableMmaped(now);
> while (true) {
> long evictableSize = evictable.size();
> long evictableMmappedSize = evictableMmapped.size();
> if (evictableSize + evictableMmappedSize <= maxTotalSize) {
> return;
> }
> ShortCircuitReplica replica;
> try {
> if (evictableSize == 0) {
> replica = (ShortCircuitReplica)evictableMmapped.get(evictableMmapped
> .firstKey());
> } else {
> replica = (ShortCircuitReplica)evictable.get(evictable.firstKey());
> }
> } catch (NoSuchElementException e) {
> break;
> }
> if (LOG.isTraceEnabled()) {
> LOG.trace(this + ": trimEvictionMaps is purging " + replica +
> StringUtils.getStackTrace(Thread.currentThread()));
> }
> purge(replica);
> }
> }
> {code}
> Our Xiaomi HDFS Team member [~leosun08] will prepare patch for this issue.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Updated] (HDFS-14541) ShortCircuitReplica#unref cost about 6% cpu and 6% heap allocation because of the frequent thrown NoSuchElementException in our HBase benchmark
[
https://issues.apache.org/jira/browse/HDFS-14541?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Zheng Hu updated HDFS-14541:
Attachment: after-cpu-flame-graph.svg
after-heap-flame-graph.svg
after-QPS.png
before-cpu-flame-graph.svg
before-heap-flame-graph.svg
before-QPS.png
> ShortCircuitReplica#unref cost about 6% cpu and 6% heap allocation because of
> the frequent thrown NoSuchElementException in our HBase benchmark
>
>
> Key: HDFS-14541
> URL: https://issues.apache.org/jira/browse/HDFS-14541
> Project: Hadoop HDFS
> Issue Type: Bug
>Reporter: Zheng Hu
>Assignee: Lisheng Sun
>Priority: Major
> Attachments: HDFS-14541.000.patch, after-QPS.png,
> after-cpu-flame-graph.svg, after-heap-flame-graph.svg,
> async-prof-pid-94152-alloc-2.svg, async-prof-pid-94152-cpu-1.svg,
> before-QPS.png, before-cpu-flame-graph.svg, before-heap-flame-graph.svg
>
>
> Our XiaoMi HBase team are evaluating the performence improvement of
> HBASE-21879, and we have few CPU flame graph & heap flame graph by using
> async-profiler, and find that there're some performence issues in DFSClient
> .
> See the attached two flame graphs, we can conclude that the try catch block
> in ShortCircuitCache#trimEvictionMaps has some serious perf problem , we
> should remove the try catch from DFSClient.
> {code}
> /**
>* Trim the eviction lists.
>*/
> private void trimEvictionMaps() {
> long now = Time.monotonicNow();
> demoteOldEvictableMmaped(now);
> while (true) {
> long evictableSize = evictable.size();
> long evictableMmappedSize = evictableMmapped.size();
> if (evictableSize + evictableMmappedSize <= maxTotalSize) {
> return;
> }
> ShortCircuitReplica replica;
> try {
> if (evictableSize == 0) {
> replica = (ShortCircuitReplica)evictableMmapped.get(evictableMmapped
> .firstKey());
> } else {
> replica = (ShortCircuitReplica)evictable.get(evictable.firstKey());
> }
> } catch (NoSuchElementException e) {
> break;
> }
> if (LOG.isTraceEnabled()) {
> LOG.trace(this + ": trimEvictionMaps is purging " + replica +
> StringUtils.getStackTrace(Thread.currentThread()));
> }
> purge(replica);
> }
> }
> {code}
> Our Xiaomi HDFS Team member [~leosun08] will prepare patch for this issue.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-14535) The default 8KB buffer in requestFileDescriptors#BufferedOutputStream is causing lots of heap allocation in HBase when using short-circut read
[ https://issues.apache.org/jira/browse/HDFS-14535?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16865286#comment-16865286 ] Zheng Hu commented on HDFS-14535: - Ping [~tlipcon] & [~jojochuang], we expect the patch can be included in the future 2.8.6 hadoop release, so we need you HDFS guys help to do the simple cherry-pick. sorry for the noise. > The default 8KB buffer in requestFileDescriptors#BufferedOutputStream is > causing lots of heap allocation in HBase when using short-circut read > -- > > Key: HDFS-14535 > URL: https://issues.apache.org/jira/browse/HDFS-14535 > Project: Hadoop HDFS > Issue Type: Bug > Components: hdfs-client >Reporter: Zheng Hu >Assignee: Zheng Hu >Priority: Major > Fix For: 3.3.0 > > Attachments: HDFS-14535.patch > > > Our HBase team are trying to read the blocks from HDFS into pooled offheap > ByteBuffers directly (HBASE-21879), and recently we had some benchmark, > found that almost 45% heap allocation from the DFS client. The heap > allocation flame graph can be see here: > https://issues.apache.org/jira/secure/attachment/12970295/async-prof-pid-25042-alloc-2.svg > After checking the code path, we found that when requesting file descriptors > from a DomainPeer, we allocated huge 8KB buffer for BufferedOutputStream, > though the protocal content was quite small and just few bytes. > It made a heavy GC pressure for HBase when cacheHitRatio < 60%, which > increased the HBase P999 latency. Actually, we can pre-allocate a small > buffer for the BufferedOutputStream, such as 512 bytes, it's enough to read > the short-circuit fd protocal content. we've created a patch like that, and > the allocation flame graph show that after the patch, the heap allocation > from DFS client dropped from 45% to 27%, that's a very good thing I think. > see: > https://issues.apache.org/jira/secure/attachment/12970475/async-prof-pid-24534-alloc-2.svg > Hope this attached patch can be merged into HDFS trunk, also Hadoop-2.8.x, > HBase will benifit a lot from this. > Thanks. > For more details, can see here: > https://issues.apache.org/jira/browse/HBASE-22387?focusedCommentId=16851639=com.atlassian.jira.plugin.system.issuetabpanels%3Acomment-tabpanel#comment-16851639 -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-14535) The default 8KB buffer in requestFileDescriptors#BufferedOutputStream is causing lots of heap allocation in HBase when using short-circut read
[ https://issues.apache.org/jira/browse/HDFS-14535?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16863655#comment-16863655 ] Zheng Hu commented on HDFS-14535: - Ping [~tlipcon] again Thanks. > The default 8KB buffer in requestFileDescriptors#BufferedOutputStream is > causing lots of heap allocation in HBase when using short-circut read > -- > > Key: HDFS-14535 > URL: https://issues.apache.org/jira/browse/HDFS-14535 > Project: Hadoop HDFS > Issue Type: Bug > Components: hdfs-client >Reporter: Zheng Hu >Assignee: Zheng Hu >Priority: Major > Fix For: 3.3.0 > > Attachments: HDFS-14535.patch > > > Our HBase team are trying to read the blocks from HDFS into pooled offheap > ByteBuffers directly (HBASE-21879), and recently we had some benchmark, > found that almost 45% heap allocation from the DFS client. The heap > allocation flame graph can be see here: > https://issues.apache.org/jira/secure/attachment/12970295/async-prof-pid-25042-alloc-2.svg > After checking the code path, we found that when requesting file descriptors > from a DomainPeer, we allocated huge 8KB buffer for BufferedOutputStream, > though the protocal content was quite small and just few bytes. > It made a heavy GC pressure for HBase when cacheHitRatio < 60%, which > increased the HBase P999 latency. Actually, we can pre-allocate a small > buffer for the BufferedOutputStream, such as 512 bytes, it's enough to read > the short-circuit fd protocal content. we've created a patch like that, and > the allocation flame graph show that after the patch, the heap allocation > from DFS client dropped from 45% to 27%, that's a very good thing I think. > see: > https://issues.apache.org/jira/secure/attachment/12970475/async-prof-pid-24534-alloc-2.svg > Hope this attached patch can be merged into HDFS trunk, also Hadoop-2.8.x, > HBase will benifit a lot from this. > Thanks. > For more details, can see here: > https://issues.apache.org/jira/browse/HBASE-22387?focusedCommentId=16851639=com.atlassian.jira.plugin.system.issuetabpanels%3Acomment-tabpanel#comment-16851639 -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Reopened] (HDFS-14535) The default 8KB buffer in requestFileDescriptors#BufferedOutputStream is causing lots of heap allocation in HBase when using short-circut read
[ https://issues.apache.org/jira/browse/HDFS-14535?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zheng Hu reopened HDFS-14535: - Reopen this JIRA, because I think need to cherry-pick to branch-2.8, so that the further 2.8.6 will include this patch. > The default 8KB buffer in requestFileDescriptors#BufferedOutputStream is > causing lots of heap allocation in HBase when using short-circut read > -- > > Key: HDFS-14535 > URL: https://issues.apache.org/jira/browse/HDFS-14535 > Project: Hadoop HDFS > Issue Type: Bug >Reporter: Zheng Hu >Assignee: Zheng Hu >Priority: Major > Fix For: 3.3.0 > > Attachments: HDFS-14535.patch > > > Our HBase team are trying to read the blocks from HDFS into pooled offheap > ByteBuffers directly (HBASE-21879), and recently we had some benchmark, > found that almost 45% heap allocation from the DFS client. The heap > allocation flame graph can be see here: > https://issues.apache.org/jira/secure/attachment/12970295/async-prof-pid-25042-alloc-2.svg > After checking the code path, we found that when requesting file descriptors > from a DomainPeer, we allocated huge 8KB buffer for BufferedOutputStream, > though the protocal content was quite small and just few bytes. > It made a heavy GC pressure for HBase when cacheHitRatio < 60%, which > increased the HBase P999 latency. Actually, we can pre-allocate a small > buffer for the BufferedOutputStream, such as 512 bytes, it's enough to read > the short-circuit fd protocal content. we've created a patch like that, and > the allocation flame graph show that after the patch, the heap allocation > from DFS client dropped from 45% to 27%, that's a very good thing I think. > see: > https://issues.apache.org/jira/secure/attachment/12970475/async-prof-pid-24534-alloc-2.svg > Hope this attached patch can be merged into HDFS trunk, also Hadoop-2.8.x, > HBase will benifit a lot from this. > Thanks. > For more details, can see here: > https://issues.apache.org/jira/browse/HBASE-22387?focusedCommentId=16851639=com.atlassian.jira.plugin.system.issuetabpanels%3Acomment-tabpanel#comment-16851639 -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-14535) The default 8KB buffer in requestFileDescriptors#BufferedOutputStream is causing lots of heap allocation in HBase when using short-circut read
[ https://issues.apache.org/jira/browse/HDFS-14535?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16861671#comment-16861671 ] Zheng Hu commented on HDFS-14535: - Ping [~tlipcon] & [~jojochuang], Thanks. > The default 8KB buffer in requestFileDescriptors#BufferedOutputStream is > causing lots of heap allocation in HBase when using short-circut read > -- > > Key: HDFS-14535 > URL: https://issues.apache.org/jira/browse/HDFS-14535 > Project: Hadoop HDFS > Issue Type: Bug >Reporter: Zheng Hu >Assignee: Zheng Hu >Priority: Major > Fix For: 3.3.0 > > Attachments: HDFS-14535.patch > > > Our HBase team are trying to read the blocks from HDFS into pooled offheap > ByteBuffers directly (HBASE-21879), and recently we had some benchmark, > found that almost 45% heap allocation from the DFS client. The heap > allocation flame graph can be see here: > https://issues.apache.org/jira/secure/attachment/12970295/async-prof-pid-25042-alloc-2.svg > After checking the code path, we found that when requesting file descriptors > from a DomainPeer, we allocated huge 8KB buffer for BufferedOutputStream, > though the protocal content was quite small and just few bytes. > It made a heavy GC pressure for HBase when cacheHitRatio < 60%, which > increased the HBase P999 latency. Actually, we can pre-allocate a small > buffer for the BufferedOutputStream, such as 512 bytes, it's enough to read > the short-circuit fd protocal content. we've created a patch like that, and > the allocation flame graph show that after the patch, the heap allocation > from DFS client dropped from 45% to 27%, that's a very good thing I think. > see: > https://issues.apache.org/jira/secure/attachment/12970475/async-prof-pid-24534-alloc-2.svg > Hope this attached patch can be merged into HDFS trunk, also Hadoop-2.8.x, > HBase will benifit a lot from this. > Thanks. > For more details, can see here: > https://issues.apache.org/jira/browse/HBASE-22387?focusedCommentId=16851639=com.atlassian.jira.plugin.system.issuetabpanels%3Acomment-tabpanel#comment-16851639 -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-14535) The default 8KB buffer in requestFileDescriptors#BufferedOutputStream is causing lots of heap allocation in HBase when using short-circut read
[ https://issues.apache.org/jira/browse/HDFS-14535?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16856285#comment-16856285 ] Zheng Hu commented on HDFS-14535: - bq. If the total number of blocks being opened by the client is not too large, then they should all fit in the cache of open FDs, and we wouldn't need to round-trip to the DN. Maybe for this workload you should consider increasing dfs.client.read.shortcircuit.streams.cache.size from its default 256 (along with increasing hbase's ulimit -n)? Thanks for your suggestion, we've disabled the blockCache of HBase to make all the read QPS to HDFS directly(because we want to show the heap allocation difference when reading into pooled DirectByteBuffers rather than an heap byte array), so there're so many requstShortCircuitFds, that's OK. BTW, I think this improvement can be applied to almost all of the branches (If the branch is not EOL) ? Mind to do the cherry-pick ? HBase team may also need an upgrading from 2.8.5 to the future 2.8.6. Thanks. > The default 8KB buffer in requestFileDescriptors#BufferedOutputStream is > causing lots of heap allocation in HBase when using short-circut read > -- > > Key: HDFS-14535 > URL: https://issues.apache.org/jira/browse/HDFS-14535 > Project: Hadoop HDFS > Issue Type: Bug >Reporter: Zheng Hu >Assignee: Zheng Hu >Priority: Major > Fix For: 3.3.0 > > Attachments: HDFS-14535.patch > > > Our HBase team are trying to read the blocks from HDFS into pooled offheap > ByteBuffers directly (HBASE-21879), and recently we had some benchmark, > found that almost 45% heap allocation from the DFS client. The heap > allocation flame graph can be see here: > https://issues.apache.org/jira/secure/attachment/12970295/async-prof-pid-25042-alloc-2.svg > After checking the code path, we found that when requesting file descriptors > from a DomainPeer, we allocated huge 8KB buffer for BufferedOutputStream, > though the protocal content was quite small and just few bytes. > It made a heavy GC pressure for HBase when cacheHitRatio < 60%, which > increased the HBase P999 latency. Actually, we can pre-allocate a small > buffer for the BufferedOutputStream, such as 512 bytes, it's enough to read > the short-circuit fd protocal content. we've created a patch like that, and > the allocation flame graph show that after the patch, the heap allocation > from DFS client dropped from 45% to 27%, that's a very good thing I think. > see: > https://issues.apache.org/jira/secure/attachment/12970475/async-prof-pid-24534-alloc-2.svg > Hope this attached patch can be merged into HDFS trunk, also Hadoop-2.8.x, > HBase will benifit a lot from this. > Thanks. > For more details, can see here: > https://issues.apache.org/jira/browse/HBASE-22387?focusedCommentId=16851639=com.atlassian.jira.plugin.system.issuetabpanels%3Acomment-tabpanel#comment-16851639 -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Updated] (HDFS-14541) ShortCircuitReplica#unref cost about 6% cpu and 6% heap allocation because of the frequent thrown NoSuchElementException in our HBase benchmark
[
https://issues.apache.org/jira/browse/HDFS-14541?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Zheng Hu updated HDFS-14541:
Description:
Our XiaoMi HBase team are evaluating the performence improvement of
HBASE-21879, and we have few CPU flame graph & heap flame graph by using
async-profiler, and find that there're some performence issues in DFSClient .
See the attached two flame graphs, we can conclude that the try catch block in
ShortCircuitCache#trimEvictionMaps has some serious perf problem , we should
remove the try catch from DFSClient.
{code}
/**
* Trim the eviction lists.
*/
private void trimEvictionMaps() {
long now = Time.monotonicNow();
demoteOldEvictableMmaped(now);
while (true) {
long evictableSize = evictable.size();
long evictableMmappedSize = evictableMmapped.size();
if (evictableSize + evictableMmappedSize <= maxTotalSize) {
return;
}
ShortCircuitReplica replica;
try {
if (evictableSize == 0) {
replica = (ShortCircuitReplica)evictableMmapped.get(evictableMmapped
.firstKey());
} else {
replica = (ShortCircuitReplica)evictable.get(evictable.firstKey());
}
} catch (NoSuchElementException e) {
break;
}
if (LOG.isTraceEnabled()) {
LOG.trace(this + ": trimEvictionMaps is purging " + replica +
StringUtils.getStackTrace(Thread.currentThread()));
}
purge(replica);
}
}
{code}
Our Xiaomi HDFS Team member [~leosun08] will prepare patch for this issue.
was:
Our XiaoMi HBase team are evaluating the performence improvement of
HBASE-21879, and we have few CPU flame graph & heap flame graph by using
async-profiler, and found that there're some performence issues in DFSClient
now .
See the attached two flame graph, we can conclude that the try catch block in
ShortCircuitCache#trimEvictionMaps has some serious perf problem now, we
should remove this from DFSClient.
{code}
/**
* Trim the eviction lists.
*/
private void trimEvictionMaps() {
long now = Time.monotonicNow();
demoteOldEvictableMmaped(now);
while (true) {
long evictableSize = evictable.size();
long evictableMmappedSize = evictableMmapped.size();
if (evictableSize + evictableMmappedSize <= maxTotalSize) {
return;
}
ShortCircuitReplica replica;
try {
if (evictableSize == 0) {
replica = (ShortCircuitReplica)evictableMmapped.get(evictableMmapped
.firstKey());
} else {
replica = (ShortCircuitReplica)evictable.get(evictable.firstKey());
}
} catch (NoSuchElementException e) {
break;
}
if (LOG.isTraceEnabled()) {
LOG.trace(this + ": trimEvictionMaps is purging " + replica +
StringUtils.getStackTrace(Thread.currentThread()));
}
purge(replica);
}
}
{code}
Our Xiaomi HDFS Team member [~leosun08] will prepare patch for this issue.
> ShortCircuitReplica#unref cost about 6% cpu and 6% heap allocation because of
> the frequent thrown NoSuchElementException in our HBase benchmark
>
>
> Key: HDFS-14541
> URL: https://issues.apache.org/jira/browse/HDFS-14541
> Project: Hadoop HDFS
> Issue Type: Bug
>Reporter: Zheng Hu
>Priority: Major
> Attachments: async-prof-pid-94152-alloc-2.svg,
> async-prof-pid-94152-cpu-1.svg
>
>
> Our XiaoMi HBase team are evaluating the performence improvement of
> HBASE-21879, and we have few CPU flame graph & heap flame graph by using
> async-profiler, and find that there're some performence issues in DFSClient
> .
> See the attached two flame graphs, we can conclude that the try catch block
> in ShortCircuitCache#trimEvictionMaps has some serious perf problem , we
> should remove the try catch from DFSClient.
> {code}
> /**
>* Trim the eviction lists.
>*/
> private void trimEvictionMaps() {
> long now = Time.monotonicNow();
> demoteOldEvictableMmaped(now);
> while (true) {
> long evictableSize = evictable.size();
> long evictableMmappedSize = evictableMmapped.size();
> if (evictableSize + evictableMmappedSize <= maxTotalSize) {
> return;
> }
> ShortCircuitReplica replica;
> try {
> if (evictableSize == 0) {
> replica = (ShortCircuitReplica)evictableMmapped.get(evictableMmapped
> .firstKey());
> } else {
> replica = (ShortCircuitReplica)evictable.get(evictable.firstKey());
> }
> } catch (NoSuchElementException e) {
> break;
> }
> if (LOG.isTraceEnabled()) {
>
[jira] [Updated] (HDFS-14541) ShortCircuitReplica#unref cost about 6% cpu and 6% heap allocation because of the frequent thrown NoSuchElementException in our HBase benchmark
[
https://issues.apache.org/jira/browse/HDFS-14541?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Zheng Hu updated HDFS-14541:
Description:
Our XiaoMi HBase team are evaluating the performence improvement of
HBASE-21879, and we have few CPU flame graph & heap flame graph by using
async-profiler, and found that there're some performence issues in DFSClient
now .
See the attached two flame graph, we can conclude that the try catch block in
ShortCircuitCache#trimEvictionMaps has some serious perf problem now, we
should remove this from DFSClient.
{code}
/**
* Trim the eviction lists.
*/
private void trimEvictionMaps() {
long now = Time.monotonicNow();
demoteOldEvictableMmaped(now);
while (true) {
long evictableSize = evictable.size();
long evictableMmappedSize = evictableMmapped.size();
if (evictableSize + evictableMmappedSize <= maxTotalSize) {
return;
}
ShortCircuitReplica replica;
try {
if (evictableSize == 0) {
replica = (ShortCircuitReplica)evictableMmapped.get(evictableMmapped
.firstKey());
} else {
replica = (ShortCircuitReplica)evictable.get(evictable.firstKey());
}
} catch (NoSuchElementException e) {
break;
}
if (LOG.isTraceEnabled()) {
LOG.trace(this + ": trimEvictionMaps is purging " + replica +
StringUtils.getStackTrace(Thread.currentThread()));
}
purge(replica);
}
}
{code}
Our Xiaomi HDFS Team member [~leosun08] will prepare patch for this issue.
was:
Our XiaoMi HBase team are evaluating the performence improvement of
HBASE-21879, and we have few CPU flame graph & heap flame graph by using
async-profiler, and found that there're some performence issues in DFSClient
now .
See the attached two flame graph, we can conclude that the try catch block in
ShortCircuitCache#trimEvictionMaps has some serious perf problem now, we
should remove this from DFSClient.
{code}
/**
* Trim the eviction lists.
*/
private void trimEvictionMaps() {
long now = Time.monotonicNow();
demoteOldEvictableMmaped(now);
while (true) {
long evictableSize = evictable.size();
long evictableMmappedSize = evictableMmapped.size();
if (evictableSize + evictableMmappedSize <= maxTotalSize) {
return;
}
ShortCircuitReplica replica;
try {
if (evictableSize == 0) {
replica = (ShortCircuitReplica)evictableMmapped.get(evictableMmapped
.firstKey());
} else {
replica = (ShortCircuitReplica)evictable.get(evictable.firstKey());
}
} catch (NoSuchElementException e) {
break;
}
if (LOG.isTraceEnabled()) {
LOG.trace(this + ": trimEvictionMaps is purging " + replica +
StringUtils.getStackTrace(Thread.currentThread()));
}
purge(replica);
}
}
{code}
> ShortCircuitReplica#unref cost about 6% cpu and 6% heap allocation because of
> the frequent thrown NoSuchElementException in our HBase benchmark
>
>
> Key: HDFS-14541
> URL: https://issues.apache.org/jira/browse/HDFS-14541
> Project: Hadoop HDFS
> Issue Type: Bug
>Reporter: Zheng Hu
>Priority: Major
> Attachments: async-prof-pid-94152-alloc-2.svg,
> async-prof-pid-94152-cpu-1.svg
>
>
> Our XiaoMi HBase team are evaluating the performence improvement of
> HBASE-21879, and we have few CPU flame graph & heap flame graph by using
> async-profiler, and found that there're some performence issues in DFSClient
> now .
> See the attached two flame graph, we can conclude that the try catch block in
> ShortCircuitCache#trimEvictionMaps has some serious perf problem now, we
> should remove this from DFSClient.
> {code}
> /**
>* Trim the eviction lists.
>*/
> private void trimEvictionMaps() {
> long now = Time.monotonicNow();
> demoteOldEvictableMmaped(now);
> while (true) {
> long evictableSize = evictable.size();
> long evictableMmappedSize = evictableMmapped.size();
> if (evictableSize + evictableMmappedSize <= maxTotalSize) {
> return;
> }
> ShortCircuitReplica replica;
> try {
> if (evictableSize == 0) {
> replica = (ShortCircuitReplica)evictableMmapped.get(evictableMmapped
> .firstKey());
> } else {
> replica = (ShortCircuitReplica)evictable.get(evictable.firstKey());
> }
> } catch (NoSuchElementException e) {
> break;
> }
> if (LOG.isTraceEnabled()) {
> LOG.trace(this + ": trimEvictionMaps is purging " + replica +
>
[jira] [Created] (HDFS-14541) ShortCircuitReplica#unref cost about 6% cpu and 6% heap allocation because of the frequent thrown NoSuchElementException in our HBase benchmark
Zheng Hu created HDFS-14541:
---
Summary: ShortCircuitReplica#unref cost about 6% cpu and 6% heap
allocation because of the frequent thrown NoSuchElementException in our HBase
benchmark
Key: HDFS-14541
URL: https://issues.apache.org/jira/browse/HDFS-14541
Project: Hadoop HDFS
Issue Type: Bug
Reporter: Zheng Hu
Attachments: async-prof-pid-94152-alloc-2.svg,
async-prof-pid-94152-cpu-1.svg
Our XiaoMi HBase team are evaluating the performence improvement of
HBASE-21879, and we have few CPU flame graph & heap flame graph by using
async-profiler, and found that there're some performence issues in DFSClient
now .
See the attached two flame graph, we can conclude that the try catch block in
ShortCircuitCache#trimEvictionMaps has some serious perf problem now, we
should remove this from DFSClient.
{code}
/**
* Trim the eviction lists.
*/
private void trimEvictionMaps() {
long now = Time.monotonicNow();
demoteOldEvictableMmaped(now);
while (true) {
long evictableSize = evictable.size();
long evictableMmappedSize = evictableMmapped.size();
if (evictableSize + evictableMmappedSize <= maxTotalSize) {
return;
}
ShortCircuitReplica replica;
try {
if (evictableSize == 0) {
replica = (ShortCircuitReplica)evictableMmapped.get(evictableMmapped
.firstKey());
} else {
replica = (ShortCircuitReplica)evictable.get(evictable.firstKey());
}
} catch (NoSuchElementException e) {
break;
}
if (LOG.isTraceEnabled()) {
LOG.trace(this + ": trimEvictionMaps is purging " + replica +
StringUtils.getStackTrace(Thread.currentThread()));
}
purge(replica);
}
}
{code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-14535) The default 8KB buffer in requestFileDescriptors#BufferedOutputStream is causing lots of heap allocation in HBase when using short-circut read
[ https://issues.apache.org/jira/browse/HDFS-14535?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16855261#comment-16855261 ] Zheng Hu commented on HDFS-14535: - bq. Emm...IIRC, seems our HBase code have a bug here, for Get operation, all of the query should share the the same reader, which means shouldn't request the short-circuit fd so frequently. OK, I considered something wrong here. It's a big HFile and has many blocks inside, for every block the Get Op wanted, DFS client will requestFileDescriptors, so the problem occur. The HBase implementation is OK. Thanks. > The default 8KB buffer in requestFileDescriptors#BufferedOutputStream is > causing lots of heap allocation in HBase when using short-circut read > -- > > Key: HDFS-14535 > URL: https://issues.apache.org/jira/browse/HDFS-14535 > Project: Hadoop HDFS > Issue Type: Bug >Reporter: Zheng Hu >Assignee: Zheng Hu >Priority: Major > Attachments: HDFS-14535.patch > > > Our HBase team are trying to read the blocks from HDFS into pooled offheap > ByteBuffers directly (HBASE-21879), and recently we had some benchmark, > found that almost 45% heap allocation from the DFS client. The heap > allocation flame graph can be see here: > https://issues.apache.org/jira/secure/attachment/12970295/async-prof-pid-25042-alloc-2.svg > After checking the code path, we found that when requesting file descriptors > from a DomainPeer, we allocated huge 8KB buffer for BufferedOutputStream, > though the protocal content was quite small and just few bytes. > It made a heavy GC pressure for HBase when cacheHitRatio < 60%, which > increased the HBase P999 latency. Actually, we can pre-allocate a small > buffer for the BufferedOutputStream, such as 512 bytes, it's enough to read > the short-circuit fd protocal content. we've created a patch like that, and > the allocation flame graph show that after the patch, the heap allocation > from DFS client dropped from 45% to 27%, that's a very good thing I think. > see: > https://issues.apache.org/jira/secure/attachment/12970475/async-prof-pid-24534-alloc-2.svg > Hope this attached patch can be merged into HDFS trunk, also Hadoop-2.8.x, > HBase will benifit a lot from this. > Thanks. > For more details, can see here: > https://issues.apache.org/jira/browse/HBASE-22387?focusedCommentId=16851639=com.atlassian.jira.plugin.system.issuetabpanels%3Acomment-tabpanel#comment-16851639 -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-14535) The default 8KB buffer in requestFileDescriptors#BufferedOutputStream is causing lots of heap allocation in HBase when using short-circut read
[ https://issues.apache.org/jira/browse/HDFS-14535?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16855258#comment-16855258 ] Zheng Hu commented on HDFS-14535: - bq. if you're missing the "open file" often enough for this to make a difference, is the short-circuit feature actually helpful for the workload? Maybe you would be better off disabling it entirely. Emm...IIRC, seems our HBase code have a bug here, for Get operation, all of the query should share the the same reader, which means shouldn't request the short-circuit fd so frequently. Any way I can check the HBase code again, and the HDFS PR can still be merged ino trunk. bq. I also noticed in the flame graph a couple of other suspicious items like Pattern.compile in DomainSocket.getEffectivePath(). That's not a heavy memory allocator but certainly seems like an easy thing to optimize out for some CPU win (in another patch). That's true, will PR for this if have any time. bq. Mind sending this as a PR for the apache/hadoop repo on github? I've created the attached PR, you can see that. Thanks. > The default 8KB buffer in requestFileDescriptors#BufferedOutputStream is > causing lots of heap allocation in HBase when using short-circut read > -- > > Key: HDFS-14535 > URL: https://issues.apache.org/jira/browse/HDFS-14535 > Project: Hadoop HDFS > Issue Type: Bug >Reporter: Zheng Hu >Assignee: Zheng Hu >Priority: Major > Attachments: HDFS-14535.patch > > > Our HBase team are trying to read the blocks from HDFS into pooled offheap > ByteBuffers directly (HBASE-21879), and recently we had some benchmark, > found that almost 45% heap allocation from the DFS client. The heap > allocation flame graph can be see here: > https://issues.apache.org/jira/secure/attachment/12970295/async-prof-pid-25042-alloc-2.svg > After checking the code path, we found that when requesting file descriptors > from a DomainPeer, we allocated huge 8KB buffer for BufferedOutputStream, > though the protocal content was quite small and just few bytes. > It made a heavy GC pressure for HBase when cacheHitRatio < 60%, which > increased the HBase P999 latency. Actually, we can pre-allocate a small > buffer for the BufferedOutputStream, such as 512 bytes, it's enough to read > the short-circuit fd protocal content. we've created a patch like that, and > the allocation flame graph show that after the patch, the heap allocation > from DFS client dropped from 45% to 27%, that's a very good thing I think. > see: > https://issues.apache.org/jira/secure/attachment/12970475/async-prof-pid-24534-alloc-2.svg > Hope this attached patch can be merged into HDFS trunk, also Hadoop-2.8.x, > HBase will benifit a lot from this. > Thanks. > For more details, can see here: > https://issues.apache.org/jira/browse/HBASE-22387?focusedCommentId=16851639=com.atlassian.jira.plugin.system.issuetabpanels%3Acomment-tabpanel#comment-16851639 -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-14535) The default 8KB buffer in requestFileDescriptors#BufferedOutputStream is causing lots of heap allocation in HBase when using short-circut read
[ https://issues.apache.org/jira/browse/HDFS-14535?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16854325#comment-16854325 ] Zheng Hu commented on HDFS-14535: - Ping [~jojochuang] & [~tlipcon] > The default 8KB buffer in requestFileDescriptors#BufferedOutputStream is > causing lots of heap allocation in HBase when using short-circut read > -- > > Key: HDFS-14535 > URL: https://issues.apache.org/jira/browse/HDFS-14535 > Project: Hadoop HDFS > Issue Type: Bug >Reporter: Zheng Hu >Assignee: Zheng Hu >Priority: Major > Attachments: HDFS-14535.patch > > > Our HBase team are trying to read the blocks from HDFS into pooled offheap > ByteBuffers directly (HBASE-21879), and recently we had some benchmark, > found that almost 45% heap allocation from the DFS client. The heap > allocation flame graph can be see here: > https://issues.apache.org/jira/secure/attachment/12970295/async-prof-pid-25042-alloc-2.svg > After checking the code path, we found that when requesting file descriptors > from a DomainPeer, we allocated huge 8KB buffer for BufferedOutputStream, > though the protocal content was quite small and just few bytes. > It made a heavy GC pressure for HBase when cacheHitRatio < 60%, which > increased the HBase P999 latency. Actually, we can pre-allocate a small > buffer for the BufferedOutputStream, such as 512 bytes, it's enough to read > the short-circuit fd protocal content. we've created a patch like that, and > the allocation flame graph show that after the patch, the heap allocation > from DFS client dropped from 45% to 27%, that's a very good thing I think. > see: > https://issues.apache.org/jira/secure/attachment/12970475/async-prof-pid-24534-alloc-2.svg > Hope this attached patch can be merged into HDFS trunk, also Hadoop-2.8.x, > HBase will benifit a lot from this. > Thanks. > For more details, can see here: > https://issues.apache.org/jira/browse/HBASE-22387?focusedCommentId=16851639=com.atlassian.jira.plugin.system.issuetabpanels%3Acomment-tabpanel#comment-16851639 -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Updated] (HDFS-14535) The default 8KB buffer in requestFileDescriptors#BufferedOutputStream is causing lots of heap allocation in HBase when using short-circut read
[ https://issues.apache.org/jira/browse/HDFS-14535?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zheng Hu updated HDFS-14535: Description: Our HBase team are trying to read the blocks from HDFS into pooled offheap ByteBuffers directly (HBASE-21879), and recently we had some benchmark, found that almost 45% heap allocation from the DFS client. The heap allocation flame graph can be see here: https://issues.apache.org/jira/secure/attachment/12970295/async-prof-pid-25042-alloc-2.svg After checking the code path, we found that when requesting file descriptors from a DomainPeer, we allocated huge 8KB buffer for BufferedOutputStream, though the protocal content was quite small and just few bytes. It made a heavy GC pressure for HBase when cacheHitRatio < 60%, which increased the HBase P999 latency. Actually, we can pre-allocate a small buffer for the BufferedOutputStream, such as 512 bytes, it's enough to read the short-circuit fd protocal content. we've created a patch like that, and the allocation flame graph show that after the patch, the heap allocation from DFS client dropped from 45% to 27%, that's a very good thing I think. see: https://issues.apache.org/jira/secure/attachment/12970475/async-prof-pid-24534-alloc-2.svg Hope this attached patch can be merged into HDFS trunk, also Hadoop-2.8.x, HBase will benifit a lot from this. Thanks. For more details, can see here: https://issues.apache.org/jira/browse/HBASE-22387?focusedCommentId=16851639=com.atlassian.jira.plugin.system.issuetabpanels%3Acomment-tabpanel#comment-16851639 was: Our HBase team are trying to read the blocks from HDFS into pooled offheap ByteBuffers directly (HBASE-21879), and recently we had some benchmark, found that almost 45% heap allocation from the DFS client. The heap allocation flame graph can be see here: https://issues.apache.org/jira/secure/attachment/12970295/async-prof-pid-25042-alloc-2.svg After checking the code path, we found that when requesting file descriptors from a DomainPeer, we allocated huge 8KB buffer for BufferedOutputStream, though the protocal content was quite small and just few bytes. It made a heavy GC pressure for HBase when cacheHitRatio < 60%, which increased the HBase P999 latency. Actually, we can pre-allocate a small buffer for the BufferedOutputStream, such as 512 bytes, it's enough to read the short-circuit fd protocal content. we've created a patch like that, and the allocation flame graph show that after the patch, the heap allocation from DFS client dropped from 45% to 27%, that's a very good thing I think. see: https://issues.apache.org/jira/secure/attachment/12970475/async-prof-pid-24534-alloc-2.svg Hope this attached patch can be merged into HDFS trunk, also Hadoop-2.8.x, HBase will benifit a lot from this. Thanks. > The default 8KB buffer in requestFileDescriptors#BufferedOutputStream is > causing lots of heap allocation in HBase when using short-circut read > -- > > Key: HDFS-14535 > URL: https://issues.apache.org/jira/browse/HDFS-14535 > Project: Hadoop HDFS > Issue Type: Bug >Reporter: Zheng Hu >Assignee: Zheng Hu >Priority: Major > Attachments: HDFS-14535.patch > > > Our HBase team are trying to read the blocks from HDFS into pooled offheap > ByteBuffers directly (HBASE-21879), and recently we had some benchmark, > found that almost 45% heap allocation from the DFS client. The heap > allocation flame graph can be see here: > https://issues.apache.org/jira/secure/attachment/12970295/async-prof-pid-25042-alloc-2.svg > After checking the code path, we found that when requesting file descriptors > from a DomainPeer, we allocated huge 8KB buffer for BufferedOutputStream, > though the protocal content was quite small and just few bytes. > It made a heavy GC pressure for HBase when cacheHitRatio < 60%, which > increased the HBase P999 latency. Actually, we can pre-allocate a small > buffer for the BufferedOutputStream, such as 512 bytes, it's enough to read > the short-circuit fd protocal content. we've created a patch like that, and > the allocation flame graph show that after the patch, the heap allocation > from DFS client dropped from 45% to 27%, that's a very good thing I think. > see: > https://issues.apache.org/jira/secure/attachment/12970475/async-prof-pid-24534-alloc-2.svg > Hope this attached patch can be merged into HDFS trunk, also Hadoop-2.8.x, > HBase will benifit a lot from this. > Thanks. > For more details, can see here: > https://issues.apache.org/jira/browse/HBASE-22387?focusedCommentId=16851639=com.atlassian.jira.plugin.system.issuetabpanels%3Acomment-tabpanel#comment-16851639
[jira] [Updated] (HDFS-14535) The default 8KB buffer in requestFileDescriptors#BufferedOutputStream is causing lots of heap allocation in HBase when using short-circut read
[ https://issues.apache.org/jira/browse/HDFS-14535?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zheng Hu updated HDFS-14535: Attachment: HDFS-14535.patch > The default 8KB buffer in requestFileDescriptors#BufferedOutputStream is > causing lots of heap allocation in HBase when using short-circut read > -- > > Key: HDFS-14535 > URL: https://issues.apache.org/jira/browse/HDFS-14535 > Project: Hadoop HDFS > Issue Type: Bug >Reporter: Zheng Hu >Assignee: Zheng Hu >Priority: Major > Attachments: HDFS-14535.patch > > > Our HBase team are trying to read the blocks from HDFS into pooled offheap > ByteBuffers directly (HBASE-21879), and recently we had some benchmark, > found that almost 45% heap allocation from the DFS client. The heap > allocation flame graph can be see here: > https://issues.apache.org/jira/secure/attachment/12970295/async-prof-pid-25042-alloc-2.svg > After checking the code path, we found that when requesting file descriptors > from a DomainPeer, we allocated huge 8KB buffer for BufferedOutputStream, > though the protocal content was quite small and just few bytes. > It made a heavy GC pressure for HBase when cacheHitRatio < 60%, which > increased the HBase P999 latency. Actually, we can pre-allocate a small > buffer for the BufferedOutputStream, such as 512 bytes, it's enough to read > the short-circuit fd protocal content. we've created a patch like that, and > the allocation flame graph show that after the patch, the heap allocation > from DFS client dropped from 45% to 27%, that's a very good thing I think. > see: > https://issues.apache.org/jira/secure/attachment/12970475/async-prof-pid-24534-alloc-2.svg > Hope this attached patch can be merged into HDFS trunk, also Hadoop-2.8.x, > HBase will benifit a lot from this. > Thanks. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Updated] (HDFS-14535) The default 8KB buffer in requestFileDescriptors#BufferedOutputStream is causing lots of heap allocation in HBase when using short-circut read
[ https://issues.apache.org/jira/browse/HDFS-14535?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zheng Hu updated HDFS-14535: Description: Our HBase team are trying to read the blocks from HDFS into pooled offheap ByteBuffers directly (HBASE-21879), and recently we had some benchmark, found that almost 45% heap allocation from the DFS client. The heap allocation flame graph can be see here: https://issues.apache.org/jira/secure/attachment/12970295/async-prof-pid-25042-alloc-2.svg After checking the code path, we found that when requesting file descriptors from a DomainPeer, we allocated huge 8KB buffer for BufferedOutputStream, though the protocal content was quite small and just few bytes. It made a heavy GC pressure for HBase when cacheHitRatio < 60%, which increased the HBase P999 latency. Actually, we can pre-allocate a small buffer for the BufferedOutputStream, such as 512 bytes, it's enough to read the short-circuit fd protocal content. we've created a patch like that, and the allocation flame graph show that after the patch, the heap allocation from DFS client dropped from 45% to 27%, that's a very good thing I think. see: https://issues.apache.org/jira/secure/attachment/12970475/async-prof-pid-24534-alloc-2.svg Hope this attached patch can be merged into HDFS trunk, also Hadoop-2.8.x, HBase will benifit a lot from this. Thanks. was: Our HBase team are trying to read the blocks from HDFS into pooled offheap ByteBuffers directly (HBASE-21879), and recently we had some benchmark, found that almost 45% heap allocation from the DFS client. The heap allocation flame graph can be see here: https://issues.apache.org/jira/secure/attachment/12970295/async-prof-pid-25042-alloc-2.svg After checking the code path, we found that when requesting file descriptors from a DomainPeer, we allocated huge 8KB buffer for BufferedOutputStream, though the protocal content was quite small and just few bytes. It made a heavy GC pressure for HBase when cacheHitRatio < 60%, which increased the HBase P999 latency. Actually, we can pre-allocate a small buffer for the BufferedOutputStream, such as 512 bytes, it's enough to read the short-circuit fd protocal content. we've created a patch like that, and the allocation flame graph show that after the patch, the heap allocation from DFS client dropped from 45% to 27%, that's a very good thing I think. Hope this attached patch can be merged into HDFS trunk, also Hadoop-2.8.x, HBase will benifit a lot from this. Thanks. > The default 8KB buffer in requestFileDescriptors#BufferedOutputStream is > causing lots of heap allocation in HBase when using short-circut read > -- > > Key: HDFS-14535 > URL: https://issues.apache.org/jira/browse/HDFS-14535 > Project: Hadoop HDFS > Issue Type: Bug >Reporter: Zheng Hu >Assignee: Zheng Hu >Priority: Major > > Our HBase team are trying to read the blocks from HDFS into pooled offheap > ByteBuffers directly (HBASE-21879), and recently we had some benchmark, > found that almost 45% heap allocation from the DFS client. The heap > allocation flame graph can be see here: > https://issues.apache.org/jira/secure/attachment/12970295/async-prof-pid-25042-alloc-2.svg > After checking the code path, we found that when requesting file descriptors > from a DomainPeer, we allocated huge 8KB buffer for BufferedOutputStream, > though the protocal content was quite small and just few bytes. > It made a heavy GC pressure for HBase when cacheHitRatio < 60%, which > increased the HBase P999 latency. Actually, we can pre-allocate a small > buffer for the BufferedOutputStream, such as 512 bytes, it's enough to read > the short-circuit fd protocal content. we've created a patch like that, and > the allocation flame graph show that after the patch, the heap allocation > from DFS client dropped from 45% to 27%, that's a very good thing I think. > see: > https://issues.apache.org/jira/secure/attachment/12970475/async-prof-pid-24534-alloc-2.svg > Hope this attached patch can be merged into HDFS trunk, also Hadoop-2.8.x, > HBase will benifit a lot from this. > Thanks. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Created] (HDFS-14535) The default 8KB buffer in requestFileDescriptors#BufferedOutputStream is causing lots of heap allocation in HBase when using short-circut read
Zheng Hu created HDFS-14535: --- Summary: The default 8KB buffer in requestFileDescriptors#BufferedOutputStream is causing lots of heap allocation in HBase when using short-circut read Key: HDFS-14535 URL: https://issues.apache.org/jira/browse/HDFS-14535 Project: Hadoop HDFS Issue Type: Bug Reporter: Zheng Hu Assignee: Zheng Hu Our HBase team are trying to read the blocks from HDFS into pooled offheap ByteBuffers directly (HBASE-21879), and recently we had some benchmark, found that almost 45% heap allocation from the DFS client. The heap allocation flame graph can be see here: https://issues.apache.org/jira/secure/attachment/12970295/async-prof-pid-25042-alloc-2.svg After checking the code path, we found that when requesting file descriptors from a DomainPeer, we allocated huge 8KB buffer for BufferedOutputStream, though the protocal content was quite small and just few bytes. It made a heavy GC pressure for HBase when cacheHitRatio < 60%, which increased the HBase P999 latency. Actually, we can pre-allocate a small buffer for the BufferedOutputStream, such as 512 bytes, it's enough to read the short-circuit fd protocal content. we've created a patch like that, and the allocation flame graph show that after the patch, the heap allocation from DFS client dropped from 45% to 27%, that's a very good thing I think. Hope this attached patch can be merged into HDFS trunk, also Hadoop-2.8.x, HBase will benifit a lot from this. Thanks. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Created] (HDFS-14483) Backport HDFS-3246 ByteBuffer pread interface to branch-2.8.x
Zheng Hu created HDFS-14483: --- Summary: Backport HDFS-3246 ByteBuffer pread interface to branch-2.8.x Key: HDFS-14483 URL: https://issues.apache.org/jira/browse/HDFS-14483 Project: Hadoop HDFS Issue Type: Task Reporter: Zheng Hu Assignee: Zheng Hu -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-3246) pRead equivalent for direct read path
[ https://issues.apache.org/jira/browse/HDFS-3246?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16835394#comment-16835394 ] Zheng Hu commented on HDFS-3246: Any plan to backport this pread ByteBuffer to hadoop 2.x ? we've a dicussion before [1], and hadoop2.x seems can include this feature, if no object, I'll file a new hdfs jira to accomplish this. FYI [~stakiar], [~jojochuang], [~tlipcon]. Thanks. 1. https://issues.apache.org/jira/browse/HBASE-21879?focusedCommentId=16767865=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-16767865 > pRead equivalent for direct read path > - > > Key: HDFS-3246 > URL: https://issues.apache.org/jira/browse/HDFS-3246 > Project: Hadoop HDFS > Issue Type: Improvement > Components: hdfs-client, performance >Affects Versions: 3.0.0-alpha1 >Reporter: Henry Robinson >Assignee: Sahil Takiar >Priority: Major > Fix For: 3.3.0 > > Attachments: HDFS-3246.001.patch, HDFS-3246.002.patch, > HDFS-3246.003.patch, HDFS-3246.004.patch, HDFS-3246.005.patch, > HDFS-3246.006.patch, HDFS-3246.007.patch > > > There is no pread equivalent in ByteBufferReadable. We should consider adding > one. It would be relatively easy to implement for the distributed case > (certainly compared to HDFS-2834), since DFSInputStream does most of the > heavy lifting. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-3246) pRead equivalent for direct read path
[ https://issues.apache.org/jira/browse/HDFS-3246?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16797952#comment-16797952 ] Zheng Hu commented on HDFS-3246: Left few comments in your PR, thanks for pinging me. > pRead equivalent for direct read path > - > > Key: HDFS-3246 > URL: https://issues.apache.org/jira/browse/HDFS-3246 > Project: Hadoop HDFS > Issue Type: Improvement > Components: hdfs-client, performance >Affects Versions: 3.0.0-alpha1 >Reporter: Henry Robinson >Assignee: Sahil Takiar >Priority: Major > Attachments: HDFS-3246.001.patch, HDFS-3246.002.patch, > HDFS-3246.003.patch, HDFS-3246.004.patch, HDFS-3246.005.patch, > HDFS-3246.006.patch, HDFS-3246.007.patch > > > There is no pread equivalent in ByteBufferReadable. We should consider adding > one. It would be relatively easy to implement for the distributed case > (certainly compared to HDFS-2834), since DFSInputStream does most of the > heavy lifting. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-3246) pRead equivalent for direct read path
[ https://issues.apache.org/jira/browse/HDFS-3246?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16790192#comment-16790192 ] Zheng Hu commented on HDFS-3246: bq. you can push PRs up to github now; yetus takes them there Yeah, seems good to create an github PR. [~stakiar], do you have some bandwidth ? maybe I can help to create an PR. > pRead equivalent for direct read path > - > > Key: HDFS-3246 > URL: https://issues.apache.org/jira/browse/HDFS-3246 > Project: Hadoop HDFS > Issue Type: Improvement > Components: hdfs-client, performance >Affects Versions: 3.0.0-alpha1 >Reporter: Henry Robinson >Assignee: Sahil Takiar >Priority: Major > Attachments: HDFS-3246.001.patch, HDFS-3246.002.patch, > HDFS-3246.003.patch, HDFS-3246.004.patch, HDFS-3246.005.patch, > HDFS-3246.006.patch, HDFS-3246.007.patch > > > There is no pread equivalent in ByteBufferReadable. We should consider adding > one. It would be relatively easy to implement for the distributed case > (certainly compared to HDFS-2834), since DFSInputStream does most of the > heavy lifting. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-3246) pRead equivalent for direct read path
[ https://issues.apache.org/jira/browse/HDFS-3246?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16789119#comment-16789119 ] Zheng Hu commented on HDFS-3246: Ping [~tlipcon] & [~stack] for reviewing this patch, Thanks. [~stakiar], it'll be better if putting this patch on review board: https://reviews.apache.org > pRead equivalent for direct read path > - > > Key: HDFS-3246 > URL: https://issues.apache.org/jira/browse/HDFS-3246 > Project: Hadoop HDFS > Issue Type: Improvement > Components: hdfs-client, performance >Affects Versions: 3.0.0-alpha1 >Reporter: Henry Robinson >Assignee: Sahil Takiar >Priority: Major > Attachments: HDFS-3246.001.patch, HDFS-3246.002.patch, > HDFS-3246.003.patch, HDFS-3246.004.patch, HDFS-3246.005.patch, > HDFS-3246.006.patch, HDFS-3246.007.patch > > > There is no pread equivalent in ByteBufferReadable. We should consider adding > one. It would be relatively easy to implement for the distributed case > (certainly compared to HDFS-2834), since DFSInputStream does most of the > heavy lifting. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-3246) pRead equivalent for direct read path
[ https://issues.apache.org/jira/browse/HDFS-3246?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16786934#comment-16786934 ] Zheng Hu commented on HDFS-3246: bq. Is it possible for HBase to run their tests against a mini-HDFS cluster, that way the tests can use DFSInputStream which does support the ByteBuffer interfaces. Yes, there're some tricks to make the UT work, such as wrap our own InpuStream which implement the ByteBuffer read by copying byte[] from heap to ByteBuffer, but we still need an compatibility branch between non-ByteBuffer read and ByteBuffer read at HBase side. btw, few comments in here: https://issues.apache.org/jira/browse/HBASE-22005?focusedCommentId=16786915=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-16786915 > pRead equivalent for direct read path > - > > Key: HDFS-3246 > URL: https://issues.apache.org/jira/browse/HDFS-3246 > Project: Hadoop HDFS > Issue Type: Improvement > Components: hdfs-client, performance >Affects Versions: 3.0.0-alpha1 >Reporter: Henry Robinson >Assignee: Sahil Takiar >Priority: Major > Attachments: HDFS-3246.001.patch, HDFS-3246.002.patch, > HDFS-3246.003.patch, HDFS-3246.004.patch, HDFS-3246.005.patch, > HDFS-3246.006.patch > > > There is no pread equivalent in ByteBufferReadable. We should consider adding > one. It would be relatively easy to implement for the distributed case > (certainly compared to HDFS-2834), since DFSInputStream does most of the > heavy lifting. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-3246) pRead equivalent for direct read path
[ https://issues.apache.org/jira/browse/HDFS-3246?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16786550#comment-16786550 ] Zheng Hu commented on HDFS-3246: I'm working on this issue HBASE-22005 now, and found our LocalFileSystem did not implement the ByteBufferReadable interface, which make many HBase UT crashed. IMO, we should support ByteBuffer read & pread in all FileSystem ? FYI [~jojochuang]. > pRead equivalent for direct read path > - > > Key: HDFS-3246 > URL: https://issues.apache.org/jira/browse/HDFS-3246 > Project: Hadoop HDFS > Issue Type: Improvement > Components: hdfs-client, performance >Affects Versions: 3.0.0-alpha1 >Reporter: Henry Robinson >Assignee: Sahil Takiar >Priority: Major > Attachments: HDFS-3246.001.patch, HDFS-3246.002.patch, > HDFS-3246.003.patch, HDFS-3246.004.patch, HDFS-3246.005.patch, > HDFS-3246.006.patch > > > There is no pread equivalent in ByteBufferReadable. We should consider adding > one. It would be relatively easy to implement for the distributed case > (certainly compared to HDFS-2834), since DFSInputStream does most of the > heavy lifting. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-3246) pRead equivalent for direct read path
[ https://issues.apache.org/jira/browse/HDFS-3246?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16781270#comment-16781270 ] Zheng Hu commented on HDFS-3246: bq. what benchmarks or tools would you run to ensure it reduces GC overhead? As the HBASE-21879 said, we are observing GC pressure at HBase side when running YCSB benchmark. Amost all of the HBase read/write path are offheap now, except here: pread a HBase data block from HDFS, because of the lack of the ByteBuffer pread interface, so HBase can only read a data block into heap (see [1]). In 100% Get case, graph [2] and [3] indicated that the p999 latency was almost the same as the G1 Young GC STW ( ~ 100ms). so we have the reason to believe that the onheap pread increase GC overhead. while if verify the reducing GC overhead by using HBase, seems not a good way , its stack is too deep and so many variables, any simple HDFS tools ? (Of course, I'll give a final benchmark after HBASE-21879 fixed, but it will take some time, and this issue need to be fixed firstly) 1. http://openinx.github.io/images/hbase-offheap-onheap.png 2. https://issues.apache.org/jira/secure/attachment/12958391/gc-data-before-HBASE-21879.png 3. https://issues.apache.org/jira/secure/attachment/12958487/QPS-latencies-before-HBASE-21879.png > pRead equivalent for direct read path > - > > Key: HDFS-3246 > URL: https://issues.apache.org/jira/browse/HDFS-3246 > Project: Hadoop HDFS > Issue Type: Improvement > Components: hdfs-client, performance >Affects Versions: 3.0.0-alpha1 >Reporter: Henry Robinson >Assignee: Sahil Takiar >Priority: Major > Attachments: HDFS-3246.001.patch, HDFS-3246.002.patch, > HDFS-3246.003.patch, HDFS-3246.004.patch > > > There is no pread equivalent in ByteBufferReadable. We should consider adding > one. It would be relatively easy to implement for the distributed case > (certainly compared to HDFS-2834), since DFSInputStream does most of the > heavy lifting. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-3246) pRead equivalent for direct read path
[ https://issues.apache.org/jira/browse/HDFS-3246?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16781226#comment-16781226 ] Zheng Hu commented on HDFS-3246: [~jojochuang], Do you know anyone who is good at the client side ? Our HBase GC optimazation does highly depends on this issue. Thanks. > pRead equivalent for direct read path > - > > Key: HDFS-3246 > URL: https://issues.apache.org/jira/browse/HDFS-3246 > Project: Hadoop HDFS > Issue Type: Improvement > Components: hdfs-client, performance >Affects Versions: 3.0.0-alpha1 >Reporter: Henry Robinson >Assignee: Sahil Takiar >Priority: Major > Attachments: HDFS-3246.001.patch, HDFS-3246.002.patch, > HDFS-3246.003.patch, HDFS-3246.004.patch > > > There is no pread equivalent in ByteBufferReadable. We should consider adding > one. It would be relatively easy to implement for the distributed case > (certainly compared to HDFS-2834), since DFSInputStream does most of the > heavy lifting. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-3246) pRead equivalent for direct read path
[ https://issues.apache.org/jira/browse/HDFS-3246?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16772810#comment-16772810 ] Zheng Hu commented on HDFS-3246: [~stakiar], any thought about those failed UT and checkstyles ? Thanks > pRead equivalent for direct read path > - > > Key: HDFS-3246 > URL: https://issues.apache.org/jira/browse/HDFS-3246 > Project: Hadoop HDFS > Issue Type: Improvement > Components: hdfs-client, performance >Affects Versions: 3.0.0-alpha1 >Reporter: Henry Robinson >Assignee: Sahil Takiar >Priority: Major > Attachments: HDFS-3246.001.patch > > > There is no pread equivalent in ByteBufferReadable. We should consider adding > one. It would be relatively easy to implement for the distributed case > (certainly compared to HDFS-2834), since DFSInputStream does most of the > heavy lifting. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Updated] (HDFS-3246) pRead equivalent for direct read path
[ https://issues.apache.org/jira/browse/HDFS-3246?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zheng Hu updated HDFS-3246: --- Target Version/s: (was: ) Status: Patch Available (was: Open) > pRead equivalent for direct read path > - > > Key: HDFS-3246 > URL: https://issues.apache.org/jira/browse/HDFS-3246 > Project: Hadoop HDFS > Issue Type: Improvement > Components: hdfs-client, performance >Affects Versions: 3.0.0-alpha1 >Reporter: Henry Robinson >Assignee: Chen Zhang >Priority: Major > Attachments: HDFS-3246.001.patch > > > There is no pread equivalent in ByteBufferReadable. We should consider adding > one. It would be relatively easy to implement for the distributed case > (certainly compared to HDFS-2834), since DFSInputStream does most of the > heavy lifting. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Updated] (HDFS-3246) pRead equivalent for direct read path
[ https://issues.apache.org/jira/browse/HDFS-3246?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zheng Hu updated HDFS-3246: --- Attachment: HDFS-3246.001.patch > pRead equivalent for direct read path > - > > Key: HDFS-3246 > URL: https://issues.apache.org/jira/browse/HDFS-3246 > Project: Hadoop HDFS > Issue Type: Improvement > Components: hdfs-client, performance >Affects Versions: 3.0.0-alpha1 >Reporter: Henry Robinson >Assignee: Chen Zhang >Priority: Major > Attachments: HDFS-3246.001.patch > > > There is no pread equivalent in ByteBufferReadable. We should consider adding > one. It would be relatively easy to implement for the distributed case > (certainly compared to HDFS-2834), since DFSInputStream does most of the > heavy lifting. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-3246) pRead equivalent for direct read path
[ https://issues.apache.org/jira/browse/HDFS-3246?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16770748#comment-16770748 ] Zheng Hu commented on HDFS-3246: Discussed with [~jojochuang] in HBASE-21879 (thanks for his helpful comment), so this feature should go into Hadoop2.x, seems we done part of work in HDFS-8901(the internal bytebuffer pread implementation), while it has not been backported to our Hadoop2.x. I mean we should make both this issue and HDFS-8901 go into Hadoop2.x. Thanks. > pRead equivalent for direct read path > - > > Key: HDFS-3246 > URL: https://issues.apache.org/jira/browse/HDFS-3246 > Project: Hadoop HDFS > Issue Type: Improvement > Components: hdfs-client, performance >Affects Versions: 3.0.0-alpha1 >Reporter: Henry Robinson >Assignee: Chen Zhang >Priority: Major > > There is no pread equivalent in ByteBufferReadable. We should consider adding > one. It would be relatively easy to implement for the distributed case > (certainly compared to HDFS-2834), since DFSInputStream does most of the > heavy lifting. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-3246) pRead equivalent for direct read path
[ https://issues.apache.org/jira/browse/HDFS-3246?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16770727#comment-16770727 ] Zheng Hu commented on HDFS-3246: [~jojochuang], Mind you help to grant the JIRA permission to [~stakiar], he is working on this feature, and out HBase will benifit a lot from his work. Thanks [~stakiar]. > pRead equivalent for direct read path > - > > Key: HDFS-3246 > URL: https://issues.apache.org/jira/browse/HDFS-3246 > Project: Hadoop HDFS > Issue Type: Improvement > Components: hdfs-client, performance >Affects Versions: 3.0.0-alpha1 >Reporter: Henry Robinson >Assignee: Chen Zhang >Priority: Major > > There is no pread equivalent in ByteBufferReadable. We should consider adding > one. It would be relatively easy to implement for the distributed case > (certainly compared to HDFS-2834), since DFSInputStream does most of the > heavy lifting. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
