[
https://issues.apache.org/jira/browse/HDFS-14541?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Zheng Hu updated HDFS-14541:
Status: Open (was: Patch Available)
We use the Github workflow now, so no need to trigger the hadoop QA
[
https://issues.apache.org/jira/browse/HDFS-14541?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16866472#comment-16866472
]
Zheng Hu commented on HDFS-14541:
-
Our XiaoMi HBase Team have made a performance testing, the only
[
https://issues.apache.org/jira/browse/HDFS-14541?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Zheng Hu updated HDFS-14541:
Attachment: after-cpu-flame-graph.svg
after-heap-flame-graph.svg
[
https://issues.apache.org/jira/browse/HDFS-14535?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16865286#comment-16865286
]
Zheng Hu commented on HDFS-14535:
-
Ping [~tlipcon] & [~jojochuang], we expect the patch can be included
[
https://issues.apache.org/jira/browse/HDFS-14535?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16863655#comment-16863655
]
Zheng Hu commented on HDFS-14535:
-
Ping [~tlipcon] again Thanks.
> The default 8KB buffer in
[
https://issues.apache.org/jira/browse/HDFS-14535?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Zheng Hu reopened HDFS-14535:
-
Reopen this JIRA, because I think need to cherry-pick to branch-2.8, so that
the further 2.8.6 will
[
https://issues.apache.org/jira/browse/HDFS-14535?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16861671#comment-16861671
]
Zheng Hu commented on HDFS-14535:
-
Ping [~tlipcon] & [~jojochuang], Thanks.
> The default 8KB buffer in
[
https://issues.apache.org/jira/browse/HDFS-14535?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16856285#comment-16856285
]
Zheng Hu commented on HDFS-14535:
-
bq. If the total number of blocks being opened by the client is not
[
https://issues.apache.org/jira/browse/HDFS-14541?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Zheng Hu updated HDFS-14541:
Description:
Our XiaoMi HBase team are evaluating the performence improvement of
HBASE-21879, and we
[
https://issues.apache.org/jira/browse/HDFS-14541?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Zheng Hu updated HDFS-14541:
Description:
Our XiaoMi HBase team are evaluating the performence improvement of
HBASE-21879, and we
Zheng Hu created HDFS-14541:
---
Summary: ShortCircuitReplica#unref cost about 6% cpu and 6% heap
allocation because of the frequent thrown NoSuchElementException in our HBase
benchmark
Key: HDFS-14541
URL:
[
https://issues.apache.org/jira/browse/HDFS-14535?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16855261#comment-16855261
]
Zheng Hu commented on HDFS-14535:
-
bq. Emm...IIRC, seems our HBase code have a bug here, for Get
[
https://issues.apache.org/jira/browse/HDFS-14535?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16855258#comment-16855258
]
Zheng Hu commented on HDFS-14535:
-
bq. if you're missing the "open file" often enough for this to make a
[
https://issues.apache.org/jira/browse/HDFS-14535?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16854325#comment-16854325
]
Zheng Hu commented on HDFS-14535:
-
Ping [~jojochuang] & [~tlipcon]
> The default 8KB buffer in
[
https://issues.apache.org/jira/browse/HDFS-14535?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Zheng Hu updated HDFS-14535:
Description:
Our HBase team are trying to read the blocks from HDFS into pooled offheap
ByteBuffers
[
https://issues.apache.org/jira/browse/HDFS-14535?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Zheng Hu updated HDFS-14535:
Attachment: HDFS-14535.patch
> The default 8KB buffer in requestFileDescriptors#BufferedOutputStream is
>
[
https://issues.apache.org/jira/browse/HDFS-14535?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Zheng Hu updated HDFS-14535:
Description:
Our HBase team are trying to read the blocks from HDFS into pooled offheap
ByteBuffers
Zheng Hu created HDFS-14535:
---
Summary: The default 8KB buffer in
requestFileDescriptors#BufferedOutputStream is causing lots of heap allocation
in HBase when using short-circut read
Key: HDFS-14535
URL:
Zheng Hu created HDFS-14483:
---
Summary: Backport HDFS-3246 ByteBuffer pread interface to
branch-2.8.x
Key: HDFS-14483
URL: https://issues.apache.org/jira/browse/HDFS-14483
Project: Hadoop HDFS
[
https://issues.apache.org/jira/browse/HDFS-3246?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16835394#comment-16835394
]
Zheng Hu commented on HDFS-3246:
Any plan to backport this pread ByteBuffer to hadoop 2.x ? we've a
[
https://issues.apache.org/jira/browse/HDFS-3246?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16797952#comment-16797952
]
Zheng Hu commented on HDFS-3246:
Left few comments in your PR, thanks for pinging me.
> pRead equivalent
[
https://issues.apache.org/jira/browse/HDFS-3246?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16790192#comment-16790192
]
Zheng Hu commented on HDFS-3246:
bq. you can push PRs up to github now; yetus takes them there
Yeah, seems
[
https://issues.apache.org/jira/browse/HDFS-3246?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16789119#comment-16789119
]
Zheng Hu commented on HDFS-3246:
Ping [~tlipcon] & [~stack] for reviewing this patch, Thanks.
[
https://issues.apache.org/jira/browse/HDFS-3246?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16786934#comment-16786934
]
Zheng Hu commented on HDFS-3246:
bq. Is it possible for HBase to run their tests against a mini-HDFS
[
https://issues.apache.org/jira/browse/HDFS-3246?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16786550#comment-16786550
]
Zheng Hu commented on HDFS-3246:
I'm working on this issue HBASE-22005 now, and found our LocalFileSystem
[
https://issues.apache.org/jira/browse/HDFS-3246?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16781270#comment-16781270
]
Zheng Hu commented on HDFS-3246:
bq. what benchmarks or tools would you run to ensure it reduces GC
[
https://issues.apache.org/jira/browse/HDFS-3246?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16781226#comment-16781226
]
Zheng Hu commented on HDFS-3246:
[~jojochuang], Do you know anyone who is good at the client side ? Our
[
https://issues.apache.org/jira/browse/HDFS-3246?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16772810#comment-16772810
]
Zheng Hu commented on HDFS-3246:
[~stakiar], any thought about those failed UT and checkstyles ? Thanks
[
https://issues.apache.org/jira/browse/HDFS-3246?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Zheng Hu updated HDFS-3246:
---
Target Version/s: (was: )
Status: Patch Available (was: Open)
> pRead equivalent for direct
[
https://issues.apache.org/jira/browse/HDFS-3246?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Zheng Hu updated HDFS-3246:
---
Attachment: HDFS-3246.001.patch
> pRead equivalent for direct read path
>
[
https://issues.apache.org/jira/browse/HDFS-3246?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16770748#comment-16770748
]
Zheng Hu commented on HDFS-3246:
Discussed with [~jojochuang] in HBASE-21879 (thanks for his helpful
[
https://issues.apache.org/jira/browse/HDFS-3246?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16770727#comment-16770727
]
Zheng Hu commented on HDFS-3246:
[~jojochuang], Mind you help to grant the JIRA permission to [~stakiar],
32 matches
Mail list logo
