[jira] [Updated] (HDFS-14313) Get hdfs used space from FsDatasetImpl#volumeMap#ReplicaInfo in memory instead of df/du

2019-04-03 Thread Lisheng Sun (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-14313?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lisheng Sun updated HDFS-14313: --- Attachment: HDFS-14313.001.patch > Get hdfs used space from FsDatasetImpl#volumeMap#ReplicaInfo in

[jira] [Updated] (HDFS-14313) Get hdfs used space from FsDatasetImpl#volumeMap#ReplicaInfo in memory instead of df/du

2019-04-03 Thread Lisheng Sun (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-14313?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lisheng Sun updated HDFS-14313: --- Attachment: HDFS-14313.000.patch > Get hdfs used space from FsDatasetImpl#volumeMap#ReplicaInfo in

[jira] [Updated] (HDFS-14313) Get hdfs used space from FsDatasetImpl#volumeMap#ReplicaInfo in memory instead of df/du

2019-04-03 Thread Lisheng Sun (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-14313?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lisheng Sun updated HDFS-14313: --- Affects Version/s: (was: 3.0.0-alpha1) 2.6.0 2.7.0

[jira] [Updated] (HDFS-14313) Get hdfs used space from FsDatasetImpl#volumeMap#ReplicaInfo instead of df/du

2019-02-26 Thread Lisheng Sun (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-14313?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lisheng Sun updated HDFS-14313: --- Summary: Get hdfs used space from FsDatasetImpl#volumeMap#ReplicaInfo instead of df/du (was: Get

[jira] [Created] (HDFS-14330) Consider StorageID to choose volume

2019-03-03 Thread Lisheng Sun (JIRA)
Lisheng Sun created HDFS-14330: -- Summary: Consider StorageID to choose volume Key: HDFS-14330 URL: https://issues.apache.org/jira/browse/HDFS-14330 Project: Hadoop HDFS Issue Type: Improvement

[jira] [Updated] (HDFS-14290) Unexpected message type: PooledUnsafeDirectByteBuf when get datanode info by DatanodeWebHdfsMethods

2019-02-19 Thread Lisheng Sun (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-14290?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lisheng Sun updated HDFS-14290: --- Status: Patch Available (was: Open) > Unexpected message type: PooledUnsafeDirectByteBuf when get

[jira] [Commented] (HDFS-14290) Unexpected message type: PooledUnsafeDirectByteBuf when get datanode info by DatanodeWebHdfsMethods

2019-02-19 Thread Lisheng Sun (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-14290?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16771772#comment-16771772 ] Lisheng Sun commented on HDFS-14290: I'll attach a patch later. > Unexpected message type:

[jira] [Updated] (HDFS-14290) Unexpected message type: PooledUnsafeDirectByteBuf when get datanode info by DatanodeWebHdfsMethods

2019-02-19 Thread Lisheng Sun (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-14290?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lisheng Sun updated HDFS-14290: --- Attachment: HDFS-14290.000.patch > Unexpected message type: PooledUnsafeDirectByteBuf when get

[jira] [Updated] (HDFS-14313) Get hdfs used space from FsDatasetImpl#volumeMap#ReplicaInfo instead of df/du

2019-02-26 Thread Lisheng Sun (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-14313?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lisheng Sun updated HDFS-14313: --- Description: There are two ways of DU/DF getting used space that are insufficient. #  Running DU

[jira] [Updated] (HDFS-14313) Get hdfs used space from FsDatasetImpl#volumeMap#ReplicaInfo in memory instead of df/du

2019-02-26 Thread Lisheng Sun (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-14313?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lisheng Sun updated HDFS-14313: --- Summary: Get hdfs used space from FsDatasetImpl#volumeMap#ReplicaInfo in memory instead of df/du

[jira] [Commented] (HDFS-14290) Unexpected message type: PooledUnsafeDirectByteBuf when get datanode info by DatanodeWebHdfsMethods

2019-02-27 Thread Lisheng Sun (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-14290?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16780024#comment-16780024 ] Lisheng Sun commented on HDFS-14290: hi [~jojochuang] [~ayushtkn] I is intended to use 

[jira] [Created] (HDFS-14313) Get hdfs used space from FsDatasetImpl#volumeMap instead of df/du

2019-02-22 Thread Lisheng Sun (JIRA)
Lisheng Sun created HDFS-14313: -- Summary: Get hdfs used space from FsDatasetImpl#volumeMap instead of df/du Key: HDFS-14313 URL: https://issues.apache.org/jira/browse/HDFS-14313 Project: Hadoop HDFS

[jira] [Created] (HDFS-14290) Unexpected message type: PooledUnsafeDirectByteBuf when get datanode info by DatanodeWebHdfsMethods

2019-02-17 Thread Lisheng Sun (JIRA)
Lisheng Sun created HDFS-14290: -- Summary: Unexpected message type: PooledUnsafeDirectByteBuf when get datanode info by DatanodeWebHdfsMethods Key: HDFS-14290 URL: https://issues.apache.org/jira/browse/HDFS-14290

[jira] [Updated] (HDFS-14290) Unexpected message type: PooledUnsafeDirectByteBuf when get datanode info by DatanodeWebHdfsMethods

2019-02-17 Thread Lisheng Sun (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-14290?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lisheng Sun updated HDFS-14290: --- Attachment: (was: image-2019-02-18-11-06-15-100.png) > Unexpected message type:

[jira] [Updated] (HDFS-14290) Unexpected message type: PooledUnsafeDirectByteBuf when get datanode info by DatanodeWebHdfsMethods

2019-02-17 Thread Lisheng Sun (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-14290?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lisheng Sun updated HDFS-14290: --- Attachment: webhdfs show.png > Unexpected message type: PooledUnsafeDirectByteBuf when get datanode

[jira] [Updated] (HDFS-14290) Unexpected message type: PooledUnsafeDirectByteBuf when get datanode info by DatanodeWebHdfsMethods

2019-02-17 Thread Lisheng Sun (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-14290?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lisheng Sun updated HDFS-14290: --- Attachment: image-2019-02-18-11-06-15-100.png > Unexpected message type: PooledUnsafeDirectByteBuf

[jira] [Updated] (HDFS-14290) Unexpected message type: PooledUnsafeDirectByteBuf when get datanode info by DatanodeWebHdfsMethods

2019-02-17 Thread Lisheng Sun (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-14290?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lisheng Sun updated HDFS-14290: --- Affects Version/s: 2.7.1 > Unexpected message type: PooledUnsafeDirectByteBuf when get datanode info

[jira] [Updated] (HDFS-14290) Unexpected message type: PooledUnsafeDirectByteBuf when get datanode info by DatanodeWebHdfsMethods

2019-02-17 Thread Lisheng Sun (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-14290?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lisheng Sun updated HDFS-14290: --- Description: The issue is there is no HttpRequestDecoder in InboundHandler of netty,  appear 

[jira] [Created] (HDFS-14289) Unexpected message type: PooledUnsafeDirectByteBuf when get datanode info by DatanodeWebHdfsMethods

2019-02-17 Thread Lisheng Sun (JIRA)
Lisheng Sun created HDFS-14289: -- Summary: Unexpected message type: PooledUnsafeDirectByteBuf when get datanode info by DatanodeWebHdfsMethods Key: HDFS-14289 URL: https://issues.apache.org/jira/browse/HDFS-14289

[jira] [Updated] (HDFS-14290) Unexpected message type: PooledUnsafeDirectByteBuf when get datanode info by DatanodeWebHdfsMethods

2019-02-17 Thread Lisheng Sun (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-14290?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lisheng Sun updated HDFS-14290: --- Description: The issue is there is no HttpRequestDecoder in InboundHandler of netty,  appear 

[jira] [Resolved] (HDFS-14289) Unexpected message type: PooledUnsafeDirectByteBuf when get datanode info by DatanodeWebHdfsMethods

2019-02-18 Thread Lisheng Sun (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-14289?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lisheng Sun resolved HDFS-14289. Resolution: Duplicate > Unexpected message type: PooledUnsafeDirectByteBuf when get datanode info

[jira] [Commented] (HDFS-14290) Unexpected message type: PooledUnsafeDirectByteBuf when get datanode info by DatanodeWebHdfsMethods

2019-02-18 Thread Lisheng Sun (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-14290?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16770989#comment-16770989 ] Lisheng Sun commented on HDFS-14290: [~ayushtkn] Sorry, I create duplicated issues. And I have closed

[jira] [Commented] (HDFS-14290) Unexpected message type: PooledUnsafeDirectByteBuf when get datanode info by DatanodeWebHdfsMethods

2019-02-19 Thread Lisheng Sun (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-14290?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16772547#comment-16772547 ] Lisheng Sun commented on HDFS-14290: Thanks [~jojochuang] for this issue. Later I will add a UT for

[jira] [Assigned] (HDFS-14290) Unexpected message type: PooledUnsafeDirectByteBuf when get datanode info by DatanodeWebHdfsMethods

2019-06-13 Thread Lisheng Sun (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-14290?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lisheng Sun reassigned HDFS-14290: -- Assignee: Lisheng Sun > Unexpected message type: PooledUnsafeDirectByteBuf when get datanode

[jira] [Assigned] (HDFS-14483) Backport HDFS-3246 ByteBuffer pread interface to branch-2.8.x

2019-06-15 Thread Lisheng Sun (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-14483?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lisheng Sun reassigned HDFS-14483: -- Assignee: Lisheng Sun (was: Zheng Hu) > Backport HDFS-3246 ByteBuffer pread interface to

[jira] [Commented] (HDFS-14313) Get hdfs used space from FsDatasetImpl#volumeMap#ReplicaInfo in memory instead of df/du

2019-06-16 Thread Lisheng Sun (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-14313?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16865274#comment-16865274 ] Lisheng Sun commented on HDFS-14313: [~jojochuang] Thanks for your review.

[jira] [Commented] (HDFS-14290) Unexpected message type: PooledUnsafeDirectByteBuf when get datanode info by DatanodeWebHdfsMethods

2019-06-10 Thread Lisheng Sun (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-14290?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16859741#comment-16859741 ] Lisheng Sun commented on HDFS-14290: [~jojochuang] [~ayushtkn] Could you please take a look? >

[jira] [Commented] (HDFS-14313) Get hdfs used space from FsDatasetImpl#volumeMap#ReplicaInfo in memory instead of df/du

2019-06-10 Thread Lisheng Sun (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-14313?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16859745#comment-16859745 ] Lisheng Sun commented on HDFS-14313: [~joepe] [~zhuqi] [~zvenczel] [~candychencan] Could you please

[jira] [Assigned] (HDFS-13694) Making md5 computing being in parallel with image loading

2019-06-18 Thread Lisheng Sun (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-13694?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lisheng Sun reassigned HDFS-13694: -- Assignee: Lisheng Sun (was: zhouyingchao) > Making md5 computing being in parallel with

[jira] [Assigned] (HDFS-13693) Remove unnecessary search in INodeDirectory.addChild during image loading

2019-06-18 Thread Lisheng Sun (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-13693?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lisheng Sun reassigned HDFS-13693: -- Assignee: Lisheng Sun (was: zhouyingchao) > Remove unnecessary search in

[jira] [Assigned] (HDFS-13700) The process of loading image can be done in a pipeline model

2019-06-18 Thread Lisheng Sun (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-13700?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lisheng Sun reassigned HDFS-13700: -- Assignee: Lisheng Sun (was: zhouyingchao) > The process of loading image can be done in a

[jira] [Updated] (HDFS-13694) Making md5 computing being in parallel with image loading

2019-06-24 Thread Lisheng Sun (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-13694?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lisheng Sun updated HDFS-13694: --- Attachment: HDFS-13694-003.patch > Making md5 computing being in parallel with image loading >

[jira] [Updated] (HDFS-13694) Making md5 computing being in parallel with image loading

2019-06-24 Thread Lisheng Sun (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-13694?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lisheng Sun updated HDFS-13694: --- Attachment: HDFS-13694-004.patch > Making md5 computing being in parallel with image loading >

[jira] [Updated] (HDFS-13694) Making md5 computing being in parallel with image loading

2019-06-25 Thread Lisheng Sun (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-13694?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lisheng Sun updated HDFS-13694: --- Attachment: HDFS-13694-005.patch > Making md5 computing being in parallel with image loading >

[jira] [Comment Edited] (HDFS-13694) Making md5 computing being in parallel with image loading

2019-06-25 Thread Lisheng Sun (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-13694?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16872073#comment-16872073 ] Lisheng Sun edited comment on HDFS-13694 at 6/25/19 6:47 AM: - [~jojochuang] 

[jira] [Commented] (HDFS-13694) Making md5 computing being in parallel with image loading

2019-06-25 Thread Lisheng Sun (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-13694?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16872073#comment-16872073 ] Lisheng Sun commented on HDFS-13694: [~jojochuang]  I fix he findbugs issue and push this patch to

[jira] [Updated] (HDFS-13694) Making md5 computing being in parallel with image loading

2019-06-25 Thread Lisheng Sun (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-13694?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lisheng Sun updated HDFS-13694: --- Attachment: (was: HDFS-13694-005.patch) > Making md5 computing being in parallel with image

[jira] [Updated] (HDFS-13694) Making md5 computing being in parallel with image loading

2019-06-25 Thread Lisheng Sun (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-13694?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lisheng Sun updated HDFS-13694: --- Attachment: HDFS-13694-005.patch > Making md5 computing being in parallel with image loading >

[jira] [Commented] (HDFS-14541) ShortCircuitReplica#unref cost about 6% cpu and 6% heap allocation because of the frequent thrown NoSuchElementException in our HBase benchmark

2019-06-12 Thread Lisheng Sun (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-14541?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16861789#comment-16861789 ] Lisheng Sun commented on HDFS-14541: Hi [~jojochuang], Mind you help to grant the JIRA permission to

[jira] [Created] (HDFS-14585) Backport HDFS-8901 Use ByteBuffer in striping positional read

2019-06-20 Thread Lisheng Sun (JIRA)
Lisheng Sun created HDFS-14585: -- Summary: Backport HDFS-8901 Use ByteBuffer in striping positional read Key: HDFS-14585 URL: https://issues.apache.org/jira/browse/HDFS-14585 Project: Hadoop HDFS

[jira] [Updated] (HDFS-14585) Backport HDFS-8901 Use ByteBuffer in striping positional read to branch-2.8.x

2019-06-20 Thread Lisheng Sun (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-14585?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lisheng Sun updated HDFS-14585: --- Summary: Backport HDFS-8901 Use ByteBuffer in striping positional read to branch-2.8.x (was:

[jira] [Updated] (HDFS-14585) Backport HDFS-8901 Use ByteBuffer in striping positional read to branch-2.8.x

2019-06-20 Thread Lisheng Sun (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-14585?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lisheng Sun updated HDFS-14585: --- Attachment: HDFS-14585.branch285.000.patch > Backport HDFS-8901 Use ByteBuffer in striping

[jira] [Updated] (HDFS-14585) Backport HDFS-8901 Use ByteBuffer in striping positional read to branch-2.8.x

2019-06-20 Thread Lisheng Sun (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-14585?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lisheng Sun updated HDFS-14585: --- Status: Patch Available (was: Open) > Backport HDFS-8901 Use ByteBuffer in striping positional read

[jira] [Updated] (HDFS-14585) Backport HDFS-8901 Use ByteBuffer in striping positional read to branch-2.8.x

2019-06-20 Thread Lisheng Sun (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-14585?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lisheng Sun updated HDFS-14585: --- Status: Open (was: Patch Available) > Backport HDFS-8901 Use ByteBuffer in striping positional read

[jira] [Updated] (HDFS-14585) Backport HDFS-8901 Use ByteBuffer in striping positional read to branch-2.8.x

2019-06-20 Thread Lisheng Sun (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-14585?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lisheng Sun updated HDFS-14585: --- Attachment: HDFS-14585.branch-2.8.5.v1.patch Status: Patch Available (was: Open) > Backport

[jira] [Commented] (HDFS-14585) Backport HDFS-8901 Use ByteBuffer in striping positional read to branch-2.8.x

2019-06-21 Thread Lisheng Sun (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-14585?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16869208#comment-16869208 ] Lisheng Sun commented on HDFS-14585: [~jojochuang] Thanks for your review.   {quote}Suggest to

[jira] [Updated] (HDFS-14585) Backport HDFS-8901 Use ByteBuffer in striping positional read to branch-2.8.x

2019-06-20 Thread Lisheng Sun (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-14585?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lisheng Sun updated HDFS-14585: --- Attachment: (was: HDFS-14585.branch-2.8.5.v1.patch) > Backport HDFS-8901 Use ByteBuffer in

[jira] [Updated] (HDFS-14585) Backport HDFS-8901 Use ByteBuffer in striping positional read to branch-2.8.x

2019-06-20 Thread Lisheng Sun (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-14585?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lisheng Sun updated HDFS-14585: --- Status: Open (was: Patch Available) > Backport HDFS-8901 Use ByteBuffer in striping positional read

[jira] [Updated] (HDFS-14585) Backport HDFS-8901 Use ByteBuffer in striping positional read to branch-2.8.x

2019-06-20 Thread Lisheng Sun (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-14585?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lisheng Sun updated HDFS-14585: --- Attachment: (was: HDFS-14585.branch285.000.patch) > Backport HDFS-8901 Use ByteBuffer in

[jira] [Commented] (HDFS-14585) Backport HDFS-8901 Use ByteBuffer in striping positional read to branch-2.8.x

2019-06-20 Thread Lisheng Sun (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-14585?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16869147#comment-16869147 ] Lisheng Sun commented on HDFS-14585: [~jojochuang] Could you please take a look ? HDFS-14483 is

[jira] [Updated] (HDFS-14483) Backport HDFS-3246 ByteBuffer pread interface to branch-2.8.x

2019-06-20 Thread Lisheng Sun (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-14483?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lisheng Sun updated HDFS-14483: --- Attachment: HDFS-14483.branch-2.8.v1.patch Status: Patch Available (was: Open) > Backport

[jira] [Updated] (HDFS-14585) Backport HDFS-8901 Use ByteBuffer in striping positional read to branch-2.8.x

2019-06-20 Thread Lisheng Sun (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-14585?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lisheng Sun updated HDFS-14585: --- Attachment: HDFS-14585.branch-2.8.v1.patch Status: Patch Available (was: Open) > Backport

[jira] [Commented] (HDFS-14313) Get hdfs used space from FsDatasetImpl#volumeMap#ReplicaInfo in memory instead of df/du

2019-06-21 Thread Lisheng Sun (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-14313?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16869213#comment-16869213 ] Lisheng Sun commented on HDFS-14313: ping [~jojochuang]    I don’t know I explain whether to clear.  

[jira] [Comment Edited] (HDFS-14585) Backport HDFS-8901 Use ByteBuffer in striping positional read to branch-2.8.x

2019-06-21 Thread Lisheng Sun (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-14585?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16869208#comment-16869208 ] Lisheng Sun edited comment on HDFS-14585 at 6/21/19 6:11 AM: - [~jojochuang]

[jira] [Comment Edited] (HDFS-14585) Backport HDFS-8901 Use ByteBuffer in striping positional read to branch-2.8.x

2019-06-21 Thread Lisheng Sun (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-14585?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16869208#comment-16869208 ] Lisheng Sun edited comment on HDFS-14585 at 6/21/19 6:03 AM: - [~jojochuang]

[jira] [Updated] (HDFS-14585) Backport HDFS-8901 Use ByteBuffer in DFSInputStream#read to branch-2 and branch2.9

2019-06-21 Thread Lisheng Sun (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-14585?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lisheng Sun updated HDFS-14585: --- Attachment: HDFS-14585.branch-2.9.v1.patch > Backport HDFS-8901 Use ByteBuffer in

[jira] [Commented] (HDFS-14313) Get hdfs used space from FsDatasetImpl#volumeMap#ReplicaInfo in memory instead of df/du

2019-06-21 Thread Lisheng Sun (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-14313?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16869322#comment-16869322 ] Lisheng Sun commented on HDFS-14313: [~jojochuang] Thans for your review. {{{quote}}}

[jira] [Updated] (HDFS-14585) Backport HDFS-8901 Use ByteBuffer in DFSInputStream#read to branch-2.8.x

2019-06-21 Thread Lisheng Sun (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-14585?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lisheng Sun updated HDFS-14585: --- Attachment: HDFS-14585.branch-2.v1.patch > Backport HDFS-8901 Use ByteBuffer in DFSInputStream#read

[jira] [Updated] (HDFS-14585) Backport HDFS-8901 Use ByteBuffer in DFSInputStream#read to branch-2 and branch2.9

2019-06-21 Thread Lisheng Sun (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-14585?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lisheng Sun updated HDFS-14585: --- Attachment: (was: HDFS-14585.branch-2.8.v1.patch) > Backport HDFS-8901 Use ByteBuffer in

[jira] [Updated] (HDFS-14585) Backport HDFS-8901 Use ByteBuffer in DFSInputStream#read to branch-2 and branch2.9

2019-06-21 Thread Lisheng Sun (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-14585?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lisheng Sun updated HDFS-14585: --- Summary: Backport HDFS-8901 Use ByteBuffer in DFSInputStream#read to branch-2 and branch2.9 (was:

[jira] [Commented] (HDFS-14585) Backport HDFS-8901 Use ByteBuffer in DFSInputStream#read to branch-2 and branch2.9

2019-06-21 Thread Lisheng Sun (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-14585?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16869296#comment-16869296 ] Lisheng Sun commented on HDFS-14585: [~jojochuang]  I backport  HDFS-8901 to branch-2 and branch-2.9.

[jira] [Comment Edited] (HDFS-14313) Get hdfs used space from FsDatasetImpl#volumeMap#ReplicaInfo in memory instead of df/du

2019-06-21 Thread Lisheng Sun (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-14313?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16869322#comment-16869322 ] Lisheng Sun edited comment on HDFS-14313 at 6/21/19 9:24 AM: - [~jojochuang]

[jira] [Comment Edited] (HDFS-14313) Get hdfs used space from FsDatasetImpl#volumeMap#ReplicaInfo in memory instead of df/du

2019-06-21 Thread Lisheng Sun (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-14313?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16869322#comment-16869322 ] Lisheng Sun edited comment on HDFS-14313 at 6/21/19 9:25 AM: - [~jojochuang]

[jira] [Updated] (HDFS-14585) Backport HDFS-8901 Use ByteBuffer in DFSInputStream#read to branch-2.8.x

2019-06-21 Thread Lisheng Sun (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-14585?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lisheng Sun updated HDFS-14585: --- Summary: Backport HDFS-8901 Use ByteBuffer in DFSInputStream#read to branch-2.8.x (was: Backport

[jira] [Updated] (HDFS-14313) Get hdfs used space from FsDatasetImpl#volumeMap#ReplicaInfo in memory instead of df/du

2019-06-23 Thread Lisheng Sun (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-14313?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lisheng Sun updated HDFS-14313: --- Attachment: HDFS-14313.002.patch > Get hdfs used space from FsDatasetImpl#volumeMap#ReplicaInfo in

[jira] [Issue Comment Deleted] (HDFS-14313) Get hdfs used space from FsDatasetImpl#volumeMap#ReplicaInfo in memory instead of df/du

2019-06-23 Thread Lisheng Sun (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-14313?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lisheng Sun updated HDFS-14313: --- Comment: was deleted (was: [~jojochuang] Thanks for your comments.  I add lock in 

[jira] [Commented] (HDFS-14313) Get hdfs used space from FsDatasetImpl#volumeMap#ReplicaInfo in memory instead of df/du

2019-06-23 Thread Lisheng Sun (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-14313?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16870797#comment-16870797 ] Lisheng Sun commented on HDFS-14313: [~jojochuang] Thanks for your comment .I add synchronized to

[jira] [Commented] (HDFS-14313) Get hdfs used space from FsDatasetImpl#volumeMap#ReplicaInfo in memory instead of df/du

2019-06-23 Thread Lisheng Sun (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-14313?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16870773#comment-16870773 ] Lisheng Sun commented on HDFS-14313: [~jojochuang] Thanks for your comments.  I add lock in 

[jira] [Updated] (HDFS-14313) Get hdfs used space from FsDatasetImpl#volumeMap#ReplicaInfo in memory instead of df/du

2019-06-23 Thread Lisheng Sun (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-14313?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lisheng Sun updated HDFS-14313: --- Attachment: HDFS-14313.003.patch > Get hdfs used space from FsDatasetImpl#volumeMap#ReplicaInfo in

[jira] [Commented] (HDFS-14585) Backport HDFS-8901 Use ByteBuffer in DFSInputStream#read to branch-2 and branch2.9

2019-06-24 Thread Lisheng Sun (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-14585?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16870833#comment-16870833 ] Lisheng Sun commented on HDFS-14585: ping [~jojochuang]  I have backported  HDFS-8901 to branch-2

[jira] [Updated] (HDFS-14541) ShortCircuitReplica#unref cost about 6% cpu and 6% heap allocation because of the frequent thrown NoSuchElementException in our HBase benchmark

2019-06-24 Thread Lisheng Sun (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-14541?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lisheng Sun updated HDFS-14541: --- Attachment: HDFS-14541.001.patch > ShortCircuitReplica#unref cost about 6% cpu and 6% heap

[jira] [Commented] (HDFS-14541) ShortCircuitReplica#unref cost about 6% cpu and 6% heap allocation because of the frequent thrown NoSuchElementException in our HBase benchmark

2019-06-24 Thread Lisheng Sun (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-14541?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16871108#comment-16871108 ] Lisheng Sun commented on HDFS-14541: [~elgoiri] [~jojochuang] Could you help review this patch? Thank

[jira] [Updated] (HDFS-13694) Making md5 computing being in parallel with image loading

2019-06-24 Thread Lisheng Sun (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-13694?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lisheng Sun updated HDFS-13694: --- Attachment: HDFS-13694-002.patch > Making md5 computing being in parallel with image loading >

[jira] [Updated] (HDFS-14541) ShortCircuitReplica#unref cost about 6% cpu and 6% heap allocation because of the frequent thrown NoSuchElementException in our HBase benchmark

2019-06-24 Thread Lisheng Sun (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-14541?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lisheng Sun updated HDFS-14541: --- Attachment: HDFS-14541.002.patch > ShortCircuitReplica#unref cost about 6% cpu and 6% heap

[jira] [Work started] (HDFS-14541) ShortCircuitReplica#unref cost about 6% cpu and 6% heap allocation because of the frequent thrown NoSuchElementException in our HBase benchmark

2019-06-18 Thread Lisheng Sun (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-14541?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Work on HDFS-14541 started by Lisheng Sun. -- > ShortCircuitReplica#unref cost about 6% cpu and 6% heap allocation because of >

[jira] [Commented] (HDFS-14541) ShortCircuitReplica#unref cost about 6% cpu and 6% heap allocation because of the frequent thrown NoSuchElementException in our HBase benchmark

2019-06-18 Thread Lisheng Sun (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-14541?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16866552#comment-16866552 ] Lisheng Sun commented on HDFS-14541: Thanks for looking into this, [~elgoiri] .Your analysis makes

[jira] [Updated] (HDFS-14541) ShortCircuitReplica#unref cost about 6% cpu and 6% heap allocation because of the frequent thrown NoSuchElementException in our HBase benchmark

2019-06-18 Thread Lisheng Sun (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-14541?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lisheng Sun updated HDFS-14541: --- Attachment: (was: HDFS-14541.001.patch) > ShortCircuitReplica#unref cost about 6% cpu and 6%

[jira] [Updated] (HDFS-14541) ShortCircuitReplica#unref cost about 6% cpu and 6% heap allocation because of the frequent thrown NoSuchElementException in our HBase benchmark

2019-06-18 Thread Lisheng Sun (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-14541?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lisheng Sun updated HDFS-14541: --- Status: Open (was: Patch Available) > ShortCircuitReplica#unref cost about 6% cpu and 6% heap

[jira] [Updated] (HDFS-14541) ShortCircuitReplica#unref cost about 6% cpu and 6% heap allocation because of the frequent thrown NoSuchElementException in our HBase benchmark

2019-06-18 Thread Lisheng Sun (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-14541?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lisheng Sun updated HDFS-14541: --- Attachment: HDFS-14541.001.patch Status: Patch Available (was: In Progress) >

[jira] [Updated] (HDFS-14541) ShortCircuitReplica#unref cost about 6% cpu and 6% heap allocation because of the frequent thrown NoSuchElementException in our HBase benchmark

2019-06-17 Thread Lisheng Sun (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-14541?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lisheng Sun updated HDFS-14541: --- Attachment: (was: HDFS-14541.000.patch) > ShortCircuitReplica#unref cost about 6% cpu and 6%

[jira] [Updated] (HDFS-14541) ShortCircuitReplica#unref cost about 6% cpu and 6% heap allocation because of the frequent thrown NoSuchElementException in our HBase benchmark

2019-06-17 Thread Lisheng Sun (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-14541?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lisheng Sun updated HDFS-14541: --- Attachment: HDFS-14541.000.patch Status: Patch Available (was: Open) >

[jira] [Updated] (HDFS-14541) ShortCircuitReplica#unref cost about 6% cpu and 6% heap allocation because of the frequent thrown NoSuchElementException in our HBase benchmark

2019-06-17 Thread Lisheng Sun (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-14541?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lisheng Sun updated HDFS-14541: --- Attachment: HDFS-14541.000.patch > ShortCircuitReplica#unref cost about 6% cpu and 6% heap

[jira] [Commented] (HDFS-14313) Get hdfs used space from FsDatasetImpl#volumeMap#ReplicaInfo in memory instead of df/du

2019-05-14 Thread Lisheng Sun (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-14313?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16839140#comment-16839140 ] Lisheng Sun commented on HDFS-14313: Test failures are not caused by this patch.  > Get hdfs used

[jira] [Commented] (HDFS-13694) Making md5 computing being in parallel with image loading

2019-07-05 Thread Lisheng Sun (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-13694?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16879580#comment-16879580 ] Lisheng Sun commented on HDFS-13694: Thank [~elgoiri]  for merging this patch.  I think we need to

[jira] [Commented] (HDFS-14483) Backport HDFS-14111,HDFS-3246 ByteBuffer pread interface to branch-2.9

2019-07-05 Thread Lisheng Sun (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-14483?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16879581#comment-16879581 ] Lisheng Sun commented on HDFS-14483: Hi [~stack]  Could you have time to continue reviewing this

[jira] [Commented] (HDFS-14313) Get hdfs used space from FsDatasetImpl#volumeMap#ReplicaInfo in memory instead of df/du

2019-07-05 Thread Lisheng Sun (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-14313?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16879582#comment-16879582 ] Lisheng Sun commented on HDFS-14313: Hi [~elgoiri] [~jojochuang] [~zvenczel] [~hexiaoqiao] Could you

[jira] [Assigned] (HDFS-13571) Dead datanode detector

2019-07-06 Thread Lisheng Sun (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-13571?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lisheng Sun reassigned HDFS-13571: -- Assignee: Lisheng Sun > Dead datanode detector > -- > >

[jira] [Assigned] (HDFS-13639) SlotReleaser is not fast enough

2019-07-06 Thread Lisheng Sun (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-13639?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lisheng Sun reassigned HDFS-13639: -- Assignee: Lisheng Sun > SlotReleaser is not fast enough > --- > >

[jira] [Commented] (HDFS-14585) Backport HDFS-8901 Use ByteBuffer in DFSInputStream#read to branch-2 and branch2.9

2019-06-26 Thread Lisheng Sun (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-14585?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16873239#comment-16873239 ] Lisheng Sun commented on HDFS-14585: [~xuzq_zander] [~jojochuang] Could you have time to  review this

[jira] [Updated] (HDFS-13694) Making md5 computing being in parallel with image loading

2019-06-27 Thread Lisheng Sun (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-13694?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lisheng Sun updated HDFS-13694: --- Description: During namenode image loading, it firstly compute the md5 and then load the image.

[jira] [Updated] (HDFS-13694) Making md5 computing being in parallel with image loading

2019-06-27 Thread Lisheng Sun (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-13694?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lisheng Sun updated HDFS-13694: --- Description: During namenode image loading, it firstly compute the md5 and then load the image.

[jira] [Updated] (HDFS-13694) Making md5 computing being in parallel with image loading

2019-06-27 Thread Lisheng Sun (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-13694?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lisheng Sun updated HDFS-13694: --- Description: During namenode image loading, it firstly compute the md5 and then load the image.

[jira] [Comment Edited] (HDFS-13694) Making md5 computing being in parallel with image loading

2019-06-28 Thread Lisheng Sun (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-13694?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16874784#comment-16874784 ] Lisheng Sun edited comment on HDFS-13694 at 6/28/19 8:47 AM: - Thank

[jira] [Updated] (HDFS-13694) Making md5 computing being in parallel with image loading

2019-06-28 Thread Lisheng Sun (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-13694?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lisheng Sun updated HDFS-13694: --- Attachment: HDFS-13694-006.patch > Making md5 computing being in parallel with image loading >

[jira] [Commented] (HDFS-13694) Making md5 computing being in parallel with image loading

2019-06-28 Thread Lisheng Sun (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-13694?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16874784#comment-16874784 ] Lisheng Sun commented on HDFS-13694: Thank [~hexiaoqiao] for your comments. I have updated this patch

[jira] [Updated] (HDFS-14585) Backport HDFS-8901 Use ByteBuffer in DFSInputStream#read to branch2.9

2019-06-28 Thread Lisheng Sun (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-14585?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lisheng Sun updated HDFS-14585: --- Summary: Backport HDFS-8901 Use ByteBuffer in DFSInputStream#read to branch2.9 (was: Backport

[jira] [Updated] (HDFS-14585) Backport HDFS-8901 Use ByteBuffer in DFSInputStream#read to branch2.9

2019-06-28 Thread Lisheng Sun (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-14585?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lisheng Sun updated HDFS-14585: --- Attachment: HDFS-14585.branch-2.9.v2.patch > Backport HDFS-8901 Use ByteBuffer in

[jira] [Updated] (HDFS-14585) Backport HDFS-8901 Use ByteBuffer in DFSInputStream#read to branch2.9

2019-06-28 Thread Lisheng Sun (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-14585?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lisheng Sun updated HDFS-14585: --- Attachment: HDFS-14585.branch-2.9.v2.patch > Backport HDFS-8901 Use ByteBuffer in

[jira] [Commented] (HDFS-14585) Backport HDFS-8901 Use ByteBuffer in DFSInputStream#read to branch2.9

2019-06-28 Thread Lisheng Sun (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-14585?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16874822#comment-16874822 ] Lisheng Sun commented on HDFS-14585: Thank [~stack] for your comments. I have updated the patch for

[jira] [Comment Edited] (HDFS-14585) Backport HDFS-8901 Use ByteBuffer in DFSInputStream#read to branch2.9

2019-06-28 Thread Lisheng Sun (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-14585?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16874822#comment-16874822 ] Lisheng Sun edited comment on HDFS-14585 at 6/28/19 10:01 AM: -- Thank

  1   2   3   4   5   6   7   8   9   10   >