[jira] [Updated] (HDFS-13693) Remove unnecessary search in INodeDirectory.addChild during image loading

2019-07-18 Thread Lisheng Sun (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-13693?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lisheng Sun updated HDFS-13693: --- Attachment: HDFS-13693-005.patch > Remove unnecessary search in INodeDirectory.addChild during image

[jira] [Updated] (HDFS-13693) Remove unnecessary search in INodeDirectory.addChild during image loading

2019-07-18 Thread Lisheng Sun (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-13693?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lisheng Sun updated HDFS-13693: --- Attachment: HDFS-13693-004.patch > Remove unnecessary search in INodeDirectory.addChild during image

[jira] [Updated] (HDFS-13693) Remove unnecessary search in INodeDirectory.addChild during image loading

2019-07-18 Thread Lisheng Sun (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-13693?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lisheng Sun updated HDFS-13693: --- Attachment: (was: HDFS-13693-004.patch) > Remove unnecessary search in INodeDirectory.addChild du

[jira] [Commented] (HDFS-13693) Remove unnecessary search in INodeDirectory.addChild during image loading

2019-07-18 Thread Lisheng Sun (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-13693?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16888483#comment-16888483 ] Lisheng Sun commented on HDFS-13693: [~jojochuang] [~hexiaoqiao] I have updated this

[jira] [Commented] (HDFS-14313) Get hdfs used space from FsDatasetImpl#volumeMap#ReplicaInfo in memory instead of df/du

2019-07-21 Thread Lisheng Sun (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-14313?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16889667#comment-16889667 ] Lisheng Sun commented on HDFS-14313: Thank [~linyiqun] for your review carefully. I h

[jira] [Commented] (HDFS-13693) Remove unnecessary search in INodeDirectory.addChild during image loading

2019-07-21 Thread Lisheng Sun (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-13693?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16889669#comment-16889669 ] Lisheng Sun commented on HDFS-13693: [~hexiaoqiao] has +1 for patch in Jira  and [~jo

[jira] [Commented] (HDFS-13693) Remove unnecessary search in INodeDirectory.addChild during image loading

2019-07-21 Thread Lisheng Sun (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-13693?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16889696#comment-16889696 ] Lisheng Sun commented on HDFS-13693: Thank [~ayushtkn] for your review.  I have delet

[jira] [Commented] (HDFS-13693) Remove unnecessary search in INodeDirectory.addChild during image loading

2019-07-21 Thread Lisheng Sun (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-13693?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16889701#comment-16889701 ] Lisheng Sun commented on HDFS-13693: Thank [~ayushtkn] for your suggestion. I must pa

[jira] [Commented] (HDFS-13693) Remove unnecessary search in INodeDirectory.addChild during image loading

2019-07-22 Thread Lisheng Sun (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-13693?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16890228#comment-16890228 ] Lisheng Sun commented on HDFS-13693: hi [~ayushtkn]  If you think this patch no probl

[jira] [Commented] (HDFS-14313) Get hdfs used space from FsDatasetImpl#volumeMap#ReplicaInfo in memory instead of df/du

2019-07-22 Thread Lisheng Sun (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-14313?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16890237#comment-16890237 ] Lisheng Sun commented on HDFS-14313: hi [~linyiqun], could you mind continuing to rei

[jira] [Commented] (HDFS-13693) Remove unnecessary search in INodeDirectory.addChild during image loading

2019-07-23 Thread Lisheng Sun (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-13693?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16890943#comment-16890943 ] Lisheng Sun commented on HDFS-13693:   {code:java} [ERROR] Failed to execute goal or

[jira] [Comment Edited] (HDFS-13693) Remove unnecessary search in INodeDirectory.addChild during image loading

2019-07-23 Thread Lisheng Sun (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-13693?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16890943#comment-16890943 ] Lisheng Sun edited comment on HDFS-13693 at 7/23/19 12:19 PM: -

[jira] [Assigned] (HDFS-13594) the lock of ShortCircuitCache is hold while close the ShortCircuitReplica

2019-07-24 Thread Lisheng Sun (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-13594?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lisheng Sun reassigned HDFS-13594: -- Assignee: Lisheng Sun > the lock of ShortCircuitCache is hold while close the ShortCircuitRepl

[jira] [Assigned] (HDFS-13564) PreAllocator for DfsClientShm

2019-07-24 Thread Lisheng Sun (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-13564?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lisheng Sun reassigned HDFS-13564: -- Assignee: Lisheng Sun > PreAllocator for DfsClientShm > - > >

[jira] [Updated] (HDFS-14313) Get hdfs used space from FsDatasetImpl#volumeMap#ReplicaInfo in memory instead of df/du

2019-07-25 Thread Lisheng Sun (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-14313?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lisheng Sun updated HDFS-14313: --- Attachment: HDFS-14313.008.patch > Get hdfs used space from FsDatasetImpl#volumeMap#ReplicaInfo in me

[jira] [Comment Edited] (HDFS-14313) Get hdfs used space from FsDatasetImpl#volumeMap#ReplicaInfo in memory instead of df/du

2019-07-25 Thread Lisheng Sun (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-14313?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16892921#comment-16892921 ] Lisheng Sun edited comment on HDFS-14313 at 7/25/19 4:24 PM: -

[jira] [Commented] (HDFS-14313) Get hdfs used space from FsDatasetImpl#volumeMap#ReplicaInfo in memory instead of df/du

2019-07-25 Thread Lisheng Sun (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-14313?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16892921#comment-16892921 ] Lisheng Sun commented on HDFS-14313: Thank [~linyiqun] for your review. I have update

[jira] [Commented] (HDFS-14313) Get hdfs used space from FsDatasetImpl#volumeMap#ReplicaInfo in memory instead of df/du

2019-07-27 Thread Lisheng Sun (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-14313?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16894465#comment-16894465 ] Lisheng Sun commented on HDFS-14313: hi [~linyiqun] [~jojochuang] Could you have time

[jira] [Assigned] (HDFS-7868) Use proper blocksize to choose target for blocks

2019-07-27 Thread Lisheng Sun (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-7868?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lisheng Sun reassigned HDFS-7868: - Assignee: Lisheng Sun (was: zhouyingchao) > Use proper blocksize to choose target for blocks > -

[jira] [Comment Edited] (HDFS-14313) Get hdfs used space from FsDatasetImpl#volumeMap#ReplicaInfo in memory instead of df/du

2019-07-28 Thread Lisheng Sun (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-14313?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16894893#comment-16894893 ] Lisheng Sun edited comment on HDFS-14313 at 7/29/19 3:11 AM: -

[jira] [Commented] (HDFS-14313) Get hdfs used space from FsDatasetImpl#volumeMap#ReplicaInfo in memory instead of df/du

2019-07-28 Thread Lisheng Sun (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-14313?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16894893#comment-16894893 ] Lisheng Sun commented on HDFS-14313: Thanx [~linyiqun] for your review. *FSCachingGe

[jira] [Updated] (HDFS-14313) Get hdfs used space from FsDatasetImpl#volumeMap#ReplicaInfo in memory instead of df/du

2019-07-28 Thread Lisheng Sun (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-14313?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lisheng Sun updated HDFS-14313: --- Attachment: HDFS-14313.009.patch > Get hdfs used space from FsDatasetImpl#volumeMap#ReplicaInfo in me

[jira] [Commented] (HDFS-14313) Get hdfs used space from FsDatasetImpl#volumeMap#ReplicaInfo in memory instead of df/du

2019-07-28 Thread Lisheng Sun (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-14313?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16894898#comment-16894898 ] Lisheng Sun commented on HDFS-14313: Updated patch according the review comments and

[jira] [Assigned] (HDFS-14644) That replication of block failed leads to decommission is blocked when the number of replicas of block is greater than the number of datanode

2019-07-30 Thread Lisheng Sun (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-14644?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lisheng Sun reassigned HDFS-14644: -- Assignee: Lisheng Sun > That replication of block failed leads to decommission is blocked when

[jira] [Commented] (HDFS-14644) That replication of block failed leads to decommission is blocked when the number of replicas of block is greater than the number of datanode

2019-07-30 Thread Lisheng Sun (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-14644?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16896099#comment-16896099 ] Lisheng Sun commented on HDFS-14644: Thanx [~sodonnell] [~jojochuang] for your sugges

[jira] [Commented] (HDFS-14313) Get hdfs used space from FsDatasetImpl#volumeMap#ReplicaInfo in memory instead of df/du

2019-07-30 Thread Lisheng Sun (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-14313?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16896696#comment-16896696 ] Lisheng Sun commented on HDFS-14313: hi [~linyiqun]  Could you have time to continue

[jira] [Updated] (HDFS-14290) Unexpected message type: PooledUnsafeDirectByteBuf when get datanode info by DatanodeWebHdfsMethods

2019-07-30 Thread Lisheng Sun (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-14290?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lisheng Sun updated HDFS-14290: --- Resolution: Not A Problem Status: Resolved (was: Patch Available) > Unexpected message type:

[jira] [Commented] (HDFS-14290) Unexpected message type: PooledUnsafeDirectByteBuf when get datanode info by DatanodeWebHdfsMethods

2019-07-30 Thread Lisheng Sun (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-14290?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16896754#comment-16896754 ] Lisheng Sun commented on HDFS-14290: Yeah. Sorry. It's not a problem. I have closed i

[jira] [Updated] (HDFS-14313) Get hdfs used space from FsDatasetImpl#volumeMap#ReplicaInfo in memory instead of df/du

2019-08-01 Thread Lisheng Sun (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-14313?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lisheng Sun updated HDFS-14313: --- Attachment: HDFS-14313.010.patch > Get hdfs used space from FsDatasetImpl#volumeMap#ReplicaInfo in me

[jira] [Commented] (HDFS-14313) Get hdfs used space from FsDatasetImpl#volumeMap#ReplicaInfo in memory instead of df/du

2019-08-01 Thread Lisheng Sun (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-14313?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16897988#comment-16897988 ] Lisheng Sun commented on HDFS-14313: Thanx for [~linyiqun] for deep review.  I have u

[jira] [Commented] (HDFS-14313) Get hdfs used space from FsDatasetImpl#volumeMap#ReplicaInfo in memory instead of df/du

2019-08-03 Thread Lisheng Sun (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-14313?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16899565#comment-16899565 ] Lisheng Sun commented on HDFS-14313: ping [~linyiqun] Could you mind taking a review

[jira] [Updated] (HDFS-13700) The process of loading image can be done in a pipeline model

2019-08-04 Thread Lisheng Sun (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-13700?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lisheng Sun updated HDFS-13700: --- Description: The process of loading a file system image involves reading inodes section, deserializi

[jira] [Updated] (HDFS-13700) The process of loading image can be done in a pipeline model

2019-08-04 Thread Lisheng Sun (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-13700?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lisheng Sun updated HDFS-13700: --- Attachment: HDFS-13700.002.patch > The process of loading image can be done in a pipeline model > ---

[jira] [Updated] (HDFS-13700) The process of loading image can be done in a pipeline model

2019-08-04 Thread Lisheng Sun (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-13700?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lisheng Sun updated HDFS-13700: --- Description: The process of loading a file system image involves reading inodes section, deserializi

[jira] [Created] (HDFS-14701) Change Log Level to warn in SlotReleaser

2019-08-05 Thread Lisheng Sun (JIRA)
Lisheng Sun created HDFS-14701: -- Summary: Change Log Level to warn in SlotReleaser Key: HDFS-14701 URL: https://issues.apache.org/jira/browse/HDFS-14701 Project: Hadoop HDFS Issue Type: Improvem

[jira] [Updated] (HDFS-14701) Change Log Level to warn in SlotReleaser

2019-08-05 Thread Lisheng Sun (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-14701?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lisheng Sun updated HDFS-14701: --- Description: {code:java} // @Override public void run() { LOG.trace("{}: about to release {}", Shor

[jira] [Updated] (HDFS-14701) Change Log Level to warn in SlotReleaser

2019-08-05 Thread Lisheng Sun (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-14701?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lisheng Sun updated HDFS-14701: --- Description: {code:java} // @Override public void run() { LOG.trace("{}: about to release {}", Shor

[jira] [Updated] (HDFS-14701) Change Log Level to warn in SlotReleaser

2019-08-05 Thread Lisheng Sun (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-14701?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lisheng Sun updated HDFS-14701: --- Description:  if the corresponding DataNode has been stopped or restarted and DFSClient close shared

[jira] [Updated] (HDFS-14701) Change Log Level to warn in SlotReleaser

2019-08-05 Thread Lisheng Sun (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-14701?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lisheng Sun updated HDFS-14701: --- Assignee: Lisheng Sun Attachment: HDFS-14701.001.patch Status: Patch Available (was: Op

[jira] [Updated] (HDFS-14313) Get hdfs used space from FsDatasetImpl#volumeMap#ReplicaInfo in memory instead of df/du

2019-08-05 Thread Lisheng Sun (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-14313?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lisheng Sun updated HDFS-14313: --- Attachment: HDFS-14313.011.patch > Get hdfs used space from FsDatasetImpl#volumeMap#ReplicaInfo in me

[jira] [Commented] (HDFS-14701) Change Log Level to warn in SlotReleaser

2019-08-05 Thread Lisheng Sun (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-14701?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16900119#comment-16900119 ] Lisheng Sun commented on HDFS-14701: hi [~ayushtkn] Could you mind help review this p

[jira] [Comment Edited] (HDFS-14701) Change Log Level to warn in SlotReleaser

2019-08-05 Thread Lisheng Sun (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-14701?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16900119#comment-16900119 ] Lisheng Sun edited comment on HDFS-14701 at 8/5/19 2:32 PM: h

[jira] [Commented] (HDFS-14313) Get hdfs used space from FsDatasetImpl#volumeMap#ReplicaInfo in memory instead of df/du

2019-08-05 Thread Lisheng Sun (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-14313?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16900515#comment-16900515 ] Lisheng Sun commented on HDFS-14313: Thanx  [~linyiqun] for your deep review. I upda

[jira] [Comment Edited] (HDFS-14313) Get hdfs used space from FsDatasetImpl#volumeMap#ReplicaInfo in memory instead of df/du

2019-08-05 Thread Lisheng Sun (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-14313?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16900515#comment-16900515 ] Lisheng Sun edited comment on HDFS-14313 at 8/6/19 1:37 AM: 

[jira] [Commented] (HDFS-14313) Get hdfs used space from FsDatasetImpl#volumeMap#ReplicaInfo in memory instead of df/du

2019-08-05 Thread Lisheng Sun (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-14313?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16900563#comment-16900563 ] Lisheng Sun commented on HDFS-14313: Thank [~jojochuang] for your suggestion.   {quo

[jira] [Updated] (HDFS-14313) Get hdfs used space from FsDatasetImpl#volumeMap#ReplicaInfo in memory instead of df/du

2019-08-06 Thread Lisheng Sun (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-14313?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lisheng Sun updated HDFS-14313: --- Attachment: HDFS-14313.012.patch > Get hdfs used space from FsDatasetImpl#volumeMap#ReplicaInfo in me

[jira] [Commented] (HDFS-14313) Get hdfs used space from FsDatasetImpl#volumeMap#ReplicaInfo in memory instead of df/du

2019-08-06 Thread Lisheng Sun (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-14313?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16900724#comment-16900724 ] Lisheng Sun commented on HDFS-14313: Thanx [~linyiqun] [~jojochuang] for your good su

[jira] [Updated] (HDFS-14313) Get hdfs used space from FsDatasetImpl#volumeMap#ReplicaInfo in memory instead of df/du

2019-08-06 Thread Lisheng Sun (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-14313?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lisheng Sun updated HDFS-14313: --- Status: Patch Available (was: Open) > Get hdfs used space from FsDatasetImpl#volumeMap#ReplicaInfo i

[jira] [Updated] (HDFS-14313) Get hdfs used space from FsDatasetImpl#volumeMap#ReplicaInfo in memory instead of df/du

2019-08-06 Thread Lisheng Sun (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-14313?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lisheng Sun updated HDFS-14313: --- Status: Open (was: Patch Available) > Get hdfs used space from FsDatasetImpl#volumeMap#ReplicaInfo i

[jira] [Commented] (HDFS-14313) Get hdfs used space from FsDatasetImpl#volumeMap#ReplicaInfo in memory instead of df/du

2019-08-06 Thread Lisheng Sun (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-14313?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16900753#comment-16900753 ] Lisheng Sun commented on HDFS-14313: {quote} HDFS-14313 does not apply to trunk. Reba

[jira] [Updated] (HDFS-14313) Get hdfs used space from FsDatasetImpl#volumeMap#ReplicaInfo in memory instead of df/du

2019-08-06 Thread Lisheng Sun (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-14313?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lisheng Sun updated HDFS-14313: --- Attachment: HDFS-14313.013.patch > Get hdfs used space from FsDatasetImpl#volumeMap#ReplicaInfo in me

[jira] [Commented] (HDFS-14313) Get hdfs used space from FsDatasetImpl#volumeMap#ReplicaInfo in memory instead of df/du

2019-08-06 Thread Lisheng Sun (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-14313?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16900863#comment-16900863 ] Lisheng Sun commented on HDFS-14313: Thank [~linyiqun] for your reminder. I rebased t

[jira] [Updated] (HDFS-14313) Get hdfs used space from FsDatasetImpl#volumeMap#ReplicaInfo in memory instead of df/du

2019-08-06 Thread Lisheng Sun (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-14313?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lisheng Sun updated HDFS-14313: --- Attachment: HDFS-14313.014.patch > Get hdfs used space from FsDatasetImpl#volumeMap#ReplicaInfo in me

[jira] [Commented] (HDFS-14313) Get hdfs used space from FsDatasetImpl#volumeMap#ReplicaInfo in memory instead of df/du

2019-08-06 Thread Lisheng Sun (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-14313?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16901146#comment-16901146 ] Lisheng Sun commented on HDFS-14313: Fix UT in hadoop.conf.TestCommonConfigurationFie

[jira] [Created] (HDFS-14708) TestLargeBlockReport#testBlockReportSucceedsWithLargerLengthLimit fails in trunk

2019-08-06 Thread Lisheng Sun (JIRA)
Lisheng Sun created HDFS-14708: -- Summary: TestLargeBlockReport#testBlockReportSucceedsWithLargerLengthLimit fails in trunk Key: HDFS-14708 URL: https://issues.apache.org/jira/browse/HDFS-14708 Project: H

[jira] [Updated] (HDFS-14708) TestLargeBlockReport#testBlockReportSucceedsWithLargerLengthLimit fails in trunk

2019-08-06 Thread Lisheng Sun (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-14708?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lisheng Sun updated HDFS-14708: --- Description: {code:java} [ERROR] testBlockReportSucceedsWithLargerLengthLimit(org.apache.hadoop.hdfs

[jira] [Assigned] (HDFS-14708) TestLargeBlockReport#testBlockReportSucceedsWithLargerLengthLimit fails in trunk

2019-08-06 Thread Lisheng Sun (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-14708?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lisheng Sun reassigned HDFS-14708: -- Assignee: Lisheng Sun > TestLargeBlockReport#testBlockReportSucceedsWithLargerLengthLimit fail

[jira] [Commented] (HDFS-14313) Get hdfs used space from FsDatasetImpl#volumeMap#ReplicaInfo in memory instead of df/du

2019-08-06 Thread Lisheng Sun (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-14313?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16901736#comment-16901736 ] Lisheng Sun commented on HDFS-14313: Thanx [~linyiqun] for all your work for this pat

[jira] [Updated] (HDFS-14313) Get hdfs used space from FsDatasetImpl#volumeMap#ReplicaInfo in memory instead of df/du

2019-08-06 Thread Lisheng Sun (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-14313?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lisheng Sun updated HDFS-14313: --- Attachment: HDFS-14313.branch-3.v1.patch > Get hdfs used space from FsDatasetImpl#volumeMap#ReplicaIn

[jira] [Commented] (HDFS-14313) Get hdfs used space from FsDatasetImpl#volumeMap#ReplicaInfo in memory instead of df/du

2019-08-06 Thread Lisheng Sun (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-14313?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16901778#comment-16901778 ] Lisheng Sun commented on HDFS-14313: I attached patch for branch-3.  you hope I attac

[jira] [Updated] (HDFS-14313) Get hdfs used space from FsDatasetImpl#volumeMap#ReplicaInfo in memory instead of df/du

2019-08-06 Thread Lisheng Sun (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-14313?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lisheng Sun updated HDFS-14313: --- Attachment: HDFS-14313.branch-3.0.v1.patch > Get hdfs used space from FsDatasetImpl#volumeMap#Replica

[jira] [Commented] (HDFS-14313) Get hdfs used space from FsDatasetImpl#volumeMap#ReplicaInfo in memory instead of df/du

2019-08-06 Thread Lisheng Sun (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-14313?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16901783#comment-16901783 ] Lisheng Sun commented on HDFS-14313: Thank [~linyiqun] for your suggestion. I rename

[jira] [Updated] (HDFS-14313) Get hdfs used space from FsDatasetImpl#volumeMap#ReplicaInfo in memory instead of df/du

2019-08-07 Thread Lisheng Sun (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-14313?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lisheng Sun updated HDFS-14313: --- Attachment: HDFS-14313-branch-2.v1.patch > Get hdfs used space from FsDatasetImpl#volumeMap#ReplicaIn

[jira] [Updated] (HDFS-14313) Get hdfs used space from FsDatasetImpl#volumeMap#ReplicaInfo in memory instead of df/du

2019-08-07 Thread Lisheng Sun (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-14313?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lisheng Sun updated HDFS-14313: --- Attachment: HDFS-14313.branch-3.0.v2.patch > Get hdfs used space from FsDatasetImpl#volumeMap#Replica

[jira] [Updated] (HDFS-14701) Change Log Level to warn in SlotReleaser

2019-08-07 Thread Lisheng Sun (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-14701?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lisheng Sun updated HDFS-14701: --- Attachment: HDFS-14701.002.patch > Change Log Level to warn in SlotReleaser > ---

[jira] [Commented] (HDFS-14701) Change Log Level to warn in SlotReleaser

2019-08-07 Thread Lisheng Sun (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-14701?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16901929#comment-16901929 ] Lisheng Sun commented on HDFS-14701: Thanx [~jojochuang] for you suggestion. I update

[jira] [Updated] (HDFS-14313) Get hdfs used space from FsDatasetImpl#volumeMap#ReplicaInfo in memory instead of df/du

2019-08-07 Thread Lisheng Sun (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-14313?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lisheng Sun updated HDFS-14313: --- Attachment: HDFS-14313-branch-2.v2.patch > Get hdfs used space from FsDatasetImpl#volumeMap#ReplicaIn

[jira] [Commented] (HDFS-14313) Get hdfs used space from FsDatasetImpl#volumeMap#ReplicaInfo in memory instead of df/du

2019-08-07 Thread Lisheng Sun (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-14313?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16902154#comment-16902154 ] Lisheng Sun commented on HDFS-14313: the branch-2.v2 for patch fixes unchecked issue

[jira] [Commented] (HDFS-13571) Dead DataNode Detector

2019-08-07 Thread Lisheng Sun (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-13571?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16902631#comment-16902631 ] Lisheng Sun commented on HDFS-13571: Sorry [~linyiqun], I have been working on this J

[jira] [Created] (HDFS-14330) Consider StorageID to choose volume

2019-03-03 Thread Lisheng Sun (JIRA)
Lisheng Sun created HDFS-14330: -- Summary: Consider StorageID to choose volume Key: HDFS-14330 URL: https://issues.apache.org/jira/browse/HDFS-14330 Project: Hadoop HDFS Issue Type: Improvement

[jira] [Assigned] (HDFS-15046) Backport HDFS-7060 to branch-2.10

2019-12-11 Thread Lisheng Sun (Jira)
[ https://issues.apache.org/jira/browse/HDFS-15046?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lisheng Sun reassigned HDFS-15046: -- Assignee: Lisheng Sun > Backport HDFS-7060 to branch-2.10 > -

[jira] [Updated] (HDFS-15046) Backport HDFS-7060 to branch-2.10

2019-12-11 Thread Lisheng Sun (Jira)
[ https://issues.apache.org/jira/browse/HDFS-15046?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lisheng Sun updated HDFS-15046: --- Attachment: HDFS-15046.branch-2.9.001.patch Status: Patch Available (was: Open) > Backport H

[jira] [Updated] (HDFS-15046) Backport HDFS-7060 to branch-2.10

2019-12-11 Thread Lisheng Sun (Jira)
[ https://issues.apache.org/jira/browse/HDFS-15046?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lisheng Sun updated HDFS-15046: --- Attachment: HDFS-15046.branch-2.9.002.patch > Backport HDFS-7060 to branch-2.10 > ---

[jira] [Updated] (HDFS-15046) Backport HDFS-7060 to branch-2.10

2019-12-11 Thread Lisheng Sun (Jira)
[ https://issues.apache.org/jira/browse/HDFS-15046?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lisheng Sun updated HDFS-15046: --- Attachment: HDFS-15046.branch-2.001.patch > Backport HDFS-7060 to branch-2.10 > -

[jira] [Updated] (HDFS-15046) Backport HDFS-7060 to branch-2.10

2020-01-14 Thread Lisheng Sun (Jira)
[ https://issues.apache.org/jira/browse/HDFS-15046?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lisheng Sun updated HDFS-15046: --- Status: Open (was: Patch Available) > Backport HDFS-7060 to branch-2.10 > --

[jira] [Updated] (HDFS-15046) Backport HDFS-7060 to branch-2.10

2020-01-14 Thread Lisheng Sun (Jira)
[ https://issues.apache.org/jira/browse/HDFS-15046?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lisheng Sun updated HDFS-15046: --- Status: Patch Available (was: Open) > Backport HDFS-7060 to branch-2.10 > --

[jira] [Updated] (HDFS-15046) Backport HDFS-7060 to branch-2.10

2020-01-15 Thread Lisheng Sun (Jira)
[ https://issues.apache.org/jira/browse/HDFS-15046?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lisheng Sun updated HDFS-15046: --- Attachment: HDFS-15046.branch-2.9.002(2).patch > Backport HDFS-7060 to branch-2.10 >

[jira] [Commented] (HDFS-15046) Backport HDFS-7060 to branch-2.10

2020-01-15 Thread Lisheng Sun (Jira)
[ https://issues.apache.org/jira/browse/HDFS-15046?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17016568#comment-17016568 ] Lisheng Sun commented on HDFS-15046: [~weichiu] Could you help review this patch? Th

[jira] [Commented] (HDFS-14651) DeadNodeDetector checks dead node periodically

2020-02-01 Thread Lisheng Sun (Jira)
[ https://issues.apache.org/jira/browse/HDFS-14651?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17028269#comment-17028269 ] Lisheng Sun commented on HDFS-14651: Thanks [~ahussein] for your questions. {quote}1.

[jira] [Created] (HDFS-15161) When evictableMmapped or evictable size is zero, do not throw NoSuchElementException in ShortCircuitCache#close()

2020-02-11 Thread Lisheng Sun (Jira)
Lisheng Sun created HDFS-15161: -- Summary: When evictableMmapped or evictable size is zero, do not throw NoSuchElementException in ShortCircuitCache#close() Key: HDFS-15161 URL: https://issues.apache.org/jira/browse/

[jira] [Updated] (HDFS-15161) When evictableMmapped or evictable size is zero, do not throw NoSuchElementException in ShortCircuitCache#close()

2020-02-11 Thread Lisheng Sun (Jira)
[ https://issues.apache.org/jira/browse/HDFS-15161?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lisheng Sun updated HDFS-15161: --- Description: detail see HDFS-14541 {code:java} /** * Close the cache and free all associated resourc

[jira] [Updated] (HDFS-15161) When evictableMmapped or evictable size is zero, do not throw NoSuchElementException in ShortCircuitCache#close()

2020-02-11 Thread Lisheng Sun (Jira)
[ https://issues.apache.org/jira/browse/HDFS-15161?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lisheng Sun updated HDFS-15161: --- Attachment: HDFS-15161.001.patch Status: Patch Available (was: Open) > When evictableMmapped

[jira] [Commented] (HDFS-15161) When evictableMmapped or evictable size is zero, do not throw NoSuchElementException in ShortCircuitCache#close()

2020-02-11 Thread Lisheng Sun (Jira)
[ https://issues.apache.org/jira/browse/HDFS-15161?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17034469#comment-17034469 ] Lisheng Sun commented on HDFS-15161: [~ayushtkn] HDFS-14541 left the the problem that

[jira] [Updated] (HDFS-15161) When evictableMmapped or evictable size is zero, do not throw NoSuchElementException in ShortCircuitCache#close()

2020-02-11 Thread Lisheng Sun (Jira)
[ https://issues.apache.org/jira/browse/HDFS-15161?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lisheng Sun updated HDFS-15161: --- Attachment: HDFS-15161.002.patch > When evictableMmapped or evictable size is zero, do not throw > N

[jira] [Created] (HDFS-15172) Remove unnecessary deadNodeDetectInterval in DeadNodeDetector#checkDeadNodes()

2020-02-15 Thread Lisheng Sun (Jira)
Lisheng Sun created HDFS-15172: -- Summary: Remove unnecessary deadNodeDetectInterval in DeadNodeDetector#checkDeadNodes() Key: HDFS-15172 URL: https://issues.apache.org/jira/browse/HDFS-15172 Project: Ha

[jira] [Updated] (HDFS-15172) Remove unnecessary deadNodeDetectInterval in DeadNodeDetector#checkDeadNodes()

2020-02-15 Thread Lisheng Sun (Jira)
[ https://issues.apache.org/jira/browse/HDFS-15172?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lisheng Sun updated HDFS-15172: --- Attachment: HDFS-15172-001.patch Status: Patch Available (was: Open) > Remove unnecessary d

[jira] [Updated] (HDFS-15172) Remove unnecessary deadNodeDetectInterval in DeadNodeDetector#checkDeadNodes()

2020-02-16 Thread Lisheng Sun (Jira)
[ https://issues.apache.org/jira/browse/HDFS-15172?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lisheng Sun updated HDFS-15172: --- Attachment: HDFS-15172-002.patch > Remove unnecessary deadNodeDetectInterval in > DeadNodeDetector#

[jira] [Updated] (HDFS-15172) Remove unnecessary deadNodeDetectInterval in DeadNodeDetector#checkDeadNodes()

2020-02-16 Thread Lisheng Sun (Jira)
[ https://issues.apache.org/jira/browse/HDFS-15172?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lisheng Sun updated HDFS-15172: --- Description: Every call to checkDeadNodes() will change the state to IDLE forcing the DeadNodeDetecto

[jira] [Assigned] (HDFS-15149) TestDeadNodeDetection test cases time-out

2020-02-16 Thread Lisheng Sun (Jira)
[ https://issues.apache.org/jira/browse/HDFS-15149?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lisheng Sun reassigned HDFS-15149: -- Assignee: Lisheng Sun (was: Ahmed Hussein) > TestDeadNodeDetection test cases time-out >

[jira] [Commented] (HDFS-15172) Remove unnecessary deadNodeDetectInterval in DeadNodeDetector#checkDeadNodes()

2020-02-16 Thread Lisheng Sun (Jira)
[ https://issues.apache.org/jira/browse/HDFS-15172?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17038073#comment-17038073 ] Lisheng Sun commented on HDFS-15172: [~elgoiri] I was intended to add this check In D

[jira] [Created] (HDFS-15174) Optimize ReplicaCachingGetSpaceUsed by reducing unnecessary io operations

2020-02-16 Thread Lisheng Sun (Jira)
Lisheng Sun created HDFS-15174: -- Summary: Optimize ReplicaCachingGetSpaceUsed by reducing unnecessary io operations Key: HDFS-15174 URL: https://issues.apache.org/jira/browse/HDFS-15174 Project: Hadoop H

[jira] [Updated] (HDFS-15149) TestDeadNodeDetection test cases time-out

2020-02-17 Thread Lisheng Sun (Jira)
[ https://issues.apache.org/jira/browse/HDFS-15149?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lisheng Sun updated HDFS-15149: --- Attachment: HDFS-15149-001.patch Status: Patch Available (was: Open) > TestDeadNodeDetection

[jira] [Commented] (HDFS-15172) Remove unnecessary deadNodeDetectInterval in DeadNodeDetector#checkDeadNodes()

2020-02-17 Thread Lisheng Sun (Jira)
[ https://issues.apache.org/jira/browse/HDFS-15172?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17038721#comment-17038721 ] Lisheng Sun commented on HDFS-15172: Thank [~elgoiri] for review. This jira solves th

[jira] [Created] (HDFS-15182) TestBlockManager#testOneOfTwoRacksDecommissioned() fail in trunk

2020-02-18 Thread Lisheng Sun (Jira)
Lisheng Sun created HDFS-15182: -- Summary: TestBlockManager#testOneOfTwoRacksDecommissioned() fail in trunk Key: HDFS-15182 URL: https://issues.apache.org/jira/browse/HDFS-15182 Project: Hadoop HDFS

[jira] [Updated] (HDFS-15182) TestBlockManager#testOneOfTwoRacksDecommissioned() fail in trunk

2020-02-18 Thread Lisheng Sun (Jira)
[ https://issues.apache.org/jira/browse/HDFS-15182?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lisheng Sun updated HDFS-15182: --- Attachment: HDFS-15182-001.patch Status: Patch Available (was: Open) > TestBlockManager#test

[jira] [Updated] (HDFS-15182) TestBlockManager#testOneOfTwoRacksDecommissioned() fail in trunk

2020-02-19 Thread Lisheng Sun (Jira)
[ https://issues.apache.org/jira/browse/HDFS-15182?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lisheng Sun updated HDFS-15182: --- Attachment: HDFS-15182-002.patch > TestBlockManager#testOneOfTwoRacksDecommissioned() fail in trunk >

[jira] [Commented] (HDFS-15182) TestBlockManager#testOneOfTwoRacksDecommissioned() fail in trunk

2020-02-19 Thread Lisheng Sun (Jira)
[ https://issues.apache.org/jira/browse/HDFS-15182?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17039993#comment-17039993 ] Lisheng Sun commented on HDFS-15182: the v002 fixed it in @Before. > TestBlockManage

[jira] [Commented] (HDFS-15149) TestDeadNodeDetection test cases time-out

2020-02-19 Thread Lisheng Sun (Jira)
[ https://issues.apache.org/jira/browse/HDFS-15149?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17040025#comment-17040025 ] Lisheng Sun commented on HDFS-15149: Thank [~ahussein] for your suggestion. {quote} T

[jira] [Commented] (HDFS-15149) TestDeadNodeDetection test cases time-out

2020-02-19 Thread Lisheng Sun (Jira)
[ https://issues.apache.org/jira/browse/HDFS-15149?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17040027#comment-17040027 ] Lisheng Sun commented on HDFS-15149: hi [~elgoiri] {quote} I like the rest of the sol

[jira] [Updated] (HDFS-15149) TestDeadNodeDetection test cases time-out

2020-02-19 Thread Lisheng Sun (Jira)
[ https://issues.apache.org/jira/browse/HDFS-15149?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lisheng Sun updated HDFS-15149: --- Attachment: HDFS-15149-002.patch > TestDeadNodeDetection test cases time-out > --

  1   2   3   4   5   6   7   8   9   10   >