[jira] [Updated] (HDFS-15386) ReplicaNotFoundException keeps happening in DN after removing multiple DN's data directories

[ https://issues.apache.org/jira/browse/HDFS-15386?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Toshihiro Suzuki updated HDFS-15386: Description: When removing volumes, we need to invalidate all the blocks in the volumes.

[jira] [Work started] (HDFS-15386) ReplicaNotFoundException keeps happening in DN after removing multiple DN's data directories

[ https://issues.apache.org/jira/browse/HDFS-15386?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Work on HDFS-15386 started by Toshihiro Suzuki. --- > ReplicaNotFoundException keeps happening in DN after removing multiple

[jira] [Updated] (HDFS-15386) ReplicaNotFoundException keeps happening in DN after removing multiple DN's data directories

[ https://issues.apache.org/jira/browse/HDFS-15386?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Toshihiro Suzuki updated HDFS-15386: Description: When removing volumes, we need to invalidate all the blocks in the volumes.

[jira] [Updated] (HDFS-15386) ReplicaNotFoundException keeps happening in DN after removing multiple DN's data directories

[ https://issues.apache.org/jira/browse/HDFS-15386?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Toshihiro Suzuki updated HDFS-15386: Summary: ReplicaNotFoundException keeps happening in DN after removing multiple DN's data

[jira] [Created] (HDFS-15386) ReplicaNotFoundException keeps happening in DN after removing multiple DN's data direcotries

Toshihiro Suzuki created HDFS-15386: --- Summary: ReplicaNotFoundException keeps happening in DN after removing multiple DN's data direcotries Key: HDFS-15386 URL: https://issues.apache.org/jira/browse/HDFS-15386

[jira] [Commented] (HDFS-14054) TestLeaseRecovery2: testHardLeaseRecoveryAfterNameNodeRestart2 and testHardLeaseRecoveryWithRenameAfterNameNodeRestart are flaky

[ https://issues.apache.org/jira/browse/HDFS-14054?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17125344#comment-17125344 ] Jim Brennan commented on HDFS-14054: [~inigoiri] Thanks for the fast response! > TestLeaseRecovery2:

[jira] [Commented] (HDFS-14054) TestLeaseRecovery2: testHardLeaseRecoveryAfterNameNodeRestart2 and testHardLeaseRecoveryWithRenameAfterNameNodeRestart are flaky

[ https://issues.apache.org/jira/browse/HDFS-14054?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17125331#comment-17125331 ] Íñigo Goiri commented on HDFS-14054: [~Jim_Brennan], I cherry picked this to branch-2.10. >

[jira] [Updated] (HDFS-14054) TestLeaseRecovery2: testHardLeaseRecoveryAfterNameNodeRestart2 and testHardLeaseRecoveryWithRenameAfterNameNodeRestart are flaky

[ https://issues.apache.org/jira/browse/HDFS-14054?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Íñigo Goiri updated HDFS-14054: --- Fix Version/s: 2.10.1 > TestLeaseRecovery2: testHardLeaseRecoveryAfterNameNodeRestart2 and >

[jira] [Commented] (HDFS-14054) TestLeaseRecovery2: testHardLeaseRecoveryAfterNameNodeRestart2 and testHardLeaseRecoveryWithRenameAfterNameNodeRestart are flaky

[ https://issues.apache.org/jira/browse/HDFS-14054?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17125323#comment-17125323 ] Jim Brennan commented on HDFS-14054: [~zvenczel], [~inigoiri] we have seen this failure in our

[jira] [Commented] (HDFS-15359) EC: Allow closing a file with committed blocks

[ https://issues.apache.org/jira/browse/HDFS-15359?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17125293#comment-17125293 ] Ayush Saxena commented on HDFS-15359: - Thanx [~vinayakumarb] for the review. [~weichiu] will be

[jira] [Commented] (HDFS-15359) EC: Allow closing a file with committed blocks

[ https://issues.apache.org/jira/browse/HDFS-15359?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17125205#comment-17125205 ] Vinayakumar B commented on HDFS-15359: -- +1 thanks [~ayushtkn] for contribution > EC: Allow closing

[jira] [Commented] (HDFS-15246) ArrayIndexOfboundsException in BlockManager CreateLocatedBlock

[ https://issues.apache.org/jira/browse/HDFS-15246?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17125141#comment-17125141 ] hemanthboyina commented on HDFS-15246: -- thanks for the review ![~elgoiri] test failures are not

[jira] [Commented] (HDFS-15385) Upgrade boost library

[ https://issues.apache.org/jira/browse/HDFS-15385?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17125121#comment-17125121 ] Gautham Banasandra commented on HDFS-15385: --- Hi all, My PR is ready. Please review at - 

[jira] [Commented] (HDFS-15372) Files in snapshots no longer see attribute provider permissions

[ https://issues.apache.org/jira/browse/HDFS-15372?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17125111#comment-17125111 ] Hadoop QA commented on HDFS-15372: -- | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote || Subsystem ||

[jira] [Created] (HDFS-15385) Upgrade boost library

Gautham Banasandra created HDFS-15385: - Summary: Upgrade boost library Key: HDFS-15385 URL: https://issues.apache.org/jira/browse/HDFS-15385 Project: Hadoop HDFS Issue Type: Improvement

[jira] [Commented] (HDFS-15384) Method getLocatedBlocks(String src, long start) of DFSClient only return partial blocks

[ https://issues.apache.org/jira/browse/HDFS-15384?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17125010#comment-17125010 ] Yang Yun commented on HDFS-15384: - Thanks [~ayushtkn] for the answer. It's reasonable to open a

[jira] [Commented] (HDFS-14546) Document block placement policies

[ https://issues.apache.org/jira/browse/HDFS-14546?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17124999#comment-17124999 ] Hadoop QA commented on HDFS-14546: -- | (/) *{color:green}+1 overall{color}* | \\ \\ || Vote || Subsystem

[jira] [Commented] (HDFS-15375) Reconstruction Work should not happen for Corrupt Block

[ https://issues.apache.org/jira/browse/HDFS-15375?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17124998#comment-17124998 ] hemanthboyina commented on HDFS-15375: -- test failures were not related {quote}We can't remove 

[jira] [Commented] (HDFS-15384) Method getLocatedBlocks(String src, long start) of DFSClient only return partial blocks

[ https://issues.apache.org/jira/browse/HDFS-15384?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17124993#comment-17124993 ] Hadoop QA commented on HDFS-15384: -- | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote || Subsystem ||

[jira] [Commented] (HDFS-14960) TestBalancerWithNodeGroup should not succeed with DFSNetworkTopology

[ https://issues.apache.org/jira/browse/HDFS-14960?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17124991#comment-17124991 ] Jim Brennan commented on HDFS-14960: Thanks [~ayushtkn]! > TestBalancerWithNodeGroup should not

[jira] [Commented] (HDFS-14546) Document block placement policies

[ https://issues.apache.org/jira/browse/HDFS-14546?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17124949#comment-17124949 ] Ayush Saxena commented on HDFS-14546: - Have updated the patch for you [~Amithsha] *Added

[jira] [Updated] (HDFS-14546) Document block placement policies

[ https://issues.apache.org/jira/browse/HDFS-14546?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ayush Saxena updated HDFS-14546: Attachment: HDFS-14546-09.patch > Document block placement policies >

[jira] [Updated] (HDFS-15372) Files in snapshots no longer see attribute provider permissions

[ https://issues.apache.org/jira/browse/HDFS-15372?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Stephen O'Donnell updated HDFS-15372: - Status: Patch Available (was: Open) > Files in snapshots no longer see attribute

[jira] [Updated] (HDFS-15372) Files in snapshots no longer see attribute provider permissions

[ https://issues.apache.org/jira/browse/HDFS-15372?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Stephen O'Donnell updated HDFS-15372: - Attachment: HDFS-15372.001.patch > Files in snapshots no longer see attribute provider

[jira] [Commented] (HDFS-15098) Add SM4 encryption method for HDFS

[ https://issues.apache.org/jira/browse/HDFS-15098?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17124910#comment-17124910 ] Hadoop QA commented on HDFS-15098: -- | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote || Subsystem ||

[jira] [Commented] (HDFS-15384) Method getLocatedBlocks(String src, long start) of DFSClient only return partial blocks

[ https://issues.apache.org/jira/browse/HDFS-15384?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17124898#comment-17124898 ] Ayush Saxena commented on HDFS-15384: - {{dfs.client.read.prefetch.size}} is a well documented

[jira] [Updated] (HDFS-15384) Method getLocatedBlocks(String src, long start) of DFSClient only return partial blocks

[ https://issues.apache.org/jira/browse/HDFS-15384?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yang Yun updated HDFS-15384: Description:     Intuitively, the method getLocatedBlocks(String src, long start) of DFSClient will

[jira] [Updated] (HDFS-15384) Method getLocatedBlocks(String src, long start) of DFSClient only return partial blocks

[ https://issues.apache.org/jira/browse/HDFS-15384?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yang Yun updated HDFS-15384: Attachment: HDFS-15384.001.patch Status: Patch Available (was: Open) > Method

[jira] [Created] (HDFS-15384) Method getLocatedBlocks(String src, long start) of DFSClient only return partial blocks

Yang Yun created HDFS-15384: --- Summary: Method getLocatedBlocks(String src, long start) of DFSClient only return partial blocks Key: HDFS-15384 URL: https://issues.apache.org/jira/browse/HDFS-15384 Project:

[jira] [Updated] (HDFS-15098) Add SM4 encryption method for HDFS

[ https://issues.apache.org/jira/browse/HDFS-15098?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] zZtai updated HDFS-15098: - Attachment: HDFS-15098.005.patch Release Note: When Openssl exists in the environment,the system will use

[jira] [Updated] (HDFS-15098) Add SM4 encryption method for HDFS

[ https://issues.apache.org/jira/browse/HDFS-15098?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] zZtai updated HDFS-15098: - Status: Open (was: Patch Available) > Add SM4 encryption method for HDFS > -- >

[jira] [Comment Edited] (HDFS-15382) Split FsDatasetImpl from blockpool lock to blockpool volume lock

[ https://issues.apache.org/jira/browse/HDFS-15382?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17124767#comment-17124767 ] Aiphago edited comment on HDFS-15382 at 6/3/20, 8:45 AM: -

[jira] [Commented] (HDFS-15382) Split FsDatasetImpl from blockpool lock to blockpool volume lock

[ https://issues.apache.org/jira/browse/HDFS-15382?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17124767#comment-17124767 ] Aiphago commented on HDFS-15382: !image-2020-06-03-1.png|width=628,height=378! The simple lock model

[jira] [Updated] (HDFS-15382) Split FsDatasetImpl from blockpool lock to blockpool volume lock

[ https://issues.apache.org/jira/browse/HDFS-15382?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aiphago updated HDFS-15382: --- Attachment: image-2020-06-03-1.png > Split FsDatasetImpl from blockpool lock to blockpool volume lock >

[jira] [Updated] (HDFS-15382) Split FsDatasetImpl from blockpool lock to blockpool volume lock

[ https://issues.apache.org/jira/browse/HDFS-15382?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aiphago updated HDFS-15382: --- Attachment: (was: 2020-06-03.png) > Split FsDatasetImpl from blockpool lock to blockpool volume lock >

[jira] [Updated] (HDFS-15382) Split FsDatasetImpl from blockpool lock to blockpool volume lock

[ https://issues.apache.org/jira/browse/HDFS-15382?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aiphago updated HDFS-15382: --- Attachment: 2020-06-03.png > Split FsDatasetImpl from blockpool lock to blockpool volume lock >

[jira] [Comment Edited] (HDFS-15382) Split FsDatasetImpl from blockpool lock to blockpool volume lock

[ https://issues.apache.org/jira/browse/HDFS-15382?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17124736#comment-17124736 ] Aiphago edited comment on HDFS-15382 at 6/3/20, 8:12 AM: - {quote}it is confused

[jira] [Commented] (HDFS-15382) Split FsDatasetImpl from blockpool lock to blockpool volume lock

[ https://issues.apache.org/jira/browse/HDFS-15382?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17124736#comment-17124736 ] Aiphago commented on HDFS-15382: {quote}it is confused with {{ReplicaCachingGetSpaceUsed}}, IIUC, 

[jira] [Commented] (HDFS-15379) DataStreamer should reset thread interrupted state in createBlockOutputStream

[ https://issues.apache.org/jira/browse/HDFS-15379?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17124728#comment-17124728 ] ludun commented on HDFS-15379: -- [~liuml07] thanks. > DataStreamer should reset thread interrupted state in

[jira] [Commented] (HDFS-11041) Unable to unregister FsDatasetState MBean if DataNode is shutdown twice

[ https://issues.apache.org/jira/browse/HDFS-11041?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17124721#comment-17124721 ] Hudson commented on HDFS-11041: --- SUCCESS: Integrated in Jenkins build Hadoop-trunk-Commit #18321 (See

[jira] [Comment Edited] (HDFS-15377) BlockScanner scans one part per round, expect full scans after several rounds

[ https://issues.apache.org/jira/browse/HDFS-15377?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17124717#comment-17124717 ] Yang Yun edited comment on HDFS-15377 at 6/3/20, 7:46 AM: -- Thanks [~liuml07] for

[jira] [Commented] (HDFS-15377) BlockScanner scans one part per round, expect full scans after several rounds

[ https://issues.apache.org/jira/browse/HDFS-15377?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17124717#comment-17124717 ] Yang Yun commented on HDFS-15377: - Thanks [~liuml07] for the question. > BlockScanner scans one part per

[jira] [Commented] (HDFS-15377) BlockScanner scans one part per round, expect full scans after several rounds

[ https://issues.apache.org/jira/browse/HDFS-15377?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17124705#comment-17124705 ] Mingliang Liu commented on HDFS-15377: -- Did not check the patch yet. Just curious, does this work

[jira] [Commented] (HDFS-14960) TestBalancerWithNodeGroup should not succeed with DFSNetworkTopology

[ https://issues.apache.org/jira/browse/HDFS-14960?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17124706#comment-17124706 ] Hudson commented on HDFS-14960: --- SUCCESS: Integrated in Jenkins build Hadoop-trunk-Commit #18320 (See

[jira] [Commented] (HDFS-15379) DataStreamer should reset thread interrupted state in createBlockOutputStream

[ https://issues.apache.org/jira/browse/HDFS-15379?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17124697#comment-17124697 ] Mingliang Liu commented on HDFS-15379: -- Added you to contributor list and assigned this JIRA to you

[jira] [Assigned] (HDFS-15379) DataStreamer should reset thread interrupted state in createBlockOutputStream

[ https://issues.apache.org/jira/browse/HDFS-15379?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mingliang Liu reassigned HDFS-15379: Assignee: ludun > DataStreamer should reset thread interrupted state in

[jira] [Updated] (HDFS-11041) Unable to unregister FsDatasetState MBean if DataNode is shutdown twice

[ https://issues.apache.org/jira/browse/HDFS-11041?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ayush Saxena updated HDFS-11041: Fix Version/s: 3.4.0 Hadoop Flags: Reviewed Resolution: Fixed Status:

[jira] [Commented] (HDFS-11041) Unable to unregister FsDatasetState MBean if DataNode is shutdown twice

[ https://issues.apache.org/jira/browse/HDFS-11041?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17124688#comment-17124688 ] Ayush Saxena commented on HDFS-11041: - Committed to trunk. Thanx [~weichiu] for the contribution!!!

[jira] [Updated] (HDFS-14960) TestBalancerWithNodeGroup should not succeed with DFSNetworkTopology

[ https://issues.apache.org/jira/browse/HDFS-14960?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ayush Saxena updated HDFS-14960: Fix Version/s: 3.4.0 Hadoop Flags: Reviewed Resolution: Fixed Status:

[jira] [Commented] (HDFS-14960) TestBalancerWithNodeGroup should not succeed with DFSNetworkTopology

[ https://issues.apache.org/jira/browse/HDFS-14960?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17124678#comment-17124678 ] Ayush Saxena commented on HDFS-14960: - Committed to trunk. Thanx [~Jim_Brennan] for the contribution

[jira] [Commented] (HDFS-15346) RBF: DistCpFedBalance implementation

[ https://issues.apache.org/jira/browse/HDFS-15346?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17124637#comment-17124637 ] Hadoop QA commented on HDFS-15346: -- | (/) *{color:green}+1 overall{color}* | \\ \\ || Vote || Subsystem