[jira] [Updated] (HDFS-16544) EC decoding failed due to invalid buffer

2022-04-20 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HDFS-16544?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran updated HDFS-16544: -- Description: In HDFS-16538 , we found an EC file decoding bug if more than one data block

[jira] [Commented] (HDFS-16548) Failed unit test testRenameMoreThanOnceAcrossSnapDirs_2

2022-04-20 Thread Hemanth Boyina (Jira)
[ https://issues.apache.org/jira/browse/HDFS-16548?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17525115#comment-17525115 ] Hemanth Boyina commented on HDFS-16548: --- thanks for reporting [~tomscut] , cc [~sodonnell]    i

[jira] [Commented] (HDFS-16548) Failed unit test testRenameMoreThanOnceAcrossSnapDirs_2

2022-04-20 Thread Stephen O'Donnell (Jira)
[ https://issues.apache.org/jira/browse/HDFS-16548?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17525133#comment-17525133 ] Stephen O'Donnell commented on HDFS-16548: -- Huum, do you see any way to fix this easily, or

[jira] [Work logged] (HDFS-16452) msync RPC should send to Acitve Namenode directly

2022-04-20 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HDFS-16452?focusedWorklogId=759242=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-759242 ] ASF GitHub Bot logged work on HDFS-16452: - Author: ASF GitHub Bot

[jira] [Updated] (HDFS-16549) Consider using volume level lock for deleting blocks

2022-04-20 Thread Yuanbo Liu (Jira)
[ https://issues.apache.org/jira/browse/HDFS-16549?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuanbo Liu updated HDFS-16549: -- Description: It's great to see the implement of fine-grain lock for DN has been committed into trunk. 

[jira] [Created] (HDFS-16549) Consider using volume level lock for deleting blocks

2022-04-20 Thread Yuanbo Liu (Jira)
Yuanbo Liu created HDFS-16549: - Summary: Consider using volume level lock for deleting blocks Key: HDFS-16549 URL: https://issues.apache.org/jira/browse/HDFS-16549 Project: Hadoop HDFS Issue

[jira] [Work logged] (HDFS-16526) Add metrics for slow DataNode

2022-04-20 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HDFS-16526?focusedWorklogId=759227=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-759227 ] ASF GitHub Bot logged work on HDFS-16526: - Author: ASF GitHub Bot

[jira] [Work logged] (HDFS-16526) Add metrics for slow DataNode

2022-04-20 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HDFS-16526?focusedWorklogId=759226=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-759226 ] ASF GitHub Bot logged work on HDFS-16526: - Author: ASF GitHub Bot

[jira] [Resolved] (HDFS-16526) Add metrics for slow DataNode

2022-04-20 Thread Ayush Saxena (Jira)
[ https://issues.apache.org/jira/browse/HDFS-16526?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ayush Saxena resolved HDFS-16526. - Fix Version/s: 3.4.0 Hadoop Flags: Reviewed Resolution: Fixed > Add metrics for slow

[jira] [Work logged] (HDFS-16528) Reconfigure slow peer enable for Namenode

2022-04-20 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HDFS-16528?focusedWorklogId=759062=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-759062 ] ASF GitHub Bot logged work on HDFS-16528: - Author: ASF GitHub Bot

[jira] [Work logged] (HDFS-16256) Minor fixes in HDFS Fedbalance document

2022-04-20 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HDFS-16256?focusedWorklogId=759153=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-759153 ] ASF GitHub Bot logged work on HDFS-16256: - Author: ASF GitHub Bot

[jira] [Work logged] (HDFS-16528) Reconfigure slow peer enable for Namenode

2022-04-20 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HDFS-16528?focusedWorklogId=759048=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-759048 ] ASF GitHub Bot logged work on HDFS-16528: - Author: ASF GitHub Bot

[jira] [Updated] (HDFS-16546) Fix UT TestOfflineImageViewer#testReverseXmlWithoutSnapshotDiffSection to branch branch-3.2

2022-04-20 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HDFS-16546?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated HDFS-16546: -- Labels: pull-request-available (was: ) > Fix UT

[jira] [Work logged] (HDFS-16546) Fix UT TestOfflineImageViewer#testReverseXmlWithoutSnapshotDiffSection to branch branch-3.2

2022-04-20 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HDFS-16546?focusedWorklogId=759047=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-759047 ] ASF GitHub Bot logged work on HDFS-16546: - Author: ASF GitHub Bot

[jira] [Updated] (HDFS-16545) Provide option to balance rack level in Balancer

2022-04-20 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HDFS-16545?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated HDFS-16545: -- Labels: pull-request-available (was: ) > Provide option to balance rack level in Balancer >

[jira] [Work logged] (HDFS-16545) Provide option to balance rack level in Balancer

2022-04-20 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HDFS-16545?focusedWorklogId=759562=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-759562 ] ASF GitHub Bot logged work on HDFS-16545: - Author: ASF GitHub Bot

[jira] [Commented] (HDFS-16531) Avoid setReplication logging an edit record if old replication equals the new value

2022-04-20 Thread Ayush Saxena (Jira)
[ https://issues.apache.org/jira/browse/HDFS-16531?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17525240#comment-17525240 ] Ayush Saxena commented on HDFS-16531: - yeps, +1 to revert. We had previous jiras trying to do such

[jira] [Updated] (HDFS-16531) Avoid setReplication logging an edit record if old replication equals the new value

2022-04-20 Thread Stephen O'Donnell (Jira)
[ https://issues.apache.org/jira/browse/HDFS-16531?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Stephen O'Donnell updated HDFS-16531: - Fix Version/s: (was: 3.4.0) (was: 3.2.4)

[jira] [Work logged] (HDFS-16528) Reconfigure slow peer enable for Namenode

2022-04-20 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HDFS-16528?focusedWorklogId=759599=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-759599 ] ASF GitHub Bot logged work on HDFS-16528: - Author: ASF GitHub Bot

[jira] [Work logged] (HDFS-16526) Add metrics for slow DataNode

2022-04-20 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HDFS-16526?focusedWorklogId=759435=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-759435 ] ASF GitHub Bot logged work on HDFS-16526: - Author: ASF GitHub Bot

[jira] [Commented] (HDFS-16531) Avoid setReplication logging an edit record if old replication equals the new value

2022-04-20 Thread Stephen O'Donnell (Jira)
[ https://issues.apache.org/jira/browse/HDFS-16531?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17525235#comment-17525235 ] Stephen O'Donnell commented on HDFS-16531: -- I worry what else might break that is not covered by

[jira] [Resolved] (HDFS-16531) Avoid setReplication logging an edit record if old replication equals the new value

2022-04-20 Thread Stephen O'Donnell (Jira)
[ https://issues.apache.org/jira/browse/HDFS-16531?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Stephen O'Donnell resolved HDFS-16531. -- Resolution: Abandoned Reverted this change down the branches. Sorry for causing the

[jira] [Work logged] (HDFS-16528) Reconfigure slow peer enable for Namenode

2022-04-20 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HDFS-16528?focusedWorklogId=759468=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-759468 ] ASF GitHub Bot logged work on HDFS-16528: - Author: ASF GitHub Bot

[jira] [Reopened] (HDFS-16531) Avoid setReplication logging an edit record if old replication equals the new value

2022-04-20 Thread Ayush Saxena (Jira)
[ https://issues.apache.org/jira/browse/HDFS-16531?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ayush Saxena reopened HDFS-16531: - > Avoid setReplication logging an edit record if old replication equals the new > value >

[jira] [Commented] (HDFS-16531) Avoid setReplication logging an edit record if old replication equals the new value

2022-04-20 Thread Ayush Saxena (Jira)
[ https://issues.apache.org/jira/browse/HDFS-16531?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17525232#comment-17525232 ] Ayush Saxena commented on HDFS-16531: - Just exploring. Guess this optimisation should work only if

[jira] [Commented] (HDFS-16531) Avoid setReplication logging an edit record if old replication equals the new value

2022-04-20 Thread Ayush Saxena (Jira)
[ https://issues.apache.org/jira/browse/HDFS-16531?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17525230#comment-17525230 ] Ayush Saxena commented on HDFS-16531: - This leads to a test failure

[jira] [Resolved] (HDFS-16548) Failed unit test testRenameMoreThanOnceAcrossSnapDirs_2

2022-04-20 Thread Ayush Saxena (Jira)
[ https://issues.apache.org/jira/browse/HDFS-16548?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ayush Saxena resolved HDFS-16548. - Resolution: Abandoned not a test issue, the prod code itself has issues, have reopened the

[jira] [Work logged] (HDFS-16550) [SBN read] Improper cache-size for journal node may cause cluster crash

2022-04-20 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HDFS-16550?focusedWorklogId=759716=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-759716 ] ASF GitHub Bot logged work on HDFS-16550: - Author: ASF GitHub Bot

[jira] [Updated] (HDFS-16550) [SBN read] Improper cache-size for journal node may cause cluster crash

2022-04-20 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HDFS-16550?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated HDFS-16550: -- Labels: pull-request-available (was: ) > [SBN read] Improper cache-size for journal node may

[jira] [Updated] (HDFS-16550) [SBN read] Improper cache-size for journal node may cause cluster crash

2022-04-20 Thread tomscut (Jira)
[ https://issues.apache.org/jira/browse/HDFS-16550?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] tomscut updated HDFS-16550: --- Description: When we introduced {*}SBN Read{*}, we encountered a situation during upgrade the JournalNodes.

[jira] [Work logged] (HDFS-16452) msync RPC should send to Acitve Namenode directly

2022-04-20 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HDFS-16452?focusedWorklogId=759718=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-759718 ] ASF GitHub Bot logged work on HDFS-16452: - Author: ASF GitHub Bot

[jira] [Work logged] (HDFS-16545) Provide option to balance rack level in Balancer

2022-04-20 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HDFS-16545?focusedWorklogId=759732=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-759732 ] ASF GitHub Bot logged work on HDFS-16545: - Author: ASF GitHub Bot

[jira] [Work logged] (HDFS-16520) Improve EC pread: avoid potential reading whole block

2022-04-20 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HDFS-16520?focusedWorklogId=759745=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-759745 ] ASF GitHub Bot logged work on HDFS-16520: - Author: ASF GitHub Bot

[jira] [Work logged] (HDFS-16520) Improve EC pread: avoid potential reading whole block

2022-04-20 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HDFS-16520?focusedWorklogId=759744=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-759744 ] ASF GitHub Bot logged work on HDFS-16520: - Author: ASF GitHub Bot

[jira] [Commented] (HDFS-16549) Consider using volume level lock for deleting blocks

2022-04-20 Thread Mingxiang Li (Jira)
[ https://issues.apache.org/jira/browse/HDFS-16549?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17525376#comment-17525376 ] Mingxiang Li commented on HDFS-16549: - in FsDatasetImpl.invalidate the scope protected by block pool

[jira] [Created] (HDFS-16550) [SBN read] Improper cache-size for journal node may cause cluster crash

2022-04-20 Thread tomscut (Jira)
tomscut created HDFS-16550: -- Summary: [SBN read] Improper cache-size for journal node may cause cluster crash Key: HDFS-16550 URL: https://issues.apache.org/jira/browse/HDFS-16550 Project: Hadoop HDFS

[jira] [Commented] (HDFS-16531) Avoid setReplication logging an edit record if old replication equals the new value

2022-04-20 Thread Xiaoqiao He (Jira)
[ https://issues.apache.org/jira/browse/HDFS-16531?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17525408#comment-17525408 ] Xiaoqiao He commented on HDFS-16531: Thanks [~ayushtkn] for your kind information. Based on above

[jira] [Resolved] (HDFS-16500) Make asynchronous blocks deletion lock and unlock durtion threshold configurable

2022-04-20 Thread Xiaoqiao He (Jira)
[ https://issues.apache.org/jira/browse/HDFS-16500?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiaoqiao He resolved HDFS-16500. Fix Version/s: 3.4.0 Hadoop Flags: Reviewed Target Version/s: (was: 3.3.1, 3.3.2)

[jira] [Work logged] (HDFS-16500) Make asynchronous blocks deletion lock and unlock durtion threshold configurable

2022-04-20 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HDFS-16500?focusedWorklogId=759752=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-759752 ] ASF GitHub Bot logged work on HDFS-16500: - Author: ASF GitHub Bot

[jira] [Work logged] (HDFS-16500) Make asynchronous blocks deletion lock and unlock durtion threshold configurable

2022-04-20 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HDFS-16500?focusedWorklogId=759748=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-759748 ] ASF GitHub Bot logged work on HDFS-16500: - Author: ASF GitHub Bot

[jira] [Updated] (HDFS-16550) [SBN read] Improper cache-size for journal node may cause cluster crash

2022-04-20 Thread tomscut (Jira)
[ https://issues.apache.org/jira/browse/HDFS-16550?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] tomscut updated HDFS-16550: --- Attachment: image-2022-04-21-12-32-56-170.png Description: When we introduced {*}SBN Read{*}, we

[jira] [Updated] (HDFS-16550) [SBN read] Improper cache-size for journal node may cause cluster crash

2022-04-20 Thread tomscut (Jira)
[ https://issues.apache.org/jira/browse/HDFS-16550?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] tomscut updated HDFS-16550: --- Description: When we introduced {*}SBN Read{*}, we encountered a situation during upgrade the JournalNodes.

[jira] [Commented] (HDFS-16531) Avoid setReplication logging an edit record if old replication equals the new value

2022-04-20 Thread Hemanth Boyina (Jira)
[ https://issues.apache.org/jira/browse/HDFS-16531?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17525412#comment-17525412 ] Hemanth Boyina commented on HDFS-16531: --- [~hexiaoqiao]{color:#172b4d}  IMO in

[jira] [Commented] (HDFS-16531) Avoid setReplication logging an edit record if old replication equals the new value

2022-04-20 Thread Xiaoqiao He (Jira)
[ https://issues.apache.org/jira/browse/HDFS-16531?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17525415#comment-17525415 ] Xiaoqiao He commented on HDFS-16531: [~hemanthboyina], Thanks very much for your reminder and the

[jira] [Created] (HDFS-16551) Backport HADOOP-17588 to 3.3 and other active old branches.

2022-04-20 Thread Renukaprasad C (Jira)
Renukaprasad C created HDFS-16551: - Summary: Backport HADOOP-17588 to 3.3 and other active old branches. Key: HDFS-16551 URL: https://issues.apache.org/jira/browse/HDFS-16551 Project: Hadoop HDFS

[jira] [Work logged] (HDFS-16544) EC decoding failed due to invalid buffer

2022-04-20 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HDFS-16544?focusedWorklogId=758977=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-758977 ] ASF GitHub Bot logged work on HDFS-16544: - Author: ASF GitHub Bot

[jira] [Resolved] (HDFS-16544) EC decoding failed due to invalid buffer

2022-04-20 Thread Takanobu Asanuma (Jira)
[ https://issues.apache.org/jira/browse/HDFS-16544?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Takanobu Asanuma resolved HDFS-16544. - Fix Version/s: 3.4.0 3.2.4 3.3.4 Assignee:

[jira] [Work logged] (HDFS-16544) EC decoding failed due to invalid buffer

2022-04-20 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HDFS-16544?focusedWorklogId=758991=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-758991 ] ASF GitHub Bot logged work on HDFS-16544: - Author: ASF GitHub Bot

[jira] [Created] (HDFS-16548) Failed unit test testRenameMoreThanOnceAcrossSnapDirs_2

2022-04-20 Thread tomscut (Jira)
tomscut created HDFS-16548: -- Summary: Failed unit test testRenameMoreThanOnceAcrossSnapDirs_2 Key: HDFS-16548 URL: https://issues.apache.org/jira/browse/HDFS-16548 Project: Hadoop HDFS Issue Type:

[jira] [Updated] (HDFS-16548) Failed unit test testRenameMoreThanOnceAcrossSnapDirs_2

2022-04-20 Thread tomscut (Jira)
[ https://issues.apache.org/jira/browse/HDFS-16548?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] tomscut updated HDFS-16548: --- Description: It seems to be related to HDFS-16531. {code:java} [ERROR] Tests run: 44, Failures: 6, Errors:

[jira] [Updated] (HDFS-16548) Failed unit test testRenameMoreThanOnceAcrossSnapDirs_2

2022-04-20 Thread tomscut (Jira)
[ https://issues.apache.org/jira/browse/HDFS-16548?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] tomscut updated HDFS-16548: --- Description: It seems to be related to this HDFS-16531. {code:java} [ERROR] Tests run: 44, Failures: 6,