[jira] [Commented] (HDFS-14775) Add Timestamp for longest FSN write/read lock held log

[ https://issues.apache.org/jira/browse/HDFS-14775?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16967196#comment-16967196 ] Xiaoqiao He commented on HDFS-14775: [^HDFS-14775.005.patch] LGTM, +1. Thanks [~zhang

[jira] [Commented] (HDFS-14117) RBF: We can only delete the files or dirs of one subcluster in a cluster with multiple subclusters when trash is enabled

[ https://issues.apache.org/jira/browse/HDFS-14117?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16969039#comment-16969039 ] Xiaoqiao He commented on HDFS-14117: Hi [~ramkumar],[~elgoiri],[~ayushtkn], any progr

[jira] [Commented] (HDFS-14882) Consider DataNode load when #getBlockLocation

[ https://issues.apache.org/jira/browse/HDFS-14882?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16969041#comment-16969041 ] Xiaoqiao He commented on HDFS-14882: Thanks [~elgoiri] for your feedback. Ping [~pift

[jira] [Commented] (HDFS-14720) DataNode shouldn't report block as bad block if the block length is Long.MAX_VALUE.

[ https://issues.apache.org/jira/browse/HDFS-14720?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16970059#comment-16970059 ] Xiaoqiao He commented on HDFS-14720: Thanks [~surendrasingh] and [~hemanthboyina], {q

[jira] [Commented] (HDFS-14529) NPE while Loading the Editlogs

[ https://issues.apache.org/jira/browse/HDFS-14529?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16970092#comment-16970092 ] Xiaoqiao He commented on HDFS-14529: I also meet another NPE at StandbyNN (build with

[jira] [Commented] (HDFS-14529) NPE while Loading the Editlogs

[ https://issues.apache.org/jira/browse/HDFS-14529?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16970116#comment-16970116 ] Xiaoqiao He commented on HDFS-14529: Thanks [~sodonnell] for your quick response. I h

[jira] [Commented] (HDFS-14720) DataNode shouldn't report block as bad block if the block length is Long.MAX_VALUE.

[ https://issues.apache.org/jira/browse/HDFS-14720?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16970332#comment-16970332 ] Xiaoqiao He commented on HDFS-14720: [^HDFS-14720.003.patch] LGTM, +1. Thanks [~heman

[jira] [Updated] (HDFS-14952) Skip safemode if blockTotal is 0 in new NN

[ https://issues.apache.org/jira/browse/HDFS-14952?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiaoqiao He updated HDFS-14952: --- Attachment: HDFS-14952.002.patch Assignee: Xiaoqiao He Status: Patch Available (was: Op

[jira] [Commented] (HDFS-14617) Improve fsimage load time by writing sub-sections to the fsimage index

[ https://issues.apache.org/jira/browse/HDFS-14617?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16973224#comment-16973224 ] Xiaoqiao He commented on HDFS-14617: Thanks [~Feng Yuan] for your pings. IIRC, it is

[jira] [Commented] (HDFS-14986) ReplicaCachingGetSpaceUsed throws ConcurrentModificationException

[ https://issues.apache.org/jira/browse/HDFS-14986?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16973305#comment-16973305 ] Xiaoqiao He commented on HDFS-14986: we meet very similar case (ConcurrentModificatio

[jira] [Commented] (HDFS-14986) ReplicaCachingGetSpaceUsed throws ConcurrentModificationException

[ https://issues.apache.org/jira/browse/HDFS-14986?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16973339#comment-16973339 ] Xiaoqiao He commented on HDFS-14986: [~linyiqun], we did some modify when backport HD

[jira] [Commented] (HDFS-14882) Consider DataNode load when #getBlockLocation

[ https://issues.apache.org/jira/browse/HDFS-14882?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16973364#comment-16973364 ] Xiaoqiao He commented on HDFS-14882: Ping [~elgoiri],[~ayushtkn] any furthermore comm

[jira] [Updated] (HDFS-14952) Skip safemode if blockTotal is 0 in new NN

[ https://issues.apache.org/jira/browse/HDFS-14952?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiaoqiao He updated HDFS-14952: --- Attachment: HDFS-14952.003.patch > Skip safemode if blockTotal is 0 in new NN > -

[jira] [Commented] (HDFS-14952) Skip safemode if blockTotal is 0 in new NN

[ https://issues.apache.org/jira/browse/HDFS-14952?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16974868#comment-16974868 ] Xiaoqiao He commented on HDFS-14952: [^HDFS-14952.003.patch] try to fix failed unit t

[jira] [Created] (HDFS-14997) BPServiceActor process command from NameNode asynchronously

Xiaoqiao He created HDFS-14997: -- Summary: BPServiceActor process command from NameNode asynchronously Key: HDFS-14997 URL: https://issues.apache.org/jira/browse/HDFS-14997 Project: Hadoop HDFS

[jira] [Updated] (HDFS-14997) BPServiceActor process command from NameNode asynchronously

[ https://issues.apache.org/jira/browse/HDFS-14997?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiaoqiao He updated HDFS-14997: --- Attachment: HDFS-14997.001.patch Assignee: Xiaoqiao He (was: Aiphago) Status: Patch Ava

[jira] [Updated] (HDFS-14997) BPServiceActor process command from NameNode asynchronously

[ https://issues.apache.org/jira/browse/HDFS-14997?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiaoqiao He updated HDFS-14997: --- Attachment: (was: HDFS-14997.001.patch) > BPServiceActor process command from NameNode asynchrono

[jira] [Updated] (HDFS-14997) BPServiceActor process command from NameNode asynchronously

[ https://issues.apache.org/jira/browse/HDFS-14997?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiaoqiao He updated HDFS-14997: --- Attachment: HDFS-14997.001.patch > BPServiceActor process command from NameNode asynchronously >

[jira] [Commented] (HDFS-14997) BPServiceActor process command from NameNode asynchronously

[ https://issues.apache.org/jira/browse/HDFS-14997?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16978517#comment-16978517 ] Xiaoqiao He commented on HDFS-14997: Thanks [~sodonnell] for your comments. {quote}We

[jira] [Commented] (HDFS-14563) Enhance interface about recommissioning/decommissioning

[ https://issues.apache.org/jira/browse/HDFS-14563?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16978954#comment-16978954 ] Xiaoqiao He commented on HDFS-14563: Thanks [~weichiu] pick up this JIRA and sorry fo

[jira] [Created] (HDFS-15000) Improve FsDatasetImpl to avoid IO operation in datasetLock

Xiaoqiao He created HDFS-15000: -- Summary: Improve FsDatasetImpl to avoid IO operation in datasetLock Key: HDFS-15000 URL: https://issues.apache.org/jira/browse/HDFS-15000 Project: Hadoop HDFS Is

[jira] [Updated] (HDFS-14997) BPServiceActor process command from NameNode asynchronously

[ https://issues.apache.org/jira/browse/HDFS-14997?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiaoqiao He updated HDFS-14997: --- Attachment: HDFS-14997.002.patch > BPServiceActor process command from NameNode asynchronously >

[jira] [Commented] (HDFS-14997) BPServiceActor process command from NameNode asynchronously

[ https://issues.apache.org/jira/browse/HDFS-14997?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16979379#comment-16979379 ] Xiaoqiao He commented on HDFS-14997: [~sodonnell], Try to upload v002 patch and add u

[jira] [Updated] (HDFS-14997) BPServiceActor process command from NameNode asynchronously

[ https://issues.apache.org/jira/browse/HDFS-14997?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiaoqiao He updated HDFS-14997: --- Attachment: HDFS-14997.003.patch > BPServiceActor process command from NameNode asynchronously >

[jira] [Commented] (HDFS-14997) BPServiceActor process command from NameNode asynchronously

[ https://issues.apache.org/jira/browse/HDFS-14997?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16980067#comment-16980067 ] Xiaoqiao He commented on HDFS-14997: Thanks [~elgoiri] for the deep reviews. {quote}D

[jira] [Updated] (HDFS-14997) BPServiceActor process command from NameNode asynchronously

[ https://issues.apache.org/jira/browse/HDFS-14997?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiaoqiao He updated HDFS-14997: --- Attachment: HDFS-14997.004.patch > BPServiceActor process command from NameNode asynchronously >

[jira] [Commented] (HDFS-14997) BPServiceActor process command from NameNode asynchronously

[ https://issues.apache.org/jira/browse/HDFS-14997?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16982686#comment-16982686 ] Xiaoqiao He commented on HDFS-14997: Thanks [~elgoiri] for your comments. v004 try to

[jira] [Updated] (HDFS-14997) BPServiceActor process command from NameNode asynchronously

[ https://issues.apache.org/jira/browse/HDFS-14997?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiaoqiao He updated HDFS-14997: --- Attachment: HDFS-14997.005.patch > BPServiceActor process command from NameNode asynchronously >

[jira] [Commented] (HDFS-14997) BPServiceActor process command from NameNode asynchronously

[ https://issues.apache.org/jira/browse/HDFS-14997?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16983253#comment-16983253 ] Xiaoqiao He commented on HDFS-14997: [~elgoiri] v005 try to fix checkstyle and run al

[jira] [Updated] (HDFS-14997) BPServiceActor process command from NameNode asynchronously

[ https://issues.apache.org/jira/browse/HDFS-14997?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiaoqiao He updated HDFS-14997: --- Attachment: (was: HDFS-14997.005.patch) > BPServiceActor process command from NameNode asynchrono

[jira] [Updated] (HDFS-14997) BPServiceActor process command from NameNode asynchronously

[ https://issues.apache.org/jira/browse/HDFS-14997?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiaoqiao He updated HDFS-14997: --- Attachment: HDFS-14997.005.patch > BPServiceActor process command from NameNode asynchronously >

[jira] [Updated] (HDFS-12733) Option to disable to namenode local edits

[ https://issues.apache.org/jira/browse/HDFS-12733?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiaoqiao He updated HDFS-12733: --- Attachment: HDFS-12733.007.patch > Option to disable to namenode local edits > --

[jira] [Commented] (HDFS-12733) Option to disable to namenode local edits

[ https://issues.apache.org/jira/browse/HDFS-12733?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16983323#comment-16983323 ] Xiaoqiao He commented on HDFS-12733: Thanks [~ayushtkn] pick up this issue. v007 try

[jira] [Updated] (HDFS-12733) Option to disable to namenode local edits

[ https://issues.apache.org/jira/browse/HDFS-12733?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiaoqiao He updated HDFS-12733: --- Attachment: HDFS-12733.008.patch > Option to disable to namenode local edits > --

[jira] [Commented] (HDFS-12733) Option to disable to namenode local edits

[ https://issues.apache.org/jira/browse/HDFS-12733?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16983480#comment-16983480 ] Xiaoqiao He commented on HDFS-12733: v008 try to fix checkstyle reported by Jenkins.

[jira] [Commented] (HDFS-14997) BPServiceActor process command from NameNode asynchronously

[ https://issues.apache.org/jira/browse/HDFS-14997?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16983557#comment-16983557 ] Xiaoqiao He commented on HDFS-14997: Check and rerun the failed unit tests at local,

[jira] [Commented] (HDFS-14997) BPServiceActor process command from NameNode asynchronously

[ https://issues.apache.org/jira/browse/HDFS-14997?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16987530#comment-16987530 ] Xiaoqiao He commented on HDFS-14997: Ping [~elgoiri],[~sodonnell],[~weichiu] any furt

[jira] [Commented] (HDFS-16855) Remove the redundant write lock in addBlockPool

[ https://issues.apache.org/jira/browse/HDFS-16855?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17641058#comment-17641058 ] Xiaoqiao He commented on HDFS-16855: [~dingshun]Thanks for your catches. Sorry I didn

[jira] [Commented] (HDFS-16855) Remove the redundant write lock in addBlockPool

[ https://issues.apache.org/jira/browse/HDFS-16855?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17641171#comment-17641171 ] Xiaoqiao He commented on HDFS-16855: [~dingshun] Thanks for the detailed explanation.

[jira] [Commented] (HDFS-16868) Audit log duplicate problem when an ACE occurs in FSNamesystem.

[ https://issues.apache.org/jira/browse/HDFS-16868?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17645937#comment-17645937 ] Xiaoqiao He commented on HDFS-16868: [~chino71] Great catch! Thanks for your report.

[jira] [Updated] (HDFS-16868) Fix audit log duplicate issue when an ACE occurs in FSNamesystem.

[ https://issues.apache.org/jira/browse/HDFS-16868?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiaoqiao He updated HDFS-16868: --- Summary: Fix audit log duplicate issue when an ACE occurs in FSNamesystem. (was: Audit log duplicate

[jira] [Resolved] (HDFS-16868) Fix audit log duplicate issue when an ACE occurs in FSNamesystem.

[ https://issues.apache.org/jira/browse/HDFS-16868?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiaoqiao He resolved HDFS-16868. Fix Version/s: 3.4.0 Hadoop Flags: Reviewed Resolution: Fixed > Fix audit log duplicate

[jira] [Updated] (HDFS-16868) Fix audit log duplicate issue when an ACE occurs in FSNamesystem.

[ https://issues.apache.org/jira/browse/HDFS-16868?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiaoqiao He updated HDFS-16868: --- Issue Type: Improvement (was: Bug) > Fix audit log duplicate issue when an ACE occurs in FSNamesyste

[jira] [Assigned] (HDFS-16898) Make write lock fine-grain in processCommandFromActor method

[ https://issues.apache.org/jira/browse/HDFS-16898?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiaoqiao He reassigned HDFS-16898: -- Assignee: ZhangHB > Make write lock fine-grain in processCommandFromActor method > ---

[jira] [Commented] (HDFS-16898) Make write lock fine-grain in processCommandFromActor method

[ https://issues.apache.org/jira/browse/HDFS-16898?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17684410#comment-17684410 ] Xiaoqiao He commented on HDFS-16898: Add ZhangHB to contributors group, and assign th

[jira] [Updated] (HDFS-16898) Remove write lock for processCommandFromActor of DataNode to reduce impact on heartbeat

[ https://issues.apache.org/jira/browse/HDFS-16898?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiaoqiao He updated HDFS-16898: --- Summary: Remove write lock for processCommandFromActor of DataNode to reduce impact on heartbeat (wa

[jira] [Resolved] (HDFS-16898) Remove write lock for processCommandFromActor of DataNode to reduce impact on heartbeat

[ https://issues.apache.org/jira/browse/HDFS-16898?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiaoqiao He resolved HDFS-16898. Fix Version/s: 3.4.0 Hadoop Flags: Reviewed Resolution: Fixed > Remove write lock for p

[jira] [Resolved] (HDFS-16939) Fix the thread safety bug in LowRedundancyBlocks

[ https://issues.apache.org/jira/browse/HDFS-16939?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiaoqiao He resolved HDFS-16939. Fix Version/s: 3.4.0 Hadoop Flags: Reviewed Resolution: Fixed > Fix the thread safety b

[jira] [Commented] (HDFS-16939) Fix the thread safety bug in LowRedundancyBlocks

[ https://issues.apache.org/jira/browse/HDFS-16939?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17698373#comment-17698373 ] Xiaoqiao He commented on HDFS-16939: [~ste...@apache.org] Thanks for your reminder, [

[jira] [Updated] (HDFS-16939) Fix the thread safety bug in LowRedundancyBlocks

[ https://issues.apache.org/jira/browse/HDFS-16939?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiaoqiao He updated HDFS-16939: --- Fix Version/s: 3.3.5 > Fix the thread safety bug in LowRedundancyBlocks > ---

[jira] [Commented] (HDFS-16939) Fix the thread safety bug in LowRedundancyBlocks

[ https://issues.apache.org/jira/browse/HDFS-16939?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17699191#comment-17699191 ] Xiaoqiao He commented on HDFS-16939: [~ste...@apache.org] This PR has been committed

[jira] [Updated] (HDFS-16948) Update log of BlockManager#chooseExcessRedundancyStriped when EC internal block is moved by balancer

[ https://issues.apache.org/jira/browse/HDFS-16948?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiaoqiao He updated HDFS-16948: --- Summary: Update log of BlockManager#chooseExcessRedundancyStriped when EC internal block is moved by

[jira] [Updated] (HDFS-16958) EC: Fix bug in processing EC excess redundancy

[ https://issues.apache.org/jira/browse/HDFS-16958?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiaoqiao He updated HDFS-16958: --- Summary: EC: Fix bug in processing EC excess redundancy (was: Fix bug in processing EC excess redun

[jira] [Resolved] (HDFS-16948) Update log of BlockManager#chooseExcessRedundancyStriped when EC internal block is moved by balancer

[ https://issues.apache.org/jira/browse/HDFS-16948?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiaoqiao He resolved HDFS-16948. Fix Version/s: 3.4.0 Hadoop Flags: Reviewed Resolution: Fixed > Update log of BlockMana

[jira] [Resolved] (HDFS-16964) Improve processing of excess redundancy after failover

[ https://issues.apache.org/jira/browse/HDFS-16964?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiaoqiao He resolved HDFS-16964. Fix Version/s: 3.4.0 Hadoop Flags: Reviewed Resolution: Fixed > Improve processing of e

[jira] [Updated] (HDFS-16974) Consider volumes average load of each DataNode when choosing target.

[ https://issues.apache.org/jira/browse/HDFS-16974?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiaoqiao He updated HDFS-16974: --- Summary: Consider volumes average load of each DataNode when choosing target. (was: Consider load of

[jira] [Resolved] (HDFS-16974) Consider volumes average load of each DataNode when choosing target.

[ https://issues.apache.org/jira/browse/HDFS-16974?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiaoqiao He resolved HDFS-16974. Fix Version/s: 3.4.0 Hadoop Flags: Reviewed Resolution: Fixed > Consider volumes averag

[jira] [Resolved] (HDFS-16986) EC: Fix locationBudget in getListing()

[ https://issues.apache.org/jira/browse/HDFS-16986?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiaoqiao He resolved HDFS-16986. Fix Version/s: 3.4.0 Hadoop Flags: Reviewed Resolution: Fixed > EC: Fix locationBudget

[jira] [Resolved] (HDFS-16997) Set the locale to avoid printing useless logs in BlockSender

[ https://issues.apache.org/jira/browse/HDFS-16997?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiaoqiao He resolved HDFS-16997. Fix Version/s: 3.4.0 Hadoop Flags: Reviewed Resolution: Fixed > Set the locale to avoid

[jira] [Resolved] (HDFS-16999) Fix wrong use of processFirstBlockReport()

[ https://issues.apache.org/jira/browse/HDFS-16999?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiaoqiao He resolved HDFS-16999. Fix Version/s: 3.4.0 Hadoop Flags: Reviewed Resolution: Fixed > Fix wrong use of proces

[jira] [Commented] (HDFS-17003) Erasure coding: invalidate wrong block after reporting bad blocks from datanode

[ https://issues.apache.org/jira/browse/HDFS-17003?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17720752#comment-17720752 ] Xiaoqiao He commented on HDFS-17003: [~zhanghaobo] Thanks for your report. Any more i

[jira] [Updated] (HDFS-16985) Fix data missing issue when delete local block file.

[ https://issues.apache.org/jira/browse/HDFS-16985?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiaoqiao He updated HDFS-16985: --- Summary: Fix data missing issue when delete local block file. (was: delete local block file when Fil

[jira] [Resolved] (HDFS-16985) Fix data missing issue when delete local block file.

[ https://issues.apache.org/jira/browse/HDFS-16985?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiaoqiao He resolved HDFS-16985. Fix Version/s: 3.4.0 Hadoop Flags: Reviewed Target Version/s: (was: 3.3.1)

[jira] [Updated] (HDFS-16909) Improve ReplicaMap#mergeAll method.

[ https://issues.apache.org/jira/browse/HDFS-16909?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiaoqiao He updated HDFS-16909: --- Fix Version/s: 3.4.0 Hadoop Flags: Reviewed Affects Version/s: (was: 3.3.4)

[jira] [Updated] (HDFS-17026) RBF: NamenodeHeartbeatService should update JMX report with configurable frequency

[ https://issues.apache.org/jira/browse/HDFS-17026?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiaoqiao He updated HDFS-17026: --- Summary: RBF: NamenodeHeartbeatService should update JMX report with configurable frequency (was: Na

[jira] [Commented] (HDFS-17026) RBF: NamenodeHeartbeatService should update JMX report with configurable frequency

[ https://issues.apache.org/jira/browse/HDFS-17026?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17726451#comment-17726451 ] Xiaoqiao He commented on HDFS-17026: {quote}We should configure the NamenodeHeartbeat

[jira] [Resolved] (HDFS-17026) RBF: NamenodeHeartbeatService should update JMX report with configurable frequency

[ https://issues.apache.org/jira/browse/HDFS-17026?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiaoqiao He resolved HDFS-17026. Fix Version/s: 3.4.0 Hadoop Flags: Reviewed Resolution: Fixed > RBF: NamenodeHeartbeatS

[jira] [Commented] (HDFS-17026) RBF: NamenodeHeartbeatService should update JMX report with configurable frequency

[ https://issues.apache.org/jira/browse/HDFS-17026?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17728183#comment-17728183 ] Xiaoqiao He commented on HDFS-17026: [~hchaverri] Please submit this PR for branch-3.

[jira] [Updated] (HDFS-17003) Erasure Coding: invalidate wrong block after reporting bad blocks from datanode

[ https://issues.apache.org/jira/browse/HDFS-17003?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiaoqiao He updated HDFS-17003: --- Summary: Erasure Coding: invalidate wrong block after reporting bad blocks from datanode (was: Erasu

[jira] [Resolved] (HDFS-17003) Erasure Coding: invalidate wrong block after reporting bad blocks from datanode

[ https://issues.apache.org/jira/browse/HDFS-17003?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiaoqiao He resolved HDFS-17003. Fix Version/s: 3.4.0 Hadoop Flags: Reviewed Resolution: Fixed > Erasure Coding: invalid

[jira] [Updated] (HDFS-17003) Erasure Coding: invalidate wrong block after reporting bad blocks from datanode

[ https://issues.apache.org/jira/browse/HDFS-17003?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiaoqiao He updated HDFS-17003: --- Component/s: namenode > Erasure Coding: invalidate wrong block after reporting bad blocks from > dat

[jira] [Commented] (HDFS-17003) Erasure Coding: invalidate wrong block after reporting bad blocks from datanode

[ https://issues.apache.org/jira/browse/HDFS-17003?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17730500#comment-17730500 ] Xiaoqiao He commented on HDFS-17003: cherry-pick to branch-3.3. > Erasure Coding: i

[jira] [Updated] (HDFS-17003) Erasure Coding: invalidate wrong block after reporting bad blocks from datanode

[ https://issues.apache.org/jira/browse/HDFS-17003?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiaoqiao He updated HDFS-17003: --- Fix Version/s: 3.3.6 > Erasure Coding: invalidate wrong block after reporting bad blocks from > data

[jira] [Resolved] (HDFS-17037) Consider nonDfsUsed when running balancer

[ https://issues.apache.org/jira/browse/HDFS-17037?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiaoqiao He resolved HDFS-17037. Fix Version/s: 3.4.0 Hadoop Flags: Reviewed Resolution: Fixed > Consider nonDfsUsed whe

[jira] [Updated] (HDFS-17037) Consider nonDfsUsed when running balancer

[ https://issues.apache.org/jira/browse/HDFS-17037?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiaoqiao He updated HDFS-17037: --- Component/s: balamcer Issue Type: Improvement (was: Bug) > Consider nonDfsUsed when running bal

[jira] [Updated] (HDFS-17037) Consider nonDfsUsed when running balancer

[ https://issues.apache.org/jira/browse/HDFS-17037?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiaoqiao He updated HDFS-17037: --- Component/s: balancer & mover (was: balamcer) > Consider nonDfsUsed when running

[jira] [Commented] (HDFS-16898) Remove write lock for processCommandFromActor of DataNode to reduce impact on heartbeat

[ https://issues.apache.org/jira/browse/HDFS-16898?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17731448#comment-17731448 ] Xiaoqiao He commented on HDFS-16898: Committed to branch-3.3. > Remove write lock fo

[jira] [Updated] (HDFS-16898) Remove write lock for processCommandFromActor of DataNode to reduce impact on heartbeat

[ https://issues.apache.org/jira/browse/HDFS-16898?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiaoqiao He updated HDFS-16898: --- Fix Version/s: 3.3.7 > Remove write lock for processCommandFromActor of DataNode to reduce impact on

[jira] [Commented] (HDFS-17046) Add metrics or delete directly when path can not parsed in Trash.

[ https://issues.apache.org/jira/browse/HDFS-17046?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17732023#comment-17732023 ] Xiaoqiao He commented on HDFS-17046: Thanks [~zhanghaobo] for your report. Do you mea

[jira] [Commented] (HDFS-17048) FSNamesystem.delete() maybe cause data residue when active namenode crash or shutdown

[ https://issues.apache.org/jira/browse/HDFS-17048?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17732434#comment-17732434 ] Xiaoqiao He commented on HDFS-17048: [~liuguanghua] Thanks for your report. Sorry I d

[jira] [Resolved] (HDFS-17049) EC: Fix duplicate block group IDs generated by SequentialBlockGroupIdGenerator

[ https://issues.apache.org/jira/browse/HDFS-17049?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiaoqiao He resolved HDFS-17049. Resolution: Not A Problem > EC: Fix duplicate block group IDs generated by SequentialBlockGroupIdGe

[jira] [Commented] (HDFS-6994) libhdfs3 - A native C/C++ HDFS client

[ https://issues.apache.org/jira/browse/HDFS-6994?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17735663#comment-17735663 ] Xiaoqiao He commented on HDFS-6994: --- Hi [~wangzw],[~cmccabe], Thanks for your great work

[jira] [Updated] (HDFS-17044) Set size of non-exist block to NO_ACK when process FBR or IBR to avoid useless report from DataNode

[ https://issues.apache.org/jira/browse/HDFS-17044?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiaoqiao He updated HDFS-17044: --- Summary: Set size of non-exist block to NO_ACK when process FBR or IBR to avoid useless report from D

[jira] [Resolved] (HDFS-17044) Set size of non-exist block to NO_ACK when process FBR or IBR to avoid useless report from DataNode

[ https://issues.apache.org/jira/browse/HDFS-17044?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiaoqiao He resolved HDFS-17044. Fix Version/s: 3.4.0 Hadoop Flags: Reviewed Resolution: Fixed > Set size of non-exist b

[jira] [Updated] (HDFS-17044) Set size of non-exist block to NO_ACK when process FBR or IBR to avoid useless report from DataNode

[ https://issues.apache.org/jira/browse/HDFS-17044?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiaoqiao He updated HDFS-17044: --- Component/s: namenode > Set size of non-exist block to NO_ACK when process FBR or IBR to avoid > use

[jira] [Resolved] (HDFS-17052) Improve BlockPlacementPolicyRackFaultTolerant to avoid choose nodes failed when no enough Rack.

[ https://issues.apache.org/jira/browse/HDFS-17052?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiaoqiao He resolved HDFS-17052. Fix Version/s: 3.4.0 Hadoop Flags: Reviewed Resolution: Fixed Committed to trunk. And m

[jira] [Commented] (HDFS-17071) Erasure coding: delete duplicated ec block with same block group id on one datanode.

[ https://issues.apache.org/jira/browse/HDFS-17071?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17740925#comment-17740925 ] Xiaoqiao He commented on HDFS-17071: I found that this JIRA report the same issue as

[jira] [Commented] (HDFS-17072) DFSAdmin: add a triggerVolumeScanner command

[ https://issues.apache.org/jira/browse/HDFS-17072?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17740926#comment-17740926 ] Xiaoqiao He commented on HDFS-17072: Thanks to involve me here. +1 from my side. > D

[jira] [Resolved] (HDFS-17073) Enhance the warning message output for BlockGroupNonStripedChecksumComputer#compute

[ https://issues.apache.org/jira/browse/HDFS-17073?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiaoqiao He resolved HDFS-17073. Fix Version/s: 3.4.0 Hadoop Flags: Reviewed Resolution: Fixed > Enhance the warning mes

[jira] [Resolved] (HDFS-17033) Update fsck to display stale state info of blocks accurately

[ https://issues.apache.org/jira/browse/HDFS-17033?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiaoqiao He resolved HDFS-17033. Fix Version/s: 3.4.0 Hadoop Flags: Reviewed Resolution: Fixed > Update fsck to display

[jira] [Created] (HDFS-17090) Decommission will be stuck for long time when restart because overlapped process Register and BlockReport.

Xiaoqiao He created HDFS-17090: -- Summary: Decommission will be stuck for long time when restart because overlapped process Register and BlockReport. Key: HDFS-17090 URL: https://issues.apache.org/jira/browse/HDFS-170

[jira] [Commented] (HDFS-17090) Decommission will be stuck for long time when restart because overlapped process Register and BlockReport.

[ https://issues.apache.org/jira/browse/HDFS-17090?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17743790#comment-17743790 ] Xiaoqiao He commented on HDFS-17090: We mark `DatanodeProtocol#registerDatanode` inte

[jira] [Commented] (HDFS-17093) In the case of all datanodes sending FBR when the namenode restarts (large clusters), there is an issue with incomplete block reporting

[ https://issues.apache.org/jira/browse/HDFS-17093?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17744424#comment-17744424 ] Xiaoqiao He commented on HDFS-17093: [~yuyanlei] Thanks for your report, Would you tr

[jira] [Commented] (HDFS-17097) Out of Memory when set dfs.namenode.blockreport.queue.size to a large number

[ https://issues.apache.org/jira/browse/HDFS-17097?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17744429#comment-17744429 ] Xiaoqiao He commented on HDFS-17097: [~FuzzingTeam] Thanks for your report. We receiv

[jira] [Comment Edited] (HDFS-17097) Out of Memory when set dfs.namenode.blockreport.queue.size to a large number

[ https://issues.apache.org/jira/browse/HDFS-17097?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17744429#comment-17744429 ] Xiaoqiao He edited comment on HDFS-17097 at 7/19/23 3:54 AM: -

[jira] [Commented] (HDFS-17093) In the case of all datanodes sending FBR when the namenode restarts (large clusters), there is an issue with incomplete block reporting

[ https://issues.apache.org/jira/browse/HDFS-17093?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17744526#comment-17744526 ] Xiaoqiao He commented on HDFS-17093: {quote}# DN wants to send 12 reports but only se

[jira] [Commented] (HDFS-17097) Out of Memory when set dfs.namenode.blockreport.queue.size to a large number

[ https://issues.apache.org/jira/browse/HDFS-17097?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17746189#comment-17746189 ] Xiaoqiao He commented on HDFS-17097: [~FuzzingTeam] Great, I try to search JIRA from

[jira] [Resolved] (HDFS-17112) Show decommission duration in JMX and HTML

[ https://issues.apache.org/jira/browse/HDFS-17112?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiaoqiao He resolved HDFS-17112. Fix Version/s: 3.4.0 Hadoop Flags: Reviewed Resolution: Fixed > Show decommission durat

[jira] [Updated] (HDFS-17112) Show decommission duration in JMX and HTML

[ https://issues.apache.org/jira/browse/HDFS-17112?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiaoqiao He updated HDFS-17112: --- Component/s: namenode > Show decommission duration in JMX and HTML >

[jira] [Updated] (HDFS-17090) Decommission will be stuck for long time when restart because overlapped process Register and BlockReport.

[ https://issues.apache.org/jira/browse/HDFS-17090?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiaoqiao He updated HDFS-17090: --- Description: I met one corner case recently, which decommission DataNode impact performance of NameN

<    5   6   7   8   9   10   11   >