Re: [PR] HDFS-17218. NameNode should process time out excess redundancy blocks [hadoop]

2023-12-04 Thread via GitHub
haiyang1987 commented on PR #6176: URL: https://github.com/apache/hadoop/pull/6176#issuecomment-1839885256 Thanks @Hexiaoqiao @zhangshuyan0 @ZanderXu for your review and merge. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub

Re: [PR] HDFS-17218. NameNode should process time out excess redundancy blocks [hadoop]

2023-12-04 Thread via GitHub
Hexiaoqiao commented on PR #6176: URL: https://github.com/apache/hadoop/pull/6176#issuecomment-1838627733 Committed to trunk. Thanks @haiyang1987 for your works. And @zhangshuyan0 @ZanderXu for your reviews. -- This is an automated message from the Apache Git Service. To respond to the

Re: [PR] HDFS-17218. NameNode should process time out excess redundancy blocks [hadoop]

2023-12-04 Thread via GitHub
Hexiaoqiao merged PR #6176: URL: https://github.com/apache/hadoop/pull/6176 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

Re: [PR] HDFS-17218. NameNode should process time out excess redundancy blocks [hadoop]

2023-11-28 Thread via GitHub
haiyang1987 commented on PR #6176: URL: https://github.com/apache/hadoop/pull/6176#issuecomment-1831105752 Thanks @Hexiaoqiao for your review! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the

Re: [PR] HDFS-17218. NameNode should process time out excess redundancy blocks [hadoop]

2023-11-28 Thread via GitHub
Hexiaoqiao commented on PR #6176: URL: https://github.com/apache/hadoop/pull/6176#issuecomment-1829998282 Let's try to checkin while wait for two workdays if no more comments. Thanks. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to

Re: [PR] HDFS-17218. NameNode should process time out excess redundancy blocks [hadoop]

2023-11-20 Thread via GitHub
hadoop-yetus commented on PR #6176: URL: https://github.com/apache/hadoop/pull/6176#issuecomment-1818909919 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok: |

Re: [PR] HDFS-17218. NameNode should process time out excess redundancy blocks [hadoop]

2023-11-19 Thread via GitHub
haiyang1987 commented on code in PR #6176: URL: https://github.com/apache/hadoop/pull/6176#discussion_r1398667667 ## hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/DFSConfigKeys.java: ## @@ -315,6 +315,13 @@ public class DFSConfigKeys extends

Re: [PR] HDFS-17218. NameNode should process time out excess redundancy blocks [hadoop]

2023-11-19 Thread via GitHub
zhangshuyan0 commented on code in PR #6176: URL: https://github.com/apache/hadoop/pull/6176#discussion_r1398373458 ## hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/DFSConfigKeys.java: ## @@ -315,6 +315,13 @@ public class DFSConfigKeys extends

Re: [PR] HDFS-17218. NameNode should process time out excess redundancy blocks [hadoop]

2023-11-16 Thread via GitHub
haiyang1987 commented on PR #6176: URL: https://github.com/apache/hadoop/pull/6176#issuecomment-1815653206 The failed unit test seems unrelated to the change, local run test is normal. -- This is an automated message from the Apache Git Service. To respond to the message, please log

Re: [PR] HDFS-17218. NameNode should process time out excess redundancy blocks [hadoop]

2023-11-16 Thread via GitHub
hadoop-yetus commented on PR #6176: URL: https://github.com/apache/hadoop/pull/6176#issuecomment-1814326717 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok: |

Re: [PR] HDFS-17218. NameNode should process time out excess redundancy blocks [hadoop]

2023-11-15 Thread via GitHub
haiyang1987 commented on code in PR #6176: URL: https://github.com/apache/hadoop/pull/6176#discussion_r1395148977 ## hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/blockmanagement/BlockManager.java: ## @@ -3040,6 +3060,100 @@ void

Re: [PR] HDFS-17218. NameNode should process time out excess redundancy blocks [hadoop]

2023-11-15 Thread via GitHub
Hexiaoqiao commented on code in PR #6176: URL: https://github.com/apache/hadoop/pull/6176#discussion_r1395128857 ## hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/blockmanagement/BlockManager.java: ## @@ -3040,6 +3060,100 @@ void

Re: [PR] HDFS-17218. NameNode should process time out excess redundancy blocks [hadoop]

2023-11-15 Thread via GitHub
haiyang1987 commented on PR #6176: URL: https://github.com/apache/hadoop/pull/6176#issuecomment-1812423707 Hi @ayushtkn @tomscut @xinglin Would you mind to also take a review this pr when you have free time? thank you very much~ -- This is an automated message from the Apache Git

Re: [PR] HDFS-17218. NameNode should process time out excess redundancy blocks [hadoop]

2023-11-14 Thread via GitHub
hadoop-yetus commented on PR #6176: URL: https://github.com/apache/hadoop/pull/6176#issuecomment-1810518324 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok: |

Re: [PR] HDFS-17218. NameNode should process time out excess redundancy blocks [hadoop]

2023-11-09 Thread via GitHub
haiyang1987 commented on PR #6176: URL: https://github.com/apache/hadoop/pull/6176#issuecomment-180365 Test failures seems unrelated. However changes in the `contains` method and `remove` method input blockInfo and the equals `method` in ExcessBlockInfo will cause some SpotBugs

Re: [PR] HDFS-17218. NameNode should process time out excess redundancy blocks [hadoop]

2023-11-07 Thread via GitHub
hadoop-yetus commented on PR #6176: URL: https://github.com/apache/hadoop/pull/6176#issuecomment-1799533592 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok: |

Re: [PR] HDFS-17218. NameNode should process time out excess redundancy blocks [hadoop]

2023-11-07 Thread via GitHub
haiyang1987 commented on PR #6176: URL: https://github.com/apache/hadoop/pull/6176#issuecomment-1798385249 Update PR. Hi @Hexiaoqiao @zhangshuyan0 please help me review it again when you have free time. Thanks~ -- This is an automated message from the Apache Git Service. To respond to

Re: [PR] HDFS-17218. NameNode should process time out excess redundancy blocks [hadoop]

2023-11-07 Thread via GitHub
haiyang1987 commented on PR #6176: URL: https://github.com/apache/hadoop/pull/6176#issuecomment-1798368726 Thanks @Hexiaoqiao for your comment, i will fix it later. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the

Re: [PR] HDFS-17218. NameNode should process time out excess redundancy blocks [hadoop]

2023-11-07 Thread via GitHub
haiyang1987 commented on code in PR #6176: URL: https://github.com/apache/hadoop/pull/6176#discussion_r1384808440 ## hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/blockmanagement/ExcessRedundancyMap.java: ## @@ -111,4 +113,56 @@ synchronized

Re: [PR] HDFS-17218. NameNode should process time out excess redundancy blocks [hadoop]

2023-11-07 Thread via GitHub
hadoop-yetus commented on PR #6176: URL: https://github.com/apache/hadoop/pull/6176#issuecomment-1798365207 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok: |

Re: [PR] HDFS-17218. NameNode should process time out excess redundancy blocks [hadoop]

2023-11-07 Thread via GitHub
haiyang1987 commented on code in PR #6176: URL: https://github.com/apache/hadoop/pull/6176#discussion_r1384798128 ## hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/blockmanagement/BlockManager.java: ## @@ -3040,6 +3058,99 @@ void

Re: [PR] HDFS-17218. NameNode should process time out excess redundancy blocks [hadoop]

2023-11-07 Thread via GitHub
haiyang1987 commented on code in PR #6176: URL: https://github.com/apache/hadoop/pull/6176#discussion_r1384797921 ## hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/blockmanagement/BlockManager.java: ## @@ -3040,6 +3058,99 @@ void

Re: [PR] HDFS-17218. NameNode should process time out excess redundancy blocks [hadoop]

2023-11-07 Thread via GitHub
Hexiaoqiao commented on code in PR #6176: URL: https://github.com/apache/hadoop/pull/6176#discussion_r1384638620 ## hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/blockmanagement/BlockManager.java: ## @@ -3040,6 +3058,99 @@ void

Re: [PR] HDFS-17218. NameNode should process time out excess redundancy blocks [hadoop]

2023-11-06 Thread via GitHub
haiyang1987 commented on code in PR #6176: URL: https://github.com/apache/hadoop/pull/6176#discussion_r1384392878 ## hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/blockmanagement/BlockManager.java: ## @@ -3040,6 +3058,98 @@ void

Re: [PR] HDFS-17218. NameNode should process time out excess redundancy blocks [hadoop]

2023-11-06 Thread via GitHub
haiyang1987 commented on code in PR #6176: URL: https://github.com/apache/hadoop/pull/6176#discussion_r1384389728 ## hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/blockmanagement/BlockManager.java: ## @@ -3040,6 +3058,98 @@ void

Re: [PR] HDFS-17218. NameNode should process time out excess redundancy blocks [hadoop]

2023-11-05 Thread via GitHub
zhangshuyan0 commented on code in PR #6176: URL: https://github.com/apache/hadoop/pull/6176#discussion_r1382557181 ## hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/blockmanagement/BlockManager.java: ## @@ -3040,6 +3058,98 @@ void

Re: [PR] HDFS-17218. NameNode should process time out excess redundancy blocks [hadoop]

2023-11-02 Thread via GitHub
haiyang1987 commented on PR #6176: URL: https://github.com/apache/hadoop/pull/6176#issuecomment-1790330475 Test failures seems unrelated. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to

Re: [PR] HDFS-17218. NameNode should process time out excess redundancy blocks [hadoop]

2023-10-31 Thread via GitHub
haiyang1987 commented on PR #6176: URL: https://github.com/apache/hadoop/pull/6176#issuecomment-1786851988 Hi @Hexiaoqiao @ayushtkn @ZanderXu @zhangshuyan0 Could you please help review this pr again when you have free time. Thanks~ -- This is an automated message from the

Re: [PR] HDFS-17218. NameNode should process time out excess redundancy blocks [hadoop]

2023-10-30 Thread via GitHub
hadoop-yetus commented on PR #6176: URL: https://github.com/apache/hadoop/pull/6176#issuecomment-1785014526 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok: |

Re: [PR] HDFS-17218. NameNode should process time out excess redundancy blocks [hadoop]

2023-10-29 Thread via GitHub
haiyang1987 closed pull request #6233: HDFS-17218. NameNode should process time out excess redundancy blocks URL: https://github.com/apache/hadoop/pull/6233 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above

Re: [PR] HDFS-17218. NameNode should process time out excess redundancy blocks [hadoop]

2023-10-29 Thread via GitHub
haiyang1987 commented on PR #6233: URL: https://github.com/apache/hadoop/pull/6233#issuecomment-1784487882 Close the current PR, the code has been submitted PR at #6176 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use

Re: [PR] HDFS-17218. NameNode should process time out excess redundancy blocks [hadoop]

2023-10-29 Thread via GitHub
haiyang1987 commented on PR #6233: URL: https://github.com/apache/hadoop/pull/6233#issuecomment-1784482645 > Hi @haiyang1987 would you mind to submit PR at #6176 if it is the same improvement, thus we could link the discussion together and will be helpful for end users or reviewers to

Re: [PR] HDFS-17218. NameNode should process time out excess redundancy blocks [hadoop]

2023-10-29 Thread via GitHub
Hexiaoqiao commented on PR #6233: URL: https://github.com/apache/hadoop/pull/6233#issuecomment-1784390596 Hi @haiyang1987 would you mind to submit PR at https://github.com/apache/hadoop/pull/6176 if it is the same improvement, thus we could link the discussion together and will be helpful

Re: [PR] HDFS-17218. NameNode should process time out excess redundancy blocks [hadoop]

2023-10-29 Thread via GitHub
haiyang1987 commented on PR #6233: URL: https://github.com/apache/hadoop/pull/6233#issuecomment-1784036840 Hi @Hexiaoqiao @ayushtkn @zhangshuyan0 @ZanderXu excuse me, could you review this PR for me when you are free? Thank you very much~ -- This is an automated message from the Apache

Re: [PR] HDFS-17218. NameNode should process time out excess redundancy blocks [hadoop]

2023-10-29 Thread via GitHub
hadoop-yetus commented on PR #6233: URL: https://github.com/apache/hadoop/pull/6233#issuecomment-1784018697 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok: |

Re: [PR] HDFS-17218. NameNode should process time out excess redundancy blocks [hadoop]

2023-10-28 Thread via GitHub
hadoop-yetus commented on PR #6233: URL: https://github.com/apache/hadoop/pull/6233#issuecomment-1783896891 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok: |

Re: [PR] HDFS-17218. NameNode should process time out excess redundancy blocks [hadoop]

2023-10-28 Thread via GitHub
hadoop-yetus commented on PR #6233: URL: https://github.com/apache/hadoop/pull/6233#issuecomment-1783894749 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok: |

[PR] HDFS-17218. NameNode should process time out excess redundancy blocks [hadoop]

2023-10-28 Thread via GitHub
haiyang1987 opened a new pull request, #6233: URL: https://github.com/apache/hadoop/pull/6233 ### Description of PR Description of PR https://issues.apache.org/jira/browse/HDFS-17218 Currently found that DN will lose all pending DNA_INVALIDATE blocks if it restarts. DN