[GitHub] [hadoop] xinglin commented on a diff in pull request #5860: HDFS-17111. RBF: Optimize msync to only call nameservices that have observer reads enabled.

2023-07-20 Thread via GitHub
xinglin commented on code in PR #5860: URL: https://github.com/apache/hadoop/pull/5860#discussion_r1270233735 ## hadoop-hdfs-project/hadoop-hdfs-rbf/src/main/java/org/apache/hadoop/hdfs/server/federation/router/RouterClientProtocol.java: ## @@ -194,6 +202,18 @@ public class

[jira] [Commented] (HADOOP-18805) s3a large file prefetch tests are too slow, don't validate data

2023-07-20 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18805?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17745383#comment-17745383 ] ASF GitHub Bot commented on HADOOP-18805: - virajjasani commented on PR #5851: URL:

[GitHub] [hadoop] virajjasani commented on pull request #5851: HADOOP-18805. S3A prefetch tests to work with small files

2023-07-20 Thread via GitHub
virajjasani commented on PR #5851: URL: https://github.com/apache/hadoop/pull/5851#issuecomment-1644973927 i plan to initiate few more followup sub-tasks: a) LRU contents unit testable with some refactor b) LRU eviction metrics c) LRU eviction and file deletion as configurable

[jira] [Commented] (HADOOP-18816) Rebuild Exceptions on Client side to get genuine exceptions

2023-07-20 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18816?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17745363#comment-17745363 ] ASF GitHub Bot commented on HADOOP-18816: - ferhui commented on PR #5863: URL:

[GitHub] [hadoop] ferhui commented on pull request #5863: HADOOP-18816. Rebuild exceptions on Client side

2023-07-20 Thread via GitHub
ferhui commented on PR #5863: URL: https://github.com/apache/hadoop/pull/5863#issuecomment-1644946546 @ZanderXu could you please take a look at this PR when you are available? Thanks -- This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [hadoop] ferhui commented on pull request #5848: MAPREDUCE-7442. Exception message is not intusive when accessing the …

2023-07-20 Thread via GitHub
ferhui commented on PR #5848: URL: https://github.com/apache/hadoop/pull/5848#issuecomment-1644941520 @liangyu-1 Thanks for your contribution. Will merge it tmr if no other comments -- This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [hadoop] hfutatzhanghb commented on pull request #5778: HDFS-17060. BlockPlacementPolicyDefault#chooseReplicaToDelete should consider datanode load.

2023-07-20 Thread via GitHub
hfutatzhanghb commented on PR #5778: URL: https://github.com/apache/hadoop/pull/5778#issuecomment-1644940335 @ayushtkn Sir, sorry for disturbing you and involving you here. Please also take a look at this PR when you have free time, Thanks a lot. -- This is an automated message from the

[GitHub] [hadoop] zhangshuyan0 commented on pull request #5866: HDFS-17112. Show decommission duration in JMX and HTML.

2023-07-20 Thread via GitHub
zhangshuyan0 commented on PR #5866: URL: https://github.com/apache/hadoop/pull/5866#issuecomment-1644927014 @Hexiaoqiao Thanks for your review. I'll add more page info and update this PR later. -- This is an automated message from the Apache Git Service. To respond to the message, please

[GitHub] [hadoop] zhangshuyan0 commented on a diff in pull request #5866: HDFS-17112. Show decommission duration in JMX and HTML.

2023-07-20 Thread via GitHub
zhangshuyan0 commented on code in PR #5866: URL: https://github.com/apache/hadoop/pull/5866#discussion_r1270185680 ## hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSNamesystem.java: ## @@ -6664,6 +6664,8 @@ public String getDecomNodes() {

[GitHub] [hadoop] hubble-insight commented on pull request #5398: HDFS-16922. The logic of IncrementalBlockReportManager#addRDBI method may cause missing blocks when cluster is busy.

2023-07-20 Thread via GitHub
hubble-insight commented on PR #5398: URL: https://github.com/apache/hadoop/pull/5398#issuecomment-1644922899 "This fix seems to only alleviate the probability of this issue occurring. If the incremental report of this chunk has already been reported to nn and is not cached in

[GitHub] [hadoop] Hexiaoqiao commented on a diff in pull request #5866: HDFS-17112. Show decommission duration in JMX and HTML.

2023-07-20 Thread via GitHub
Hexiaoqiao commented on code in PR #5866: URL: https://github.com/apache/hadoop/pull/5866#discussion_r1270178517 ## hadoop-hdfs-project/hadoop-hdfs/src/main/webapps/hdfs/dfshealth.html: ## @@ -414,6 +415,7 @@ {underReplicatedBlocks} {decommissionOnlyReplicas}

[GitHub] [hadoop] hfutatzhanghb commented on a diff in pull request #5866: HDFS-17112. Show decommission duration in JMX and HTML.

2023-07-20 Thread via GitHub
hfutatzhanghb commented on code in PR #5866: URL: https://github.com/apache/hadoop/pull/5866#discussion_r1270166295 ## hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSNamesystem.java: ## @@ -6664,6 +6664,8 @@ public String getDecomNodes()

[GitHub] [hadoop] hfutatzhanghb commented on a diff in pull request #5866: HDFS-17112. Show decommission duration in JMX and HTML.

2023-07-20 Thread via GitHub
hfutatzhanghb commented on code in PR #5866: URL: https://github.com/apache/hadoop/pull/5866#discussion_r1270163504 ## hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSNamesystem.java: ## @@ -6664,6 +6664,8 @@ public String getDecomNodes()

[GitHub] [hadoop] zhangshuyan0 commented on a diff in pull request #5866: HDFS-17112. Show decommission duration in JMX and HTML.

2023-07-20 Thread via GitHub
zhangshuyan0 commented on code in PR #5866: URL: https://github.com/apache/hadoop/pull/5866#discussion_r1270162405 ## hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSNamesystem.java: ## @@ -6664,6 +6664,8 @@ public String getDecomNodes() {

[GitHub] [hadoop] simbadzina commented on a diff in pull request #5860: HDFS-17111. RBF: Optimize msync to only call nameservices that have observer reads enabled.

2023-07-20 Thread via GitHub
simbadzina commented on code in PR #5860: URL: https://github.com/apache/hadoop/pull/5860#discussion_r1270153376 ## hadoop-hdfs-project/hadoop-hdfs-rbf/src/main/java/org/apache/hadoop/hdfs/server/federation/router/RouterClientProtocol.java: ## @@ -194,6 +202,18 @@ public class

[GitHub] [hadoop] xinglin commented on a diff in pull request #5860: HDFS-17111. RBF: Optimize msync to only call nameservices that have observer reads enabled.

2023-07-20 Thread via GitHub
xinglin commented on code in PR #5860: URL: https://github.com/apache/hadoop/pull/5860#discussion_r1270146327 ## hadoop-hdfs-project/hadoop-hdfs-rbf/src/main/java/org/apache/hadoop/hdfs/server/federation/router/RouterClientProtocol.java: ## @@ -194,6 +202,18 @@ public class

[GitHub] [hadoop] xinglin commented on a diff in pull request #5860: HDFS-17111. RBF: Optimize msync to only call nameservices that have observer reads enabled.

2023-07-20 Thread via GitHub
xinglin commented on code in PR #5860: URL: https://github.com/apache/hadoop/pull/5860#discussion_r1270146327 ## hadoop-hdfs-project/hadoop-hdfs-rbf/src/main/java/org/apache/hadoop/hdfs/server/federation/router/RouterClientProtocol.java: ## @@ -194,6 +202,18 @@ public class

[GitHub] [hadoop] hadoop-yetus commented on pull request #5860: HDFS-17111. RBF: Optimize msync to only call nameservices that have observer reads enabled.

2023-07-20 Thread via GitHub
hadoop-yetus commented on PR #5860: URL: https://github.com/apache/hadoop/pull/5860#issuecomment-1644855434 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok: |

[GitHub] [hadoop] hadoop-yetus commented on pull request #5860: HDFS-17111. RBF: Optimize msync to only call nameservices that have observer reads enabled.

2023-07-20 Thread via GitHub
hadoop-yetus commented on PR #5860: URL: https://github.com/apache/hadoop/pull/5860#issuecomment-1644796572 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok: |

[GitHub] [hadoop] hadoop-yetus commented on pull request #5860: HDFS-17111. RBF: Optimize msync to only call nameservices that have observer reads enabled.

2023-07-20 Thread via GitHub
hadoop-yetus commented on PR #5860: URL: https://github.com/apache/hadoop/pull/5860#issuecomment-1644769849 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok: |

[GitHub] [hadoop] hadoop-yetus commented on pull request #5860: HDFS-17111. RBF: Optimize msync to only call nameservices that have observer reads enabled.

2023-07-20 Thread via GitHub
hadoop-yetus commented on PR #5860: URL: https://github.com/apache/hadoop/pull/5860#issuecomment-1644767505 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok: |

[jira] [Commented] (HADOOP-18073) Upgrade AWS SDK to v2

2023-07-20 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18073?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17745299#comment-17745299 ] ASF GitHub Bot commented on HADOOP-18073: - hadoop-yetus commented on PR #5858: URL:

[GitHub] [hadoop] hadoop-yetus commented on pull request #5858: HADOOP-18073. Upgrade S3A in 3.3 branch to AWS SDK V2.(#5707)

2023-07-20 Thread via GitHub
hadoop-yetus commented on PR #5858: URL: https://github.com/apache/hadoop/pull/5858#issuecomment-1644742615 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok: |

[jira] [Commented] (HADOOP-18805) s3a large file prefetch tests are too slow, don't validate data

2023-07-20 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18805?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17745295#comment-17745295 ] ASF GitHub Bot commented on HADOOP-18805: - virajjasani commented on PR #5851: URL:

[GitHub] [hadoop] virajjasani commented on pull request #5851: HADOOP-18805. S3A prefetch tests to work with small files

2023-07-20 Thread via GitHub
virajjasani commented on PR #5851: URL: https://github.com/apache/hadoop/pull/5851#issuecomment-1644716330 us-west-2: `$ mvn clean verify -Dparallel-tests -DtestsThreadCount=8 -Dscale -Dprefetch` ``` [INFO]

[jira] [Commented] (HADOOP-18778) Test failures with CSE enabled

2023-07-20 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18778?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17745293#comment-17745293 ] ASF GitHub Bot commented on HADOOP-18778: - mukund-thakur commented on PR #5763: URL:

[GitHub] [hadoop] mukund-thakur commented on pull request #5763: HADOOP-18778. Fixes failing tests when CSE is enabled.

2023-07-20 Thread via GitHub
mukund-thakur commented on PR #5763: URL: https://github.com/apache/hadoop/pull/5763#issuecomment-1644715347 Looks good to me. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific

[GitHub] [hadoop] hadoop-yetus commented on pull request #5860: HDFS-17111. RBF: Optimize msync to only call nameservices that have observer reads enabled.

2023-07-20 Thread via GitHub
hadoop-yetus commented on PR #5860: URL: https://github.com/apache/hadoop/pull/5860#issuecomment-1644701615 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok: |

[jira] [Commented] (HADOOP-18805) s3a large file prefetch tests are too slow, don't validate data

2023-07-20 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18805?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17745289#comment-17745289 ] ASF GitHub Bot commented on HADOOP-18805: - hadoop-yetus commented on PR #5851: URL:

[GitHub] [hadoop] hadoop-yetus commented on pull request #5851: HADOOP-18805. S3A prefetch tests to work with small files

2023-07-20 Thread via GitHub
hadoop-yetus commented on PR #5851: URL: https://github.com/apache/hadoop/pull/5851#issuecomment-1644691602 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok: |

[jira] [Commented] (HADOOP-18184) s3a prefetching stream to support unbuffer()

2023-07-20 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18184?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17745282#comment-17745282 ] ASF GitHub Bot commented on HADOOP-18184: - hadoop-yetus commented on PR #5832: URL:

[GitHub] [hadoop] hadoop-yetus commented on pull request #5832: HADOOP-18184. S3A Prefetching unbuffer.

2023-07-20 Thread via GitHub
hadoop-yetus commented on PR #5832: URL: https://github.com/apache/hadoop/pull/5832#issuecomment-1644647399 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok: |

[jira] [Commented] (HADOOP-18805) s3a large file prefetch tests are too slow, don't validate data

2023-07-20 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18805?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17745275#comment-17745275 ] ASF GitHub Bot commented on HADOOP-18805: - hadoop-yetus commented on PR #5851: URL:

[GitHub] [hadoop] hadoop-yetus commented on pull request #5851: HADOOP-18805. S3A prefetch tests to work with small files

2023-07-20 Thread via GitHub
hadoop-yetus commented on PR #5851: URL: https://github.com/apache/hadoop/pull/5851#issuecomment-1644618133 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok: |

[GitHub] [hadoop] simbadzina commented on a diff in pull request #5860: HDFS-17111. RBF: Optimize msync to only call nameservices that have observer reads enabled.

2023-07-20 Thread via GitHub
simbadzina commented on code in PR #5860: URL: https://github.com/apache/hadoop/pull/5860#discussion_r1269958645 ## hadoop-hdfs-project/hadoop-hdfs-rbf/src/main/java/org/apache/hadoop/hdfs/server/federation/router/RouterClientProtocol.java: ## @@ -1928,9 +1929,17 @@ public

[GitHub] [hadoop] simbadzina commented on a diff in pull request #5860: HDFS-17111. RBF: Optimize msync to only call nameservices that have observer reads enabled.

2023-07-20 Thread via GitHub
simbadzina commented on code in PR #5860: URL: https://github.com/apache/hadoop/pull/5860#discussion_r1269958411 ## hadoop-hdfs-project/hadoop-hdfs-rbf/src/main/java/org/apache/hadoop/hdfs/server/federation/router/RouterClientProtocol.java: ## @@ -1928,9 +1928,13 @@ public

[GitHub] [hadoop] hadoop-yetus commented on pull request #5832: HADOOP-18184. S3A Prefetching unbuffer.

2023-07-20 Thread via GitHub
hadoop-yetus commented on PR #5832: URL: https://github.com/apache/hadoop/pull/5832#issuecomment-1644594922 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok: |

[jira] [Commented] (HADOOP-18184) s3a prefetching stream to support unbuffer()

2023-07-20 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18184?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17745271#comment-17745271 ] ASF GitHub Bot commented on HADOOP-18184: - hadoop-yetus commented on PR #5832: URL:

[GitHub] [hadoop] simbadzina commented on a diff in pull request #5860: HDFS-17111. RBF: Optimize msync to only call nameservices that have observer reads enabled.

2023-07-20 Thread via GitHub
simbadzina commented on code in PR #5860: URL: https://github.com/apache/hadoop/pull/5860#discussion_r1269919579 ## hadoop-hdfs-project/hadoop-hdfs-rbf/src/main/java/org/apache/hadoop/hdfs/server/federation/router/RouterClientProtocol.java: ## @@ -1928,9 +1929,17 @@ public

[jira] [Commented] (HADOOP-18807) Close child file systems in ViewFileSystem when cache is disabled.

2023-07-20 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18807?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17745251#comment-17745251 ] ASF GitHub Bot commented on HADOOP-18807: - hadoop-yetus commented on PR #5865: URL:

[GitHub] [hadoop] xinglin commented on a diff in pull request #5860: HDFS-17111. RBF: Optimize msync to only call nameservices that have observer reads enabled.

2023-07-20 Thread via GitHub
xinglin commented on code in PR #5860: URL: https://github.com/apache/hadoop/pull/5860#discussion_r1269897343 ## hadoop-hdfs-project/hadoop-hdfs-rbf/src/main/java/org/apache/hadoop/hdfs/server/federation/router/RouterClientProtocol.java: ## @@ -1928,9 +1928,13 @@ public

[GitHub] [hadoop] hadoop-yetus commented on pull request #5865: HADOOP-18807. Close child file systems in ViewFileSystem when cache is disabled.

2023-07-20 Thread via GitHub
hadoop-yetus commented on PR #5865: URL: https://github.com/apache/hadoop/pull/5865#issuecomment-1644506749 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok: |

[GitHub] [hadoop] hadoop-yetus commented on pull request #5591: HDFS-16991. fix testMkdirsRaceWithObserverRead

2023-07-20 Thread via GitHub
hadoop-yetus commented on PR #5591: URL: https://github.com/apache/hadoop/pull/5591#issuecomment-1644503356 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok: |

[GitHub] [hadoop] hadoop-yetus commented on pull request #5863: HADOOP-18816. Rebuild exceptions on Client side

2023-07-20 Thread via GitHub
hadoop-yetus commented on PR #5863: URL: https://github.com/apache/hadoop/pull/5863#issuecomment-1644500065 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok: |

[jira] [Commented] (HADOOP-18816) Rebuild Exceptions on Client side to get genuine exceptions

2023-07-20 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18816?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17745248#comment-17745248 ] ASF GitHub Bot commented on HADOOP-18816: - hadoop-yetus commented on PR #5863: URL:

[GitHub] [hadoop] simbadzina commented on a diff in pull request #5860: HDFS-17111. RBF: Optimize msync to only call nameservices that have observer reads enabled.

2023-07-20 Thread via GitHub
simbadzina commented on code in PR #5860: URL: https://github.com/apache/hadoop/pull/5860#discussion_r1269883206 ## hadoop-hdfs-project/hadoop-hdfs-rbf/src/main/java/org/apache/hadoop/hdfs/server/federation/router/RouterRpcClient.java: ## @@ -1783,16 +1783,26 @@ &&

[GitHub] [hadoop] hadoop-yetus commented on pull request #5591: HDFS-16991. fix testMkdirsRaceWithObserverRead

2023-07-20 Thread via GitHub
hadoop-yetus commented on PR #5591: URL: https://github.com/apache/hadoop/pull/5591#issuecomment-1644478437 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok: |

[jira] [Commented] (HADOOP-18805) s3a large file prefetch tests are too slow, don't validate data

2023-07-20 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18805?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17745243#comment-17745243 ] ASF GitHub Bot commented on HADOOP-18805: - hadoop-yetus commented on PR #5851: URL:

[GitHub] [hadoop] hadoop-yetus commented on pull request #5851: HADOOP-18805. S3A prefetch tests to work with small files

2023-07-20 Thread via GitHub
hadoop-yetus commented on PR #5851: URL: https://github.com/apache/hadoop/pull/5851#issuecomment-1644475917 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok: |

[GitHub] [hadoop] ayushtkn merged pull request #5822: HDFS-17074. Remove incorrect comment in TestRedudantBlocks#setup.

2023-07-20 Thread via GitHub
ayushtkn merged PR #5822: URL: https://github.com/apache/hadoop/pull/5822 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

[GitHub] [hadoop] ayushtkn merged pull request #5861: YARN-11540. Fix typo: form -> from

2023-07-20 Thread via GitHub
ayushtkn merged PR #5861: URL: https://github.com/apache/hadoop/pull/5861 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

[jira] [Updated] (HADOOP-18820) AWS SDK v2: cut the v1 bridging support

2023-07-20 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18820?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran updated HADOOP-18820: Summary: AWS SDK v2: cut the v1 bridging support (was: AWS SDK v2: cut the v1

[jira] [Commented] (HADOOP-18073) Upgrade AWS SDK to v2

2023-07-20 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18073?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17745226#comment-17745226 ] ASF GitHub Bot commented on HADOOP-18073: - steveloughran commented on PR #5858: URL:

[GitHub] [hadoop] steveloughran commented on pull request #5858: HADOOP-18073. Upgrade S3A in 3.3 branch to AWS SDK V2.(#5707)

2023-07-20 Thread via GitHub
steveloughran commented on PR #5858: URL: https://github.com/apache/hadoop/pull/5858#issuecomment-162385 will manually merge if yetus is happy -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to

[jira] [Commented] (HADOOP-18073) Upgrade AWS SDK to v2

2023-07-20 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18073?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17745224#comment-17745224 ] ASF GitHub Bot commented on HADOOP-18073: - steveloughran closed pull request #5857:

[jira] [Commented] (HADOOP-18073) Upgrade AWS SDK to v2

2023-07-20 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18073?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17745223#comment-17745223 ] ASF GitHub Bot commented on HADOOP-18073: - steveloughran commented on PR #5857: URL:

[GitHub] [hadoop] steveloughran closed pull request #5857: HADOOP-18073. Upgrade AWS SDK to v2 in S3A [work in progress]

2023-07-20 Thread via GitHub
steveloughran closed pull request #5857: HADOOP-18073. Upgrade AWS SDK to v2 in S3A [work in progress] URL: https://github.com/apache/hadoop/pull/5857 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go

[GitHub] [hadoop] steveloughran commented on pull request #5857: HADOOP-18073. Upgrade AWS SDK to v2 in S3A [work in progress]

2023-07-20 Thread via GitHub
steveloughran commented on PR #5857: URL: https://github.com/apache/hadoop/pull/5857#issuecomment-160801 updated the feature branch; closing this -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go

[jira] [Commented] (HADOOP-18073) Upgrade AWS SDK to v2

2023-07-20 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18073?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17745218#comment-17745218 ] ASF GitHub Bot commented on HADOOP-18073: - steveloughran commented on PR #5857: URL:

[GitHub] [hadoop] steveloughran commented on pull request #5857: HADOOP-18073. Upgrade AWS SDK to v2 in S3A [work in progress]

2023-07-20 Thread via GitHub
steveloughran commented on PR #5857: URL: https://github.com/apache/hadoop/pull/5857#issuecomment-1644427636 synced feature branch up with this rebase -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to

[GitHub] [hadoop] xinglin commented on pull request #5803: HDFS-17067 Use BlockingThreadPoolExecutorService for nnProbingThreadPool in ObserverReadProxy

2023-07-20 Thread via GitHub
xinglin commented on PR #5803: URL: https://github.com/apache/hadoop/pull/5803#issuecomment-1644420686 thanks @goiri for committing this PR to trunk. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go

[jira] [Commented] (HADOOP-18073) Upgrade AWS SDK to v2

2023-07-20 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18073?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17745208#comment-17745208 ] ASF GitHub Bot commented on HADOOP-18073: - hadoop-yetus commented on PR #5857: URL:

[GitHub] [hadoop] hadoop-yetus commented on pull request #5857: HADOOP-18073. Upgrade AWS SDK to v2 in S3A [work in progress]

2023-07-20 Thread via GitHub
hadoop-yetus commented on PR #5857: URL: https://github.com/apache/hadoop/pull/5857#issuecomment-1644416613 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok: |

[jira] [Commented] (HADOOP-18805) s3a large file prefetch tests are too slow, don't validate data

2023-07-20 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18805?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17745203#comment-17745203 ] ASF GitHub Bot commented on HADOOP-18805: - hadoop-yetus commented on PR #5851: URL:

[GitHub] [hadoop] hadoop-yetus commented on pull request #5851: HADOOP-18805. S3A prefetch tests to work with small files

2023-07-20 Thread via GitHub
hadoop-yetus commented on PR #5851: URL: https://github.com/apache/hadoop/pull/5851#issuecomment-1644388460 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok: |

[GitHub] [hadoop] hadoop-yetus commented on pull request #5597: HDFS-16993. Datanode supports configure TopN DatanodeNetworkCounts

2023-07-20 Thread via GitHub
hadoop-yetus commented on PR #5597: URL: https://github.com/apache/hadoop/pull/5597#issuecomment-1644388189 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok: |

[GitHub] [hadoop] hadoop-yetus commented on pull request #5866: HDFS-17112. Show decommission duration in JMX and HTML.

2023-07-20 Thread via GitHub
hadoop-yetus commented on PR #5866: URL: https://github.com/apache/hadoop/pull/5866#issuecomment-1644370875 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok: |

[GitHub] [hadoop] hadoop-yetus commented on pull request #5868: YARN-11539. Fix leaf-templates in Flexible AQC.

2023-07-20 Thread via GitHub
hadoop-yetus commented on PR #5868: URL: https://github.com/apache/hadoop/pull/5868#issuecomment-1644368238 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok: |

[GitHub] [hadoop] hadoop-yetus commented on pull request #5864: YARN-11534. Fixed exception handling when container signalling is interrupted

2023-07-20 Thread via GitHub
hadoop-yetus commented on PR #5864: URL: https://github.com/apache/hadoop/pull/5864#issuecomment-1644367335 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok: |

[GitHub] [hadoop] mkuchenbecker commented on a diff in pull request #5860: HDFS-17111. RBF: Optimize msync to only call nameservices that have observer reads enabled.

2023-07-20 Thread via GitHub
mkuchenbecker commented on code in PR #5860: URL: https://github.com/apache/hadoop/pull/5860#discussion_r1269785090 ## hadoop-hdfs-project/hadoop-hdfs-rbf/src/main/java/org/apache/hadoop/hdfs/server/federation/router/RouterClientProtocol.java: ## @@ -1928,9 +1929,17 @@ public

[GitHub] [hadoop] mkuchenbecker commented on a diff in pull request #5860: HDFS-17111. RBF: Optimize msync to only call nameservices that have observer reads enabled.

2023-07-20 Thread via GitHub
mkuchenbecker commented on code in PR #5860: URL: https://github.com/apache/hadoop/pull/5860#discussion_r1269784340 ## hadoop-hdfs-project/hadoop-hdfs-rbf/src/main/java/org/apache/hadoop/hdfs/server/federation/router/RouterRpcClient.java: ## @@ -1783,16 +1783,26 @@ &&

[jira] [Commented] (HADOOP-18752) Change fs.s3a.directory.marker.retention to "keep"

2023-07-20 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18752?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17745188#comment-17745188 ] Steve Loughran commented on HADOOP-18752: - I have pulled in everything but the default change

[jira] [Commented] (HADOOP-18752) Change fs.s3a.directory.marker.retention to "keep"

2023-07-20 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18752?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17745187#comment-17745187 ] ASF GitHub Bot commented on HADOOP-18752: - steveloughran merged PR #5859: URL:

[jira] [Commented] (HADOOP-18752) Change fs.s3a.directory.marker.retention to "keep"

2023-07-20 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18752?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17745186#comment-17745186 ] ASF GitHub Bot commented on HADOOP-18752: - steveloughran commented on PR #5859: URL:

[GitHub] [hadoop] goiri merged pull request #5803: HDFS-17067 Use BlockingThreadPoolExecutorService for nnProbingThreadPool in ObserverReadProxy

2023-07-20 Thread via GitHub
goiri merged PR #5803: URL: https://github.com/apache/hadoop/pull/5803 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

[GitHub] [hadoop] steveloughran merged pull request #5859: HADOOP-18752. Change fs.s3a.directory.marker.retention to "keep" -everything but the switch

2023-07-20 Thread via GitHub
steveloughran merged PR #5859: URL: https://github.com/apache/hadoop/pull/5859 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

[GitHub] [hadoop] steveloughran commented on pull request #5859: HADOOP-18752. Change fs.s3a.directory.marker.retention to "keep" -everything but the switch

2023-07-20 Thread via GitHub
steveloughran commented on PR #5859: URL: https://github.com/apache/hadoop/pull/5859#issuecomment-1644342424 OK, backport has gone in cleanly. This does not change the default value, as discussed, even though I'd like to. -- This is an automated message from the Apache Git Service. To

[GitHub] [hadoop] hadoop-yetus commented on pull request #5864: YARN-11534. Fixed exception handling when container signalling is interrupted

2023-07-20 Thread via GitHub
hadoop-yetus commented on PR #5864: URL: https://github.com/apache/hadoop/pull/5864#issuecomment-1644335119 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok: |

[GitHub] [hadoop] hadoop-yetus commented on pull request #5597: HDFS-16993. Datanode supports configure TopN DatanodeNetworkCounts

2023-07-20 Thread via GitHub
hadoop-yetus commented on PR #5597: URL: https://github.com/apache/hadoop/pull/5597#issuecomment-1644290834 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok: |

[jira] [Commented] (HADOOP-18752) Change fs.s3a.directory.marker.retention to "keep"

2023-07-20 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18752?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17745175#comment-17745175 ] ASF GitHub Bot commented on HADOOP-18752: - hadoop-yetus commented on PR #5859: URL:

[GitHub] [hadoop] hadoop-yetus commented on pull request #5859: HADOOP-18752. Change fs.s3a.directory.marker.retention to "keep" -everything but the switch

2023-07-20 Thread via GitHub
hadoop-yetus commented on PR #5859: URL: https://github.com/apache/hadoop/pull/5859#issuecomment-1644290192 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok: |

[jira] [Created] (HADOOP-18822) Out of Memory when mistakenly set decay-scheduler.metrics.top.user.count to a large number

2023-07-20 Thread ConfX (Jira)
ConfX created HADOOP-18822: -- Summary: Out of Memory when mistakenly set decay-scheduler.metrics.top.user.count to a large number Key: HADOOP-18822 URL: https://issues.apache.org/jira/browse/HADOOP-18822

[GitHub] [hadoop] steveloughran merged pull request #5839: HDFS-13916. Distcp SnapshotDiff to support WebHDFS.

2023-07-20 Thread via GitHub
steveloughran merged PR #5839: URL: https://github.com/apache/hadoop/pull/5839 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

[jira] [Created] (HADOOP-18821) Walking up the ldap hierarchy may trigger null pointer exception

2023-07-20 Thread ConfX (Jira)
ConfX created HADOOP-18821: -- Summary: Walking up the ldap hierarchy may trigger null pointer exception Key: HADOOP-18821 URL: https://issues.apache.org/jira/browse/HADOOP-18821 Project: Hadoop Common

[jira] [Created] (HADOOP-18820) AWS SDK v2: cut the v1 assistance

2023-07-20 Thread Steve Loughran (Jira)
Steve Loughran created HADOOP-18820: --- Summary: AWS SDK v2: cut the v1 assistance Key: HADOOP-18820 URL: https://issues.apache.org/jira/browse/HADOOP-18820 Project: Hadoop Common Issue

[jira] [Commented] (HADOOP-18747) AWS SDK V2 - sigv2 support

2023-07-20 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18747?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17745162#comment-17745162 ] Steve Loughran commented on HADOOP-18747: - # should this be a blocker for the merge to trunk?

[jira] [Created] (HADOOP-18819) AWS SDK v2 build complaints

2023-07-20 Thread Steve Loughran (Jira)
Steve Loughran created HADOOP-18819: --- Summary: AWS SDK v2 build complaints Key: HADOOP-18819 URL: https://issues.apache.org/jira/browse/HADOOP-18819 Project: Hadoop Common Issue Type:

[jira] [Comment Edited] (HADOOP-18073) Upgrade AWS SDK to v2

2023-07-20 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18073?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17745161#comment-17745161 ] Steve Loughran edited comment on HADOOP-18073 at 7/20/23 4:21 PM: --

[jira] [Created] (HADOOP-18818) Merge aws v2 upgrade feature branch into trunk

2023-07-20 Thread Steve Loughran (Jira)
Steve Loughran created HADOOP-18818: --- Summary: Merge aws v2 upgrade feature branch into trunk Key: HADOOP-18818 URL: https://issues.apache.org/jira/browse/HADOOP-18818 Project: Hadoop Common

[jira] [Commented] (HADOOP-18073) Upgrade AWS SDK to v2

2023-07-20 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18073?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17745161#comment-17745161 ] Steve Loughran commented on HADOOP-18073: - rebase branches hightlight spotbugs and javadoc

[GitHub] [hadoop] steveloughran commented on pull request #5867: HADOOP-18817.Exclude aws-java-sdk-swf-libraries to avoid compilation …

2023-07-20 Thread via GitHub
steveloughran commented on PR #5867: URL: https://github.com/apache/hadoop/pull/5867#issuecomment-1644215527 see discussion on the jira; really love to know how you see this even though I don't. also:

[jira] [Commented] (HADOOP-18817) Exclude aws-java-sdk-swf-libraries packages to avoid compilation failures

2023-07-20 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18817?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17745159#comment-17745159 ] ASF GitHub Bot commented on HADOOP-18817: - steveloughran commented on PR #5867: URL:

[GitHub] [hadoop] brumi1024 opened a new pull request, #5868: YARN-11539. Fix leaf-templates in Flexible AQC.

2023-07-20 Thread via GitHub
brumi1024 opened a new pull request, #5868: URL: https://github.com/apache/hadoop/pull/5868 ### For code changes: - [ ] Does the title or this PR starts with the corresponding JIRA issue id (e.g. 'HADOOP-17799. Your PR title ...')? - [ ] Object storage: have the

[jira] [Commented] (HADOOP-18812) list aws SDK v2 libraries in LICENSE.binary

2023-07-20 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18812?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17745136#comment-17745136 ] Steve Loughran commented on HADOOP-18812: - {code} [INFO] Scanning for projects... [INFO]

[jira] [Assigned] (HADOOP-18812) list aws SDK v2 libraries in LICENSE.binary

2023-07-20 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18812?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran reassigned HADOOP-18812: --- Assignee: Steve Loughran > list aws SDK v2 libraries in LICENSE.binary >

[jira] [Commented] (HADOOP-18817) Exclude aws-java-sdk-swf-libraries packages to avoid compilation failures

2023-07-20 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18817?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17745133#comment-17745133 ] ASF GitHub Bot commented on HADOOP-18817: - hadoop-yetus commented on PR #5867: URL:

[GitHub] [hadoop] hadoop-yetus commented on pull request #5867: HADOOP-18817.Exclude aws-java-sdk-swf-libraries to avoid compilation …

2023-07-20 Thread via GitHub
hadoop-yetus commented on PR #5867: URL: https://github.com/apache/hadoop/pull/5867#issuecomment-1644139568 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok: |

[jira] [Commented] (HADOOP-18817) Exclude aws-java-sdk-swf-libraries packages to avoid compilation failures

2023-07-20 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18817?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17745132#comment-17745132 ] Steve Loughran commented on HADOOP-18817: - And mvnrepo says no too.

[GitHub] [hadoop] simbadzina commented on pull request #5860: HDFS-17111. RBF: Optimize msync to only call nameservices that have observer reads enabled.

2023-07-20 Thread via GitHub
simbadzina commented on PR #5860: URL: https://github.com/apache/hadoop/pull/5860#issuecomment-1644104524 @virajjasani @goiri @hchaverri could you please help review this when you have bandwidth. -- This is an automated message from the Apache Git Service. To respond to the message,

[GitHub] [hadoop] xinglin commented on pull request #5803: HDFS-17067 Use BlockingThreadPoolExecutorService for nnProbingThreadPool in ObserverReadProxy

2023-07-20 Thread via GitHub
xinglin commented on PR #5803: URL: https://github.com/apache/hadoop/pull/5803#issuecomment-1644060205 Thanks @mccormickt12 for reviewing and approving the PR! @goiri, could you take a look? thanks, -- This is an automated message from the Apache Git Service. To respond to the

  1   2   3   >