[jira] [Commented] (HADOOP-18207) Introduce hadoop-logging module

2023-07-18 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18207?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=1771#comment-1771 ] ASF GitHub Bot commented on HADOOP-18207: - hadoop-yetus commented on PR #5717:

[GitHub] [hadoop] hadoop-yetus commented on pull request #5717: HADOOP-18207. Introduce hadoop-logging module

2023-07-18 Thread via GitHub
hadoop-yetus commented on PR #5717: URL: https://github.com/apache/hadoop/pull/5717#issuecomment-1641452395 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok: | reexe

[GitHub] [hadoop] YuanbenWang commented on pull request #5849: HDFS-17091. Blocks on DECOMMISSIONING DNs should be sorted properly in LocatedBlocks.

2023-07-18 Thread via GitHub
YuanbenWang commented on PR #5849: URL: https://github.com/apache/hadoop/pull/5849#issuecomment-1641306654 @jojochuang @slfan1989 @ayushtkn Thank you for your review~! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use th

[GitHub] [hadoop] slfan1989 commented on pull request #5846: HDFS-17088. Improve the debug verifyEC and dfsrouteradmin commands in HDFSCommands.md

2023-07-18 Thread via GitHub
slfan1989 commented on PR #5846: URL: https://github.com/apache/hadoop/pull/5846#issuecomment-1641268449 @haiyang1987 Thanks for your contribution! @ayushtkn @goiri @tomscut Thanks for the review! -- This is an automated message from the Apache Git Service. To respond to the message, plea

[GitHub] [hadoop] slfan1989 merged pull request #5846: HDFS-17088. Improve the debug verifyEC and dfsrouteradmin commands in HDFSCommands.md

2023-07-18 Thread via GitHub
slfan1989 merged PR #5846: URL: https://github.com/apache/hadoop/pull/5846 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: common-issues-unsubscr...@had

[GitHub] [hadoop] slfan1989 commented on pull request #5849: HDFS-17091. Blocks on DECOMMISSIONING DNs should be sorted properly in LocatedBlocks.

2023-07-18 Thread via GitHub
slfan1989 commented on PR #5849: URL: https://github.com/apache/hadoop/pull/5849#issuecomment-1641267247 @YuanbenWang Thanks for your contribution! @ayushtkn @jojochuang Thanks for helping review! -- This is an automated message from the Apache Git Service. To respond to the message, ple

[GitHub] [hadoop] slfan1989 merged pull request #5849: HDFS-17091. Blocks on DECOMMISSIONING DNs should be sorted properly in LocatedBlocks.

2023-07-18 Thread via GitHub
slfan1989 merged PR #5849: URL: https://github.com/apache/hadoop/pull/5849 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: common-issues-unsubscr...@had

[GitHub] [hadoop] hadoop-yetus commented on pull request #2821: HDFS-15925. The lack of packet-level mirrorError state synchronization in BlockReceiver can cause the HDFS client to hang

2023-07-18 Thread via GitHub
hadoop-yetus commented on PR #2821: URL: https://github.com/apache/hadoop/pull/2821#issuecomment-1640951200 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok: | reexe

[GitHub] [hadoop] hadoop-yetus commented on pull request #5810: YARN-7707. BackPort [GPG] Policy generator framework.

2023-07-18 Thread via GitHub
hadoop-yetus commented on PR #5810: URL: https://github.com/apache/hadoop/pull/5810#issuecomment-1640797603 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok: | reex

[GitHub] [hadoop] hadoop-yetus commented on pull request #5841: YARN-11531. [Federation] Code cleanup for NodeManager#amrmproxy.

2023-07-18 Thread via GitHub
hadoop-yetus commented on PR #5841: URL: https://github.com/apache/hadoop/pull/5841#issuecomment-1640752706 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok: | reex

[GitHub] [hadoop] hadoop-yetus commented on pull request #5637: YARN-11491. [Federation] Use ZookeeperFederationStateStore as the DefaultStateStore.

2023-07-18 Thread via GitHub
hadoop-yetus commented on PR #5637: URL: https://github.com/apache/hadoop/pull/5637#issuecomment-1640746032 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok: | reexe

[jira] [Commented] (HADOOP-18706) Improve S3ABlockOutputStream recovery

2023-07-18 Thread Chris Bevard (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18706?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17744294#comment-17744294 ] Chris Bevard commented on HADOOP-18706: --- [~ste...@apache.org]  > Improve S3ABloc

[jira] [Commented] (HADOOP-18807) Close child file systems in ViewFileSystem when cache is disabled.

2023-07-18 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18807?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17744293#comment-17744293 ] ASF GitHub Bot commented on HADOOP-18807: - steveloughran commented on code in P

[GitHub] [hadoop] steveloughran commented on a diff in pull request #5847: HADOOP-18807. Close child file systems in ViewFileSystem when cache is disabled.

2023-07-18 Thread via GitHub
steveloughran commented on code in PR #5847: URL: https://github.com/apache/hadoop/pull/5847#discussion_r1267028844 ## hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/fs/viewfs/TestViewFileSystemClose.java: ## @@ -17,26 +17,26 @@ */ package org.apache.hadoop.f

[GitHub] [hadoop] hadoop-yetus commented on pull request #5854: HDFS-17094. EC: Fix bug in block recovery when there are stale datanodes.

2023-07-18 Thread via GitHub
hadoop-yetus commented on PR #5854: URL: https://github.com/apache/hadoop/pull/5854#issuecomment-1640555209 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok: | reex

[jira] [Commented] (HADOOP-18807) Close child file systems in ViewFileSystem when cache is disabled.

2023-07-18 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18807?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17744291#comment-17744291 ] ASF GitHub Bot commented on HADOOP-18807: - steveloughran commented on code in P

[GitHub] [hadoop] steveloughran commented on a diff in pull request #5847: HADOOP-18807. Close child file systems in ViewFileSystem when cache is disabled.

2023-07-18 Thread via GitHub
steveloughran commented on code in PR #5847: URL: https://github.com/apache/hadoop/pull/5847#discussion_r1267026982 ## hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/fs/viewfs/TestViewFileSystemClose.java: ## @@ -0,0 +1,64 @@ +/** + * Licensed to the Apache Soft

[jira] [Commented] (HADOOP-18807) Close child file systems in ViewFileSystem when cache is disabled.

2023-07-18 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18807?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17744290#comment-17744290 ] ASF GitHub Bot commented on HADOOP-18807: - steveloughran commented on code in P

[GitHub] [hadoop] steveloughran commented on a diff in pull request #5847: HADOOP-18807. Close child file systems in ViewFileSystem when cache is disabled.

2023-07-18 Thread via GitHub
steveloughran commented on code in PR #5847: URL: https://github.com/apache/hadoop/pull/5847#discussion_r1267026055 ## hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/viewfs/ViewFileSystem.java: ## @@ -1926,12 +1926,41 @@ enum RenameStrategy { SAME_FI

[jira] [Commented] (HADOOP-18805) s3a large file prefetch tests are too slow, don't validate data

2023-07-18 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18805?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17744283#comment-17744283 ] ASF GitHub Bot commented on HADOOP-18805: - virajjasani commented on PR #5851: U

[GitHub] [hadoop] virajjasani commented on pull request #5851: HADOOP-18805. S3A prefetch LRU test to work with small files

2023-07-18 Thread via GitHub
virajjasani commented on PR #5851: URL: https://github.com/apache/hadoop/pull/5851#issuecomment-1640511615 > this jira would cover the original large file test, as it has similar issues you mean `ITestS3APrefetchingInputStream` correct? let me try including that as well. -- This i

[jira] [Updated] (HADOOP-18804) Out of Memory when mistakenly set io.file.buffer.size to a large number

2023-07-18 Thread ConfX (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18804?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ConfX updated HADOOP-18804: --- Attachment: reproduce.sh > Out of Memory when mistakenly set io.file.buffer.size to a large number > --

[GitHub] [hadoop] slfan1989 commented on a diff in pull request #5841: YARN-11531. [Federation] Code cleanup for NodeManager#amrmproxy.

2023-07-18 Thread via GitHub
slfan1989 commented on code in PR #5841: URL: https://github.com/apache/hadoop/pull/5841#discussion_r1266945158 ## hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/java/org/apache/hadoop/yarn/server/nodemanager/amrmproxy/TestFederationIntercepto

[GitHub] [hadoop] slfan1989 commented on a diff in pull request #5841: YARN-11531. [Federation] Code cleanup for NodeManager#amrmproxy.

2023-07-18 Thread via GitHub
slfan1989 commented on code in PR #5841: URL: https://github.com/apache/hadoop/pull/5841#discussion_r1266944879 ## hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/amrmproxy/AMRMProxyApplicationCont

[jira] [Commented] (HADOOP-12677) DecompressorStream throws IndexOutOfBoundsException when calling skip(long)

2023-07-18 Thread Xiang Li (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-12677?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17744264#comment-17744264 ] Xiang Li commented on HADOOP-12677: --- We exposed this problem when trying to read Spar

[GitHub] [hadoop] brumi1024 commented on a diff in pull request #5784: YARN-9877 - Intermittent TIME_OUT of LogAggregationReport

2023-07-18 Thread via GitHub
brumi1024 commented on code in PR #5784: URL: https://github.com/apache/hadoop/pull/5784#discussion_r1266613396 ## hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/rmapp/RMAppImpl.java:

[GitHub] [hadoop] brumi1024 commented on a diff in pull request #5784: YARN-9877 - Intermittent TIME_OUT of LogAggregationReport

2023-07-18 Thread via GitHub
brumi1024 commented on code in PR #5784: URL: https://github.com/apache/hadoop/pull/5784#discussion_r1266829431 ## hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/rmapp/RMAppImpl.java:

[GitHub] [hadoop] brumi1024 commented on a diff in pull request #5784: YARN-9877 - Intermittent TIME_OUT of LogAggregationReport

2023-07-18 Thread via GitHub
brumi1024 commented on code in PR #5784: URL: https://github.com/apache/hadoop/pull/5784#discussion_r1266889475 ## hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/rmapp/RMAppRunningOnNodeEv

[GitHub] [hadoop] ferdelyi commented on a diff in pull request #5784: YARN-9877 - Intermittent TIME_OUT of LogAggregationReport

2023-07-18 Thread via GitHub
ferdelyi commented on code in PR #5784: URL: https://github.com/apache/hadoop/pull/5784#discussion_r1266883249 ## hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/rmapp/RMAppRunningOnNodeEve

[GitHub] [hadoop] hadoop-yetus commented on pull request #5841: YARN-11531. [Federation] Code cleanup for NodeManager#amrmproxy.

2023-07-18 Thread via GitHub
hadoop-yetus commented on PR #5841: URL: https://github.com/apache/hadoop/pull/5841#issuecomment-1640362747 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok: | reex

[GitHub] [hadoop] hadoop-yetus commented on pull request #5784: YARN-9877 - Intermittent TIME_OUT of LogAggregationReport

2023-07-18 Thread via GitHub
hadoop-yetus commented on PR #5784: URL: https://github.com/apache/hadoop/pull/5784#issuecomment-1640352968 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok: | reex

[GitHub] [hadoop] brumi1024 commented on a diff in pull request #5784: YARN-9877 - Intermittent TIME_OUT of LogAggregationReport

2023-07-18 Thread via GitHub
brumi1024 commented on code in PR #5784: URL: https://github.com/apache/hadoop/pull/5784#discussion_r1266850170 ## hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/rmapp/RMAppRunningOnNodeEv

[GitHub] [hadoop] brumi1024 commented on a diff in pull request #5784: YARN-9877 - Intermittent TIME_OUT of LogAggregationReport

2023-07-18 Thread via GitHub
brumi1024 commented on code in PR #5784: URL: https://github.com/apache/hadoop/pull/5784#discussion_r1266829431 ## hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/rmapp/RMAppImpl.java:

[jira] [Resolved] (HADOOP-18808) LogExactlyOnce to add a debug() method

2023-07-18 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18808?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran resolved HADOOP-18808. - Fix Version/s: 3.3.9 Resolution: Fixed > LogExactlyOnce to add a debug() method

[jira] [Commented] (HADOOP-18808) LogExactlyOnce to add a debug() method

2023-07-18 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18808?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17744222#comment-17744222 ] ASF GitHub Bot commented on HADOOP-18808: - steveloughran merged PR #5850: URL:

[GitHub] [hadoop] steveloughran merged pull request #5850: HADOOP-18808. LogExactlyOnce to add a debug() method

2023-07-18 Thread via GitHub
steveloughran merged PR #5850: URL: https://github.com/apache/hadoop/pull/5850 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: common-issues-unsubscr...

[jira] [Updated] (HADOOP-18809) s3a prefetch read/write file operations should guard channel close

2023-07-18 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18809?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran updated HADOOP-18809: Component/s: fs/s3 > s3a prefetch read/write file operations should guard channel close >

[jira] [Updated] (HADOOP-18809) s3a prefetch read/write file operations should guard channel close

2023-07-18 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18809?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran updated HADOOP-18809: Affects Version/s: 3.3.9 > s3a prefetch read/write file operations should guard channel c

[jira] [Resolved] (HADOOP-18809) s3a prefetch read/write file operations should guard channel close

2023-07-18 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18809?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran resolved HADOOP-18809. - Fix Version/s: 3.3.9 Resolution: Fixed > s3a prefetch read/write file operations

[jira] [Commented] (HADOOP-18809) s3a prefetch read/write file operations should guard channel close

2023-07-18 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18809?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17744220#comment-17744220 ] ASF GitHub Bot commented on HADOOP-18809: - steveloughran merged PR #5853: URL:

[GitHub] [hadoop] steveloughran merged pull request #5853: HADOOP-18809. s3a prefetch read/write file operations should guard channel close

2023-07-18 Thread via GitHub
steveloughran merged PR #5853: URL: https://github.com/apache/hadoop/pull/5853 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: common-issues-unsubscr...

[jira] [Commented] (HADOOP-18487) protobuf-2.5.0 dependencies => provided

2023-07-18 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18487?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17744219#comment-17744219 ] ASF GitHub Bot commented on HADOOP-18487: - steveloughran commented on PR #4996:

[GitHub] [hadoop] steveloughran commented on pull request #4996: HADOOP-18487. protobuf 2.5.0 marked as provided.

2023-07-18 Thread via GitHub
steveloughran commented on PR #4996: URL: https://github.com/apache/hadoop/pull/4996#issuecomment-1640209168 rbf compile failure is very much related ``` rbf: [ERROR] COMPILATION ERROR : [INFO] - [ERROR] /home/jenkins/j

[jira] [Commented] (HADOOP-18487) protobuf-2.5.0 dependencies => provided

2023-07-18 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18487?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17744217#comment-17744217 ] ASF GitHub Bot commented on HADOOP-18487: - steveloughran commented on PR #4996:

[GitHub] [hadoop] steveloughran commented on pull request #4996: HADOOP-18487. protobuf 2.5.0 marked as provided.

2023-07-18 Thread via GitHub
steveloughran commented on PR #4996: URL: https://github.com/apache/hadoop/pull/4996#issuecomment-1640205050 Failures are all in TestMRJobsWithProfiler, covered exactly in MAPREDUCE-7436 and therefore unrelated. This PR is ready to merge -- This is an automated message from the Apa

[jira] [Commented] (HADOOP-18807) Close child file systems in ViewFileSystem when cache is disabled.

2023-07-18 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18807?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17744212#comment-17744212 ] ASF GitHub Bot commented on HADOOP-18807: - Hexiaoqiao commented on code in PR #

[GitHub] [hadoop] Hexiaoqiao commented on a diff in pull request #5847: HADOOP-18807. Close child file systems in ViewFileSystem when cache is disabled.

2023-07-18 Thread via GitHub
Hexiaoqiao commented on code in PR #5847: URL: https://github.com/apache/hadoop/pull/5847#discussion_r1266746765 ## hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/viewfs/ViewFileSystem.java: ## @@ -1926,12 +1926,41 @@ enum RenameStrategy { SAME_FILES

[GitHub] [hadoop] Hexiaoqiao commented on a diff in pull request #5854: HDFS-17094. EC: Fix bug in block recovery when there are stale datanodes.

2023-07-18 Thread via GitHub
Hexiaoqiao commented on code in PR #5854: URL: https://github.com/apache/hadoop/pull/5854#discussion_r1266726577 ## hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestLeaseRecoveryStriped.java: ## @@ -17,6 +17,10 @@ */ package org.apache.hadoop.hdfs; +

[GitHub] [hadoop] ferdelyi commented on a diff in pull request #5784: YARN-9877 - Intermittent TIME_OUT of LogAggregationReport

2023-07-18 Thread via GitHub
ferdelyi commented on code in PR #5784: URL: https://github.com/apache/hadoop/pull/5784#discussion_r1266723998 ## hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/rmapp/RMAppRunningOnNodeEve

[jira] [Commented] (HADOOP-18772) remove use of com.google.common.base.Charsets;

2023-07-18 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18772?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17744198#comment-17744198 ] Steve Loughran commented on HADOOP-18772: - go for it' i'd just missed the origi

[jira] [Commented] (HADOOP-18291) S3A prefetch - Implement LRU cache for SingleFilePerBlockCache

2023-07-18 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18291?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17744197#comment-17744197 ] Steve Loughran commented on HADOOP-18291: - bq. multiple addendum-addendum commi

[jira] [Commented] (HADOOP-18805) s3a large file prefetch tests are too slow, don't validate data

2023-07-18 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18805?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17744196#comment-17744196 ] ASF GitHub Bot commented on HADOOP-18805: - steveloughran commented on code in P

[GitHub] [hadoop] steveloughran commented on a diff in pull request #5851: HADOOP-18805. S3A prefetch LRU test to work with small files

2023-07-18 Thread via GitHub
steveloughran commented on code in PR #5851: URL: https://github.com/apache/hadoop/pull/5851#discussion_r1266680532 ## hadoop-tools/hadoop-aws/src/test/java/org/apache/hadoop/fs/s3a/ITestS3APrefetchingLruEviction.java: ## @@ -78,39 +77,44 @@ public ITestS3APrefetchingLruEviction

[GitHub] [hadoop] hadoop-yetus commented on pull request #5846: HDFS-17088. Improve the debug verifyEC and dfsrouteradmin commands in HDFSCommands.md

2023-07-18 Thread via GitHub
hadoop-yetus commented on PR #5846: URL: https://github.com/apache/hadoop/pull/5846#issuecomment-1640053229 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok: | reex

[GitHub] [hadoop] fanluoo commented on pull request #5591: HDFS-16991. fix testMkdirsRaceWithObserverRead

2023-07-18 Thread via GitHub
fanluoo commented on PR #5591: URL: https://github.com/apache/hadoop/pull/5591#issuecomment-1640019438 @ayushtkn could you help to review this pr ? is there anything else that i can improve -- This is an automated message from the Apache Git Service. To respond to the message, please log

[GitHub] [hadoop] brumi1024 commented on a diff in pull request #5784: YARN-9877 - Intermittent TIME_OUT of LogAggregationReport

2023-07-18 Thread via GitHub
brumi1024 commented on code in PR #5784: URL: https://github.com/apache/hadoop/pull/5784#discussion_r1266612668 ## hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/rmapp/RMAppRunningOnNodeEv

[jira] [Commented] (HADOOP-18807) Close child file systems in ViewFileSystem when cache is disabled.

2023-07-18 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18807?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17744168#comment-17744168 ] ASF GitHub Bot commented on HADOOP-18807: - hadoop-yetus commented on PR #5847:

[GitHub] [hadoop] hadoop-yetus commented on pull request #5847: HADOOP-18807. Close child file systems in ViewFileSystem when cache is disabled.

2023-07-18 Thread via GitHub
hadoop-yetus commented on PR #5847: URL: https://github.com/apache/hadoop/pull/5847#issuecomment-1639994871 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok: | reexe

[GitHub] [hadoop] ferdelyi commented on pull request #5784: YARN-9877 - Intermittent TIME_OUT of LogAggregationReport

2023-07-18 Thread via GitHub
ferdelyi commented on PR #5784: URL: https://github.com/apache/hadoop/pull/5784#issuecomment-1639994837 LGTM -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe,

[jira] [Commented] (HADOOP-18757) S3A Committer finalizing the commits in a single thread

2023-07-18 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18757?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17744164#comment-17744164 ] ASF GitHub Bot commented on HADOOP-18757: - modithah commented on code in PR #57

[GitHub] [hadoop] modithah commented on a diff in pull request #5706: HADOOP-18757: Bump corePoolSize of HadoopThreadPoolExecutor in s3a committer

2023-07-18 Thread via GitHub
modithah commented on code in PR #5706: URL: https://github.com/apache/hadoop/pull/5706#discussion_r1266594931 ## hadoop-tools/hadoop-aws/src/test/java/org/apache/hadoop/fs/s3a/commit/staging/integration/ITestStagingCommitProtocolFailure.java: ## @@ -32,8 +32,7 @@ import org.ap

[GitHub] [hadoop] K0K0V0K commented on a diff in pull request #5784: YARN-9877 - Intermittent TIME_OUT of LogAggregationReport

2023-07-18 Thread via GitHub
K0K0V0K commented on code in PR #5784: URL: https://github.com/apache/hadoop/pull/5784#discussion_r1266592765 ## hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/rmapp/RMAppImpl.java: ##

[GitHub] [hadoop] zhangshuyan0 opened a new pull request, #5854: HDFS-17094. EC: Fix bug in block recovery when there are stale datanodes.

2023-07-18 Thread via GitHub
zhangshuyan0 opened a new pull request, #5854: URL: https://github.com/apache/hadoop/pull/5854 ### Description of PR When a block recovery occurs, `RecoveryTaskStriped` in datanode expects `rBlock.getLocations()` and `rBlock. getBlockIndices()` to be in one-to-one correspondence. However

[GitHub] [hadoop] ferdelyi commented on a diff in pull request #5784: YARN-9877 - Intermittent TIME_OUT of LogAggregationReport

2023-07-18 Thread via GitHub
ferdelyi commented on code in PR #5784: URL: https://github.com/apache/hadoop/pull/5784#discussion_r1266568192 ## hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/rmapp/RMAppImpl.java: #

[GitHub] [hadoop] haiyang1987 commented on pull request #5846: HDFS-17088. Improve the debug verifyEC and dfsrouteradmin commands in HDFSCommands.md

2023-07-18 Thread via GitHub
haiyang1987 commented on PR #5846: URL: https://github.com/apache/hadoop/pull/5846#issuecomment-1639885700 Update PR, add -dumpState. @slfan1989 @goiri @ayushtkn sir please help me review this pr again when you have free time. Thanks a lot!!! -- This is an automated message from

[jira] [Commented] (HADOOP-18757) S3A Committer finalizing the commits in a single thread

2023-07-18 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18757?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17744139#comment-17744139 ] ASF GitHub Bot commented on HADOOP-18757: - hadoop-yetus commented on PR #5706:

[GitHub] [hadoop] hadoop-yetus commented on pull request #5706: HADOOP-18757: Bump corePoolSize of HadoopThreadPoolExecutor in s3a committer

2023-07-18 Thread via GitHub
hadoop-yetus commented on PR #5706: URL: https://github.com/apache/hadoop/pull/5706#issuecomment-1639878208 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok: | reex

[GitHub] [hadoop] haiyang1987 commented on pull request #5846: HDFS-17088. Improve the debug verifyEC and dfsrouteradmin commands in HDFSCommands.md

2023-07-18 Thread via GitHub
haiyang1987 commented on PR #5846: URL: https://github.com/apache/hadoop/pull/5846#issuecomment-1639852392 > dfsrouteradmin missed `-dumpState` as well, we should add that as well https://github.com/apache/hadoop/blob/trunk/hadoop-hdfs-project/hadoop-hdfs-rbf/src/test/java/org/apache/hadoop/

[GitHub] [hadoop] haiyang1987 commented on pull request #5845: HDFS-17087. Add Throttler for datanode reading block

2023-07-18 Thread via GitHub
haiyang1987 commented on PR #5845: URL: https://github.com/apache/hadoop/pull/5845#issuecomment-1639833566 > can we add a test as well, may be like one in: [HDFS-14795](https://issues.apache.org/jira/browse/HDFS-14795) yes. thanks sir help me review, i will add test ut later. -- Th

[GitHub] [hadoop] slfan1989 commented on a diff in pull request #5810: YARN-7707. BackPort [GPG] Policy generator framework.

2023-07-18 Thread via GitHub
slfan1989 commented on code in PR #5810: URL: https://github.com/apache/hadoop/pull/5810#discussion_r1266392811 ## hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-globalpolicygenerator/src/main/java/org/apache/hadoop/yarn/server/globalpolicygenerator/GPGPolicyFacad

[jira] [Commented] (HADOOP-18291) S3A prefetch - Implement LRU cache for SingleFilePerBlockCache

2023-07-18 Thread Quan Li (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18291?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17744079#comment-17744079 ] Quan Li commented on HADOOP-18291: -- They were test related only, same exeception as yo

[GitHub] [hadoop] slfan1989 commented on a diff in pull request #5810: YARN-7707. BackPort [GPG] Policy generator framework.

2023-07-18 Thread via GitHub
slfan1989 commented on code in PR #5810: URL: https://github.com/apache/hadoop/pull/5810#discussion_r1266333007 ## hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/resources/yarn-default.xml: ## @@ -5426,4 +5426,36 @@ 100ms + + + The interval at