[GitHub] [hadoop] slfan1989 commented on a diff in pull request #5565: YARN-11467. RM failover may fail when the nodes.exclude-path file does not exist

2023-04-26 Thread via GitHub
slfan1989 commented on code in PR #5565: URL: https://github.com/apache/hadoop/pull/5565#discussion_r1178658969 ## hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/TestRMHA.java: ##

[GitHub] [hadoop] slfan1989 commented on a diff in pull request #5565: YARN-11467. RM failover may fail when the nodes.exclude-path file does not exist

2023-04-26 Thread via GitHub
slfan1989 commented on code in PR #5565: URL: https://github.com/apache/hadoop/pull/5565#discussion_r1178657862 ## hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/TestRMHA.java: ##

[GitHub] [hadoop] cxzl25 opened a new pull request, #5599: YARN-11476. Add NodeManager metric for event queue size of dispatcher

2023-04-26 Thread via GitHub
cxzl25 opened a new pull request, #5599: URL: https://github.com/apache/hadoop/pull/5599 ### Description of PR NodeManager has two dispatchers, we can add metrics to observe the dispatcher queue size. ### How was this patch tested? ### For code changes: - [ ]

[GitHub] [hadoop] hadoop-yetus commented on pull request #5598: YARN-11437. [hotfix][Federation] SQLFederationStateStore Support Version.

2023-04-26 Thread via GitHub
hadoop-yetus commented on PR #5598: URL: https://github.com/apache/hadoop/pull/5598#issuecomment-1524696999 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok: |

[GitHub] [hadoop] hadoop-yetus commented on pull request #5577: YARN-11474.The yarn queue list is displayed on the CLI

2023-04-26 Thread via GitHub
hadoop-yetus commented on PR #5577: URL: https://github.com/apache/hadoop/pull/5577#issuecomment-1524688841 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok: |

[GitHub] [hadoop] hadoop-yetus commented on pull request #5577: YARN-11474.The yarn queue list is displayed on the CLI

2023-04-26 Thread via GitHub
hadoop-yetus commented on PR #5577: URL: https://github.com/apache/hadoop/pull/5577#issuecomment-1524684886 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok: |

[GitHub] [hadoop] hfutatzhanghb commented on pull request #5593: HDFS-16989. Large scale block transfer causes too many excess blocks

2023-04-26 Thread via GitHub
hfutatzhanghb commented on PR #5593: URL: https://github.com/apache/hadoop/pull/5593#issuecomment-1524651963 > we can reproduce this case by execute `hdfs dfs -setrep 3 /largedir` on a large capacity dir(1.0PB) with replication factor 2. -- This is an automated message from the

[GitHub] [hadoop] simbadzina commented on pull request #5598: YARN-11437. [hotfix][Federation] SQLFederationStateStore Support Version.

2023-04-26 Thread via GitHub
simbadzina commented on PR #5598: URL: https://github.com/apache/hadoop/pull/5598#issuecomment-1524561408 There are still 3 unresolved checkstyle issues. Since this PR is to clean-up checkstyle errors, we shouldn't introduce new one.

[GitHub] [hadoop] slfan1989 commented on a diff in pull request #5596: YARN-11379. [Federation] Support mapAttributesToNodes、getGroupsForUser API's for Federation.

2023-04-26 Thread via GitHub
slfan1989 commented on code in PR #5596: URL: https://github.com/apache/hadoop/pull/5596#discussion_r1178561074 ## hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-router/src/main/java/org/apache/hadoop/yarn/server/router/rmadmin/FederationRMAdminInterceptor.java:

[GitHub] [hadoop] slfan1989 commented on pull request #5577: YARN-11474.The yarn queue list is displayed on the CLI

2023-04-26 Thread via GitHub
slfan1989 commented on PR #5577: URL: https://github.com/apache/hadoop/pull/5577#issuecomment-1524470331 @yl09099 Thanks for your contribution! merged. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to

[jira] [Commented] (HADOOP-18603) NPE in LdapAuthenticationHandler as disableHostNameVerification is never initialized

2023-04-26 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18603?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17716935#comment-17716935 ] ASF GitHub Bot commented on HADOOP-18603: - simbadzina commented on PR #5581: URL:

[GitHub] [hadoop] simbadzina commented on pull request #5581: HADOOP-18603 NPE in LdapAuthenticationHandler as disableHostNameVerif…

2023-04-26 Thread via GitHub
simbadzina commented on PR #5581: URL: https://github.com/apache/hadoop/pull/5581#issuecomment-1524300983 Could you please add a unit test that fails without your change and passes with the change. -- This is an automated message from the Apache Git Service. To respond to the message,

[jira] [Commented] (HADOOP-18706) The temporary files for disk-block buffer aren't unique enough to recover partial uploads. 

2023-04-26 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18706?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17716891#comment-17716891 ] ASF GitHub Bot commented on HADOOP-18706: - hadoop-yetus commented on PR #5563: URL:

[GitHub] [hadoop] hadoop-yetus commented on pull request #5563: HADOOP-18706: Improve S3ABlockOutputStream recovery

2023-04-26 Thread via GitHub
hadoop-yetus commented on PR #5563: URL: https://github.com/apache/hadoop/pull/5563#issuecomment-1523955619 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok: |

[GitHub] [hadoop] goiri commented on a diff in pull request #5596: YARN-11379. [Federation] Support mapAttributesToNodes、getGroupsForUser API's for Federation.

2023-04-26 Thread via GitHub
goiri commented on code in PR #5596: URL: https://github.com/apache/hadoop/pull/5596#discussion_r1178310665 ## hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-router/src/main/java/org/apache/hadoop/yarn/server/router/rmadmin/FederationRMAdminInterceptor.java:

[GitHub] [hadoop] hadoop-yetus commented on pull request #5332: YARN-11041. Replace all occurences of queuePath with the new QueuePath class - followup

2023-04-26 Thread via GitHub
hadoop-yetus commented on PR #5332: URL: https://github.com/apache/hadoop/pull/5332#issuecomment-1523887426 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok: |

[jira] [Commented] (HADOOP-18706) The temporary files for disk-block buffer aren't unique enough to recover partial uploads. 

2023-04-26 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18706?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17716847#comment-17716847 ] ASF GitHub Bot commented on HADOOP-18706: - cbevard1 commented on code in PR #5563: URL:

[GitHub] [hadoop] cbevard1 commented on a diff in pull request #5563: HADOOP-18706: Improve S3ABlockOutputStream recovery

2023-04-26 Thread via GitHub
cbevard1 commented on code in PR #5563: URL: https://github.com/apache/hadoop/pull/5563#discussion_r1178227622 ## hadoop-tools/hadoop-aws/src/test/java/org/apache/hadoop/fs/s3a/ITestS3ABlockOutputArray.java: ## @@ -79,6 +80,42 @@ public void testRegularUpload() throws

[jira] [Commented] (HADOOP-18703) Backport S3A prefetching stream to branch-3.3

2023-04-26 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18703?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17716843#comment-17716843 ] Steve Loughran commented on HADOOP-18703: - feature branch has all the commits that I think are

[jira] [Commented] (HADOOP-18291) SingleFilePerBlockCache does not have a limit

2023-04-26 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18291?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17716841#comment-17716841 ] Steve Loughran commented on HADOOP-18291: - something in the storecontext should

[jira] [Updated] (HADOOP-18283) Review s3a prefetching input stream retry code; synchronization

2023-04-26 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18283?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran updated HADOOP-18283: Description: Need to review S3A prefetching stream retry logic * no attempt to retry on

[jira] [Updated] (HADOOP-18283) Review s3a prefetching input stream retry code; synchronization

2023-04-26 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18283?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran updated HADOOP-18283: Summary: Review s3a prefetching input stream retry code; synchronization (was: Review

[jira] [Commented] (HADOOP-18697) Fix transient failure of ITestS3APrefetchingInputStream#testRandomReadLargeFile

2023-04-26 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18697?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17716831#comment-17716831 ] ASF GitHub Bot commented on HADOOP-18697: - steveloughran commented on code in PR #5580: URL:

[GitHub] [hadoop] steveloughran commented on a diff in pull request #5580: HADOOP-18697. Fix transient failure of ITestS3APrefetchingInputStream#testRandomReadLargeFile

2023-04-26 Thread via GitHub
steveloughran commented on code in PR #5580: URL: https://github.com/apache/hadoop/pull/5580#discussion_r1178178480 ## hadoop-tools/hadoop-aws/src/test/java/org/apache/hadoop/fs/s3a/ITestS3APrefetchingInputStream.java: ## @@ -202,13 +203,31 @@ public void

[GitHub] [hadoop] hadoop-yetus commented on pull request #5596: YARN-11379. [Federation] Support mapAttributesToNodes、getGroupsForUser API's for Federation.

2023-04-26 Thread via GitHub
hadoop-yetus commented on PR #5596: URL: https://github.com/apache/hadoop/pull/5596#issuecomment-1523750376 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok: |

[GitHub] [hadoop] virajjasani commented on pull request #5554: HDFS-16978. RBF: Admin command to support bulk add of mount points

2023-04-26 Thread via GitHub
virajjasani commented on PR #5554: URL: https://github.com/apache/hadoop/pull/5554#issuecomment-1523700673 javadoc warnings are not for patch code, they are only for generated proto code -- This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [hadoop] hadoop-yetus commented on pull request #5594: HDFS-16992. Change log level of some logs in FSNamesystem#internalReleaseLease method.

2023-04-26 Thread via GitHub
hadoop-yetus commented on PR #5594: URL: https://github.com/apache/hadoop/pull/5594#issuecomment-1523688163 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok: |

[GitHub] [hadoop] hadoop-yetus commented on pull request #5519: MAPREDUCE-7435. Manifest Committer OOM on abfs

2023-04-26 Thread via GitHub
hadoop-yetus commented on PR #5519: URL: https://github.com/apache/hadoop/pull/5519#issuecomment-1523685056 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok: |

[GitHub] [hadoop] hadoop-yetus commented on pull request #5583: HDFS-16987. [BugFix] MarkBlockAsCorrupt should not mark a replica as corrupted if the DN has a newest replica

2023-04-26 Thread via GitHub
hadoop-yetus commented on PR #5583: URL: https://github.com/apache/hadoop/pull/5583#issuecomment-1523678996 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok: |

[GitHub] [hadoop] hadoop-yetus commented on pull request #5591: HDFS-16991. fix testMkdirsRaceWithObserverRead

2023-04-26 Thread via GitHub
hadoop-yetus commented on PR #5591: URL: https://github.com/apache/hadoop/pull/5591#issuecomment-1523677733 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok: |

[GitHub] [hadoop] hadoop-yetus commented on pull request #5594: HDFS-16992. Change log level of some logs in FSNamesystem#internalReleaseLease method.

2023-04-26 Thread via GitHub
hadoop-yetus commented on PR #5594: URL: https://github.com/apache/hadoop/pull/5594#issuecomment-1523673141 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok: |

[jira] [Commented] (HADOOP-18718) Fix several maven build warnings

2023-04-26 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18718?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17716784#comment-17716784 ] ASF GitHub Bot commented on HADOOP-18718: - dongjoon-hyun commented on PR #5592: URL:

[GitHub] [hadoop] dongjoon-hyun commented on pull request #5592: HADOOP-18718. Fix several maven build warnings

2023-04-26 Thread via GitHub
dongjoon-hyun commented on PR #5592: URL: https://github.com/apache/hadoop/pull/5592#issuecomment-1523661404 Ya, I thought like that. Thank you, @steveloughran and @ayushtkn . -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub

[GitHub] [hadoop-thirdparty] dmmkr commented on pull request #22: HADOOP-18721. Upgrade guava to 31.1

2023-04-26 Thread via GitHub
dmmkr commented on PR #22: URL: https://github.com/apache/hadoop-thirdparty/pull/22#issuecomment-1523658895 Hadoop trunk build is successful with guava-31.1-hadoop-thirdparty -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub

[jira] [Commented] (HADOOP-18695) S3A: reject multipart copy requests when disabled

2023-04-26 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18695?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17716781#comment-17716781 ] ASF GitHub Bot commented on HADOOP-18695: - hadoop-yetus commented on PR #5548: URL:

[GitHub] [hadoop] hadoop-yetus commented on pull request #5548: HADOOP-18695. S3A: reject multipart copy requests when disabled

2023-04-26 Thread via GitHub
hadoop-yetus commented on PR #5548: URL: https://github.com/apache/hadoop/pull/5548#issuecomment-1523654004 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok: |

[GitHub] [hadoop] hadoop-yetus commented on pull request #5597: HDFS-16993. Datanode supports configure TopN DatanodeNetworkCounts

2023-04-26 Thread via GitHub
hadoop-yetus commented on PR #5597: URL: https://github.com/apache/hadoop/pull/5597#issuecomment-1523650228 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok: |

[GitHub] [hadoop] hadoop-yetus commented on pull request #5593: HDFS-16989. Large scale block transfer causes too many excess blocks

2023-04-26 Thread via GitHub
hadoop-yetus commented on PR #5593: URL: https://github.com/apache/hadoop/pull/5593#issuecomment-1523544877 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok: |

[jira] [Created] (HADOOP-18722) Optimise S3A delete objects when multiObjectDelete is disabled

2023-04-26 Thread Mehakmeet Singh (Jira)
Mehakmeet Singh created HADOOP-18722: Summary: Optimise S3A delete objects when multiObjectDelete is disabled Key: HADOOP-18722 URL: https://issues.apache.org/jira/browse/HADOOP-18722 Project:

[GitHub] [hadoop] hadoop-yetus commented on pull request #5594: HDFS-16992. Change log level of some logs in FSNamesystem#internalReleaseLease method.

2023-04-26 Thread via GitHub
hadoop-yetus commented on PR #5594: URL: https://github.com/apache/hadoop/pull/5594#issuecomment-1523492794 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok: |

[GitHub] [hadoop] hadoop-yetus commented on pull request #5594: HDFS-16992. Change log level of some logs in FSNamesystem#internalReleaseLease method.

2023-04-26 Thread via GitHub
hadoop-yetus commented on PR #5594: URL: https://github.com/apache/hadoop/pull/5594#issuecomment-1523489130 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok: |

[jira] [Updated] (HADOOP-18721) Upgrade guava to 31.1

2023-04-26 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18721?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated HADOOP-18721: Labels: pull-request-available (was: ) > Upgrade guava to 31.1 > -

[GitHub] [hadoop-thirdparty] dmmkr opened a new pull request, #22: HADOOP-18721. Upgrade guava to 31.1

2023-04-26 Thread via GitHub
dmmkr opened a new pull request, #22: URL: https://github.com/apache/hadoop-thirdparty/pull/22 ## NOTICE Please create an issue in ASF JIRA before opening a pull request, and you need to set the title of the pull request which starts with the corresponding JIRA issue number.

[jira] [Created] (HADOOP-18721) Upgrade guava to 31.1

2023-04-26 Thread D M Murali Krishna Reddy (Jira)
D M Murali Krishna Reddy created HADOOP-18721: - Summary: Upgrade guava to 31.1 Key: HADOOP-18721 URL: https://issues.apache.org/jira/browse/HADOOP-18721 Project: Hadoop Common

[jira] [Commented] (HADOOP-18695) S3A: reject multipart copy requests when disabled

2023-04-26 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18695?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17716740#comment-17716740 ] ASF GitHub Bot commented on HADOOP-18695: - steveloughran commented on PR #5548: URL:

[GitHub] [hadoop] steveloughran commented on pull request #5548: HADOOP-18695. S3A: reject multipart copy requests when disabled

2023-04-26 Thread via GitHub
steveloughran commented on PR #5548: URL: https://github.com/apache/hadoop/pull/5548#issuecomment-1523468362 one more checkstyle update; full test run of ` -Dparallel-tests -DtestsThreadCount=8 -Dscale` against s3 london. all good -- This is an automated message from the Apache Git

[jira] [Commented] (HADOOP-18565) AWS SDK V2 - Complete outstanding items

2023-04-26 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18565?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17716721#comment-17716721 ] ASF GitHub Bot commented on HADOOP-18565: - steveloughran commented on PR #5421: URL:

[GitHub] [hadoop] steveloughran commented on pull request #5421: HADOOP-18565. Completes outstanding items for the SDK V2 upgrade.

2023-04-26 Thread via GitHub
steveloughran commented on PR #5421: URL: https://github.com/apache/hadoop/pull/5421#issuecomment-1523433345 oh, and based some other changes, spotbugs warning about extant spotbugs doesn't mean you've added a new one, just that there is one to get rid of. yetus can still warn there even

[jira] [Commented] (HADOOP-18695) S3A: reject multipart copy requests when disabled

2023-04-26 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18695?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17716720#comment-17716720 ] ASF GitHub Bot commented on HADOOP-18695: - steveloughran commented on PR #5548: URL:

[GitHub] [hadoop] steveloughran commented on pull request #5548: HADOOP-18695. S3A: reject multipart copy requests when disabled

2023-04-26 Thread via GitHub
steveloughran commented on PR #5548: URL: https://github.com/apache/hadoop/pull/5548#issuecomment-1523430441 checkstyle still unhappy. the file length one is something i can ignore for a test case, but I will improve it slightly anyway. a lot of the lines are just multiline assertj

[GitHub] [hadoop] steveloughran commented on pull request #5519: MAPREDUCE-7435. Manifest Committer OOM on abfs

2023-04-26 Thread via GitHub
steveloughran commented on PR #5519: URL: https://github.com/apache/hadoop/pull/5519#issuecomment-1523419702 latest patch tested against azure cardiff, timeout unrelated ``` [ERROR] Errors: [ERROR]

[jira] [Updated] (HADOOP-18712) Upgrade to jetty 9.4.51 due to cve

2023-04-26 Thread Ayush Saxena (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18712?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ayush Saxena updated HADOOP-18712: -- Fix Version/s: 3.3.9 > Upgrade to jetty 9.4.51 due to cve >

[GitHub] [hadoop] ayushtkn merged pull request #5585: Hadoop 18712. upgrade to jetty 9.4.51 due to cve

2023-04-26 Thread via GitHub
ayushtkn merged PR #5585: URL: https://github.com/apache/hadoop/pull/5585 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

[GitHub] [hadoop] ayushtkn commented on pull request #5585: Hadoop 18712. upgrade to jetty 9.4.51 due to cve

2023-04-26 Thread via GitHub
ayushtkn commented on PR #5585: URL: https://github.com/apache/hadoop/pull/5585#issuecomment-1523412130 Failing tests passed locally ``` [INFO] --- [INFO] T E S T S [INFO] ---

[GitHub] [hadoop] hadoop-yetus commented on pull request #5598: YARN-11437. [hotfix][Federation] SQLFederationStateStore Support Version.

2023-04-26 Thread via GitHub
hadoop-yetus commented on PR #5598: URL: https://github.com/apache/hadoop/pull/5598#issuecomment-1523393115 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok: |

[jira] [Commented] (HADOOP-18718) Fix several maven build warnings

2023-04-26 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18718?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17716709#comment-17716709 ] ASF GitHub Bot commented on HADOOP-18718: - ayushtkn commented on PR #5592: URL:

[GitHub] [hadoop] ayushtkn commented on pull request #5592: HADOOP-18718. Fix several maven build warnings

2023-04-26 Thread via GitHub
ayushtkn commented on PR #5592: URL: https://github.com/apache/hadoop/pull/5592#issuecomment-1523387368 Should be good, if the build doesn't complain. The findbug thing looks like a leftover to me. We don't run neither setup findbugs now in our jenkins job, we just run spotbugs

[GitHub] [hadoop] hadoop-yetus commented on pull request #5596: YARN-11379. [Federation] Support mapAttributesToNodes、getGroupsForUser API's for Federation.

2023-04-26 Thread via GitHub
hadoop-yetus commented on PR #5596: URL: https://github.com/apache/hadoop/pull/5596#issuecomment-1523375951 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok: |

[jira] [Commented] (HADOOP-18695) S3A: reject multipart copy requests when disabled

2023-04-26 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18695?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17716700#comment-17716700 ] ASF GitHub Bot commented on HADOOP-18695: - hadoop-yetus commented on PR #5548: URL:

[GitHub] [hadoop] hadoop-yetus commented on pull request #5548: HADOOP-18695. S3A: reject multipart copy requests when disabled

2023-04-26 Thread via GitHub
hadoop-yetus commented on PR #5548: URL: https://github.com/apache/hadoop/pull/5548#issuecomment-1523366082 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok: |

[GitHub] [hadoop] caoergou commented on pull request #5293: YARN-11414. ClusterMetricsInfo shows wrong availableMB when node labels enabled

2023-04-26 Thread via GitHub
caoergou commented on PR #5293: URL: https://github.com/apache/hadoop/pull/5293#issuecomment-1523331494 still nobody review for that? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific

[GitHub] [hadoop] hadoop-yetus commented on pull request #3618: YARN-11000. Replace queue resource calculation logic in updateClusterResource

2023-04-26 Thread via GitHub
hadoop-yetus commented on PR #3618: URL: https://github.com/apache/hadoop/pull/3618#issuecomment-1523266310 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok: |

[GitHub] [hadoop] steveloughran commented on pull request #5519: MAPREDUCE-7435. Manifest Committer OOM on abfs

2023-04-26 Thread via GitHub
steveloughran commented on PR #5519: URL: https://github.com/apache/hadoop/pull/5519#issuecomment-1523265310 ok, rebasing and pushing up with a commit to address most of the changes. Not addressed: having timeouts on the offer/take of the EntryWriter in the queue. I agree it is

[GitHub] [hadoop] steveloughran commented on a diff in pull request #5519: MAPREDUCE-7435. Manifest Committer OOM on abfs

2023-04-26 Thread via GitHub
steveloughran commented on code in PR #5519: URL: https://github.com/apache/hadoop/pull/5519#discussion_r1177727496 ##

[GitHub] [hadoop] steveloughran commented on a diff in pull request #5519: MAPREDUCE-7435. Manifest Committer OOM on abfs

2023-04-26 Thread via GitHub
steveloughran commented on code in PR #5519: URL: https://github.com/apache/hadoop/pull/5519#discussion_r1177725376 ##

[GitHub] [hadoop] steveloughran commented on a diff in pull request #5519: MAPREDUCE-7435. Manifest Committer OOM on abfs

2023-04-26 Thread via GitHub
steveloughran commented on code in PR #5519: URL: https://github.com/apache/hadoop/pull/5519#discussion_r1177721915 ##

[GitHub] [hadoop] steveloughran commented on a diff in pull request #5519: MAPREDUCE-7435. Manifest Committer OOM on abfs

2023-04-26 Thread via GitHub
steveloughran commented on code in PR #5519: URL: https://github.com/apache/hadoop/pull/5519#discussion_r1177721536 ##

[GitHub] [hadoop] steveloughran commented on a diff in pull request #5519: MAPREDUCE-7435. Manifest Committer OOM on abfs

2023-04-26 Thread via GitHub
steveloughran commented on code in PR #5519: URL: https://github.com/apache/hadoop/pull/5519#discussion_r1177720020 ##

[GitHub] [hadoop] steveloughran commented on a diff in pull request #5519: MAPREDUCE-7435. Manifest Committer OOM on abfs

2023-04-26 Thread via GitHub
steveloughran commented on code in PR #5519: URL: https://github.com/apache/hadoop/pull/5519#discussion_r1177718895 ##

[GitHub] [hadoop] slfan1989 opened a new pull request, #5598: YARN-11437. [hotfix][Federation] SQLFederationStateStore Support Version.

2023-04-26 Thread via GitHub
slfan1989 opened a new pull request, #5598: URL: https://github.com/apache/hadoop/pull/5598 JIRA: YARN-11437. [hotfix][Federation] SQLFederationStateStore Support Version. Fix CheckStyle Issue. -- This is an automated message from the Apache Git Service. To respond to the message,

[GitHub] [hadoop] steveloughran commented on a diff in pull request #5519: MAPREDUCE-7435. Manifest Committer OOM on abfs

2023-04-26 Thread via GitHub
steveloughran commented on code in PR #5519: URL: https://github.com/apache/hadoop/pull/5519#discussion_r1177718242 ##

[GitHub] [hadoop] hfutatzhanghb opened a new pull request, #5597: HDFS-16993. Datanode supports configure TopN DatanodeNetworkCounts

2023-04-26 Thread via GitHub
hfutatzhanghb opened a new pull request, #5597: URL: https://github.com/apache/hadoop/pull/5597 ### Description of PR In our prod environment, we try to collect datanode metrics every 15s through jmx_exporter. we found the datanodenetworkerror metric generates a lot. for example,

[GitHub] [hadoop] slfan1989 opened a new pull request, #5596: YARN-11379. [Federation] Support mapAttributesToNodes、getGroupsForUser API's for Federation.

2023-04-26 Thread via GitHub
slfan1989 opened a new pull request, #5596: URL: https://github.com/apache/hadoop/pull/5596 JIRA: YARN-11379. [Federation] Support mapAttributesToNodes、getGroupsForUser API's for Federation. -- This is an automated message from the Apache Git Service. To respond to the message, please

[jira] [Created] (HADOOP-18720) Publish test jar for hadoop-aws project

2023-04-26 Thread Ujjawal Kumar (Jira)
Ujjawal Kumar created HADOOP-18720: -- Summary: Publish test jar for hadoop-aws project Key: HADOOP-18720 URL: https://issues.apache.org/jira/browse/HADOOP-18720 Project: Hadoop Common Issue

[GitHub] [hadoop] hadoop-yetus commented on pull request #5564: HDFS-16985. delete local block file when FileNotFoundException occurred may lead to missing block.

2023-04-26 Thread via GitHub
hadoop-yetus commented on PR #5564: URL: https://github.com/apache/hadoop/pull/5564#issuecomment-1523093653 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok: |

[GitHub] [hadoop] hadoop-yetus commented on pull request #5583: HDFS-16987. [BugFix] MarkBlockAsCorrupt should not mark a replica as corrupted if the DN has a newest replica

2023-04-26 Thread via GitHub
hadoop-yetus commented on PR #5583: URL: https://github.com/apache/hadoop/pull/5583#issuecomment-1523089889 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok: |

[jira] [Commented] (HADOOP-18695) S3A: reject multipart copy requests when disabled

2023-04-26 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18695?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17716619#comment-17716619 ] ASF GitHub Bot commented on HADOOP-18695: - steveloughran commented on PR #5548: URL:

[GitHub] [hadoop] steveloughran commented on pull request #5548: HADOOP-18695. S3A: reject multipart copy requests when disabled

2023-04-26 Thread via GitHub
steveloughran commented on PR #5548: URL: https://github.com/apache/hadoop/pull/5548#issuecomment-1523060709 little bit of indentation ``` ./hadoop-tools/hadoop-aws/src/test/java/org/apache/hadoop/fs/s3a/scale/AbstractSTestS3AHugeFiles.java:160: @Test:3: Method length is 154 lines

[jira] [Commented] (HADOOP-18718) Fix several maven build warnings

2023-04-26 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18718?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17716618#comment-17716618 ] ASF GitHub Bot commented on HADOOP-18718: - steveloughran commented on PR #5592: URL:

[GitHub] [hadoop] steveloughran commented on pull request #5592: HADOOP-18718. Fix several maven build warnings

2023-04-26 Thread via GitHub
steveloughran commented on PR #5592: URL: https://github.com/apache/hadoop/pull/5592#issuecomment-1523055074 looks good. worried that a move to spotbugs may have surprises, but it is only in 1 module and that already has a spotbugs by the look of things -- This is an automated message

[GitHub] [hadoop] pjfanning opened a new pull request, #5595: Hadoop-18658 upgrade snakeyaml to 2.0

2023-04-26 Thread via GitHub
pjfanning opened a new pull request, #5595: URL: https://github.com/apache/hadoop/pull/5595 ### Description of PR ### How was this patch tested? ### For code changes: - [x] Does the title or this PR starts with the corresponding JIRA issue id (e.g.

[jira] [Resolved] (HADOOP-18719) upgrade snakeyaml to 2.0 (fixes CVE-2022-1471)

2023-04-26 Thread PJ Fanning (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18719?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] PJ Fanning resolved HADOOP-18719. - Resolution: Duplicate > upgrade snakeyaml to 2.0 (fixes CVE-2022-1471) >

[jira] [Created] (HADOOP-18719) upgrade snakeyaml to 2.0 (fixes CVE-2022-1471)

2023-04-26 Thread PJ Fanning (Jira)
PJ Fanning created HADOOP-18719: --- Summary: upgrade snakeyaml to 2.0 (fixes CVE-2022-1471) Key: HADOOP-18719 URL: https://issues.apache.org/jira/browse/HADOOP-18719 Project: Hadoop Common Issue

[GitHub] [hadoop] pjfanning commented on pull request #5585: Hadoop 18712. upgrade to jetty 9.4.51 due to cve

2023-04-26 Thread via GitHub
pjfanning commented on PR #5585: URL: https://github.com/apache/hadoop/pull/5585#issuecomment-1523000989 @ayushtkn I did a new run - there was a merge conflict that needed fixing -- This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [hadoop] hadoop-yetus commented on pull request #5585: Hadoop 18712. upgrade to jetty 9.4.51 due to cve

2023-04-26 Thread via GitHub
hadoop-yetus commented on PR #5585: URL: https://github.com/apache/hadoop/pull/5585#issuecomment-1522990034 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok: |

[GitHub] [hadoop] hfutatzhanghb opened a new pull request, #5594: HDFS-16992. Change log level of some logs in FSNamesystem#internalReleaseLease method.

2023-04-26 Thread via GitHub
hfutatzhanghb opened a new pull request, #5594: URL: https://github.com/apache/hadoop/pull/5594 recently, we found lots of logs like below which trigger logWarn alarm: ``` WARN org.apache.hadoop.hdfs.StateChange: BLOCK* internalReleaseLease: All existing blocks are COMPLETE, lease

[jira] [Commented] (HADOOP-18359) Update commons-cli from 1.2 to 1.5.

2023-04-26 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18359?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17716592#comment-17716592 ] ASF GitHub Bot commented on HADOOP-18359: - hadoop-yetus commented on PR #5095: URL:

[GitHub] [hadoop] hadoop-yetus commented on pull request #5095: HADOOP-18359. Update commons-cli from 1.2 to 1.5.

2023-04-26 Thread via GitHub
hadoop-yetus commented on PR #5095: URL: https://github.com/apache/hadoop/pull/5095#issuecomment-1522957405 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok: |

[GitHub] [hadoop] hadoop-yetus commented on pull request #5554: HDFS-16978. RBF: Admin command to support bulk add of mount points

2023-04-26 Thread via GitHub
hadoop-yetus commented on PR #5554: URL: https://github.com/apache/hadoop/pull/5554#issuecomment-1522920461 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok: |

[GitHub] [hadoop] hfutatzhanghb commented on pull request #5593: HDFS-16989. Large scale block transfer causes too many excess blocks

2023-04-26 Thread via GitHub
hfutatzhanghb commented on PR #5593: URL: https://github.com/apache/hadoop/pull/5593#issuecomment-1522898614 hi @ayushtkn @tomscut , sorry for disturbing you. now i have not add UT for this change. could you please take a look at

[GitHub] [hadoop] hfutatzhanghb opened a new pull request, #5593: HDFS-16989. Large scale block transfer causes too many excess blocks

2023-04-26 Thread via GitHub
hfutatzhanghb opened a new pull request, #5593: URL: https://github.com/apache/hadoop/pull/5593 https://issues.apache.org/jira/browse/HDFS-16989 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to