Re: [PR] HADOOP-19124. Update org.ehcache from 3.3.1 to 3.8.2. [hadoop]

2024-03-26 Thread via GitHub
hadoop-yetus commented on PR #6665: URL: https://github.com/apache/hadoop/pull/6665#issuecomment-2022016531 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok: |

Re: [PR] YARN-11664: Remove HDFS Binaries/Jars Dependency From Yarn [hadoop]

2024-03-26 Thread via GitHub
shameersss1 commented on PR #6631: URL: https://github.com/apache/hadoop/pull/6631#issuecomment-2022015512 > looks ok to me, but hdfs mail list should be invited to comment. > > made some minor suggestions I have send an email to hdfs mailing list asking their opinion as well.

Re: [PR] HADOOP-19071. Update maven-surefire-plugin from 3.0.0 to 3.2.5. [hadoop]

2024-03-26 Thread via GitHub
slfan1989 commented on code in PR #6664: URL: https://github.com/apache/hadoop/pull/6664#discussion_r1540513670 ## dev-support/bin/hadoop.sh: ## @@ -614,7 +615,7 @@ function shadedclient_rebuild echo_and_redirect "${logfile}" \ "${MAVEN}" "${MAVEN_ARGS[@]}" verify -fae

[jira] [Commented] (HADOOP-19071) Update maven-surefire-plugin from 3.0.0 to 3.2.5

2024-03-26 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-19071?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17831177#comment-17831177 ] ASF GitHub Bot commented on HADOOP-19071: - slfan1989 commented on code in PR #6664: URL:

[jira] [Commented] (HADOOP-19071) Update maven-surefire-plugin from 3.0.0 to 3.2.5

2024-03-26 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-19071?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17831178#comment-17831178 ] ASF GitHub Bot commented on HADOOP-19071: - slfan1989 commented on code in PR #6664: URL:

Re: [PR] HDFS-17216. Distcp: When handle the small files, the bandwidth parameter will be invalid, fix this bug. [hadoop]

2024-03-26 Thread via GitHub
hadoop-yetus commented on PR #6138: URL: https://github.com/apache/hadoop/pull/6138#issuecomment-2022013520 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok: |

Re: [PR] HADOOP-19071. Update maven-surefire-plugin from 3.0.0 to 3.2.5. [hadoop]

2024-03-26 Thread via GitHub
slfan1989 commented on code in PR #6664: URL: https://github.com/apache/hadoop/pull/6664#discussion_r1540513670 ## dev-support/bin/hadoop.sh: ## @@ -614,7 +615,7 @@ function shadedclient_rebuild echo_and_redirect "${logfile}" \ "${MAVEN}" "${MAVEN_ARGS[@]}" verify -fae

Re: [PR] HADOOP-19071. Update maven-surefire-plugin from 3.0.0 to 3.2.5. [hadoop]

2024-03-26 Thread via GitHub
hadoop-yetus commented on PR #6664: URL: https://github.com/apache/hadoop/pull/6664#issuecomment-2022007319 (!) A patch to the testing environment has been detected. Re-executing against the patched versions to perform further tests. The console is at

Re: [PR] HDFS-17439. Support -nonSuperUser for NNThroughputBenchmark: useful f… [hadoop]

2024-03-26 Thread via GitHub
hadoop-yetus commented on PR #6677: URL: https://github.com/apache/hadoop/pull/6677#issuecomment-2022006984 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok: |

Re: [PR] YARN-11664: Remove HDFS Binaries/Jars Dependency From Yarn [hadoop]

2024-03-26 Thread via GitHub
shameersss1 commented on code in PR #6631: URL: https://github.com/apache/hadoop/pull/6631#discussion_r1540493331 ## hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/io/IOStreamPair.java: ## @@ -15,15 +15,14 @@ * See the License for the specific language

Re: [PR] YARN-11664: Remove HDFS Binaries/Jars Dependency From Yarn [hadoop]

2024-03-26 Thread via GitHub
shameersss1 commented on code in PR #6631: URL: https://github.com/apache/hadoop/pull/6631#discussion_r1540456198 ## hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/Constants.java: ## @@ -0,0 +1,35 @@ +/* + * Licensed to the Apache Software Foundation (ASF)

[PR] HDFS-17420. EditLogTailer and FSEditLogLoader support FGL [hadoop]

2024-03-26 Thread via GitHub
ZanderXu opened a new pull request, #6679: URL: https://github.com/apache/hadoop/pull/6679 EditLogTailer and FSEditLogLoader support FGL. Both EditLogTailer and FSEditLogLoader need Global lock. -- This is an automated message from the Apache Git Service. To respond to the message,

[jira] [Updated] (HADOOP-19130) FTPFileSystem rename with full qualified path broken

2024-03-26 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-19130?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated HADOOP-19130: Labels: pull-request-available (was: ) > FTPFileSystem rename with full qualified path

[jira] [Commented] (HADOOP-19130) FTPFileSystem rename with full qualified path broken

2024-03-26 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-19130?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17831137#comment-17831137 ] ASF GitHub Bot commented on HADOOP-19130: - zj619 opened a new pull request, #6678: URL:

[PR] HADOOP-19130. FTPFileSystem rename with full qualified path broken [hadoop]

2024-03-26 Thread via GitHub
zj619 opened a new pull request, #6678: URL: https://github.com/apache/hadoop/pull/6678 ### Description of PR For more information about this PR, please refer to the following issue: [HADOOP-19130](https://issues.apache.org/jira/browse/HADOOP-19130) fix bug in

[jira] [Commented] (HADOOP-19116) update to zookeeper client 3.8.4 due to CVE-2024-23944

2024-03-26 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-19116?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17831135#comment-17831135 ] ASF GitHub Bot commented on HADOOP-19116: - hadoop-yetus commented on PR #6675: URL:

Re: [PR] HADOOP-19116. Update to zookeeper client 3.8.4 due to CVE-2024-23944 [hadoop]

2024-03-26 Thread via GitHub
hadoop-yetus commented on PR #6675: URL: https://github.com/apache/hadoop/pull/6675#issuecomment-2021802435 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok: |

[jira] [Updated] (HADOOP-19130) FTPFileSystem rename with full qualified path broken

2024-03-26 Thread shawn (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-19130?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] shawn updated HADOOP-19130: --- External issue ID: (was: https://issues.apache.org/jira/browse/HADOOP-8653) > FTPFileSystem rename with

[jira] [Created] (HADOOP-19130) FTPFileSystem rename with full qualified path broken

2024-03-26 Thread shawn (Jira)
shawn created HADOOP-19130: -- Summary: FTPFileSystem rename with full qualified path broken Key: HADOOP-19130 URL: https://issues.apache.org/jira/browse/HADOOP-19130 Project: Hadoop Common Issue

Re: [PR] HDFS-17412. [FGL] Client RPCs involving maintenance supports fine-grained lock [hadoop]

2024-03-26 Thread via GitHub
ferhui commented on PR #6667: URL: https://github.com/apache/hadoop/pull/6667#issuecomment-2021773879 @ZanderXu there are conflicts after merging other PRs. could you solve them? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to

Re: [PR] HDFS-17417. [FGL] HeartbeatManager and DatanodeAdminMonitor support fine-grained locking [hadoop]

2024-03-26 Thread via GitHub
ferhui commented on PR #6656: URL: https://github.com/apache/hadoop/pull/6656#issuecomment-2021771515 @ZanderXu Thanks for contribution. @huangzhaobo99 Thanks for review. Merged. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to

Re: [PR] HDFS-17417. [FGL] HeartbeatManager and DatanodeAdminMonitor support fine-grained locking [hadoop]

2024-03-26 Thread via GitHub
ferhui merged PR #6656: URL: https://github.com/apache/hadoop/pull/6656 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

Re: [PR] HDFS-17415. [FGL] RPCs in NamenodeProtocol support fine-grained lock [hadoop]

2024-03-26 Thread via GitHub
ferhui commented on PR #6654: URL: https://github.com/apache/hadoop/pull/6654#issuecomment-2021770281 Thanks for contribution. Merged. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the

Re: [PR] HDFS-17415. [FGL] RPCs in NamenodeProtocol support fine-grained lock [hadoop]

2024-03-26 Thread via GitHub
ferhui merged PR #6654: URL: https://github.com/apache/hadoop/pull/6654 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

Re: [PR] HDFS-17410. [FGL] Client RPCs that changes file attributes supports fine-grained lock [hadoop]

2024-03-26 Thread via GitHub
ferhui commented on PR #6634: URL: https://github.com/apache/hadoop/pull/6634#issuecomment-2021767985 Thanks for contribution. Merged. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the

Re: [PR] HDFS-17410. [FGL] Client RPCs that changes file attributes supports fine-grained lock [hadoop]

2024-03-26 Thread via GitHub
ferhui merged PR #6634: URL: https://github.com/apache/hadoop/pull/6634 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

[jira] [Commented] (HADOOP-19093) Improve rate limiting through ABFS in Manifest Committer

2024-03-26 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-19093?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17831118#comment-17831118 ] ASF GitHub Bot commented on HADOOP-19093: - hadoop-yetus commented on PR #6596: URL:

Re: [PR] HADOOP-19093. [ABFS] Improve rate limiting through ABFS in Manifest Committer [hadoop]

2024-03-26 Thread via GitHub
hadoop-yetus commented on PR #6596: URL: https://github.com/apache/hadoop/pull/6596#issuecomment-2021753216 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok: |

Re: [PR] HDFS-17429. Fixing wrong log file name in datatransfer Sender.java [hadoop]

2024-03-26 Thread via GitHub
slfan1989 commented on PR #6670: URL: https://github.com/apache/hadoop/pull/6670#issuecomment-2021608269 @wzk784533 Thanks for the contribution! merged into trunk. @ayushtkn Thanks for the review! -- This is an automated message from the Apache Git Service. To respond to the message,

Re: [PR] HDFS-17429. Fixing wrong log file name in datatransfer Sender.java [hadoop]

2024-03-26 Thread via GitHub
slfan1989 merged PR #6670: URL: https://github.com/apache/hadoop/pull/6670 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

Re: [PR] YARN-11663. [Federation] Add Cache Entity Nums Limit. [hadoop]

2024-03-26 Thread via GitHub
slfan1989 commented on code in PR #6662: URL: https://github.com/apache/hadoop/pull/6662#discussion_r1540208851 ## hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/conf/YarnConfiguration.java: ## @@ -4031,6 +4031,11 @@ public static boolean

[jira] [Commented] (HADOOP-19096) [ABFS] Enhancing Client-Side Throttling Metrics Updation Logic

2024-03-26 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-19096?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17831098#comment-17831098 ] ASF GitHub Bot commented on HADOOP-19096: - mukund-thakur commented on code in PR #6276: URL:

Re: [PR] HADOOP-19096: [ABFS] [CST Optimization] Enhancing Client-Side Throttling Metrics Updating Logic [hadoop]

2024-03-26 Thread via GitHub
mukund-thakur commented on code in PR #6276: URL: https://github.com/apache/hadoop/pull/6276#discussion_r1540183142 ## hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azurebfs/services/AbfsRestOperation.java: ## @@ -283,7 +285,7 @@ String getClientLatency() {

[jira] [Commented] (HADOOP-19096) [ABFS] Enhancing Client-Side Throttling Metrics Updation Logic

2024-03-26 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-19096?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17831097#comment-17831097 ] ASF GitHub Bot commented on HADOOP-19096: - mukund-thakur commented on code in PR #6276: URL:

Re: [PR] HADOOP-19096: [ABFS] [CST Optimization] Enhancing Client-Side Throttling Metrics Updating Logic [hadoop]

2024-03-26 Thread via GitHub
mukund-thakur commented on code in PR #6276: URL: https://github.com/apache/hadoop/pull/6276#discussion_r1540181623 ## hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azurebfs/services/AbfsRestOperation.java: ## @@ -321,7 +323,27 @@ private boolean

[jira] [Commented] (HADOOP-19096) [ABFS] Enhancing Client-Side Throttling Metrics Updation Logic

2024-03-26 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-19096?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17831096#comment-17831096 ] ASF GitHub Bot commented on HADOOP-19096: - mukund-thakur commented on code in PR #6276: URL:

Re: [PR] HADOOP-19096: [ABFS] [CST Optimization] Enhancing Client-Side Throttling Metrics Updating Logic [hadoop]

2024-03-26 Thread via GitHub
mukund-thakur commented on code in PR #6276: URL: https://github.com/apache/hadoop/pull/6276#discussion_r1540174936 ## hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azurebfs/services/AbfsRestOperation.java: ## @@ -321,7 +323,27 @@ private boolean

[jira] [Commented] (HADOOP-19096) [ABFS] Enhancing Client-Side Throttling Metrics Updation Logic

2024-03-26 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-19096?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17831095#comment-17831095 ] ASF GitHub Bot commented on HADOOP-19096: - mukund-thakur commented on code in PR #6276: URL:

Re: [PR] HADOOP-19096: [ABFS] [CST Optimization] Enhancing Client-Side Throttling Metrics Updating Logic [hadoop]

2024-03-26 Thread via GitHub
mukund-thakur commented on code in PR #6276: URL: https://github.com/apache/hadoop/pull/6276#discussion_r1540174681 ## hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azurebfs/services/AbfsRestOperation.java: ## @@ -333,57 +355,44 @@ private boolean

Re: [PR] HDFS-17439. Support -nonSuperUser for NNThroughputBenchmark: useful f… [hadoop]

2024-03-26 Thread via GitHub
hadoop-yetus commented on PR #6677: URL: https://github.com/apache/hadoop/pull/6677#issuecomment-2021543381 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok: |

[jira] [Commented] (HADOOP-19093) Improve rate limiting through ABFS in Manifest Committer

2024-03-26 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-19093?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17831082#comment-17831082 ] ASF GitHub Bot commented on HADOOP-19093: - steveloughran commented on PR #6596: URL:

Re: [PR] HADOOP-19093. [ABFS] Improve rate limiting through ABFS in Manifest Committer [hadoop]

2024-03-26 Thread via GitHub
steveloughran commented on PR #6596: URL: https://github.com/apache/hadoop/pull/6596#issuecomment-2021481003 I am having fun testing at scale with a throttled store. ``` [ERROR] testSeekZeroByteFile(org.apache.hadoop.fs.azurebfs.contract.ITestAbfsFileSystemContractSeek) Time

[jira] [Commented] (HADOOP-19098) Vector IO: consistent specified rejection of overlapping ranges

2024-03-26 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-19098?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17831079#comment-17831079 ] ASF GitHub Bot commented on HADOOP-19098: - hadoop-yetus commented on PR #6604: URL:

Re: [PR] HADOOP-19098 Vector IO: consistent specified rejection of overlapping ranges [hadoop]

2024-03-26 Thread via GitHub
hadoop-yetus commented on PR #6604: URL: https://github.com/apache/hadoop/pull/6604#issuecomment-2021454419 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok: |

Re: [PR] YARN-11664: Remove HDFS Binaries/Jars Dependency From Yarn [hadoop]

2024-03-26 Thread via GitHub
steveloughran commented on code in PR #6631: URL: https://github.com/apache/hadoop/pull/6631#discussion_r1539932538 ## hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/io/IOStreamPair.java: ## @@ -15,15 +15,14 @@ * See the License for the specific language

[jira] [Commented] (HADOOP-19114) upgrade to commons-compress 1.26.1 due to cves

2024-03-26 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-19114?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17831046#comment-17831046 ] ASF GitHub Bot commented on HADOOP-19114: - pjfanning commented on PR #6636: URL:

Re: [PR] HADOOP-19114. Upgrade to commons-compress 1.26.1 due to CVEs. [hadoop]

2024-03-26 Thread via GitHub
pjfanning commented on PR #6636: URL: https://github.com/apache/hadoop/pull/6636#issuecomment-2021243187 createArchiveEntry returns a TarArchiveEntry - so I just need to change the variable type I'm checking to see if there might be similar issues in other parts of Hadoop but so

[jira] [Commented] (HADOOP-19114) upgrade to commons-compress 1.26.1 due to cves

2024-03-26 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-19114?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17831044#comment-17831044 ] ASF GitHub Bot commented on HADOOP-19114: - steveloughran commented on PR #6636: URL:

Re: [PR] HADOOP-19114. Upgrade to commons-compress 1.26.1 due to CVEs. [hadoop]

2024-03-26 Thread via GitHub
steveloughran commented on PR #6636: URL: https://github.com/apache/hadoop/pull/6636#issuecomment-2021234507 looking at the code which doesn't compile ```java TarArchiveOutputStream out = new TarArchiveOutputStream(targetStream) ... try (FileInputStream inputStream =

[jira] [Commented] (HADOOP-19114) upgrade to commons-compress 1.26.1 due to cves

2024-03-26 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-19114?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17831040#comment-17831040 ] ASF GitHub Bot commented on HADOOP-19114: - pjfanning commented on PR #6636: URL:

Re: [PR] HADOOP-19114. Upgrade to commons-compress 1.26.1 due to CVEs. [hadoop]

2024-03-26 Thread via GitHub
pjfanning commented on PR #6636: URL: https://github.com/apache/hadoop/pull/6636#issuecomment-2021227384 @steveloughran I'll have a look. Looks like I will need to uptake some API changes in commons-compress to get the code to compile again. -- This is an automated message from the

[jira] [Commented] (HADOOP-19114) upgrade to commons-compress 1.26.1 due to cves

2024-03-26 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-19114?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17831038#comment-17831038 ] ASF GitHub Bot commented on HADOOP-19114: - steveloughran commented on PR #6636: URL:

Re: [PR] HADOOP-19114. Upgrade to commons-compress 1.26.1 due to CVEs. [hadoop]

2024-03-26 Thread via GitHub
steveloughran commented on PR #6636: URL: https://github.com/apache/hadoop/pull/6636#issuecomment-2021221194 not a happy build ``` [ERROR] Failed to execute goal org.apache.maven.plugins:maven-compiler-plugin:3.10.1:compile (default-compile) on project

Re: [PR] HDFS-17410. [FGL] Client RPCs that changes file attributes supports fine-grained lock [hadoop]

2024-03-26 Thread via GitHub
hadoop-yetus commented on PR #6634: URL: https://github.com/apache/hadoop/pull/6634#issuecomment-2021155840 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok: |

[jira] [Resolved] (HADOOP-19047) Support InMemory Tracking Of S3A Magic Commits

2024-03-26 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-19047?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran resolved HADOOP-19047. - Fix Version/s: 3.5.0 3.4.1 Resolution: Fixed > Support

[jira] [Commented] (HADOOP-19096) [ABFS] Enhancing Client-Side Throttling Metrics Updation Logic

2024-03-26 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-19096?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17831006#comment-17831006 ] ASF GitHub Bot commented on HADOOP-19096: - rakeshadr commented on code in PR #6276: URL:

Re: [PR] HADOOP-19096: [ABFS] [CST Optimization] Enhancing Client-Side Throttling Metrics Updating Logic [hadoop]

2024-03-26 Thread via GitHub
rakeshadr commented on code in PR #6276: URL: https://github.com/apache/hadoop/pull/6276#discussion_r1539600695 ## hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azurebfs/services/retryReasonCategories/ServerErrorRetryReason.java: ## @@ -56,10 +58,14 @@ String

[jira] [Commented] (HADOOP-19047) Support InMemory Tracking Of S3A Magic Commits

2024-03-26 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-19047?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17831004#comment-17831004 ] ASF GitHub Bot commented on HADOOP-19047: - hadoop-yetus commented on PR #6468: URL:

Re: [PR] HADOOP-19047: Support InMemory Tracking Of S3A Magic Commits [hadoop]

2024-03-26 Thread via GitHub
hadoop-yetus commented on PR #6468: URL: https://github.com/apache/hadoop/pull/6468#issuecomment-2020947224 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok: |

Re: [PR] YARN-11663. [Federation] Add Cache Entity Nums Limit. [hadoop]

2024-03-26 Thread via GitHub
dineshchitlangia commented on code in PR #6662: URL: https://github.com/apache/hadoop/pull/6662#discussion_r1539567214 ## hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/conf/YarnConfiguration.java: ## @@ -4031,6 +4031,11 @@ public static

[PR] HDFS-17439. Support -nonSuperUser for NNThroughputBenchmark: useful f… [hadoop]

2024-03-26 Thread via GitHub
fateh288 opened a new pull request, #6677: URL: https://github.com/apache/hadoop/pull/6677 …or testing auth frameworks such as Ranger ### Description of PR ### How was this patch tested? 1. Unit tests defined in TestNNThroughputBenchmark worked after the patch

Re: [PR] HDFS-17412. [FGL] Client RPCs involving maintenance supports fine-grained lock [hadoop]

2024-03-26 Thread via GitHub
hadoop-yetus commented on PR #6667: URL: https://github.com/apache/hadoop/pull/6667#issuecomment-2020823299 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok: |

[jira] [Commented] (HADOOP-19047) Support InMemory Tracking Of S3A Magic Commits

2024-03-26 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-19047?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17830988#comment-17830988 ] ASF GitHub Bot commented on HADOOP-19047: - steveloughran merged PR #6468: URL:

Re: [PR] HADOOP-19047: Support InMemory Tracking Of S3A Magic Commits [hadoop]

2024-03-26 Thread via GitHub
steveloughran merged PR #6468: URL: https://github.com/apache/hadoop/pull/6468 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

Re: [PR] HDFS-17412. [FGL] Client RPCs involving maintenance supports fine-grained lock [hadoop]

2024-03-26 Thread via GitHub
hadoop-yetus commented on PR #6667: URL: https://github.com/apache/hadoop/pull/6667#issuecomment-2020685124 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok: |

Re: [PR] YARN-11387. [GPG] YARN GPG mistakenly deleted applicationid. [hadoop]

2024-03-26 Thread via GitHub
slfan1989 commented on code in PR #6660: URL: https://github.com/apache/hadoop/pull/6660#discussion_r1539407981 ##

Re: [PR] YARN-11663. [Federation] Add Cache Entity Nums Limit. [hadoop]

2024-03-26 Thread via GitHub
slfan1989 commented on PR #6662: URL: https://github.com/apache/hadoop/pull/6662#issuecomment-2020639828 @goiri Can you help review this PR? Thank you very much! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL

[jira] [Commented] (HADOOP-19072) S3A: expand optimisations on stores with "fs.s3a.create.performance"

2024-03-26 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-19072?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17830962#comment-17830962 ] ASF GitHub Bot commented on HADOOP-19072: - hadoop-yetus commented on PR #6543: URL:

Re: [PR] HADOOP-19072. S3A: expand optimisations on stores with "fs.s3a.create.performance" [hadoop]

2024-03-26 Thread via GitHub
hadoop-yetus commented on PR #6543: URL: https://github.com/apache/hadoop/pull/6543#issuecomment-2020614729 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok: |

Re: [PR] HDFS-17415. [FGL] RPCs in NamenodeProtocol support fine-grained lock [hadoop]

2024-03-26 Thread via GitHub
hadoop-yetus commented on PR #6654: URL: https://github.com/apache/hadoop/pull/6654#issuecomment-2020574709 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok: |

Re: [PR] HADOOP-19047: Support InMemory Tracking Of S3A Magic Commits [hadoop]

2024-03-26 Thread via GitHub
shameersss1 commented on PR #6468: URL: https://github.com/apache/hadoop/pull/6468#issuecomment-2020548007 @steveloughran - I have addressed your comments. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL

[jira] [Commented] (HADOOP-19047) Support InMemory Tracking Of S3A Magic Commits

2024-03-26 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-19047?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17830950#comment-17830950 ] ASF GitHub Bot commented on HADOOP-19047: - shameersss1 commented on PR #6468: URL:

Re: [PR] [ABFS] Test Fixes [hadoop]

2024-03-26 Thread via GitHub
hadoop-yetus commented on PR #6676: URL: https://github.com/apache/hadoop/pull/6676#issuecomment-2020457859 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok: |

Re: [PR] YARN-11261 Avoid unnecessary reconstruction of ConfigurationProperties [hadoop]

2024-03-26 Thread via GitHub
hadoop-yetus commented on PR #6652: URL: https://github.com/apache/hadoop/pull/6652#issuecomment-2020426986 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok: |

[jira] [Commented] (HADOOP-19116) update to zookeeper client 3.8.4 due to CVE-2024-23944

2024-03-26 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-19116?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17830925#comment-17830925 ] ASF GitHub Bot commented on HADOOP-19116: - hadoop-yetus commented on PR #6675: URL:

Re: [PR] HADOOP-19116. Update to zookeeper client 3.8.4 due to CVE-2024-23944 [hadoop]

2024-03-26 Thread via GitHub
hadoop-yetus commented on PR #6675: URL: https://github.com/apache/hadoop/pull/6675#issuecomment-2020394416 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok: |

[jira] [Commented] (HADOOP-19124) Update org.ehcache from 3.3.1 to 3.8.2.

2024-03-26 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-19124?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17830912#comment-17830912 ] ASF GitHub Bot commented on HADOOP-19124: - hadoop-yetus commented on PR #6665: URL:

Re: [PR] HADOOP-19124. Update org.ehcache from 3.3.1 to 3.8.2. [hadoop]

2024-03-26 Thread via GitHub
hadoop-yetus commented on PR #6665: URL: https://github.com/apache/hadoop/pull/6665#issuecomment-2020301592 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok: |

Re: [PR] HDFS-17368. HA: Standby should exit safemode when resources are available. [hadoop]

2024-03-26 Thread via GitHub
zhuzilong2013 commented on PR #6518: URL: https://github.com/apache/hadoop/pull/6518#issuecomment-2020272541 Thanks @Hexiaoqiao for your review and merge~ -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above

[jira] [Commented] (HADOOP-19047) Support InMemory Tracking Of S3A Magic Commits

2024-03-26 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-19047?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17830903#comment-17830903 ] ASF GitHub Bot commented on HADOOP-19047: - hadoop-yetus commented on PR #6468: URL:

Re: [PR] HADOOP-19047: Support InMemory Tracking Of S3A Magic Commits [hadoop]

2024-03-26 Thread via GitHub
hadoop-yetus commented on PR #6468: URL: https://github.com/apache/hadoop/pull/6468#issuecomment-2020230242 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok: |

[jira] [Commented] (HADOOP-19098) Vector IO: consistent specified rejection of overlapping ranges

2024-03-26 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-19098?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17830898#comment-17830898 ] ASF GitHub Bot commented on HADOOP-19098: - steveloughran commented on code in PR #6604: URL:

Re: [PR] HADOOP-19098 Vector IO: consistent specified rejection of overlapping ranges [hadoop]

2024-03-26 Thread via GitHub
steveloughran commented on code in PR #6604: URL: https://github.com/apache/hadoop/pull/6604#discussion_r1539064441 ## hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/VectoredReadUtils.java: ## @@ -133,26 +172,42 @@ private static void

[jira] [Commented] (HADOOP-19047) Support InMemory Tracking Of S3A Magic Commits

2024-03-26 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-19047?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17830894#comment-17830894 ] ASF GitHub Bot commented on HADOOP-19047: - shameersss1 commented on code in PR #6468: URL:

Re: [PR] HADOOP-19047: Support InMemory Tracking Of S3A Magic Commits [hadoop]

2024-03-26 Thread via GitHub
shameersss1 commented on code in PR #6468: URL: https://github.com/apache/hadoop/pull/6468#discussion_r1539051935 ## hadoop-tools/hadoop-aws/src/main/java/org/apache/hadoop/fs/s3a/commit/impl/CommitOperations.java: ## @@ -584,7 +584,7 @@ public SinglePendingCommit

[jira] [Updated] (HADOOP-18980) S3A credential provider remapping: make extensible

2024-03-26 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18980?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran updated HADOOP-18980: Description: A new option fs.s3a.aws.credentials.provider.mapping takes a key value pair

Re: [PR] HADOOP-18980. Invalid inputs for getTrimmedStringCollectionSplitByEquals (ADDENDUM) [hadoop]

2024-03-26 Thread via GitHub
steveloughran commented on PR #6546: URL: https://github.com/apache/hadoop/pull/6546#issuecomment-2020161193 in trunk; testing cherrypick to 3.4 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to

Re: [PR] HADOOP-18980. Invalid inputs for getTrimmedStringCollectionSplitByEquals (ADDENDUM) [hadoop]

2024-03-26 Thread via GitHub
steveloughran merged PR #6546: URL: https://github.com/apache/hadoop/pull/6546 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

[jira] [Commented] (HADOOP-19047) Support InMemory Tracking Of S3A Magic Commits

2024-03-26 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-19047?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17830882#comment-17830882 ] ASF GitHub Bot commented on HADOOP-19047: - steveloughran commented on code in PR #6468: URL:

Re: [PR] HADOOP-19047: Support InMemory Tracking Of S3A Magic Commits [hadoop]

2024-03-26 Thread via GitHub
steveloughran commented on code in PR #6468: URL: https://github.com/apache/hadoop/pull/6468#discussion_r1539027620 ## hadoop-tools/hadoop-aws/src/main/java/org/apache/hadoop/fs/s3a/commit/impl/CommitOperations.java: ## @@ -584,7 +584,7 @@ public SinglePendingCommit

[PR] [ABFS] Test Fixes [hadoop]

2024-03-26 Thread via GitHub
anujmodi2021 opened a new pull request, #6676: URL: https://github.com/apache/hadoop/pull/6676 ### Description of PR ### How was this patch tested? ### For code changes: - [ ] Does the title or this PR starts with the corresponding JIRA issue id (e.g.

[jira] [Updated] (HADOOP-19129) ABFS: Fixing Test Script Bug and Some Known test Failures in ABFS Test Suite

2024-03-26 Thread Anuj Modi (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-19129?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Anuj Modi updated HADOOP-19129: --- Description: Test Script used by ABFS to validate changes has following two issues: # When there

Re: [PR] HDFS-17410. [FGL] Client RPCs that changes file attributes supports fine-grained lock [hadoop]

2024-03-26 Thread via GitHub
hadoop-yetus commented on PR #6634: URL: https://github.com/apache/hadoop/pull/6634#issuecomment-2020088958 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok: |

Re: [PR] HDFS-17408:Reduce the number of quota calculations in FSDirRenameOp [hadoop]

2024-03-26 Thread via GitHub
Hexiaoqiao commented on code in PR #6653: URL: https://github.com/apache/hadoop/pull/6653#discussion_r1538927685 ## hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSDirRenameOp.java: ## @@ -681,14 +703,46 @@ private static class

[jira] [Updated] (HADOOP-19129) ABFS: Fixing Test Script Bug and Some Known test Failures in ABFS Test Suite

2024-03-26 Thread Anuj Modi (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-19129?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Anuj Modi updated HADOOP-19129: --- Description: Test Script used by ABFS to validate changes has following two issues: # When there

[jira] [Updated] (HADOOP-19129) ABFS: Fixing Test Script Bug and Some Known test Failures in ABFS Test Suite

2024-03-26 Thread Anuj Modi (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-19129?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Anuj Modi updated HADOOP-19129: --- Description: Test Script used by ABFS to validate changes has following two issues: # When there

[jira] [Updated] (HADOOP-19129) ABFS: Fixing Test Script Bug and Some Known test Failures in ABFS Test Suite

2024-03-26 Thread Anuj Modi (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-19129?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Anuj Modi updated HADOOP-19129: --- Description: Test Script used by ABFS to validate changes has following two issues: # When there

Re: [PR] HDFS-17408:Reduce the number of quota calculations in FSDirRenameOp [hadoop]

2024-03-26 Thread via GitHub
ThinkerLei commented on code in PR #6653: URL: https://github.com/apache/hadoop/pull/6653#discussion_r1538895771 ## hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSDirRenameOp.java: ## @@ -681,14 +703,46 @@ private static class

Re: [PR] HADOOP-19047: Support InMemory Tracking Of S3A Magic Commits [hadoop]

2024-03-26 Thread via GitHub
shameersss1 commented on PR #6468: URL: https://github.com/apache/hadoop/pull/6468#issuecomment-2019942477 @steveloughran - Thanks a lot for the detailed review. I have addressed your comments. > note, you will need to a followup in the docs -but we can get this in and tested while

[jira] [Commented] (HADOOP-19047) Support InMemory Tracking Of S3A Magic Commits

2024-03-26 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-19047?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17830856#comment-17830856 ] ASF GitHub Bot commented on HADOOP-19047: - shameersss1 commented on PR #6468: URL:

  1   2   >