Re: [PR] YARN-11670. Add CallerContext in NodeManager [hadoop]

2024-03-28 Thread via GitHub
yangjiandan commented on PR #6688: URL: https://github.com/apache/hadoop/pull/6688#issuecomment-2026710223 @dineshchitlangia Thank you very much for your review. In my test environment, when running MR jobs, some of the HDFS-audit logs are as follows: ` 2024-03-28 22:31:09,716 INFO

Re: [PR] HDFS-17447. Add logging to print the implementation class of AccessControlEnforcer, print ACEnforcer when AccessControlException occurs. [hadoop]

2024-03-28 Thread via GitHub
hadoop-yetus commented on PR #6687: URL: https://github.com/apache/hadoop/pull/6687#issuecomment-2026691032 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok: |

Re: [PR] Add getJournalSyncerStarted jmx metrics, to Indicates whether the JournalSyncer thread has been started. [hadoop]

2024-03-28 Thread via GitHub
hadoop-yetus commented on PR #6684: URL: https://github.com/apache/hadoop/pull/6684#issuecomment-2026683164 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok: |

Re: [PR] Add getJournalSyncerStarted jmx metrics, to Indicates whether the JournalSyncer thread has been started. [hadoop]

2024-03-28 Thread via GitHub
slfan1989 commented on PR #6684: URL: https://github.com/apache/hadoop/pull/6684#issuecomment-2026626932 @xiaojunxiang2023 Thanks for the contribution! LGTM. We need to check the hadoop.hdfs.qjournal.server.TestJournalNodeMXBean unit test, it seems to be related to the

Re: [PR] HADOOP-19124. Update org.ehcache from 3.3.1 to 3.8.2. [hadoop]

2024-03-28 Thread via GitHub
slfan1989 commented on PR #6665: URL: https://github.com/apache/hadoop/pull/6665#issuecomment-2026622450 @dineshchitlangia Thanks for the review and merge! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above

[jira] [Commented] (HADOOP-19124) Update org.ehcache from 3.3.1 to 3.8.2.

2024-03-28 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-19124?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17832034#comment-17832034 ] ASF GitHub Bot commented on HADOOP-19124: - slfan1989 commented on PR #6665: URL:

Re: [PR] Add getJournalSyncerStarted jmx metrics, to Indicates whether the JournalSyncer thread has been started. [hadoop]

2024-03-28 Thread via GitHub
hadoop-yetus commented on PR #6684: URL: https://github.com/apache/hadoop/pull/6684#issuecomment-2026619693 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok: |

Re: [PR] YARN-11670. Add CallerContext in NodeManager [hadoop]

2024-03-28 Thread via GitHub
hadoop-yetus commented on PR #6688: URL: https://github.com/apache/hadoop/pull/6688#issuecomment-2026618787 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok: |

Re: [PR] HDFS-17446. The DataNode adds a log to report its progress during addToReplicasMap execution. [hadoop]

2024-03-28 Thread via GitHub
hadoop-yetus commented on PR #6685: URL: https://github.com/apache/hadoop/pull/6685#issuecomment-2026565855 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok: |

[jira] [Resolved] (HADOOP-19041) further use of StandardCharsets

2024-03-28 Thread Dinesh Chitlangia (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-19041?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dinesh Chitlangia resolved HADOOP-19041. Fix Version/s: 3.5.0 Assignee: PJ Fanning Resolution: Fixed

Re: [PR] HADOOP-19041. Use StandardCharsets in more places [hadoop]

2024-03-28 Thread via GitHub
dineshchitlangia merged PR #6449: URL: https://github.com/apache/hadoop/pull/6449 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

Re: [PR] HADOOP-19041. Use StandardCharsets in more places [hadoop]

2024-03-28 Thread via GitHub
dineshchitlangia commented on PR #6449: URL: https://github.com/apache/hadoop/pull/6449#issuecomment-2026552685 @pjfanning thanks for your contribution. Do you mind filing a jira for the 2 spotbugs raised in the CI here? -- This is an automated message from the Apache Git Service. To

Re: [PR] HDFS-17447. Add logging to print the implementation class of AccessControlEnforcer, print ACEnforcer when AccessControlException occurs. [hadoop]

2024-03-28 Thread via GitHub
xiaojunxiang2023 closed pull request #6687: HDFS-17447. Add logging to print the implementation class of AccessControlEnforcer, print ACEnforcer when AccessControlException occurs. URL: https://github.com/apache/hadoop/pull/6687 -- This is an automated message from the Apache Git Service.

[PR] YARN-11670. Add CallerContext in NodeManager [hadoop]

2024-03-28 Thread via GitHub
yangjiandan opened a new pull request, #6688: URL: https://github.com/apache/hadoop/pull/6688 ### Description of PR Currently, MR and Spark have added caller context, enabling tracing of HDFS/ResourceManager operators from Spark apps and MapReduce apps. However, operators from

[jira] [Commented] (HADOOP-19124) Update org.ehcache from 3.3.1 to 3.8.2.

2024-03-28 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-19124?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17832015#comment-17832015 ] ASF GitHub Bot commented on HADOOP-19124: - dineshchitlangia merged PR #6665: URL:

[jira] [Resolved] (HADOOP-19124) Update org.ehcache from 3.3.1 to 3.8.2.

2024-03-28 Thread Dinesh Chitlangia (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-19124?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dinesh Chitlangia resolved HADOOP-19124. Fix Version/s: 3.5.0 Resolution: Fixed Thanks [~slfan1989] for the

[jira] [Commented] (HADOOP-19124) Update org.ehcache from 3.3.1 to 3.8.2.

2024-03-28 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-19124?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17832014#comment-17832014 ] ASF GitHub Bot commented on HADOOP-19124: - dineshchitlangia commented on PR #6665: URL:

Re: [PR] HADOOP-19124. Update org.ehcache from 3.3.1 to 3.8.2. [hadoop]

2024-03-28 Thread via GitHub
dineshchitlangia merged PR #6665: URL: https://github.com/apache/hadoop/pull/6665 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

Re: [PR] HADOOP-19124. Update org.ehcache from 3.3.1 to 3.8.2. [hadoop]

2024-03-28 Thread via GitHub
dineshchitlangia commented on PR #6665: URL: https://github.com/apache/hadoop/pull/6665#issuecomment-2026465277 Thanks @slfan1989 for the contribution -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to

Re: [PR] YARN-11668. Fix RM crash for potential concurrent modification exception when updating node attributes [hadoop]

2024-03-28 Thread via GitHub
slfan1989 commented on PR #6681: URL: https://github.com/apache/hadoop/pull/6681#issuecomment-2026455866 @zuston Thanks for the contribution! merged into trunk. @dineshchitlangia Thanks for the review! -- This is an automated message from the Apache Git Service. To respond to the

Re: [PR] YARN-11668. Fix RM crash for potential concurrent modification exception when updating node attributes [hadoop]

2024-03-28 Thread via GitHub
slfan1989 merged PR #6681: URL: https://github.com/apache/hadoop/pull/6681 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

Re: [PR] HADOOP-19115. Upgrade to nimbus-jose-jwt 9.37.2 due to CVE-2023-52428. (#6637) [hadoop]

2024-03-28 Thread via GitHub
slfan1989 commented on PR #6682: URL: https://github.com/apache/hadoop/pull/6682#issuecomment-2026453796 @pjfanning Thanks for the contribution! @dineshchitlangia Thanks for the review! -- This is an automated message from the Apache Git Service. To respond to the message, please log on

Re: [PR] HADOOP-19124. Update org.ehcache from 3.3.1 to 3.8.2. [hadoop]

2024-03-28 Thread via GitHub
slfan1989 commented on PR #6665: URL: https://github.com/apache/hadoop/pull/6665#issuecomment-2026452247 @dineshchitlangia Can you help review this pr again? Thank you very much! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to

Re: [PR] HADOOP-19115. Upgrade to nimbus-jose-jwt 9.37.2 due to CVE-2023-52428. (#6637) [hadoop]

2024-03-28 Thread via GitHub
slfan1989 merged PR #6682: URL: https://github.com/apache/hadoop/pull/6682 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

[jira] [Commented] (HADOOP-19124) Update org.ehcache from 3.3.1 to 3.8.2.

2024-03-28 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-19124?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17832008#comment-17832008 ] ASF GitHub Bot commented on HADOOP-19124: - slfan1989 commented on PR #6665: URL:

[PR] HDFS-17447. Add an implementation class that prints AccessControlEnforcer when an ACE exception occurs. [hadoop]

2024-03-28 Thread via GitHub
xiaojunxiang2023 opened a new pull request, #6687: URL: https://github.com/apache/hadoop/pull/6687 As we know, when Ranger authentication is enabled in the HDFS, if Ranger authentication fails to match, it is degraded to the FSPermissionChecker authentication of the HDFS ACL. If an

Re: [PR] HDFS-17446. The DataNode adds a log to report its progress during addToReplicasMap execution. [hadoop]

2024-03-28 Thread via GitHub
hadoop-yetus commented on PR #6685: URL: https://github.com/apache/hadoop/pull/6685#issuecomment-2026311691 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok: |

Re: [PR] HDFS-17446. The DataNode adds a log to report its progress during addToReplicasMap execution. [hadoop]

2024-03-28 Thread via GitHub
hadoop-yetus commented on PR #6685: URL: https://github.com/apache/hadoop/pull/6685#issuecomment-2026305411 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok: |

[jira] [Commented] (HADOOP-18508) support multiple s3a integration test runs on same bucket in parallel

2024-03-28 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18508?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17831983#comment-17831983 ] ASF GitHub Bot commented on HADOOP-18508: - hadoop-yetus commented on PR #5081: URL:

Re: [PR] HADOOP-18508. Support multiple s3a integration test runs on same bucket in parallel [hadoop]

2024-03-28 Thread via GitHub
hadoop-yetus commented on PR #5081: URL: https://github.com/apache/hadoop/pull/5081#issuecomment-2026154983 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok: |

[jira] [Commented] (HADOOP-19131) Assist reflection IO with WrappedOperations class

2024-03-28 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-19131?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17831967#comment-17831967 ] ASF GitHub Bot commented on HADOOP-19131: - hadoop-yetus commented on PR #6686: URL:

Re: [PR] HADOOP-19131. Assist reflection IO with WrappedOperations class [hadoop]

2024-03-28 Thread via GitHub
hadoop-yetus commented on PR #6686: URL: https://github.com/apache/hadoop/pull/6686#issuecomment-2026084008 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok: |

[jira] [Commented] (HADOOP-19107) Drop support for HBase v1

2024-03-28 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-19107?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17831958#comment-17831958 ] ASF GitHub Bot commented on HADOOP-19107: - hadoop-yetus commented on PR #6629: URL:

Re: [PR] HADOOP-19107. Drop support for HBase v1 & upgrade HBase v2 [hadoop]

2024-03-28 Thread via GitHub
hadoop-yetus commented on PR #6629: URL: https://github.com/apache/hadoop/pull/6629#issuecomment-2026043729 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok: |

[jira] [Commented] (HADOOP-18508) support multiple s3a integration test runs on same bucket in parallel

2024-03-28 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18508?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17831925#comment-17831925 ] ASF GitHub Bot commented on HADOOP-18508: - hadoop-yetus commented on PR #5081: URL:

Re: [PR] HADOOP-18508. Support multiple s3a integration test runs on same bucket in parallel [hadoop]

2024-03-28 Thread via GitHub
hadoop-yetus commented on PR #5081: URL: https://github.com/apache/hadoop/pull/5081#issuecomment-2025872096 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok: |

Re: [PR] Add getJournalSyncerStarted jmx metrics, to Indicates whether the JournalSyncer thread has been started. [hadoop]

2024-03-28 Thread via GitHub
hadoop-yetus commented on PR #6684: URL: https://github.com/apache/hadoop/pull/6684#issuecomment-2025857179 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok: |

Re: [PR] HDFS-17389. [FGL] Client RPCs involving read process supports fine-grained lock [hadoop]

2024-03-28 Thread via GitHub
hadoop-yetus commented on PR #6590: URL: https://github.com/apache/hadoop/pull/6590#issuecomment-2025749407 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok: |

Re: [PR] HDFS-17388. [FGL] Client RPCs involving write process supports fine-grained lock [hadoop]

2024-03-28 Thread via GitHub
hadoop-yetus commented on PR #6589: URL: https://github.com/apache/hadoop/pull/6589#issuecomment-2025736207 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok: |

[jira] [Commented] (HADOOP-19072) S3A: expand optimisations on stores with "fs.s3a.create.performance"

2024-03-28 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-19072?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17831907#comment-17831907 ] ASF GitHub Bot commented on HADOOP-19072: - steveloughran commented on PR #6543: URL:

Re: [PR] HADOOP-19079. Check class is an exception class before constructing an instance [hadoop]

2024-03-28 Thread via GitHub
pjfanning commented on code in PR #6557: URL: https://github.com/apache/hadoop/pull/6557#discussion_r1543349031 ## hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/util/TestHttpExceptionUtils.java: ## @@ -87,35 +89,31 @@ public void testValidateResponseOK()

[jira] [Commented] (HADOOP-19079) check that class that is loaded is really an exception

2024-03-28 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-19079?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17831906#comment-17831906 ] ASF GitHub Bot commented on HADOOP-19079: - pjfanning commented on code in PR #6557: URL:

[jira] [Commented] (HADOOP-19072) S3A: expand optimisations on stores with "fs.s3a.create.performance"

2024-03-28 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-19072?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17831904#comment-17831904 ] ASF GitHub Bot commented on HADOOP-19072: - steveloughran commented on code in PR #6543: URL:

Re: [PR] HADOOP-19072. S3A: expand optimisations on stores with "fs.s3a.create.performance" [hadoop]

2024-03-28 Thread via GitHub
steveloughran commented on PR #6543: URL: https://github.com/apache/hadoop/pull/6543#issuecomment-2025732364 reviewed, i'm just wondering how to make the test the cleanest. Going to invite reviews from @shameersss1 @ahmarsuhail @HarshitGupta11 @mukund-thakur as they've been looking

[jira] [Commented] (HADOOP-19077) remove use of javax.ws.rs.core.HttpHeaders

2024-03-28 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-19077?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17831905#comment-17831905 ] ASF GitHub Bot commented on HADOOP-19077: - hadoop-yetus commented on PR #6554: URL:

Re: [PR] HADOOP-19077. Remove use of javax.ws.rs.core.HttpHeaders [hadoop]

2024-03-28 Thread via GitHub
hadoop-yetus commented on PR #6554: URL: https://github.com/apache/hadoop/pull/6554#issuecomment-2025729834 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok: |

Re: [PR] HADOOP-19072. S3A: expand optimisations on stores with "fs.s3a.create.performance" [hadoop]

2024-03-28 Thread via GitHub
steveloughran commented on code in PR #6543: URL: https://github.com/apache/hadoop/pull/6543#discussion_r1543344114 ## hadoop-tools/hadoop-aws/src/test/java/org/apache/hadoop/fs/contract/s3a/ITestS3AContractMkdirWithCreatePerf.java: ## @@ -0,0 +1,69 @@ +/* + * Licensed to the

[jira] [Commented] (HADOOP-19079) check that class that is loaded is really an exception

2024-03-28 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-19079?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17831902#comment-17831902 ] ASF GitHub Bot commented on HADOOP-19079: - steveloughran commented on code in PR #6557: URL:

Re: [PR] HADOOP-19079. Check class is an exception class before constructing an instance [hadoop]

2024-03-28 Thread via GitHub
steveloughran commented on code in PR #6557: URL: https://github.com/apache/hadoop/pull/6557#discussion_r1543332155 ## hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/util/TestHttpExceptionUtils.java: ## @@ -87,35 +89,31 @@ public void

Re: [PR] YARN-11664: Remove HDFS Binaries/Jars Dependency From Yarn [hadoop]

2024-03-28 Thread via GitHub
steveloughran commented on PR #6631: URL: https://github.com/apache/hadoop/pull/6631#issuecomment-2025705847 waiting to see what hdfs people say; mentioned internally. now, there is a way to do this with a smaller diff, specifically, move the IOPair class into hadoop common *but keep

[jira] [Commented] (HADOOP-18184) s3a prefetching stream to support unbuffer()

2024-03-28 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18184?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17831900#comment-17831900 ] ASF GitHub Bot commented on HADOOP-18184: - hadoop-yetus commented on PR #5832: URL:

Re: [PR] HADOOP-18184. S3A Prefetching unbuffer. [hadoop]

2024-03-28 Thread via GitHub
hadoop-yetus commented on PR #5832: URL: https://github.com/apache/hadoop/pull/5832#issuecomment-2025705076 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok: |

[jira] [Commented] (HADOOP-18135) Produce Windows binaries of Hadoop

2024-03-28 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18135?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17831899#comment-17831899 ] ASF GitHub Bot commented on HADOOP-18135: - steveloughran commented on PR #6673: URL:

Re: [PR] HADOOP-18135. Produce Windows binaries of Hadoop [hadoop]

2024-03-28 Thread via GitHub
steveloughran commented on PR #6673: URL: https://github.com/apache/hadoop/pull/6673#issuecomment-2025701608 docs are good, but still shellcheck issues ``` dev-support/bin/create-release:540:1: note: The mentioned syntax error was in this brace group. [SC1009]

Re: [PR] HADOOP-18679. Add API for bulk/paged object deletion [hadoop]

2024-03-28 Thread via GitHub
steveloughran commented on PR #6494: URL: https://github.com/apache/hadoop/pull/6494#issuecomment-2025696873 In #6686 I'm creating a new utils class for reflection access, nothing else. And proposing that all tests of the API use reflection to be really confident it works and that there's

[jira] [Commented] (HADOOP-18679) Add API for bulk/paged object deletion

2024-03-28 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18679?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17831898#comment-17831898 ] ASF GitHub Bot commented on HADOOP-18679: - steveloughran commented on PR #6494: URL:

[jira] [Updated] (HADOOP-19131) Assist reflection IO with WrappedOperations class

2024-03-28 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-19131?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran updated HADOOP-19131: Summary: Assist reflection IO with WrappedOperations class (was: Assist reflection iO

[jira] [Updated] (HADOOP-19131) Assist reflection IO with WrappedOperations class

2024-03-28 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-19131?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated HADOOP-19131: Labels: pull-request-available (was: ) > Assist reflection IO with WrappedOperations

[jira] [Commented] (HADOOP-19131) Assist reflection IO with WrappedOperations class

2024-03-28 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-19131?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17831896#comment-17831896 ] ASF GitHub Bot commented on HADOOP-19131: - steveloughran opened a new pull request, #6686: URL:

[jira] [Updated] (HADOOP-19131) Assist reflection IO with WrappedOperations class

2024-03-28 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-19131?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran updated HADOOP-19131: Description: parquet, avro etc are still stuck building with older hadoop releases.

[jira] [Created] (HADOOP-19131) Assist reflection iO with WrappedOperations class

2024-03-28 Thread Steve Loughran (Jira)
Steve Loughran created HADOOP-19131: --- Summary: Assist reflection iO with WrappedOperations class Key: HADOOP-19131 URL: https://issues.apache.org/jira/browse/HADOOP-19131 Project: Hadoop Common

[jira] [Assigned] (HADOOP-19131) Assist reflection iO with WrappedOperations class

2024-03-28 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-19131?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran reassigned HADOOP-19131: --- Assignee: Steve Loughran > Assist reflection iO with WrappedOperations class >

[PR] HDFS-17446. The DataNode adds a log to report its progress during addToReplicasMap execution. [hadoop]

2024-03-28 Thread via GitHub
xiaojunxiang2023 opened a new pull request, #6685: URL: https://github.com/apache/hadoop/pull/6685 Datanodes do not print logs when addToReplicasMap is just started, so we will not be aware of its progress, or even wait up to 4 hours for a large cluster. This is very confusing. A

[jira] [Commented] (HADOOP-18679) Add API for bulk/paged object deletion

2024-03-28 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18679?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17831878#comment-17831878 ] ASF GitHub Bot commented on HADOOP-18679: - steveloughran commented on PR #6494: URL:

Re: [PR] HADOOP-18679. Add API for bulk/paged object deletion [hadoop]

2024-03-28 Thread via GitHub
steveloughran commented on PR #6494: URL: https://github.com/apache/hadoop/pull/6494#issuecomment-2025591287 FYI i want to pull the rate limiter API of #6596 in here too; we'd have a rate limiter in s3a store which if enabled would limit #of deletes which can be issued on a bucket. Ideally

[jira] [Commented] (HADOOP-19093) Improve rate limiting through ABFS in Manifest Committer

2024-03-28 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-19093?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17831875#comment-17831875 ] ASF GitHub Bot commented on HADOOP-19093: - hadoop-yetus commented on PR #6596: URL:

Re: [PR] HADOOP-19093. [ABFS] Improve rate limiting through ABFS in Manifest Committer [hadoop]

2024-03-28 Thread via GitHub
hadoop-yetus commented on PR #6596: URL: https://github.com/apache/hadoop/pull/6596#issuecomment-2025587698 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok: |

[jira] [Commented] (HADOOP-19072) S3A: expand optimisations on stores with "fs.s3a.create.performance"

2024-03-28 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-19072?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17831874#comment-17831874 ] ASF GitHub Bot commented on HADOOP-19072: - virajjasani commented on PR #6543: URL:

Re: [PR] HADOOP-19072. S3A: expand optimisations on stores with "fs.s3a.create.performance" [hadoop]

2024-03-28 Thread via GitHub
virajjasani commented on PR #6543: URL: https://github.com/apache/hadoop/pull/6543#issuecomment-2025582097 @steveloughran could you please take another look? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL

Re: [PR] HDFS-17389. [FGL] Client RPCs involving read process supports fine-grained lock [hadoop]

2024-03-28 Thread via GitHub
hadoop-yetus commented on PR #6590: URL: https://github.com/apache/hadoop/pull/6590#issuecomment-2025507325 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok: |

Re: [PR] HADOOP-19129: [ABFS] Test Fixes and Test Script Bug Fixes [hadoop]

2024-03-28 Thread via GitHub
hadoop-yetus commented on PR #6676: URL: https://github.com/apache/hadoop/pull/6676#issuecomment-2025498665 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok: |

[jira] [Commented] (HADOOP-19129) ABFS: Fixing Test Script Bug and Some Known test Failures in ABFS Test Suite

2024-03-28 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-19129?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17831865#comment-17831865 ] ASF GitHub Bot commented on HADOOP-19129: - hadoop-yetus commented on PR #6676: URL:

[jira] [Commented] (HADOOP-19124) Update org.ehcache from 3.3.1 to 3.8.2.

2024-03-28 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-19124?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17831864#comment-17831864 ] ASF GitHub Bot commented on HADOOP-19124: - hadoop-yetus commented on PR #6665: URL:

Re: [PR] HADOOP-19124. Update org.ehcache from 3.3.1 to 3.8.2. [hadoop]

2024-03-28 Thread via GitHub
hadoop-yetus commented on PR #6665: URL: https://github.com/apache/hadoop/pull/6665#issuecomment-2025490497 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok: |

Re: [PR] HDFS-17443. add null check for fileSys and cluster in TestNameEditsConfigs#testNameEditsConfigsFailure before shutting down [hadoop]

2024-03-28 Thread via GitHub
dineshchitlangia merged PR #6683: URL: https://github.com/apache/hadoop/pull/6683 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

Re: [PR] YARN-11582. Improve WebUI diagnosticMessage to show AM Container resource request size. [hadoop]

2024-03-28 Thread via GitHub
hiwangzhihui commented on code in PR #6139: URL: https://github.com/apache/hadoop/pull/6139#discussion_r1542224470 ##

Re: [PR] HDFS-17216. Distcp: When handle the small files, the bandwidth parameter will be invalid, fix this bug. [hadoop]

2024-03-28 Thread via GitHub
xiaojunxiang2023 commented on PR #6138: URL: https://github.com/apache/hadoop/pull/6138#issuecomment-2025369238 > +1 LGTM, thanks @xiaojunxiang2023 for the contribution and @hiwangzhihui for review. @dineshchitlangia @hiwangzhihui thank you very much. -- This is an automated

Re: [PR] HDFS-17216. Distcp: When handle the small files, the bandwidth parameter will be invalid, fix this bug. [hadoop]

2024-03-28 Thread via GitHub
dineshchitlangia merged PR #6138: URL: https://github.com/apache/hadoop/pull/6138 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

Re: [PR] HDFS-17216. Distcp: When handle the small files, the bandwidth parameter will be invalid, fix this bug. [hadoop]

2024-03-28 Thread via GitHub
hiwangzhihui commented on PR #6138: URL: https://github.com/apache/hadoop/pull/6138#issuecomment-2025324133 > Overall, LGTM. Pls address the issues from CI run - blank line, checkstyle and review test failures. hi,@dineshchitlangia . Now it has passed the Jenkins Check, please help

[jira] [Commented] (HADOOP-18184) s3a prefetching stream to support unbuffer()

2024-03-28 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18184?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17831814#comment-17831814 ] ASF GitHub Bot commented on HADOOP-18184: - hadoop-yetus commented on PR #5832: URL:

Re: [PR] HADOOP-18184. S3A Prefetching unbuffer. [hadoop]

2024-03-28 Thread via GitHub
hadoop-yetus commented on PR #5832: URL: https://github.com/apache/hadoop/pull/5832#issuecomment-2025253679 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok: |

[jira] [Updated] (HADOOP-19129) ABFS: Fixing Test Script Bug and Some Known test Failures in ABFS Test Suite

2024-03-28 Thread Anuj Modi (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-19129?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Anuj Modi updated HADOOP-19129: --- Description: Test Script used by ABFS to validate changes has following two issues: # When there

[jira] [Updated] (HADOOP-19129) ABFS: Fixing Test Script Bug and Some Known test Failures in ABFS Test Suite

2024-03-28 Thread Anuj Modi (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-19129?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Anuj Modi updated HADOOP-19129: --- Description: Test Script used by ABFS to validate changes has following two issues: # When there

[jira] [Commented] (HADOOP-19120) [ABFS]: ApacheHttpClient adaptation as network library

2024-03-28 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-19120?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17831781#comment-17831781 ] ASF GitHub Bot commented on HADOOP-19120: - anmolanmol1234 commented on code in PR #6633: URL:

Re: [PR] HADOOP-19120. ApacheHttpClient adaptation in ABFS. [hadoop]

2024-03-28 Thread via GitHub
anmolanmol1234 commented on code in PR #6633: URL: https://github.com/apache/hadoop/pull/6633#discussion_r1542941161 ## hadoop-tools/hadoop-azure/src/test/java/org/apache/hadoop/fs/azurebfs/services/TestApacheHttpClientFallback.java: ## @@ -0,0 +1,205 @@ +/** + * Licensed to

[jira] [Commented] (HADOOP-19120) [ABFS]: ApacheHttpClient adaptation as network library

2024-03-28 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-19120?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17831779#comment-17831779 ] ASF GitHub Bot commented on HADOOP-19120: - anmolanmol1234 commented on code in PR #6633: URL:

Re: [PR] HADOOP-19120. ApacheHttpClient adaptation in ABFS. [hadoop]

2024-03-28 Thread via GitHub
anmolanmol1234 commented on code in PR #6633: URL: https://github.com/apache/hadoop/pull/6633#discussion_r1542935703 ## hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azurebfs/services/kac/package-info.java: ## @@ -0,0 +1,22 @@ +/* + * Licensed to the Apache

[jira] [Commented] (HADOOP-19120) [ABFS]: ApacheHttpClient adaptation as network library

2024-03-28 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-19120?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17831778#comment-17831778 ] ASF GitHub Bot commented on HADOOP-19120: - anmolanmol1234 commented on PR #6633: URL:

Re: [PR] HADOOP-19120. ApacheHttpClient adaptation in ABFS. [hadoop]

2024-03-28 Thread via GitHub
anmolanmol1234 commented on PR #6633: URL: https://github.com/apache/hadoop/pull/6633#issuecomment-2025139312 1. Add more tests around verifying the sendTime and receiveTime for ManagedClientContext. 2. Tests around connections in KAC getting stale and returning false. 3. -- This

[jira] [Commented] (HADOOP-19120) [ABFS]: ApacheHttpClient adaptation as network library

2024-03-28 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-19120?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17831772#comment-17831772 ] ASF GitHub Bot commented on HADOOP-19120: - anmolanmol1234 commented on code in PR #6633: URL:

Re: [PR] HADOOP-19120. ApacheHttpClient adaptation in ABFS. [hadoop]

2024-03-28 Thread via GitHub
anmolanmol1234 commented on code in PR #6633: URL: https://github.com/apache/hadoop/pull/6633#discussion_r1542920863 ## hadoop-tools/hadoop-azure/src/test/java/org/apache/hadoop/fs/azurebfs/services/ITestAbfsHttpClientRequestExecutor.java: ## @@ -0,0 +1,178 @@ +/** + * Licensed

Re: [PR] Add getJournalSyncerStarted jmx metrics, to Indicates whether the JournalSyncer thread has been started. [hadoop]

2024-03-28 Thread via GitHub
hadoop-yetus commented on PR #6684: URL: https://github.com/apache/hadoop/pull/6684#issuecomment-2025121120 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok: |

Re: [PR] Add getJournalSyncerStarted jmx metrics, to Indicates whether the JournalSyncer thread has been started. [hadoop]

2024-03-28 Thread via GitHub
hadoop-yetus commented on PR #6684: URL: https://github.com/apache/hadoop/pull/6684#issuecomment-2025110477 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok: |

[jira] [Commented] (HADOOP-18679) Add API for bulk/paged object deletion

2024-03-28 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18679?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17831757#comment-17831757 ] ASF GitHub Bot commented on HADOOP-18679: - ahmarsuhail commented on code in PR #6494: URL:

Re: [PR] HADOOP-18679. Add API for bulk/paged object deletion [hadoop]

2024-03-28 Thread via GitHub
ahmarsuhail commented on code in PR #6494: URL: https://github.com/apache/hadoop/pull/6494#discussion_r1542647008 ## hadoop-common-project/hadoop-common/src/site/markdown/filesystem/bulkdelete.md: ## @@ -0,0 +1,284 @@ + + +# interface `BulkDelete` + + + +The `BulkDelete`

[jira] [Commented] (HADOOP-19120) [ABFS]: ApacheHttpClient adaptation as network library

2024-03-28 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-19120?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17831751#comment-17831751 ] ASF GitHub Bot commented on HADOOP-19120: - anmolanmol1234 commented on code in PR #6633: URL:

[jira] [Commented] (HADOOP-19120) [ABFS]: ApacheHttpClient adaptation as network library

2024-03-28 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-19120?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17831752#comment-17831752 ] ASF GitHub Bot commented on HADOOP-19120: - anmolanmol1234 commented on code in PR #6633: URL:

Re: [PR] HADOOP-19120. ApacheHttpClient adaptation in ABFS. [hadoop]

2024-03-28 Thread via GitHub
anmolanmol1234 commented on code in PR #6633: URL: https://github.com/apache/hadoop/pull/6633#discussion_r1542816451 ## hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azurebfs/services/ApacheHttpClientHealthMonitor.java: ## @@ -0,0 +1,33 @@ +/** + * Licensed to

Re: [PR] HADOOP-19120. ApacheHttpClient adaptation in ABFS. [hadoop]

2024-03-28 Thread via GitHub
anmolanmol1234 commented on code in PR #6633: URL: https://github.com/apache/hadoop/pull/6633#discussion_r1542814746 ## hadoop-tools/hadoop-azure/src/test/java/org/apache/hadoop/fs/azurebfs/ITestAbfsReadWriteAndSeek.java: ## @@ -55,22 +56,47 @@ public class

[jira] [Commented] (HADOOP-19120) [ABFS]: ApacheHttpClient adaptation as network library

2024-03-28 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-19120?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17831746#comment-17831746 ] ASF GitHub Bot commented on HADOOP-19120: - anmolanmol1234 commented on code in PR #6633: URL:

  1   2   >