[PR] YARN-11693. Refactor Container Scheduler [hadoop]

2024-04-30 Thread via GitHub
mohitgaggar opened a new pull request, #6786: URL: https://github.com/apache/hadoop/pull/6786 …, running container manager and container queue manager classes ### Description of PR ### How was this patch tested? ### For code changes: - [x ] Does

[jira] [Commented] (HADOOP-18610) ABFS OAuth2 Token Provider to support Azure Workload Identity for AKS

2024-04-30 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18610?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17842326#comment-17842326 ] ASF GitHub Bot commented on HADOOP-18610: - creste commented on PR #5953: URL:

[PR] MAPREDUCE-7475. Fixed non-idempotent unit tests [hadoop]

2024-04-30 Thread via GitHub
kaiyaok2 opened a new pull request, #6785: URL: https://github.com/apache/hadoop/pull/6785 ### Description of PR As described in https://issues.apache.org/jira/browse/MAPREDUCE-7475: 2 tests are not idempotent and fails upon repeated execution within the same JVM instance due to

[jira] [Commented] (HADOOP-19146) noaa-cors-pds bucket access with global endpoint fails

2024-04-30 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-19146?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17842336#comment-17842336 ] ASF GitHub Bot commented on HADOOP-19146: - steveloughran merged PR #6723: URL:

Re: [PR] HADOOP-19146 noaa-cors-pds bucket access with global endpoint fails [hadoop]

2024-04-30 Thread via GitHub
steveloughran merged PR #6723: URL: https://github.com/apache/hadoop/pull/6723 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

Re: [PR] YARN-11685. Create a config to enable/disable cgroup v2 functionality [hadoop]

2024-04-30 Thread via GitHub
brumi1024 merged PR #6770: URL: https://github.com/apache/hadoop/pull/6770 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

Re: [PR] YARN-11685. Create a config to enable/disable cgroup v2 functionality [hadoop]

2024-04-30 Thread via GitHub
brumi1024 commented on PR #6770: URL: https://github.com/apache/hadoop/pull/6770#issuecomment-2084814001 Thanks @p-szucs for the fixes, the latest state LGTM. Merging to trunk. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub

Re: [PR] HADOOP-18610. ABFS OAuth2 Token Provider support for Azure Workload Identity [hadoop]

2024-04-30 Thread via GitHub
creste commented on PR #5953: URL: https://github.com/apache/hadoop/pull/5953#issuecomment-2084911705 @snvijaya - My team is no longer using Hadoop and has moved on to another project so I am unable to commit to completing this PR. Feel free to address the feedback and make any changes

Re: [PR] HADOOP-19146 noaa-cors-pds bucket access with global endpoint fails [hadoop]

2024-04-30 Thread via GitHub
steveloughran commented on PR #6723: URL: https://github.com/apache/hadoop/pull/6723#issuecomment-2085237232 merged to trunk and 3.4; doesn't go in to 3.3.x and I'm not sure whether its worth the effort -- This is an automated message from the Apache Git Service. To respond to the

[jira] [Commented] (HADOOP-19146) noaa-cors-pds bucket access with global endpoint fails

2024-04-30 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-19146?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17842367#comment-17842367 ] ASF GitHub Bot commented on HADOOP-19146: - steveloughran commented on PR #6723: URL:

Re: [PR] HADOOP-18516: [ABFS][Authentication] Support Fixed SAS Token for ABFS Authentication [hadoop]

2024-04-30 Thread via GitHub
anujmodi2021 commented on PR #6552: URL: https://github.com/apache/hadoop/pull/6552#issuecomment-2085142098 -- AGGREGATED TEST RESULT HNS-OAuth

Re: [PR] YARN-11693. Refactor Container Scheduler [hadoop]

2024-04-30 Thread via GitHub
hadoop-yetus commented on PR #6786: URL: https://github.com/apache/hadoop/pull/6786#issuecomment-2085033047 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok: |

Re: [PR] HADOOP-18516: [ABFS][Authentication] Support Fixed SAS Token for ABFS Authentication [hadoop]

2024-04-30 Thread via GitHub
anujmodi2021 commented on PR #6552: URL: https://github.com/apache/hadoop/pull/6552#issuecomment-2085145286 Thanks @steveloughran for the review. This is good to be merged now. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to

[jira] [Commented] (HADOOP-18516) [ABFS]: Support fixed SAS token config in addition to Custom SASTokenProvider Implementation

2024-04-30 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18516?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17842351#comment-17842351 ] ASF GitHub Bot commented on HADOOP-18516: - anujmodi2021 commented on PR #6552: URL:

[jira] [Commented] (HADOOP-18516) [ABFS]: Support fixed SAS token config in addition to Custom SASTokenProvider Implementation

2024-04-30 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18516?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17842352#comment-17842352 ] ASF GitHub Bot commented on HADOOP-18516: - anujmodi2021 commented on PR #6552: URL:

[jira] [Updated] (HADOOP-19146) noaa-cors-pds bucket access with global endpoint fails

2024-04-30 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-19146?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran updated HADOOP-19146: Priority: Minor (was: Major) > noaa-cors-pds bucket access with global endpoint fails >

[jira] [Resolved] (HADOOP-19146) noaa-cors-pds bucket access with global endpoint fails

2024-04-30 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-19146?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran resolved HADOOP-19146. - Fix Version/s: 3.5.0 3.4.1 Resolution: Fixed > noaa-cors-pds

Re: [PR] HADOOP-19152. Do not hard code security providers. [hadoop]

2024-04-30 Thread via GitHub
steveloughran commented on code in PR #6739: URL: https://github.com/apache/hadoop/pull/6739#discussion_r1584770438 ## hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/crypto/TestCryptoUtils.java: ## @@ -0,0 +1,86 @@ +/* + * Licensed to the Apache Software

Re: [PR] HADOOP-18516: [ABFS][Authentication] Support Fixed SAS Token for ABFS Authentication [hadoop]

2024-04-30 Thread via GitHub
hadoop-yetus commented on PR #6552: URL: https://github.com/apache/hadoop/pull/6552#issuecomment-2085299632 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok: |

[jira] [Commented] (HADOOP-18516) [ABFS]: Support fixed SAS token config in addition to Custom SASTokenProvider Implementation

2024-04-30 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18516?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17842376#comment-17842376 ] ASF GitHub Bot commented on HADOOP-18516: - hadoop-yetus commented on PR #6552: URL:

Re: [PR] HADOOP-18679. Add API for bulk/paged object deletion [hadoop]

2024-04-30 Thread via GitHub
steveloughran commented on code in PR #6726: URL: https://github.com/apache/hadoop/pull/6726#discussion_r1584787287 ## hadoop-common-project/hadoop-common/src/site/markdown/filesystem/bulkdelete.md: ## @@ -0,0 +1,136 @@ + + +# interface `BulkDelete` + + + +The `BulkDelete`

[jira] [Commented] (HADOOP-18679) Add API for bulk/paged object deletion

2024-04-30 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18679?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17842385#comment-17842385 ] ASF GitHub Bot commented on HADOOP-18679: - steveloughran commented on code in PR #6726: URL:

[jira] [Commented] (HADOOP-19152) Do not hard code security providers.

2024-04-30 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-19152?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17842371#comment-17842371 ] ASF GitHub Bot commented on HADOOP-19152: - steveloughran commented on code in PR #6739: URL:

Re: [PR] MAPREDUCE-7474. Improve Manifest committer resilience [hadoop]

2024-04-30 Thread via GitHub
steveloughran commented on code in PR #6716: URL: https://github.com/apache/hadoop/pull/6716#discussion_r1584841337 ##

Re: [PR] MAPREDUCE-7474. Improve Manifest committer resilience [hadoop]

2024-04-30 Thread via GitHub
steveloughran commented on code in PR #6716: URL: https://github.com/apache/hadoop/pull/6716#discussion_r1584877464 ## hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/site/markdown/manifest_committer.md: ## @@ -523,7 +509,7 @@ And optional

Re: [PR] MAPREDUCE-7474. Improve Manifest committer resilience [hadoop]

2024-04-30 Thread via GitHub
steveloughran commented on PR #6716: URL: https://github.com/apache/hadoop/pull/6716#issuecomment-2086367792 I've now moved to commitFile() to rename the task manifest, after doing a getFileStatus() call first...which means its iO cost is the same as a rename with recovery enabled. it does

Re: [PR] HADOOP-19156. ZooKeeper based state stores use different ZK address c… [hadoop]

2024-04-30 Thread via GitHub
hadoop-yetus commented on PR #6767: URL: https://github.com/apache/hadoop/pull/6767#issuecomment-2086063685 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok: |

[jira] [Commented] (HADOOP-19156) ZooKeeper based state stores use different ZK address configs

2024-04-30 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-19156?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17842452#comment-17842452 ] ASF GitHub Bot commented on HADOOP-19156: - hadoop-yetus commented on PR #6767: URL:

[jira] [Assigned] (HADOOP-18610) ABFS OAuth2 Token Provider to support Azure Workload Identity for AKS

2024-04-30 Thread Sneha Vijayarajan (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18610?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sneha Vijayarajan reassigned HADOOP-18610: -- Assignee: Anuj Modi > ABFS OAuth2 Token Provider to support Azure Workload

Re: [PR] HADOOP-18610. ABFS OAuth2 Token Provider support for Azure Workload Identity [hadoop]

2024-04-30 Thread via GitHub
snvijaya commented on PR #5953: URL: https://github.com/apache/hadoop/pull/5953#issuecomment-2087970204 Thanks @creste . @anujmodi2021 Will pick this up. As the PR is raised from a forked repo, we will not be able to make changes to the same dev branch from which the PR is raised. Will

[jira] [Commented] (HADOOP-18610) ABFS OAuth2 Token Provider to support Azure Workload Identity for AKS

2024-04-30 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18610?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17842543#comment-17842543 ] ASF GitHub Bot commented on HADOOP-18610: - snvijaya commented on PR #5953: URL:

[jira] [Commented] (HADOOP-18610) ABFS OAuth2 Token Provider to support Azure Workload Identity for AKS

2024-04-30 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18610?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17842553#comment-17842553 ] ASF GitHub Bot commented on HADOOP-18610: - anujmodi2021 opened a new pull request, #6787: URL: