[jira] [Commented] (HADOOP-18458) AliyunOSS: AliyunOSSBlockOutputStream to support heap/off-heap buffer before uploading data to OSS

2023-03-27 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18458?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17705836#comment-17705836 ] ASF GitHub Bot commented on HADOOP-18458: - wujinhu commented on PR #4912: URL:

[GitHub] [hadoop] wujinhu commented on pull request #4912: HADOOP-18458: AliyunOSSBlockOutputStream to support heap/off-heap buffer before uploading data to OSS

2023-03-27 Thread via GitHub
wujinhu commented on PR #4912: URL: https://github.com/apache/hadoop/pull/4912#issuecomment-1486297215 Thanks @ChenSammi and @steveloughran :) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the

[jira] [Commented] (HADOOP-18458) AliyunOSS: AliyunOSSBlockOutputStream to support heap/off-heap buffer before uploading data to OSS

2023-03-27 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18458?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17705832#comment-17705832 ] ASF GitHub Bot commented on HADOOP-18458: - ChenSammi merged PR #4912: URL: http

[GitHub] [hadoop] ChenSammi merged pull request #4912: HADOOP-18458: AliyunOSSBlockOutputStream to support heap/off-heap buffer before uploading data to OSS

2023-03-27 Thread via GitHub
ChenSammi merged PR #4912: URL: https://github.com/apache/hadoop/pull/4912 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: common-issues-unsubscr...@had

[jira] [Commented] (HADOOP-18458) AliyunOSS: AliyunOSSBlockOutputStream to support heap/off-heap buffer before uploading data to OSS

2023-03-27 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18458?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17705831#comment-17705831 ] ASF GitHub Bot commented on HADOOP-18458: - ChenSammi commented on PR #4912: URL

[GitHub] [hadoop] ChenSammi commented on pull request #4912: HADOOP-18458: AliyunOSSBlockOutputStream to support heap/off-heap buffer before uploading data to OSS

2023-03-27 Thread via GitHub
ChenSammi commented on PR #4912: URL: https://github.com/apache/hadoop/pull/4912#issuecomment-1486285589 The overall patch LGTM +1. Thanks @wujinhu for the contribution and @steveloughran for the code review. -- This is an automated message from the Apache Git Service. To respond to t

[jira] [Resolved] (HADOOP-18681) Upgrade hadoop3 docker scripts to use 3.3.5

2023-03-27 Thread Ayush Saxena (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18681?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ayush Saxena resolved HADOOP-18681. --- Hadoop Flags: Reviewed Resolution: Fixed > Upgrade hadoop3 docker scripts to use 3.3.

[jira] [Commented] (HADOOP-18681) Upgrade hadoop3 docker scripts to use 3.3.5

2023-03-27 Thread Ayush Saxena (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18681?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17705830#comment-17705830 ] Ayush Saxena commented on HADOOP-18681: --- Committed. Thanx [~cnauroth] for the rev

[jira] [Commented] (HADOOP-18681) Upgrade hadoop3 docker scripts to use 3.3.5

2023-03-27 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18681?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17705829#comment-17705829 ] ASF GitHub Bot commented on HADOOP-18681: - ayushtkn merged PR #5514: URL: https

[GitHub] [hadoop] ayushtkn merged pull request #5514: HADOOP-18681. Upgrade hadoop3 docker scripts to use 3.3.5.

2023-03-27 Thread via GitHub
ayushtkn merged PR #5514: URL: https://github.com/apache/hadoop/pull/5514 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: common-issues-unsubscr...@hado

[GitHub] [hadoop] sreeb-msft commented on a diff in pull request #5488: HADOOP-18012. ABFS: Using Source Path eTags for Rename Idemptonency checks

2023-03-27 Thread via GitHub
sreeb-msft commented on code in PR #5488: URL: https://github.com/apache/hadoop/pull/5488#discussion_r1150069760 ## hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azurebfs/services/AbfsClient.java: ## @@ -556,20 +588,23 @@ public AbfsClientRenameResult renamePath(

[jira] [Commented] (HADOOP-18012) ABFS: Enable config controlled ETag check for Rename idempotency

2023-03-27 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18012?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17705820#comment-17705820 ] ASF GitHub Bot commented on HADOOP-18012: - sreeb-msft commented on code in PR #

[GitHub] [hadoop] hadoop-yetus commented on pull request #5335: YARN-11426. Improve YARN NodeLabel Memory Display.

2023-03-27 Thread via GitHub
hadoop-yetus commented on PR #5335: URL: https://github.com/apache/hadoop/pull/5335#issuecomment-1486262422 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok: | reex

[GitHub] [hadoop] hadoop-yetus commented on pull request #5476: YARN-11446. [Federation] Add updateSchedulerConfiguration, getSchedulerConfiguration REST APIs for Router.

2023-03-27 Thread via GitHub
hadoop-yetus commented on PR #5476: URL: https://github.com/apache/hadoop/pull/5476#issuecomment-1486178513 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok: | reex

[jira] [Commented] (HADOOP-18458) AliyunOSS: AliyunOSSBlockOutputStream to support heap/off-heap buffer before uploading data to OSS

2023-03-27 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18458?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17705784#comment-17705784 ] ASF GitHub Bot commented on HADOOP-18458: - hadoop-yetus commented on PR #4912:

[GitHub] [hadoop] hadoop-yetus commented on pull request #4912: HADOOP-18458: AliyunOSSBlockOutputStream to support heap/off-heap buffer before uploading data to OSS

2023-03-27 Thread via GitHub
hadoop-yetus commented on PR #4912: URL: https://github.com/apache/hadoop/pull/4912#issuecomment-1486158215 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok: | reex

[GitHub] [hadoop] zhangshuyan0 commented on pull request #5510: HDFS-16964. Improve processing of excess redundancy after failover.

2023-03-27 Thread via GitHub
zhangshuyan0 commented on PR #5510: URL: https://github.com/apache/hadoop/pull/5510#issuecomment-1486118554 @Hexiaoqiao Thanks. I've checked it out. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go t

[GitHub] [hadoop] slfan1989 commented on pull request #5496: YARN-11376. [Federation] Support updateNodeResource、refreshNodesResources API's for Federation.

2023-03-27 Thread via GitHub
slfan1989 commented on PR #5496: URL: https://github.com/apache/hadoop/pull/5496#issuecomment-1486072013 @goiri Thank you very much for your help in reviewing the code! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use t

[jira] [Commented] (HADOOP-18548) Hadoop Archive tool (HAR) should acquire delegation tokens from source and destination file systems

2023-03-27 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18548?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17705735#comment-17705735 ] ASF GitHub Bot commented on HADOOP-18548: - cnauroth commented on code in PR #53

[GitHub] [hadoop] cnauroth commented on a diff in pull request #5355: HADOOP-18548. Added source and dest file systems for HAR tool

2023-03-27 Thread via GitHub
cnauroth commented on code in PR #5355: URL: https://github.com/apache/hadoop/pull/5355#discussion_r1149889446 ## hadoop-tools/hadoop-archives/src/main/java/org/apache/hadoop/tools/HadoopArchives.java: ## @@ -487,6 +489,11 @@ void archive(Path parentPath, List srcPaths,

[GitHub] [hadoop] goiri commented on a diff in pull request #5420: YARN-11442. Refactor FederationInterceptorREST Code.

2023-03-27 Thread via GitHub
goiri commented on code in PR #5420: URL: https://github.com/apache/hadoop/pull/5420#discussion_r1149515622 ## hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-router/src/main/java/org/apache/hadoop/yarn/server/router/webapp/FederationInterceptorREST.java: #

[GitHub] [hadoop] goiri commented on a diff in pull request #4963: YARN-11326. [Federation] Add RM FederationStateStoreService Metrics.

2023-03-27 Thread via GitHub
goiri commented on code in PR #4963: URL: https://github.com/apache/hadoop/pull/4963#discussion_r1149505746 ## hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/federation/FederationStateStor

[GitHub] [hadoop] goiri merged pull request #5496: YARN-11376. [Federation] Support updateNodeResource、refreshNodesResources API's for Federation.

2023-03-27 Thread via GitHub
goiri merged PR #5496: URL: https://github.com/apache/hadoop/pull/5496 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.

[GitHub] [hadoop] hadoop-yetus commented on pull request #5483: HDFS-16954. RBF: rename a dir that has multiple destinations to a dir that just…

2023-03-27 Thread via GitHub
hadoop-yetus commented on PR #5483: URL: https://github.com/apache/hadoop/pull/5483#issuecomment-1485390250 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok: | reexe

[jira] [Commented] (HADOOP-18637) S3A to support upload of files greater than 2 GB using DiskBlocks

2023-03-27 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18637?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17705399#comment-17705399 ] ASF GitHub Bot commented on HADOOP-18637: - hadoop-yetus commented on PR #5481:

[GitHub] [hadoop] hadoop-yetus commented on pull request #5481: HADOOP-18637:S3A to support upload of files greater than 2 GB using DiskBlocks

2023-03-27 Thread via GitHub
hadoop-yetus commented on PR #5481: URL: https://github.com/apache/hadoop/pull/5481#issuecomment-1485290836 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok: | reex

[GitHub] [hadoop] hadoop-yetus commented on pull request #5515: YARN-11037. Add configurable logic to split resource request to the least loaded SC.

2023-03-27 Thread via GitHub
hadoop-yetus commented on PR #5515: URL: https://github.com/apache/hadoop/pull/5515#issuecomment-1485206947 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok: | reexe

[jira] [Resolved] (HADOOP-18672) ask: abfs connector to support checksum

2023-03-27 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18672?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran resolved HADOOP-18672. - Resolution: Information Provided > ask: abfs connector to support checksum > --

[jira] [Commented] (HADOOP-18012) ABFS: Enable config controlled ETag check for Rename idempotency

2023-03-27 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18012?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17705349#comment-17705349 ] ASF GitHub Bot commented on HADOOP-18012: - sreeb-msft commented on code in PR #

[jira] [Commented] (HADOOP-18012) ABFS: Enable config controlled ETag check for Rename idempotency

2023-03-27 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18012?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17705350#comment-17705350 ] ASF GitHub Bot commented on HADOOP-18012: - sreeb-msft commented on code in PR #

[GitHub] [hadoop] sreeb-msft commented on a diff in pull request #5488: HADOOP-18012. ABFS: Using Source Path eTags for Rename Idemptonency checks

2023-03-27 Thread via GitHub
sreeb-msft commented on code in PR #5488: URL: https://github.com/apache/hadoop/pull/5488#discussion_r1149233939 ## hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azurebfs/services/AbfsClient.java: ## @@ -556,20 +588,23 @@ public AbfsClientRenameResult renamePath(

[GitHub] [hadoop] sreeb-msft commented on a diff in pull request #5488: HADOOP-18012. ABFS: Using Source Path eTags for Rename Idemptonency checks

2023-03-27 Thread via GitHub
sreeb-msft commented on code in PR #5488: URL: https://github.com/apache/hadoop/pull/5488#discussion_r1149233538 ## hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azurebfs/services/AbfsClient.java: ## @@ -578,26 +613,49 @@ public AbfsClientRenameResult renamePath(

[GitHub] [hadoop] hadoop-yetus commented on pull request #5465: YARN-11427. + YARN-11404. Pull up the versioned imports in pom of hadoop-mapreduce-client-app to hadoop-project pom

2023-03-27 Thread via GitHub
hadoop-yetus commented on PR #5465: URL: https://github.com/apache/hadoop/pull/5465#issuecomment-1485069177 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok: | reexe

[jira] [Commented] (HADOOP-18606) Add reason in in x-ms-client-request-id on a retry API call.

2023-03-27 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18606?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17705337#comment-17705337 ] ASF GitHub Bot commented on HADOOP-18606: - saxenapranav commented on PR #5461:

[GitHub] [hadoop] saxenapranav commented on pull request #5461: HADOOP-18606. Backport Merged pr https://github.com/apache/hadoop/pull/5299 in branch-3.3

2023-03-27 Thread via GitHub
saxenapranav commented on PR #5461: URL: https://github.com/apache/hadoop/pull/5461#issuecomment-1485062964 Hi @steveloughran , requesting you to kindly review the PR please. This is pre-requisite for merging changes from following PRs in 3.3: 1. https://github.com/apache/hadoop/pull/5437

[jira] [Commented] (HADOOP-18565) AWS SDK V2 - Complete outstanding items

2023-03-27 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18565?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17705329#comment-17705329 ] ASF GitHub Bot commented on HADOOP-18565: - ahmarsuhail commented on PR #5421: U

[GitHub] [hadoop] ahmarsuhail commented on pull request #5421: HADOOP-18565. Completes outstanding items for the SDK V2 upgrade.

2023-03-27 Thread via GitHub
ahmarsuhail commented on PR #5421: URL: https://github.com/apache/hadoop/pull/5421#issuecomment-1485030132 Sounds good, I've created [this](https://issues.apache.org/jira/browse/HADOOP-18683) JIRA for adding in a new config option. Are we ok to merge this PR? Also, I was wondering,

[jira] [Updated] (HADOOP-18683) Add new store vendor config option

2023-03-27 Thread Ahmar Suhail (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18683?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ahmar Suhail updated HADOOP-18683: -- Affects Version/s: 3.3.5 > Add new store vendor config option > -

[jira] [Created] (HADOOP-18683) Add new store vendor config option

2023-03-27 Thread Ahmar Suhail (Jira)
Ahmar Suhail created HADOOP-18683: - Summary: Add new store vendor config option Key: HADOOP-18683 URL: https://issues.apache.org/jira/browse/HADOOP-18683 Project: Hadoop Common Issue Type: Su

[jira] [Updated] (HADOOP-18683) Add new store vendor config option

2023-03-27 Thread Ahmar Suhail (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18683?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ahmar Suhail updated HADOOP-18683: -- Component/s: fs/s3 > Add new store vendor config option > --

[jira] [Commented] (HADOOP-18679) Add API for bulk/paged object deletion

2023-03-27 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18679?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17705326#comment-17705326 ] Steve Loughran commented on HADOOP-18679: - actually, i'd have the builder have

[GitHub] [hadoop] slfan1989 opened a new pull request, #5515: YARN-11037. Add configurable logic to split resource request to the least loaded SC.

2023-03-27 Thread via GitHub
slfan1989 opened a new pull request, #5515: URL: https://github.com/apache/hadoop/pull/5515 JIRA: YARN-11037. Add configurable logic to split resource request to the least loaded SC. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to G

[jira] [Updated] (HADOOP-18678) Broken links for arm build of hadoop 3.3.5

2023-03-27 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18678?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran updated HADOOP-18678: Fix Version/s: 3.3.5 > Broken links for arm build of hadoop 3.3.5 > -

[jira] [Updated] (HADOOP-18146) ABFS: Add changes for expect hundred continue header with append requests

2023-03-27 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18146?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran updated HADOOP-18146: Affects Version/s: 3.3.5 (was: 3.3.1) > ABFS: Add changes for

[jira] [Updated] (HADOOP-18146) ABFS: Add changes for expect hundred continue header with append requests

2023-03-27 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18146?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran updated HADOOP-18146: Target Version/s: 3.3.9 > ABFS: Add changes for expect hundred continue header with appen

[jira] [Updated] (HADOOP-18146) ABFS: Add changes for expect hundred continue header with append requests

2023-03-27 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18146?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran updated HADOOP-18146: Component/s: fs/azure > ABFS: Add changes for expect hundred continue header with append

[jira] [Updated] (HADOOP-18146) ABFS: Add changes for expect hundred continue header with append requests

2023-03-27 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18146?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran updated HADOOP-18146: Fix Version/s: 3.4.0 > ABFS: Add changes for expect hundred continue header with append r

[jira] [Commented] (HADOOP-18146) ABFS: Add changes for expect hundred continue header with append requests

2023-03-27 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18146?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17705319#comment-17705319 ] ASF GitHub Bot commented on HADOOP-18146: - steveloughran commented on PR #4039:

[GitHub] [hadoop] steveloughran commented on pull request #4039: HADOOP-18146: ABFS: Added changes for expect hundred continue header

2023-03-27 Thread via GitHub
steveloughran commented on PR #4039: URL: https://github.com/apache/hadoop/pull/4039#issuecomment-1484996081 merged to trunk with a quick description. @anmolanmol1234 -can you cherrypick and test for 3.3, put up as a new PR and I will merge there too, so we can get it out sooner --

[jira] [Commented] (HADOOP-18146) ABFS: Add changes for expect hundred continue header with append requests

2023-03-27 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18146?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17705318#comment-17705318 ] ASF GitHub Bot commented on HADOOP-18146: - steveloughran merged PR #4039: URL:

[GitHub] [hadoop] steveloughran merged pull request #4039: HADOOP-18146: ABFS: Added changes for expect hundred continue header

2023-03-27 Thread via GitHub
steveloughran merged PR #4039: URL: https://github.com/apache/hadoop/pull/4039 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: common-issues-unsubscr...

[jira] [Commented] (HADOOP-18207) Introduce hadoop-logging module

2023-03-27 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18207?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17705317#comment-17705317 ] ASF GitHub Bot commented on HADOOP-18207: - steveloughran commented on PR #5503:

[GitHub] [hadoop] steveloughran commented on pull request #5503: HADOOP-18207. Introduce hadoop-logging module

2023-03-27 Thread via GitHub
steveloughran commented on PR #5503: URL: https://github.com/apache/hadoop/pull/5503#issuecomment-1484990379 can you chase other reviewers; i'm too caught up in internal release and various storage problems to look at this for a least a week -- This is an automated message from the Apache

[jira] [Commented] (HADOOP-18565) AWS SDK V2 - Complete outstanding items

2023-03-27 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18565?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17705314#comment-17705314 ] ASF GitHub Bot commented on HADOOP-18565: - steveloughran commented on PR #5421:

[GitHub] [hadoop] steveloughran commented on pull request #5421: HADOOP-18565. Completes outstanding items for the SDK V2 upgrade.

2023-03-27 Thread via GitHub
steveloughran commented on PR #5421: URL: https://github.com/apache/hadoop/pull/5421#issuecomment-1484983979 `Endpoint is not set || Endpoint is set && ends in amazonaws.com || ARN is set` is roughly what storediag does: https://github.com/steveloughran/cloudstore/blob/trunk/src/main/java/o

[GitHub] [hadoop] steveloughran commented on pull request #5378: MAPREDUCE-7432. Make manifest committer default on abfs and gcs stores

2023-03-27 Thread via GitHub
steveloughran commented on PR #5378: URL: https://github.com/apache/hadoop/pull/5378#issuecomment-1484965578 not going to merge this just yet; been getting complaints about memory use in some jobs during commit. I think I will have to merge manifest load with the file commit phase, which is

[jira] [Commented] (HADOOP-18675) CachedSASToken noisy log errors when SAS token has YYYY-MM-DD expiration

2023-03-27 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18675?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17705303#comment-17705303 ] ASF GitHub Bot commented on HADOOP-18675: - steveloughran commented on code in P

[GitHub] [hadoop] steveloughran commented on a diff in pull request #5508: HADOOP-18675: Fix CachedSASToken noisy log errors when SAS token has YYYY-MM-DD expiration

2023-03-27 Thread via GitHub
steveloughran commented on code in PR #5508: URL: https://github.com/apache/hadoop/pull/5508#discussion_r1149159845 ## hadoop-tools/hadoop-azure/src/test/java/org/apache/hadoop/fs/azurebfs/utils/TestCachedSASToken.java: ## @@ -78,6 +78,31 @@ public void testUpdateAndGet() throws

[GitHub] [hadoop] hadoop-yetus commented on pull request #5510: HDFS-16964. Improve processing of excess redundancy after failover.

2023-03-27 Thread via GitHub
hadoop-yetus commented on PR #5510: URL: https://github.com/apache/hadoop/pull/5510#issuecomment-1484863393 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok: | reexe

[GitHub] [hadoop] tasanuma commented on pull request #5489: HDFS-16958. EC: Fix bug in processing EC excess redundancy.

2023-03-27 Thread via GitHub
tasanuma commented on PR #5489: URL: https://github.com/apache/hadoop/pull/5489#issuecomment-1484859166 @zhangshuyan0 Thanks. I will close the jira. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go t

[GitHub] [hadoop] hadoop-yetus commented on pull request #5363: YARN-11424. [Federation] Router Supports DeregisterSubCluster.

2023-03-27 Thread via GitHub
hadoop-yetus commented on PR #5363: URL: https://github.com/apache/hadoop/pull/5363#issuecomment-1484844306 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok: | reexe

[GitHub] [hadoop] zhangshuyan0 commented on pull request #5489: HDFS-16958. EC: Fix bug in processing EC excess redundancy.

2023-03-27 Thread via GitHub
zhangshuyan0 commented on PR #5489: URL: https://github.com/apache/hadoop/pull/5489#issuecomment-1484829413 @tasanuma Yes, current is correct. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the

[GitHub] [hadoop] slfan1989 commented on pull request #5335: YARN-11426. Improve YARN NodeLabel Memory Display.

2023-03-27 Thread via GitHub
slfan1989 commented on PR #5335: URL: https://github.com/apache/hadoop/pull/5335#issuecomment-1484800229 @cnauroth Can you help to review this PR again? Thank you very much! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and

[GitHub] [hadoop] tasanuma commented on pull request #5489: HDFS-16958. EC: Fix bug in processing EC excess redundancy.

2023-03-27 Thread via GitHub
tasanuma commented on PR #5489: URL: https://github.com/apache/hadoop/pull/5489#issuecomment-1484795648 @zhangshuyan0 Thanks for your explanation. So the current implementation is correct? -- This is an automated message from the Apache Git Service. To respond to the message, please log o

[jira] [Commented] (HADOOP-18637) S3A to support upload of files greater than 2 GB using DiskBlocks

2023-03-27 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18637?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17705254#comment-17705254 ] ASF GitHub Bot commented on HADOOP-18637: - HarshitGupta11 commented on code in

[GitHub] [hadoop] HarshitGupta11 commented on a diff in pull request #5481: HADOOP-18637:S3A to support upload of files greater than 2 GB using DiskBlocks

2023-03-27 Thread via GitHub
HarshitGupta11 commented on code in PR #5481: URL: https://github.com/apache/hadoop/pull/5481#discussion_r1149011685 ## hadoop-tools/hadoop-aws/src/main/java/org/apache/hadoop/fs/s3a/S3AUtils.java: ## @@ -1031,6 +1031,19 @@ public static long getMultipartSizeProperty(Configurat

[jira] [Commented] (HADOOP-18637) S3A to support upload of files greater than 2 GB using DiskBlocks

2023-03-27 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18637?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17705251#comment-17705251 ] ASF GitHub Bot commented on HADOOP-18637: - HarshitGupta11 commented on code in

[GitHub] [hadoop] HarshitGupta11 commented on a diff in pull request #5481: HADOOP-18637:S3A to support upload of files greater than 2 GB using DiskBlocks

2023-03-27 Thread via GitHub
HarshitGupta11 commented on code in PR #5481: URL: https://github.com/apache/hadoop/pull/5481#discussion_r1149003025 ## hadoop-tools/hadoop-aws/src/main/java/org/apache/hadoop/fs/s3a/S3ABlockOutputStream.java: ## @@ -369,6 +373,9 @@ private synchronized void uploadCurrentBlock(b

[jira] [Resolved] (HADOOP-18676) Include jettison as direct dependency of hadoop-common

2023-03-27 Thread Andras Katona (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18676?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andras Katona resolved HADOOP-18676. Resolution: Fixed > Include jettison as direct dependency of hadoop-common >

[jira] [Updated] (HADOOP-18676) Include jettison as direct dependency of hadoop-common

2023-03-27 Thread Andras Katona (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18676?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andras Katona updated HADOOP-18676: --- Fix Version/s: 3.4.0 > Include jettison as direct dependency of hadoop-common > ---

[jira] [Updated] (HADOOP-18639) DockerContainerDeletionTask is not removed from the Nodemanager's statestore when the task is completed.

2023-03-27 Thread Sejin Hwang (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18639?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sejin Hwang updated HADOOP-18639: - Affects Version/s: 3.1.2 > DockerContainerDeletionTask is not removed from the Nodemanager's st

[jira] [Commented] (HADOOP-18676) Include jettison as direct dependency of hadoop-common

2023-03-27 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18676?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17705229#comment-17705229 ] ASF GitHub Bot commented on HADOOP-18676: - brumi1024 merged PR #5513: URL: http

[GitHub] [hadoop] brumi1024 merged pull request #5513: HADOOP-18676. jettison dependency override in hadoop-common lib

2023-03-27 Thread via GitHub
brumi1024 merged PR #5513: URL: https://github.com/apache/hadoop/pull/5513 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: common-issues-unsubscr...@had

[GitHub] [hadoop] zhangshuyan0 commented on pull request #5489: HDFS-16958. EC: Fix bug in processing EC excess redundancy.

2023-03-27 Thread via GitHub
zhangshuyan0 commented on PR #5489: URL: https://github.com/apache/hadoop/pull/5489#issuecomment-1484670476 @tasanuma Thanks for your comment. I think I was wrong before. The chosen of `excessTypes` should based on healthy replicas. Assume that the storage policy is ALL_SSD, the real tot

[GitHub] [hadoop] hadoop-yetus commented on pull request #5510: HDFS-16964. Improve processing of excess redundancy after failover.

2023-03-27 Thread via GitHub
hadoop-yetus commented on PR #5510: URL: https://github.com/apache/hadoop/pull/5510#issuecomment-1484657510 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok: | reexe

[GitHub] [hadoop] hadoop-yetus commented on pull request #4963: YARN-11326. [Federation] Add RM FederationStateStoreService Metrics.

2023-03-27 Thread via GitHub
hadoop-yetus commented on PR #4963: URL: https://github.com/apache/hadoop/pull/4963#issuecomment-1484633675 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok: | reex