[jira] [Commented] (HADOOP-18249) Fix getUri() in HttpRequest has been deprecated

2022-05-24 Thread Hemanth Boyina (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18249?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17541638#comment-17541638 ] Hemanth Boyina commented on HADOOP-18249: - committed to trunk thanks for the contribution

[jira] [Commented] (HADOOP-18140) Update default value of hadoop.ssl.enabled.protocols in the EncryptedShuffle doc

2022-05-24 Thread Samrat Deb (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18140?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17541677#comment-17541677 ] Samrat Deb commented on HADOOP-18140: - I checked the code . looks like the file in github trunk is

[jira] [Work logged] (HADOOP-18249) Fix getUri() in HttpRequest has been deprecated

2022-05-24 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18249?focusedWorklogId=774233=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-774233 ] ASF GitHub Bot logged work on HADOOP-18249: --- Author: ASF GitHub Bot

[jira] [Created] (HADOOP-18255) fsdatainputstreambuilder.md refers to hadoop 3.3.3, when it shouldn't

2022-05-24 Thread Steve Loughran (Jira)
Steve Loughran created HADOOP-18255: --- Summary: fsdatainputstreambuilder.md refers to hadoop 3.3.3, when it shouldn't Key: HADOOP-18255 URL: https://issues.apache.org/jira/browse/HADOOP-18255

[jira] [Commented] (HADOOP-18140) Update default value of hadoop.ssl.enabled.protocols in the EncryptedShuffle doc

2022-05-24 Thread Wei-Chiu Chuang (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18140?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17541689#comment-17541689 ] Wei-Chiu Chuang commented on HADOOP-18140: -- Uh. you're right. This is fixed by HADOOP-16549.

[jira] [Assigned] (HADOOP-17726) Replace Sets#newHashSet() and newTreeSet() with constructors directly

2022-05-24 Thread Samrat Deb (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-17726?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Samrat Deb reassigned HADOOP-17726: --- Assignee: Samrat Deb > Replace Sets#newHashSet() and newTreeSet() with constructors

[jira] [Work logged] (HADOOP-18107) Vectored IO support for large S3 files.

2022-05-24 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18107?focusedWorklogId=774225=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-774225 ] ASF GitHub Bot logged work on HADOOP-18107: --- Author: ASF GitHub Bot

[GitHub] [hadoop] hemanthboyina merged pull request #4335: HADOOP-18249. Fix getUri() in HttpRequest has been deprecated.

2022-05-24 Thread GitBox
hemanthboyina merged PR #4335: URL: https://github.com/apache/hadoop/pull/4335 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

[jira] [Work logged] (HADOOP-17461) Add thread-level IOStatistics Context

2022-05-24 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-17461?focusedWorklogId=774151=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-774151 ] ASF GitHub Bot logged work on HADOOP-17461: --- Author: ASF GitHub Bot

[jira] [Assigned] (HADOOP-17726) Replace Sets#newHashSet() and newTreeSet() with constructors directly

2022-05-24 Thread Samrat Deb (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-17726?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Samrat Deb reassigned HADOOP-17726: --- Assignee: (was: Samrat Deb) > Replace Sets#newHashSet() and newTreeSet() with

[jira] [Work logged] (HADOOP-18107) Vectored IO support for large S3 files.

2022-05-24 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18107?focusedWorklogId=774223=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-774223 ] ASF GitHub Bot logged work on HADOOP-18107: --- Author: ASF GitHub Bot

[GitHub] [hadoop] steveloughran commented on a diff in pull request #4273: HADOOP-18107 Adding scale test for vectored reads for large file

2022-05-24 Thread GitBox
steveloughran commented on code in PR #4273: URL: https://github.com/apache/hadoop/pull/4273#discussion_r880943791 ## hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/fs/contract/ContractTestUtils.java: ## @@ -1095,6 +1102,54 @@ public static void

[GitHub] [hadoop] hadoop-yetus commented on pull request #4352: HADOOP-17461. Thread-level IOStatistics in S3A

2022-05-24 Thread GitBox
hadoop-yetus commented on PR #4352: URL: https://github.com/apache/hadoop/pull/4352#issuecomment-1136252339 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok: |

[jira] [Work logged] (HADOOP-18249) Fix getUri() in HttpRequest has been deprecated

2022-05-24 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18249?focusedWorklogId=774150=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-774150 ] ASF GitHub Bot logged work on HADOOP-18249: --- Author: ASF GitHub Bot

[jira] [Resolved] (HADOOP-18249) Fix getUri() in HttpRequest has been deprecated

2022-05-24 Thread Hemanth Boyina (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18249?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hemanth Boyina resolved HADOOP-18249. - Resolution: Fixed > Fix getUri() in HttpRequest has been deprecated >

[GitHub] [hadoop] steveloughran commented on a diff in pull request #4273: HADOOP-18107 Adding scale test for vectored reads for large file

2022-05-24 Thread GitBox
steveloughran commented on code in PR #4273: URL: https://github.com/apache/hadoop/pull/4273#discussion_r880944821 ## hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/fs/contract/ContractTestUtils.java: ## @@ -1095,6 +1102,54 @@ public static void

[jira] [Work logged] (HADOOP-18107) Vectored IO support for large S3 files.

2022-05-24 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18107?focusedWorklogId=774224=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-774224 ] ASF GitHub Bot logged work on HADOOP-18107: --- Author: ASF GitHub Bot

[GitHub] [hadoop] steveloughran commented on pull request #4273: HADOOP-18107 Adding scale test for vectored reads for large file

2022-05-24 Thread GitBox
steveloughran commented on PR #4273: URL: https://github.com/apache/hadoop/pull/4273#issuecomment-1136425987 +1 pending that move into a constant, just for clarity. thanks (btw, suggest another rebase to get rid of those javadoc issues) -- This is an automated message from the

[GitHub] [hadoop] slfan1989 commented on pull request #4335: HADOOP-18249. Fix getUri() in HttpRequest has been deprecated.

2022-05-24 Thread GitBox
slfan1989 commented on PR #4335: URL: https://github.com/apache/hadoop/pull/4335#issuecomment-1136439304 @hemanthboyina Thank you very much for your help reviewing the code! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and

[jira] [Commented] (HADOOP-17726) Replace Sets#newHashSet() and newTreeSet() with constructors directly

2022-05-24 Thread Viraj Jasani (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-17726?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17541718#comment-17541718 ] Viraj Jasani commented on HADOOP-17726: --- Sure [~samrat007] please go ahead, no concerns from my

[jira] [Commented] (HADOOP-17343) Upgrade aws-java-sdk to 1.11.901

2022-05-24 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-17343?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17541666#comment-17541666 ] Steve Loughran commented on HADOOP-17343: - bq. Can we backport to branch-3.2? what is your

[jira] [Resolved] (HADOOP-18140) Update default value of hadoop.ssl.enabled.protocols in the EncryptedShuffle doc

2022-05-24 Thread Wei-Chiu Chuang (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18140?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wei-Chiu Chuang resolved HADOOP-18140. -- Resolution: Duplicate > Update default value of hadoop.ssl.enabled.protocols in the

[jira] [Comment Edited] (HADOOP-17726) Replace Sets#newHashSet() and newTreeSet() with constructors directly

2022-05-24 Thread Samrat Deb (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-17726?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17541692#comment-17541692 ] Samrat Deb edited comment on HADOOP-17726 at 5/24/22 8:24 PM: -- hi

[jira] [Commented] (HADOOP-18249) Fix getUri() in HttpRequest has been deprecated

2022-05-24 Thread fanshilun (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18249?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17541702#comment-17541702 ] fanshilun commented on HADOOP-18249: Hi, [~hemanthboyina] ,Thanks you very much! > Fix getUri() in

[GitHub] [hadoop] mukund-thakur commented on pull request #4273: HADOOP-18107 Adding scale test for vectored reads for large file

2022-05-24 Thread GitBox
mukund-thakur commented on PR #4273: URL: https://github.com/apache/hadoop/pull/4273#issuecomment-1136457258 > (btw, suggest another rebase to get rid of those javadoc issues) Will do the rebase of feature branch in one go after the smaller patches are merged. -- This

[jira] [Commented] (HADOOP-17726) Replace Sets#newHashSet() and newTreeSet() with constructors directly

2022-05-24 Thread Samrat Deb (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-17726?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17541692#comment-17541692 ] Samrat Deb commented on HADOOP-17726: - hi [~vjasani] ,  I am willing to pick this task as a newbie

[jira] [Work logged] (HADOOP-18105) Implement a variant of ElasticByteBufferPool which uses weak references for garbage collection.

2022-05-24 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18105?focusedWorklogId=774231=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-774231 ] ASF GitHub Bot logged work on HADOOP-18105: --- Author: ASF GitHub Bot

[GitHub] [hadoop] steveloughran commented on a diff in pull request #4263: HADOOP-18105 Implement buffer pooling with weak references

2022-05-24 Thread GitBox
steveloughran commented on code in PR #4263: URL: https://github.com/apache/hadoop/pull/4263#discussion_r880947333 ## hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/io/WeakReferencedElasticByteBufferPool.java: ## @@ -0,0 +1,126 @@ +/** + * Licensed to the

[GitHub] [hadoop] slfan1989 commented on pull request #4349: HDFS-16590. Fix Junit Test Deprecated assertThat

2022-05-24 Thread GitBox
slfan1989 commented on PR #4349: URL: https://github.com/apache/hadoop/pull/4349#issuecomment-1136434600 > The new javac warnings make sense, so they are OK to me. > > Unit test failures need looking at, I'm not sure if they are occasional issues. I don't usually review HDFS patches.

[GitHub] [hadoop] hadoop-yetus commented on pull request #4348: HDFS-16586. Purge FsDatasetAsyncDiskService threadgroup; it causes BP…

2022-05-24 Thread GitBox
hadoop-yetus commented on PR #4348: URL: https://github.com/apache/hadoop/pull/4348#issuecomment-1136459189 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok: |

[GitHub] [hadoop] slfan1989 commented on pull request #4349: HDFS-16590. Fix Junit Test Deprecated assertThat

2022-05-24 Thread GitBox
slfan1989 commented on PR #4349: URL: https://github.com/apache/hadoop/pull/4349#issuecomment-1136477439 I run this junit test and found that it can be successful. ``` [INFO] --- [INFO] T E S T S [INFO]

[GitHub] [hadoop] slfan1989 commented on a diff in pull request #4341: YARN-10487. Support getQueueUserAcls, listReservations, getApplicationAttempts, getContainerReport, getContainers, getResourceTyp

2022-05-24 Thread GitBox
slfan1989 commented on code in PR #4341: URL: https://github.com/apache/hadoop/pull/4341#discussion_r881159169 ##

[GitHub] [hadoop] hadoop-yetus commented on pull request #4341: YARN-10487. Support getQueueUserAcls, listReservations, getApplicationAttempts, getContainerReport, getContainers, getResourceTypeInfo A

2022-05-24 Thread GitBox
hadoop-yetus commented on PR #4341: URL: https://github.com/apache/hadoop/pull/4341#issuecomment-1136702589 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok: |

[jira] [Work logged] (HADOOP-18107) Vectored IO support for large S3 files.

2022-05-24 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18107?focusedWorklogId=774245=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-774245 ] ASF GitHub Bot logged work on HADOOP-18107: --- Author: ASF GitHub Bot

[GitHub] [hadoop] hadoop-yetus commented on pull request #4347: HDFS-16586. Purge FsDatasetAsyncDiskService threadgroup; it causes BP…

2022-05-24 Thread GitBox
hadoop-yetus commented on PR #4347: URL: https://github.com/apache/hadoop/pull/4347#issuecomment-1136471915 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok: |

[GitHub] [hadoop] hadoop-yetus commented on pull request #4341: YARN-10487. Support getQueueUserAcls, listReservations, getApplicationAttempts, getContainerReport, getContainers, getResourceTypeInfo A

2022-05-24 Thread GitBox
hadoop-yetus commented on PR #4341: URL: https://github.com/apache/hadoop/pull/4341#issuecomment-1136699843 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok: |

[jira] [Assigned] (HADOOP-18255) fsdatainputstreambuilder.md refers to hadoop 3.3.3, when it shouldn't

2022-05-24 Thread Ashutosh Gupta (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18255?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ashutosh Gupta reassigned HADOOP-18255: --- Assignee: Ashutosh Gupta > fsdatainputstreambuilder.md refers to hadoop 3.3.3,

[GitHub] [hadoop] goiri commented on a diff in pull request #4341: YARN-10487. Support getQueueUserAcls, listReservations, getApplicationAttempts, getContainerReport, getContainers, getResourceTypeInf

2022-05-24 Thread GitBox
goiri commented on code in PR #4341: URL: https://github.com/apache/hadoop/pull/4341#discussion_r881077343 ## hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-router/src/main/java/org/apache/hadoop/yarn/server/router/clientrm/RouterYarnClientUtils.java: ##

[GitHub] [hadoop] slfan1989 commented on a diff in pull request #4341: YARN-10487. Support getQueueUserAcls, listReservations, getApplicationAttempts, getContainerReport, getContainers, getResourceTyp

2022-05-24 Thread GitBox
slfan1989 commented on code in PR #4341: URL: https://github.com/apache/hadoop/pull/4341#discussion_r881103578 ## hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-router/src/main/java/org/apache/hadoop/yarn/server/router/clientrm/RouterYarnClientUtils.java:

[GitHub] [hadoop] slfan1989 commented on a diff in pull request #4341: YARN-10487. Support getQueueUserAcls, listReservations, getApplicationAttempts, getContainerReport, getContainers, getResourceTyp

2022-05-24 Thread GitBox
slfan1989 commented on code in PR #4341: URL: https://github.com/apache/hadoop/pull/4341#discussion_r881106167 ## hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-router/src/main/java/org/apache/hadoop/yarn/server/router/RouterMetrics.java: ## @@ -238,6

[jira] [Work logged] (HADOOP-18107) Vectored IO support for large S3 files.

2022-05-24 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18107?focusedWorklogId=774330=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-774330 ] ASF GitHub Bot logged work on HADOOP-18107: --- Author: ASF GitHub Bot

[GitHub] [hadoop] hadoop-yetus commented on pull request #4273: HADOOP-18107 Adding scale test for vectored reads for large file

2022-05-24 Thread GitBox
hadoop-yetus commented on PR #4273: URL: https://github.com/apache/hadoop/pull/4273#issuecomment-1136634303 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok: |

[GitHub] [hadoop] hadoop-yetus commented on pull request #4341: YARN-10487. Support getQueueUserAcls, listReservations, getApplicationAttempts, getContainerReport, getContainers, getResourceTypeInfo A

2022-05-24 Thread GitBox
hadoop-yetus commented on PR #4341: URL: https://github.com/apache/hadoop/pull/4341#issuecomment-1136684524 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok: |

[GitHub] [hadoop] hadoop-yetus commented on pull request #4216: HDFS-16555. rename mixlead method name in DistCpOptions

2022-05-24 Thread GitBox
hadoop-yetus commented on PR #4216: URL: https://github.com/apache/hadoop/pull/4216#issuecomment-1136691451 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok: |

[GitHub] [hadoop] ZanderXu commented on pull request #3806: HDFS-16386.Reduce DataNode load when FsDatasetAsyncDiskService is working.

2022-05-24 Thread GitBox
ZanderXu commented on PR #3806: URL: https://github.com/apache/hadoop/pull/3806#issuecomment-1136695695 Thanks @jianghuazhu for you comment. - I have a question, if the queue is unbounded, will the number of active thread in the ThreadPool be greater than the number of core thread? -

[jira] [Work logged] (HADOOP-18242) ABFS Rename Failure when tracking metadata is in incomplete state

2022-05-24 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18242?focusedWorklogId=774253=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-774253 ] ASF GitHub Bot logged work on HADOOP-18242: --- Author: ASF GitHub Bot

[GitHub] [hadoop] steveloughran commented on a diff in pull request #4216: HDFS-16555. rename mixlead method name in DistCpOptions

2022-05-24 Thread GitBox
steveloughran commented on code in PR #4216: URL: https://github.com/apache/hadoop/pull/4216#discussion_r880984625 ## hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop/tools/DistCpOptions.java: ## @@ -684,11 +684,23 @@ public Builder withAppend(boolean newAppend) {

[GitHub] [hadoop] slfan1989 commented on pull request #4336: YARN-11137. Improve log message in FederationClientInterceptor

2022-05-24 Thread GitBox
slfan1989 commented on PR #4336: URL: https://github.com/apache/hadoop/pull/4336#issuecomment-1136480996 Hi, @goiri @tomscut please help to review the code , thank you very match! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [hadoop] GuoPhilipse commented on a diff in pull request #4216: HDFS-16555. rename mixlead method name in DistCpOptions

2022-05-24 Thread GitBox
GuoPhilipse commented on code in PR #4216: URL: https://github.com/apache/hadoop/pull/4216#discussion_r881089014 ## hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop/tools/DistCpOptions.java: ## @@ -684,11 +684,23 @@ public Builder withAppend(boolean newAppend) {

[GitHub] [hadoop] jianghuazhu commented on pull request #3806: HDFS-16386.Reduce DataNode load when FsDatasetAsyncDiskService is working.

2022-05-24 Thread GitBox
jianghuazhu commented on PR #3806: URL: https://github.com/apache/hadoop/pull/3806#issuecomment-1136659630 Thanks @ZanderXu for following. Here are some explanations: 1. The main job of FsDatasetAsyncDiskService is to delete the replica files synchronously or asynchronously. The copy

[GitHub] [hadoop] zhengchenyu commented on pull request #4308: YARN-11148. In federation and security mode, nm recover may fail.

2022-05-24 Thread GitBox
zhengchenyu commented on PR #4308: URL: https://github.com/apache/hadoop/pull/4308#issuecomment-1136701598 > @zhengchenyu Thanks for your contribution. It makes sense at first glance. How about to add unit test to cover this case? I think we should apply

[GitHub] [hadoop] ZanderXu commented on pull request #3806: HDFS-16386.Reduce DataNode load when FsDatasetAsyncDiskService is working.

2022-05-24 Thread GitBox
ZanderXu commented on PR #3806: URL: https://github.com/apache/hadoop/pull/3806#issuecomment-1136727763 Thanks, and i will create a new PR to do it. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go

[GitHub] [hadoop] steveloughran commented on a diff in pull request #4331: HADOOP-18242. ABFS Rename Failure when tracking metadata is in an incomplete state

2022-05-24 Thread GitBox
steveloughran commented on code in PR #4331: URL: https://github.com/apache/hadoop/pull/4331#discussion_r880982514 ## hadoop-tools/hadoop-azure/src/test/java/org/apache/hadoop/fs/azurebfs/ITestAzureBlobFileSystemRename.java: ## @@ -167,4 +169,30 @@ public void

[GitHub] [hadoop] goiri merged pull request #4336: YARN-11137. Improve log message in FederationClientInterceptor

2022-05-24 Thread GitBox
goiri merged PR #4336: URL: https://github.com/apache/hadoop/pull/4336 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

[jira] [Work logged] (HADOOP-18107) Vectored IO support for large S3 files.

2022-05-24 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18107?focusedWorklogId=774319=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-774319 ] ASF GitHub Bot logged work on HADOOP-18107: --- Author: ASF GitHub Bot

[GitHub] [hadoop] hadoop-yetus commented on pull request #4273: HADOOP-18107 Adding scale test for vectored reads for large file

2022-05-24 Thread GitBox
hadoop-yetus commented on PR #4273: URL: https://github.com/apache/hadoop/pull/4273#issuecomment-1136593177 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok: |

[GitHub] [hadoop] ZanderXu opened a new pull request, #4353: HDFS-16593. Correct inaccurate BlocksRemoved metric on DataNode side

2022-05-24 Thread GitBox
ZanderXu opened a new pull request, #4353: URL: https://github.com/apache/hadoop/pull/4353 Correct inaccurate BlocksRemoved metric on DataNode side -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go

[GitHub] [hadoop] jianghuazhu commented on pull request #3806: HDFS-16386.Reduce DataNode load when FsDatasetAsyncDiskService is working.

2022-05-24 Thread GitBox
jianghuazhu commented on PR #3806: URL: https://github.com/apache/hadoop/pull/3806#issuecomment-1136708070 @ZanderXu , nice to communicate with you. I suggest that the number of active threads here needs to be set reasonably, according to the load capacity of the cluster. -- This is

[GitHub] [hadoop] hadoop-yetus commented on pull request #4332: HDFS-16583. DatanodeAdminDefaultMonitor can get stuck in an infinite loop holding the write lock

2022-05-24 Thread GitBox
hadoop-yetus commented on PR #4332: URL: https://github.com/apache/hadoop/pull/4332#issuecomment-1136461239 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok: |

[GitHub] [hadoop] hadoop-yetus commented on pull request #4340: HDFS-15878 RBF: Flaky test TestRouterWebHDFSContractCreate>AbstractCo…

2022-05-24 Thread GitBox
hadoop-yetus commented on PR #4340: URL: https://github.com/apache/hadoop/pull/4340#issuecomment-1135447208 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok: |

[jira] [Work logged] (HADOOP-18248) Fix Junit Test Deprecated assertThat

2022-05-24 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18248?focusedWorklogId=773909=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-773909 ] ASF GitHub Bot logged work on HADOOP-18248: --- Author: ASF GitHub Bot

[GitHub] [hadoop] dannycjones commented on pull request #4334: HADOOP-18248. Fix Junit Test Deprecated assertThat.

2022-05-24 Thread GitBox
dannycjones commented on PR #4334: URL: https://github.com/apache/hadoop/pull/4334#issuecomment-1135552621 @slfan1989 sounds good. I assume the patch in the new PR will be similar - if so, feel free to mention me and I will try and review in good time. -- This is an automated message

[GitHub] [hadoop] slfan1989 commented on pull request #4334: HADOOP-18248. Fix Junit Test Deprecated assertThat.

2022-05-24 Thread GitBox
slfan1989 commented on PR #4334: URL: https://github.com/apache/hadoop/pull/4334#issuecomment-1135564137 > @slfan1989 sounds good. I assume the patch in the new PR will be similar - if so, feel free to mention me and I will try and review in good time. Hi, @dannycjones , thank you ,

[jira] [Updated] (HADOOP-13091) DistCp masks potential CRC check failures

2022-05-24 Thread Masatake Iwasaki (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-13091?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Masatake Iwasaki updated HADOOP-13091: -- Target Version/s: 2.10.3 (was: 2.10.2) > DistCp masks potential CRC check failures

[jira] [Commented] (HADOOP-13091) DistCp masks potential CRC check failures

2022-05-24 Thread Masatake Iwasaki (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-13091?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17541343#comment-17541343 ] Masatake Iwasaki commented on HADOOP-13091: --- updated the target version for preparing 2.10.2

[jira] [Commented] (HADOOP-16039) backport HADOOP-15965, "Upgrade ADLS SDK to 2.3.3" to branch-2

2022-05-24 Thread Masatake Iwasaki (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-16039?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17541342#comment-17541342 ] Masatake Iwasaki commented on HADOOP-16039: --- updated the target version for preparing 2.10.2

[jira] [Updated] (HADOOP-16039) backport HADOOP-15965, "Upgrade ADLS SDK to 2.3.3" to branch-2

2022-05-24 Thread Masatake Iwasaki (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-16039?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Masatake Iwasaki updated HADOOP-16039: -- Target Version/s: 2.10.3 (was: 2.10.2) > backport HADOOP-15965, "Upgrade ADLS SDK

[jira] [Updated] (HADOOP-10738) Dynamically adjust distcp configuration by adding distcp-site.xml into code base

2022-05-24 Thread Masatake Iwasaki (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-10738?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Masatake Iwasaki updated HADOOP-10738: -- Target Version/s: 2.10.3 (was: 2.10.2) > Dynamically adjust distcp configuration by

[jira] [Commented] (HADOOP-10738) Dynamically adjust distcp configuration by adding distcp-site.xml into code base

2022-05-24 Thread Masatake Iwasaki (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-10738?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17541344#comment-17541344 ] Masatake Iwasaki commented on HADOOP-10738: --- updated the target version for preparing 2.10.2

[GitHub] [hadoop] hadoop-yetus commented on pull request #4345: backport of security patches to branch 3.3.3

2022-05-24 Thread GitBox
hadoop-yetus commented on PR #4345: URL: https://github.com/apache/hadoop/pull/4345#issuecomment-1135473426 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok: |

[jira] [Updated] (HADOOP-18203) Backport HADOOP-15033 to branch-2.10

2022-05-24 Thread Masatake Iwasaki (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18203?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Masatake Iwasaki updated HADOOP-18203: -- Target Version/s: 2.10.3 (was: 2.10.2) > Backport HADOOP-15033 to branch-2.10 >

[jira] [Commented] (HADOOP-18203) Backport HADOOP-15033 to branch-2.10

2022-05-24 Thread Masatake Iwasaki (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18203?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17541341#comment-17541341 ] Masatake Iwasaki commented on HADOOP-18203: --- updated the target version for preparing 2.10.2

[jira] [Work logged] (HADOOP-18248) Fix Junit Test Deprecated assertThat

2022-05-24 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18248?focusedWorklogId=773917=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-773917 ] ASF GitHub Bot logged work on HADOOP-18248: --- Author: ASF GitHub Bot

[GitHub] [hadoop] klaus-xiong closed pull request #4303: MAPREDUCE-7378. Change job temporary dir name to avoid delete by other jobs

2022-05-24 Thread GitBox
klaus-xiong closed pull request #4303: MAPREDUCE-7378. Change job temporary dir name to avoid delete by other jobs URL: https://github.com/apache/hadoop/pull/4303 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL

[jira] [Assigned] (HADOOP-17726) Replace Sets#newHashSet() and newTreeSet() with constructors directly

2022-05-24 Thread Samrat Deb (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-17726?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Samrat Deb reassigned HADOOP-17726: --- Assignee: Samrat Deb > Replace Sets#newHashSet() and newTreeSet() with constructors

[jira] [Work logged] (HADOOP-18244) Fix Hadoop-Common JavaDoc Error on branch-3.3

2022-05-24 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18244?focusedWorklogId=773926=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-773926 ] ASF GitHub Bot logged work on HADOOP-18244: --- Author: ASF GitHub Bot

[GitHub] [hadoop] slfan1989 commented on pull request #4327: HADOOP-18244. Fix Hadoop-Common JavaDoc Error on branch-3.3

2022-05-24 Thread GitBox
slfan1989 commented on PR #4327: URL: https://github.com/apache/hadoop/pull/4327#issuecomment-1135601667 Hi, @steveloughran, please help to review the code, thank you very much! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub

[GitHub] [hadoop] jianghuazhu commented on pull request #4351: HDFS-16592.Fix typo for BalancingPolicy.

2022-05-24 Thread GitBox
jianghuazhu commented on PR #4351: URL: https://github.com/apache/hadoop/pull/4351#issuecomment-1135682996 Here are some failing unit tests such as: TestWebHDFS TestUnderReplicatedBlocks TestExternalStoragePolicySatisfier TestIncrementalBlockReports TestRedudantBlocks

[jira] [Commented] (HADOOP-18140) Update default value of hadoop.ssl.enabled.protocols in the EncryptedShuffle doc

2022-05-24 Thread Samrat Deb (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18140?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17541404#comment-17541404 ] Samrat Deb commented on HADOOP-18140: - can i pick this task up ?  [~weichiu]  > Update default

[GitHub] [hadoop] hadoop-yetus commented on pull request #4347: HDFS-16586. Purge FsDatasetAsyncDiskService threadgroup; it causes BP…

2022-05-24 Thread GitBox
hadoop-yetus commented on PR #4347: URL: https://github.com/apache/hadoop/pull/4347#issuecomment-1135730139 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok: |

[GitHub] [hadoop] hadoop-yetus commented on pull request #4351: HDFS-16592.Fix typo for BalancingPolicy.

2022-05-24 Thread GitBox
hadoop-yetus commented on PR #4351: URL: https://github.com/apache/hadoop/pull/4351#issuecomment-1135664263 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok: |

[jira] [Commented] (HADOOP-18140) Update default value of hadoop.ssl.enabled.protocols in the EncryptedShuffle doc

2022-05-24 Thread Akira Ajisaka (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18140?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17541415#comment-17541415 ] Akira Ajisaka commented on HADOOP-18140: Sure. Assigned. > Update default value of

[jira] [Assigned] (HADOOP-18140) Update default value of hadoop.ssl.enabled.protocols in the EncryptedShuffle doc

2022-05-24 Thread Akira Ajisaka (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18140?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Akira Ajisaka reassigned HADOOP-18140: -- Assignee: Samrat Deb > Update default value of hadoop.ssl.enabled.protocols in the

[GitHub] [hadoop] steveloughran commented on pull request #4303: MAPREDUCE-7378. Change job temporary dir name to avoid delete by other jobs

2022-05-24 Thread GitBox
steveloughran commented on PR #4303: URL: https://github.com/apache/hadoop/pull/4303#issuecomment-1135590790 thanks for your work anyway. we do plan to make a hadoop release with the new committer later this year (it's shipping in cloudera cloud releases in preview mode, so i'll be

[GitHub] [hadoop] hadoop-yetus commented on pull request #4348: HDFS-16586. Purge FsDatasetAsyncDiskService threadgroup; it causes BP…

2022-05-24 Thread GitBox
hadoop-yetus commented on PR #4348: URL: https://github.com/apache/hadoop/pull/4348#issuecomment-1135653728 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok: |

[GitHub] [hadoop] ZanderXu commented on pull request #3806: HDFS-16386.Reduce DataNode load when FsDatasetAsyncDiskService is working.

2022-05-24 Thread GitBox
ZanderXu commented on PR #3806: URL: https://github.com/apache/hadoop/pull/3806#issuecomment-1135779919 @jianghuazhu I'm so sorry to discuss this issue again. Setting smaller MAX THREAD can reduce memory usage? [HDFS-16386](https://issues.apache.org/jira/browse/HDFS-16386) -- This

[GitHub] [hadoop] ZanderXu commented on pull request #3806: HDFS-16386.Reduce DataNode load when FsDatasetAsyncDiskService is working.

2022-05-24 Thread GitBox
ZanderXu commented on PR #3806: URL: https://github.com/apache/hadoop/pull/3806#issuecomment-1135792522 `ThreadPoolExecutor executor = new ThreadPoolExecutor( CORE_THREADS_PER_VOLUME, maxNumThreadsPerVolume, THREADS_KEEP_ALIVE_SECONDS, TimeUnit.SECONDS, new

[jira] [Work logged] (HADOOP-17461) Add thread-level IOStatistics Context

2022-05-24 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-17461?focusedWorklogId=774010=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-774010 ] ASF GitHub Bot logged work on HADOOP-17461: --- Author: ASF GitHub Bot

[jira] [Updated] (HADOOP-17461) Add thread-level IOStatistics Context

2022-05-24 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-17461?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated HADOOP-17461: Labels: pull-request-available (was: ) > Add thread-level IOStatistics Context >

[jira] [Commented] (HADOOP-17695) adls test suite TestAdlContractGetFileStatusLive failing with no assertJ on the classpath

2022-05-24 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-17695?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17541453#comment-17541453 ] Steve Loughran commented on HADOOP-17695: - this happens because test artifact exports are *not*

[GitHub] [hadoop] mehakmeet opened a new pull request, #4352: HADOOP-17461. Thread-level IOStatistics in S3A

2022-05-24 Thread GitBox
mehakmeet opened a new pull request, #4352: URL: https://github.com/apache/hadoop/pull/4352 ### Description of PR Adding Thread-level IOStatsitics in hadoop-common and implementing it in S3A Streams. ### How was this patch tested? Region: ap-south-1 `mvn clean verify

[jira] [Created] (HADOOP-18253) Hadoop 3.3.3 Spark write Mode.Overwrite breaks partitioned tables

2022-05-24 Thread Aaron Whiteway (Jira)
Aaron Whiteway created HADOOP-18253: --- Summary: Hadoop 3.3.3 Spark write Mode.Overwrite breaks partitioned tables Key: HADOOP-18253 URL: https://issues.apache.org/jira/browse/HADOOP-18253 Project:

[jira] [Created] (HADOOP-18252) Hadoop 3.3.3 Spark write Mode.Overwrite breaks partitioned tables

2022-05-24 Thread Aaron Whiteway (Jira)
Aaron Whiteway created HADOOP-18252: --- Summary: Hadoop 3.3.3 Spark write Mode.Overwrite breaks partitioned tables Key: HADOOP-18252 URL: https://issues.apache.org/jira/browse/HADOOP-18252 Project:

[GitHub] [hadoop] hadoop-yetus commented on pull request #4341: YARN-10487. Support getQueueUserAcls, listReservations, getApplicationAttempts, getContainerReport, getContainers, getResourceTypeInfo A

2022-05-24 Thread GitBox
hadoop-yetus commented on PR #4341: URL: https://github.com/apache/hadoop/pull/4341#issuecomment-1136052655 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok: |

[jira] [Created] (HADOOP-18254) Add in configuration option to enable prefetching

2022-05-24 Thread Ahmar Suhail (Jira)
Ahmar Suhail created HADOOP-18254: - Summary: Add in configuration option to enable prefetching Key: HADOOP-18254 URL: https://issues.apache.org/jira/browse/HADOOP-18254 Project: Hadoop Common

[GitHub] [hadoop] Hexiaoqiao commented on pull request #4342: HDFS-16588.Backport HDFS-16584 to branch-3.3.

2022-05-24 Thread GitBox
Hexiaoqiao commented on PR #4342: URL: https://github.com/apache/hadoop/pull/4342#issuecomment-1136099049 Committed to branch-3.3. Thanks @jianghuazhu for your contribution. Thanks @tomscut for your review. -- This is an automated message from the Apache Git Service. To respond to the

[GitHub] [hadoop] Hexiaoqiao merged pull request #4342: HDFS-16588.Backport HDFS-16584 to branch-3.3.

2022-05-24 Thread GitBox
Hexiaoqiao merged PR #4342: URL: https://github.com/apache/hadoop/pull/4342 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

[GitHub] [hadoop] hadoop-yetus commented on pull request #4349: HDFS-16590. Fix Junit Test Deprecated assertThat

2022-05-24 Thread GitBox
hadoop-yetus commented on PR #4349: URL: https://github.com/apache/hadoop/pull/4349#issuecomment-1136104471 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok: |

[jira] [Resolved] (HADOOP-18252) Hadoop 3.3.3 Spark write Mode.Overwrite breaks partitioned tables

2022-05-24 Thread Aaron Whiteway (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18252?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aaron Whiteway resolved HADOOP-18252. - Resolution: Invalid > Hadoop 3.3.3 Spark write Mode.Overwrite breaks partitioned

[jira] [Resolved] (HADOOP-18253) Hadoop 3.3.3 Spark write Mode.Overwrite breaks partitioned tables

2022-05-24 Thread Aaron Whiteway (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18253?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aaron Whiteway resolved HADOOP-18253. - Resolution: Invalid > Hadoop 3.3.3 Spark write Mode.Overwrite breaks partitioned

  1   2   >