[jira] [Moved] (HADOOP-17866) YarnClient Caching Addresses

2021-08-25 Thread Prabhu Joseph (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-17866?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Prabhu Joseph moved YARN-10857 to HADOOP-17866: --- Component/s: (was: yarn) (was: client)

[jira] [Assigned] (HADOOP-17865) [KMS] Some error messages are not logged in the KMS server

2021-08-25 Thread Wei-Chiu Chuang (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-17865?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wei-Chiu Chuang reassigned HADOOP-17865: Assignee: Wei-Chiu Chuang > [KMS] Some error messages are not logged in the KMS

[jira] [Created] (HADOOP-17865) [KMS] Some error messages are not logged in the KMS server

2021-08-25 Thread Wei-Chiu Chuang (Jira)
Wei-Chiu Chuang created HADOOP-17865: Summary: [KMS] Some error messages are not logged in the KMS server Key: HADOOP-17865 URL: https://issues.apache.org/jira/browse/HADOOP-17865 Project: Hadoop

[GitHub] [hadoop] Hexiaoqiao commented on pull request #3320: HDFS-16182.numOfReplicas is given the wrong value in BlockPlacementPolicyDefault$chooseTarget can cause DataStreamer to fail with Heteroge

2021-08-25 Thread GitBox
Hexiaoqiao commented on pull request #3320: URL: https://github.com/apache/hadoop/pull/3320#issuecomment-906072715 @Neilxzn Please fix checkstyle and check failed unit tests first. Thanks -- This is an automated message from the Apache Git Service. To respond to the message, please log

[GitHub] [hadoop] Hexiaoqiao commented on pull request #3200: HDFS-15160. branch-3.2. ReplicaMap, Disk Balancer, Directory Scanner and various FsDatasetImpl methods should use datanode readlock.

2021-08-25 Thread GitBox
Hexiaoqiao commented on pull request #3200: URL: https://github.com/apache/hadoop/pull/3200#issuecomment-906071805 @jojochuang Would you mind to have another check? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the

[GitHub] [hadoop] jojochuang commented on a change in pull request #3322: HDFS-6874. Add GETFILEBLOCKLOCATIONS operation to HttpFS.

2021-08-25 Thread GitBox
jojochuang commented on a change in pull request #3322: URL: https://github.com/apache/hadoop/pull/3322#discussion_r696260085 ## File path: hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/web/JsonUtilClient.java ## @@ -908,4 +911,51 @@ private

[GitHub] [hadoop] Hexiaoqiao commented on pull request #3200: HDFS-15160. branch-3.2. ReplicaMap, Disk Balancer, Directory Scanner and various FsDatasetImpl methods should use datanode readlock.

2021-08-25 Thread GitBox
Hexiaoqiao commented on pull request #3200: URL: https://github.com/apache/hadoop/pull/3200#issuecomment-906060946 Thanks @amahussein for your works. I trigger Yetus and it seems works well. Failed unit test `TestDirectoryScanner ` looks not related to changes. AND verify this feature

[GitHub] [hadoop] Neilxzn commented on pull request #3320: HDFS-16182.numOfReplicas is given the wrong value in BlockPlacementPolicyDefault$chooseTarget can cause DataStreamer to fail with Heterogeneo

2021-08-25 Thread GitBox
Neilxzn commented on pull request #3320: URL: https://github.com/apache/hadoop/pull/3320#issuecomment-906027161 Failed junit tests seem unrelated. And I try these tests in IDEA locally, these tests run pass. - hadoop.hdfs.server.namenode.ha.TestEditLogTailer -

[GitHub] [hadoop] jojochuang commented on a change in pull request #3322: HDFS-6874. Add GETFILEBLOCKLOCATIONS operation to HttpFS.

2021-08-25 Thread GitBox
jojochuang commented on a change in pull request #3322: URL: https://github.com/apache/hadoop/pull/3322#discussion_r696240622 ## File path: hadoop-hdfs-project/hadoop-hdfs-httpfs/src/test/java/org/apache/hadoop/fs/http/server/TestHttpFSServer.java ## @@ -2002,4 +2003,38 @@

[GitHub] [hadoop] jianghuazhu commented on a change in pull request #3302: HDFS-16173.Improve CopyCommands#Put#executor queue configurability.

2021-08-25 Thread GitBox
jianghuazhu commented on a change in pull request #3302: URL: https://github.com/apache/hadoop/pull/3302#discussion_r696235839 ## File path: hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/shell/CopyCommands.java ## @@ -329,6 +340,25 @@ private void

[GitHub] [hadoop] fengnanli commented on pull request #3284: HDFS-16157. Support configuring DNS record to get list of journal nodes.

2021-08-25 Thread GitBox
fengnanli commented on pull request #3284: URL: https://github.com/apache/hadoop/pull/3284#issuecomment-905970607 Thanks for the work @LeonGao91 and review from @Hexiaoqiao -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub

[GitHub] [hadoop] fengnanli merged pull request #3284: HDFS-16157. Support configuring DNS record to get list of journal nodes.

2021-08-25 Thread GitBox
fengnanli merged pull request #3284: URL: https://github.com/apache/hadoop/pull/3284 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

[GitHub] [hadoop] hadoop-yetus commented on pull request #3320: HDFS-16182.numOfReplicas is given the wrong value in BlockPlacementPolicyDefault$chooseTarget can cause DataStreamer to fail with Hetero

2021-08-25 Thread GitBox
hadoop-yetus commented on pull request #3320: URL: https://github.com/apache/hadoop/pull/3320#issuecomment-905964389 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0

[GitHub] [hadoop] hadoop-yetus commented on pull request #3314: YARN-10891. Extend QueueInfo with max-parallel-apps in CS.

2021-08-25 Thread GitBox
hadoop-yetus commented on pull request #3314: URL: https://github.com/apache/hadoop/pull/3314#issuecomment-905915020 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0

[jira] [Commented] (HADOOP-17857) Check real user ACLs in addition to proxied user ACLs

2021-08-25 Thread Hadoop QA (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-17857?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17404740#comment-17404740 ] Hadoop QA commented on HADOOP-17857: | (/) *{color:green}+1 overall{color}* | \\ \\ || Vote ||

[jira] [Work logged] (HADOOP-17682) ABFS: Support FileStatus input to OpenFileWithOptions() via OpenFileParameters

2021-08-25 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-17682?focusedWorklogId=641945=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-641945 ] ASF GitHub Bot logged work on HADOOP-17682: --- Author: ASF GitHub Bot

[GitHub] [hadoop] steveloughran commented on a change in pull request #2975: HADOOP-17682. ABFS: Support FileStatus input to OpenFileWithOptions() via OpenFileParameters

2021-08-25 Thread GitBox
steveloughran commented on a change in pull request #2975: URL: https://github.com/apache/hadoop/pull/2975#discussion_r696063619 ## File path: hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azurebfs/AzureBlobFileSystemStore.java ## @@ -669,44 +672,64 @@ public

[jira] [Work logged] (HADOOP-17682) ABFS: Support FileStatus input to OpenFileWithOptions() via OpenFileParameters

2021-08-25 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-17682?focusedWorklogId=641936=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-641936 ] ASF GitHub Bot logged work on HADOOP-17682: --- Author: ASF GitHub Bot

[jira] [Work logged] (HADOOP-17682) ABFS: Support FileStatus input to OpenFileWithOptions() via OpenFileParameters

2021-08-25 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-17682?focusedWorklogId=641935=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-641935 ] ASF GitHub Bot logged work on HADOOP-17682: --- Author: ASF GitHub Bot

[jira] [Work logged] (HADOOP-17682) ABFS: Support FileStatus input to OpenFileWithOptions() via OpenFileParameters

2021-08-25 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-17682?focusedWorklogId=641934=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-641934 ] ASF GitHub Bot logged work on HADOOP-17682: --- Author: ASF GitHub Bot

[GitHub] [hadoop] hadoop-yetus removed a comment on pull request #2975: HADOOP-17682. ABFS: Support FileStatus input to OpenFileWithOptions() via OpenFileParameters

2021-08-25 Thread GitBox
hadoop-yetus removed a comment on pull request #2975: URL: https://github.com/apache/hadoop/pull/2975#issuecomment-49397 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment.

[GitHub] [hadoop] hadoop-yetus removed a comment on pull request #2975: HADOOP-17682. ABFS: Support FileStatus input to OpenFileWithOptions() via OpenFileParameters

2021-08-25 Thread GitBox
hadoop-yetus removed a comment on pull request #2975: URL: https://github.com/apache/hadoop/pull/2975#issuecomment-888794912 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment.

[GitHub] [hadoop] hadoop-yetus removed a comment on pull request #2975: HADOOP-17682. ABFS: Support FileStatus input to OpenFileWithOptions() via OpenFileParameters

2021-08-25 Thread GitBox
hadoop-yetus removed a comment on pull request #2975: URL: https://github.com/apache/hadoop/pull/2975#issuecomment-879797819 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment.

[jira] [Work logged] (HADOOP-17682) ABFS: Support FileStatus input to OpenFileWithOptions() via OpenFileParameters

2021-08-25 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-17682?focusedWorklogId=641932=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-641932 ] ASF GitHub Bot logged work on HADOOP-17682: --- Author: ASF GitHub Bot

[jira] [Work logged] (HADOOP-17682) ABFS: Support FileStatus input to OpenFileWithOptions() via OpenFileParameters

2021-08-25 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-17682?focusedWorklogId=641933=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-641933 ] ASF GitHub Bot logged work on HADOOP-17682: --- Author: ASF GitHub Bot

[GitHub] [hadoop] hadoop-yetus removed a comment on pull request #2975: HADOOP-17682. ABFS: Support FileStatus input to OpenFileWithOptions() via OpenFileParameters

2021-08-25 Thread GitBox
hadoop-yetus removed a comment on pull request #2975: URL: https://github.com/apache/hadoop/pull/2975#issuecomment-879494586 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:|

[GitHub] [hadoop] hadoop-yetus removed a comment on pull request #2975: HADOOP-17682. ABFS: Support FileStatus input to OpenFileWithOptions() via OpenFileParameters

2021-08-25 Thread GitBox
hadoop-yetus removed a comment on pull request #2975: URL: https://github.com/apache/hadoop/pull/2975#issuecomment-833554277 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment.

[jira] [Work logged] (HADOOP-17682) ABFS: Support FileStatus input to OpenFileWithOptions() via OpenFileParameters

2021-08-25 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-17682?focusedWorklogId=641930=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-641930 ] ASF GitHub Bot logged work on HADOOP-17682: --- Author: ASF GitHub Bot

[jira] [Work logged] (HADOOP-17682) ABFS: Support FileStatus input to OpenFileWithOptions() via OpenFileParameters

2021-08-25 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-17682?focusedWorklogId=641931=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-641931 ] ASF GitHub Bot logged work on HADOOP-17682: --- Author: ASF GitHub Bot

[jira] [Work logged] (HADOOP-17682) ABFS: Support FileStatus input to OpenFileWithOptions() via OpenFileParameters

2021-08-25 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-17682?focusedWorklogId=641929=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-641929 ] ASF GitHub Bot logged work on HADOOP-17682: --- Author: ASF GitHub Bot

[GitHub] [hadoop] hadoop-yetus removed a comment on pull request #2975: HADOOP-17682. ABFS: Support FileStatus input to OpenFileWithOptions() via OpenFileParameters

2021-08-25 Thread GitBox
hadoop-yetus removed a comment on pull request #2975: URL: https://github.com/apache/hadoop/pull/2975#issuecomment-833013347 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment.

[GitHub] [hadoop] hadoop-yetus removed a comment on pull request #2975: HADOOP-17682. ABFS: Support FileStatus input to OpenFileWithOptions() via OpenFileParameters

2021-08-25 Thread GitBox
hadoop-yetus removed a comment on pull request #2975: URL: https://github.com/apache/hadoop/pull/2975#issuecomment-831949886 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment.

[GitHub] [hadoop] hadoop-yetus removed a comment on pull request #2975: HADOOP-17682. ABFS: Support FileStatus input to OpenFileWithOptions() via OpenFileParameters

2021-08-25 Thread GitBox
hadoop-yetus removed a comment on pull request #2975: URL: https://github.com/apache/hadoop/pull/2975#issuecomment-831945482 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:|

[jira] [Commented] (HADOOP-17857) Check real user ACLs in addition to proxied user ACLs

2021-08-25 Thread Eric Payne (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-17857?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17404659#comment-17404659 ] Eric Payne commented on HADOOP-17857: - [~zhoukang] , [~snemeth], I believe that this JIRA is the

[jira] [Commented] (HADOOP-17857) Check real user ACLs in addition to proxied user ACLs

2021-08-25 Thread Eric Payne (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-17857?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17404656#comment-17404656 ] Eric Payne commented on HADOOP-17857: - Thanks [~ahussein] for the review and comments! I have

[jira] [Updated] (HADOOP-17857) Check real user ACLs in addition to proxied user ACLs

2021-08-25 Thread Eric Payne (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-17857?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Eric Payne updated HADOOP-17857: Attachment: HADOOP-17857.002.patch > Check real user ACLs in addition to proxied user ACLs >

[jira] [Work logged] (HADOOP-17833) Improve Magic Committer Performance

2021-08-25 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-17833?focusedWorklogId=641861=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-641861 ] ASF GitHub Bot logged work on HADOOP-17833: --- Author: ASF GitHub Bot

[GitHub] [hadoop] hadoop-yetus commented on pull request #3289: HADOOP-17833. Improve Magic Committer performance

2021-08-25 Thread GitBox
hadoop-yetus commented on pull request #3289: URL: https://github.com/apache/hadoop/pull/3289#issuecomment-905742205 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0

[GitHub] [hadoop] Neilxzn commented on pull request #3320: HDFS-16182.numOfReplicas is given the wrong value in BlockPlacementPolicyDefault$chooseTarget can cause DataStreamer to fail with Heterogeneo

2021-08-25 Thread GitBox
Neilxzn commented on pull request #3320: URL: https://github.com/apache/hadoop/pull/3320#issuecomment-905714180 > Thanks @Neilxzn for your works. Great catch here! LGTM. +1 once leaved nits comments fixed. I think it is a bug just when fallback storage policy happens. -- This is

[GitHub] [hadoop] Neilxzn removed a comment on pull request #3320: HDFS-16182.numOfReplicas is given the wrong value in BlockPlacementPolicyDefault$chooseTarget can cause DataStreamer to fail with Het

2021-08-25 Thread GitBox
Neilxzn removed a comment on pull request #3320: URL: https://github.com/apache/hadoop/pull/3320#issuecomment-905710927 Just wonder if this is common bug for every chooseTarget invokes when assign requiredStorageTypes.size() to numOfReplicas here? or just for set storagePolicy? -- This

[GitHub] [hadoop] Neilxzn commented on pull request #3320: HDFS-16182.numOfReplicas is given the wrong value in BlockPlacementPolicyDefault$chooseTarget can cause DataStreamer to fail with Heterogeneo

2021-08-25 Thread GitBox
Neilxzn commented on pull request #3320: URL: https://github.com/apache/hadoop/pull/3320#issuecomment-905710927 Just wonder if this is common bug for every chooseTarget invokes when assign requiredStorageTypes.size() to numOfReplicas here? or just for set storagePolicy? -- This is an

[GitHub] [hadoop] Neilxzn commented on a change in pull request #3320: HDFS-16182.numOfReplicas is given the wrong value in BlockPlacementPolicyDefault$chooseTarget can cause DataStreamer to fail with

2021-08-25 Thread GitBox
Neilxzn commented on a change in pull request #3320: URL: https://github.com/apache/hadoop/pull/3320#discussion_r695940727 ## File path: hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestBlockStoragePolicy.java ## @@ -1337,6 +1337,50 @@ public void

[GitHub] [hadoop] hadoop-yetus commented on pull request #3314: YARN-10891. Extend QueueInfo with max-parallel-apps in CS.

2021-08-25 Thread GitBox
hadoop-yetus commented on pull request #3314: URL: https://github.com/apache/hadoop/pull/3314#issuecomment-905697347 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0

[GitHub] [hadoop] hadoop-yetus commented on pull request #3333: YARN-10576. Update Capacity Scheduler documentation about JSON-based …

2021-08-25 Thread GitBox
hadoop-yetus commented on pull request #: URL: https://github.com/apache/hadoop/pull/#issuecomment-905681089 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0

[GitHub] [hadoop] goiri commented on a change in pull request #3328: HDFS-16183. RBF: Delete unnecessary metric of proxyOpComplete in routerRpcClient

2021-08-25 Thread GitBox
goiri commented on a change in pull request #3328: URL: https://github.com/apache/hadoop/pull/3328#discussion_r695904216 ## File path: hadoop-hdfs-project/hadoop-hdfs-rbf/src/main/java/org/apache/hadoop/hdfs/server/federation/router/RouterRpcClient.java ## @@ -517,7 +517,6 @@

[GitHub] [hadoop] brumi1024 opened a new pull request #3333: YARN-10576. Update Capacity Scheduler documentation about JSON-based …

2021-08-25 Thread GitBox
brumi1024 opened a new pull request #: URL: https://github.com/apache/hadoop/pull/ …placement mapping ### Description of PR ### How was this patch tested? ### For code changes: - [ ] Does the title or this PR starts with the corresponding

[GitHub] [hadoop] hadoop-yetus commented on pull request #3200: HDFS-15160. branch-3.2. ReplicaMap, Disk Balancer, Directory Scanner and various FsDatasetImpl methods should use datanode readlock.

2021-08-25 Thread GitBox
hadoop-yetus commented on pull request #3200: URL: https://github.com/apache/hadoop/pull/3200#issuecomment-905580662 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0

[GitHub] [hadoop] hadoop-yetus commented on pull request #3320: HDFS-16182.numOfReplicas is given the wrong value in BlockPlacementPolicyDefault$chooseTarget can cause DataStreamer to fail with Hetero

2021-08-25 Thread GitBox
hadoop-yetus commented on pull request #3320: URL: https://github.com/apache/hadoop/pull/3320#issuecomment-905536745 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0

[jira] [Work logged] (HADOOP-17833) Improve Magic Committer Performance

2021-08-25 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-17833?focusedWorklogId=641680=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-641680 ] ASF GitHub Bot logged work on HADOOP-17833: --- Author: ASF GitHub Bot

[GitHub] [hadoop] hadoop-yetus removed a comment on pull request #3289: HADOOP-17833. Improve Magic Committer performance

2021-08-25 Thread GitBox
hadoop-yetus removed a comment on pull request #3289: URL: https://github.com/apache/hadoop/pull/3289#issuecomment-904031779 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:|

[jira] [Work logged] (HADOOP-17863) ABFS: Fix compiler deprecation warning in TextFileBasedIdentityHandler

2021-08-25 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-17863?focusedWorklogId=641675=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-641675 ] ASF GitHub Bot logged work on HADOOP-17863: --- Author: ASF GitHub Bot

[GitHub] [hadoop] hadoop-yetus commented on pull request #3332: HADOOP-17863. ABFS: Fix compiler deprecation warning in TextFileBasedIdentityHandler

2021-08-25 Thread GitBox
hadoop-yetus commented on pull request #3332: URL: https://github.com/apache/hadoop/pull/3332#issuecomment-905489502 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0

[jira] [Commented] (HADOOP-14693) Upgrade JUnit from 4 to 5

2021-08-25 Thread Akira Ajisaka (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-14693?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17404429#comment-17404429 ] Akira Ajisaka commented on HADOOP-14693: If we are going to create JUnit 5 based

[jira] [Commented] (HADOOP-14693) Upgrade JUnit from 4 to 5

2021-08-25 Thread Akira Ajisaka (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-14693?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17404427#comment-17404427 ] Akira Ajisaka commented on HADOOP-14693: Hi [~smeng] I appreciated that you are trying to

[GitHub] [hadoop] aajisaka commented on a change in pull request #3302: HDFS-16173.Improve CopyCommands#Put#executor queue configurability.

2021-08-25 Thread GitBox
aajisaka commented on a change in pull request #3302: URL: https://github.com/apache/hadoop/pull/3302#discussion_r695701899 ## File path: hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/shell/CopyCommands.java ## @@ -329,6 +340,25 @@ private void

[jira] [Updated] (HADOOP-15939) Filter overlapping objenesis class in hadoop-client-minicluster

2021-08-25 Thread Akira Ajisaka (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-15939?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Akira Ajisaka updated HADOOP-15939: --- Fix Version/s: 3.2.3 Backported to branch-3.2 and branch-3.2.3. > Filter overlapping

[jira] [Created] (HADOOP-17864) ABFS: Fork AbfsHttpOperation to add alternate connection

2021-08-25 Thread Sneha Vijayarajan (Jira)
Sneha Vijayarajan created HADOOP-17864: -- Summary: ABFS: Fork AbfsHttpOperation to add alternate connection Key: HADOOP-17864 URL: https://issues.apache.org/jira/browse/HADOOP-17864 Project:

[GitHub] [hadoop] bbeaudreault commented on a change in pull request #3271: HDFS-16155: Allow configurable exponential backoff in DFSInputStream refetchLocations

2021-08-25 Thread GitBox
bbeaudreault commented on a change in pull request #3271: URL: https://github.com/apache/hadoop/pull/3271#discussion_r695678157 ## File path: hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/client/impl/TestFetchBlockLocationsRetryer.java ## @@ -0,0 +1,107

[GitHub] [hadoop] bbeaudreault commented on a change in pull request #3271: HDFS-16155: Allow configurable exponential backoff in DFSInputStream refetchLocations

2021-08-25 Thread GitBox
bbeaudreault commented on a change in pull request #3271: URL: https://github.com/apache/hadoop/pull/3271#discussion_r695678157 ## File path: hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/client/impl/TestFetchBlockLocationsRetryer.java ## @@ -0,0 +1,107

[jira] [Work logged] (HADOOP-17863) ABFS: Fix compiler deprecation warning in TextFileBasedIdentityHandler

2021-08-25 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-17863?focusedWorklogId=641628=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-641628 ] ASF GitHub Bot logged work on HADOOP-17863: --- Author: ASF GitHub Bot

[jira] [Updated] (HADOOP-17863) ABFS: Fix compiler deprecation warning in TextFileBasedIdentityHandler

2021-08-25 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-17863?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated HADOOP-17863: Labels: pull-request-available (was: ) > ABFS: Fix compiler deprecation warning in

[GitHub] [hadoop] sumangala-patki opened a new pull request #3332: HADOOP-17863. ABFS: Fix compiler deprecation warning in TextFileBasedIdentityHandler

2021-08-25 Thread GitBox
sumangala-patki opened a new pull request #3332: URL: https://github.com/apache/hadoop/pull/3332 TextFileBasedIdentityHandler uses an instance of LineIterator whose closeQuietly method has been deprecated, resulting in compiler warnings during yetus runs. This PR fixes it by leveraging a

[jira] [Work logged] (HADOOP-17862) ABFS: Fix unchecked cast compiler warning for AbfsListStatusRemoteIterator

2021-08-25 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-17862?focusedWorklogId=641625=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-641625 ] ASF GitHub Bot logged work on HADOOP-17862: --- Author: ASF GitHub Bot

[GitHub] [hadoop] sumangala-patki commented on pull request #3331: HADOOP-17862. ABFS: Fix unchecked cast compiler warning for AbfsListStatusRemoteIterator

2021-08-25 Thread GitBox
sumangala-patki commented on pull request #3331: URL: https://github.com/apache/hadoop/pull/3331#issuecomment-905424970 TEST RESULTS HNS Account Location: East US 2 NonHNS Account Location: East US 2, Central US ``` AppendBlob HNS OAuth [WARNING] Tests run: 106,

[jira] [Created] (HADOOP-17863) ABFS: Fix compiler deprecation warning in TextFileBasedIdentityHandler

2021-08-25 Thread Sumangala Patki (Jira)
Sumangala Patki created HADOOP-17863: Summary: ABFS: Fix compiler deprecation warning in TextFileBasedIdentityHandler Key: HADOOP-17863 URL: https://issues.apache.org/jira/browse/HADOOP-17863

[jira] [Work logged] (HADOOP-17862) ABFS: Fix unchecked cast compiler warning for AbfsListStatusRemoteIterator

2021-08-25 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-17862?focusedWorklogId=641622=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-641622 ] ASF GitHub Bot logged work on HADOOP-17862: --- Author: ASF GitHub Bot

[GitHub] [hadoop] hadoop-yetus commented on pull request #3331: HADOOP-17862. ABFS: Fix unchecked cast compiler warning for AbfsListStatusRemoteIterator

2021-08-25 Thread GitBox
hadoop-yetus commented on pull request #3331: URL: https://github.com/apache/hadoop/pull/3331#issuecomment-905419314 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0

[GitHub] [hadoop] bbeaudreault commented on a change in pull request #3271: HDFS-16155: Allow configurable exponential backoff in DFSInputStream refetchLocations

2021-08-25 Thread GitBox
bbeaudreault commented on a change in pull request #3271: URL: https://github.com/apache/hadoop/pull/3271#discussion_r695662268 ## File path: hadoop-hdfs-project/hadoop-hdfs/src/main/resources/hdfs-default.xml ## @@ -4440,8 +4440,29 @@ 3000 Base time window in ms

[GitHub] [hadoop] hadoop-yetus commented on pull request #3314: YARN-10891. Extend QueueInfo with max-parallel-apps in CS.

2021-08-25 Thread GitBox
hadoop-yetus commented on pull request #3314: URL: https://github.com/apache/hadoop/pull/3314#issuecomment-905414155 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0

[jira] [Commented] (HADOOP-17847) S3AInstrumentation Closing output stream statistics while data is still marked as pending upload in OutputStreamStatistics

2021-08-25 Thread Chris Nix (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-17847?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17404352#comment-17404352 ] Chris Nix commented on HADOOP-17847: Not wishing to divert the original report regarding spark, but

[GitHub] [hadoop] Hexiaoqiao commented on a change in pull request #3271: HDFS-16155: Allow configurable exponential backoff in DFSInputStream refetchLocations

2021-08-25 Thread GitBox
Hexiaoqiao commented on a change in pull request #3271: URL: https://github.com/apache/hadoop/pull/3271#discussion_r695628043 ## File path: hadoop-hdfs-project/hadoop-hdfs/src/main/resources/hdfs-default.xml ## @@ -4440,8 +4440,29 @@ 3000 Base time window in ms for

[jira] [Work logged] (HADOOP-17861) improve YARN Registry DNS Server qps

2021-08-25 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-17861?focusedWorklogId=641608=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-641608 ] ASF GitHub Bot logged work on HADOOP-17861: --- Author: ASF GitHub Bot

[GitHub] [hadoop] hadoop-yetus commented on pull request #3326: HADOOP-17861. improve YARN Registry DNS Server qps

2021-08-25 Thread GitBox
hadoop-yetus commented on pull request #3326: URL: https://github.com/apache/hadoop/pull/3326#issuecomment-905386948 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0

[GitHub] [hadoop] jojochuang commented on a change in pull request #3322: HDFS-6874. Add GETFILEBLOCKLOCATIONS operation to HttpFS.

2021-08-25 Thread GitBox
jojochuang commented on a change in pull request #3322: URL: https://github.com/apache/hadoop/pull/3322#discussion_r695375296 ## File path: hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/web/JsonUtilClient.java ## @@ -908,4 +911,51 @@ private

[GitHub] [hadoop] hadoop-yetus commented on pull request #3330: YARN-10522. Document for Flexible Auto Queue Creation in Capacity Scheduler.

2021-08-25 Thread GitBox
hadoop-yetus commented on pull request #3330: URL: https://github.com/apache/hadoop/pull/3330#issuecomment-905382812 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0

[GitHub] [hadoop] Hexiaoqiao commented on a change in pull request #3320: HDFS-16182.numOfReplicas is given the wrong value in BlockPlacementPolicyDefault$chooseTarget can cause DataStreamer to fail w

2021-08-25 Thread GitBox
Hexiaoqiao commented on a change in pull request #3320: URL: https://github.com/apache/hadoop/pull/3320#discussion_r695602966 ## File path: hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestBlockStoragePolicy.java ## @@ -1337,6 +1337,50 @@ public void

[GitHub] [hadoop] jianghuazhu commented on pull request #3302: HDFS-16173.Improve CopyCommands#Put#executor queue configurability.

2021-08-25 Thread GitBox
jianghuazhu commented on pull request #3302: URL: https://github.com/apache/hadoop/pull/3302#issuecomment-905351084 Thanks @jojochuang for the comment. I will continue to work. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [hadoop] jojochuang merged pull request #3307: HDFS-16175.Improve the configurable value of Server #PURGE_INTERVAL_NANOS.

2021-08-25 Thread GitBox
jojochuang merged pull request #3307: URL: https://github.com/apache/hadoop/pull/3307 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

[GitHub] [hadoop] jojochuang commented on pull request #3307: HDFS-16175.Improve the configurable value of Server #PURGE_INTERVAL_NANOS.

2021-08-25 Thread GitBox
jojochuang commented on pull request #3307: URL: https://github.com/apache/hadoop/pull/3307#issuecomment-905340718 Merging it -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific

[GitHub] [hadoop] jojochuang commented on pull request #3302: HDFS-16173.Improve CopyCommands#Put#executor queue configurability.

2021-08-25 Thread GitBox
jojochuang commented on pull request #3302: URL: https://github.com/apache/hadoop/pull/3302#issuecomment-905339585 @jianghuazhu could you help verify TestCLI isn't broken by this change? We'll have to verify it manually. Thanks. -- This is an automated message from the Apache Git

[jira] [Commented] (HADOOP-17848) Hadoop NativeAzureFileSystem append removes ownership set on the file

2021-08-25 Thread Prabhu Joseph (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-17848?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17404300#comment-17404300 ] Prabhu Joseph commented on HADOOP-17848: [~anoop.hbase] bq. The call

[jira] [Updated] (HADOOP-17848) Hadoop NativeAzureFileSystem append removes ownership set on the file

2021-08-25 Thread Prabhu Joseph (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-17848?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Prabhu Joseph updated HADOOP-17848: --- Description: *Repro:* Create Operation sets ownership whereas append operation removes the

[GitHub] [hadoop] jianghuazhu commented on pull request #3307: HDFS-16175.Improve the configurable value of Server #PURGE_INTERVAL_NANOS.

2021-08-25 Thread GitBox
jianghuazhu commented on pull request #3307: URL: https://github.com/apache/hadoop/pull/3307#issuecomment-905322240 @ayushtkn , thanks for your help. I don't know if this PR request can be merged into the trunk branch. If can, can you help me deal with it? thank you very much. (I'm

[GitHub] [hadoop] jianghuazhu commented on pull request #3302: HDFS-16173.Improve CopyCommands#Put#executor queue configurability.

2021-08-25 Thread GitBox
jianghuazhu commented on pull request #3302: URL: https://github.com/apache/hadoop/pull/3302#issuecomment-905321805 @ayushtkn , thanks for your help. I don't know if this PR request can be merged into the trunk branch. If can, can you help me deal with it? thank you very much. (I'm

[GitHub] [hadoop] hadoop-yetus commented on pull request #3200: HDFS-15160. branch-3.2. ReplicaMap, Disk Balancer, Directory Scanner and various FsDatasetImpl methods should use datanode readlock.

2021-08-25 Thread GitBox
hadoop-yetus commented on pull request #3200: URL: https://github.com/apache/hadoop/pull/3200#issuecomment-905303834 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0

[GitHub] [hadoop] tasanuma commented on pull request #3329: HDFS-16184. De-flake TestBlockScanner#testSkipRecentAccessFile

2021-08-25 Thread GitBox
tasanuma commented on pull request #3329: URL: https://github.com/apache/hadoop/pull/3329#issuecomment-905302618 Merged. Thanks for your contribution, @virajjasani. Thanks for your review, @ayushtkn. > Also, it's good to see two consecutive +1s from QA bot for hadoop-hdfs. Haven't

[GitHub] [hadoop] tasanuma merged pull request #3329: HDFS-16184. De-flake TestBlockScanner#testSkipRecentAccessFile

2021-08-25 Thread GitBox
tasanuma merged pull request #3329: URL: https://github.com/apache/hadoop/pull/3329 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

[jira] [Work logged] (HADOOP-17862) ABFS: Fix unchecked cast compiler warning for AbfsListStatusRemoteIterator

2021-08-25 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-17862?focusedWorklogId=641555=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-641555 ] ASF GitHub Bot logged work on HADOOP-17862: --- Author: ASF GitHub Bot

[GitHub] [hadoop] hadoop-yetus commented on pull request #3331: HADOOP-17862. ABFS: Fix unchecked cast compiler warning for AbfsListStatusRemoteIterator

2021-08-25 Thread GitBox
hadoop-yetus commented on pull request #3331: URL: https://github.com/apache/hadoop/pull/3331#issuecomment-905296286 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0

[GitHub] [hadoop] virajjasani commented on pull request #3329: HDFS-16184. De-flake TestBlockScanner#testSkipRecentAccessFile

2021-08-25 Thread GitBox
virajjasani commented on pull request #3329: URL: https://github.com/apache/hadoop/pull/3329#issuecomment-905248069 Also, it's good to see two consecutive +1s from QA bot for hadoop-hdfs. Haven't seen more than single +1 from QA build for long time on trunk. -- This is an automated

[GitHub] [hadoop] virajjasani commented on pull request #3329: HDFS-16184. De-flake TestBlockScanner#testSkipRecentAccessFile

2021-08-25 Thread GitBox
virajjasani commented on pull request #3329: URL: https://github.com/apache/hadoop/pull/3329#issuecomment-905246751 Thanks to both of you for quick reviews @tasanuma @ayushtkn !! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to

[jira] [Work logged] (HADOOP-17862) ABFS: Fix unchecked cast compiler warning for AbfsListStatusRemoteIterator

2021-08-25 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-17862?focusedWorklogId=641533=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-641533 ] ASF GitHub Bot logged work on HADOOP-17862: --- Author: ASF GitHub Bot

[GitHub] [hadoop] hadoop-yetus commented on pull request #3331: HADOOP-17862. ABFS: Fix unchecked cast compiler warning for AbfsListStatusRemoteIterator

2021-08-25 Thread GitBox
hadoop-yetus commented on pull request #3331: URL: https://github.com/apache/hadoop/pull/3331#issuecomment-905245795 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0

[jira] [Work logged] (HADOOP-17858) Avoid possible class loading deadlock with VerifierNone initialization

2021-08-25 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-17858?focusedWorklogId=641530=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-641530 ] ASF GitHub Bot logged work on HADOOP-17858: --- Author: ASF GitHub Bot

[GitHub] [hadoop] virajjasani commented on pull request #3321: HADOOP-17858. Avoid possible class loading deadlock with VerifierNone initialization

2021-08-25 Thread GitBox
virajjasani commented on pull request #3321: URL: https://github.com/apache/hadoop/pull/3321#issuecomment-905244311 Thank you @tasanuma! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the

[GitHub] [hadoop] virajjasani commented on pull request #3329: HDFS-16184. De-flake TestBlockScanner#testSkipRecentAccessFile

2021-08-25 Thread GitBox
virajjasani commented on pull request #3329: URL: https://github.com/apache/hadoop/pull/3329#issuecomment-905241473 Thanks @ayushtkn for detailed review with cross-verification. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [hadoop] Neilxzn commented on a change in pull request #3320: HDFS-16182.numOfReplicas is given the wrong value in BlockPlacementPolicyDefault$chooseTarget can cause DataStreamer to fail with

2021-08-25 Thread GitBox
Neilxzn commented on a change in pull request #3320: URL: https://github.com/apache/hadoop/pull/3320#discussion_r695451621 ## File path: hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/blockmanagement/BlockPlacementPolicyDefault.java ## @@ -469,7