[jira] [Work logged] (HADOOP-17998) Enable get command run with multi-thread

2021-11-10 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-17998?focusedWorklogId=680178=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-680178 ] ASF GitHub Bot logged work on HADOOP-17998: --- Author: ASF GitHub Bot

[GitHub] [hadoop] hadoop-yetus commented on pull request #3645: HADOOP-17998. Enable get command run with multi-thread.

2021-11-10 Thread GitBox
hadoop-yetus commented on pull request #3645: URL: https://github.com/apache/hadoop/pull/3645#issuecomment-966055357 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0

[jira] [Work logged] (HADOOP-18003) Add a method appendIfAbsent for CallerContext

2021-11-10 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18003?focusedWorklogId=680169=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-680169 ] ASF GitHub Bot logged work on HADOOP-18003: --- Author: ASF GitHub Bot

[GitHub] [hadoop] tomscut commented on pull request #3644: HADOOP-18003. Add a method appendIfAbsent for CallerContext

2021-11-10 Thread GitBox
tomscut commented on pull request #3644: URL: https://github.com/apache/hadoop/pull/3644#issuecomment-966014434 Hi @goiri @aajisaka @tasanuma @jojochuang @iwasakims @ferhui , please take a look at this. Thank you very much. -- This is an automated message from the Apache Git Service. To

[jira] [Work logged] (HADOOP-18003) Add a method appendIfAbsent for CallerContext

2021-11-10 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18003?focusedWorklogId=680168=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-680168 ] ASF GitHub Bot logged work on HADOOP-18003: --- Author: ASF GitHub Bot

[GitHub] [hadoop] hadoop-yetus commented on pull request #3644: HADOOP-18003. Add a method appendIfAbsent for CallerContext

2021-11-10 Thread GitBox
hadoop-yetus commented on pull request #3644: URL: https://github.com/apache/hadoop/pull/3644#issuecomment-966011391 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0

[GitHub] [hadoop] ashutoshcipher commented on pull request #3592: YARN-10991. Fix to ignore the grouping "[]" for resourcesStr in parse…

2021-11-10 Thread GitBox
ashutoshcipher commented on pull request #3592: URL: https://github.com/apache/hadoop/pull/3592#issuecomment-966010320 @aajisaka Thanks for the review. I will work on adding the regression test. -- This is an automated message from the Apache Git Service. To respond to the message,

[jira] [Commented] (HADOOP-17998) Enable get command run with multi-thread

2021-11-10 Thread Chengwei Wang (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-17998?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17442076#comment-17442076 ] Chengwei Wang commented on HADOOP-17998: Hi [~ste...@apache.org] , PR

[jira] [Updated] (HADOOP-17998) Enable get command run with multi-thread

2021-11-10 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-17998?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated HADOOP-17998: Labels: pull-request-available (was: ) > Enable get command run with multi-thread >

[jira] [Work logged] (HADOOP-17998) Enable get command run with multi-thread

2021-11-10 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-17998?focusedWorklogId=680144=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-680144 ] ASF GitHub Bot logged work on HADOOP-17998: --- Author: ASF GitHub Bot

[GitHub] [hadoop] smarthanwang opened a new pull request #3645: HADOOP-17998. Enable get command run with multi-thread.

2021-11-10 Thread GitBox
smarthanwang opened a new pull request #3645: URL: https://github.com/apache/hadoop/pull/3645 ### Description CopyFromLocal/Put is enabled to run with multi-thread with HDFS-11786 and HADOOP-14698, and make put dirs or multiple files faster.So, It's necessary to enable get and

[GitHub] [hadoop] aajisaka commented on pull request #3592: YARN-10991. Fix to ignore the grouping "[]" for resourcesStr in parse…

2021-11-10 Thread GitBox
aajisaka commented on pull request #3592: URL: https://github.com/apache/hadoop/pull/3592#issuecomment-965970130 @ashutoshcipher Thank you for your fix. The fix looks good to me. Would you add a regression test for this issue? I think you can create a new test class (such as

[jira] [Work logged] (HADOOP-17997) RBF: Namespace usage of mount table with multi subclusters can exceed quota

2021-11-10 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-17997?focusedWorklogId=680115=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-680115 ] ASF GitHub Bot logged work on HADOOP-17997: --- Author: ASF GitHub Bot

[GitHub] [hadoop] xicm commented on a change in pull request #3634: HADOOP-17997. RBF: Namespace usage of mount table with multi subclust…

2021-11-10 Thread GitBox
xicm commented on a change in pull request #3634: URL: https://github.com/apache/hadoop/pull/3634#discussion_r747147592 ## File path: hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/Quota.java ## @@ -50,7 +50,7 @@ public static Counts

[jira] [Work logged] (HADOOP-18003) Add a method appendIfAbsent for CallerContext

2021-11-10 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18003?focusedWorklogId=680111=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-680111 ] ASF GitHub Bot logged work on HADOOP-18003: --- Author: ASF GitHub Bot

[jira] [Updated] (HADOOP-18003) Add a method appendIfAbsent for CallerContext

2021-11-10 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18003?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated HADOOP-18003: Labels: pull-request-available (was: ) > Add a method appendIfAbsent for CallerContext

[GitHub] [hadoop] tomscut opened a new pull request #3644: HADOOP-18003. Add a method appendIfAbsent for CallerContext

2021-11-10 Thread GitBox
tomscut opened a new pull request #3644: URL: https://github.com/apache/hadoop/pull/3644 JIRA: [HADOOP-18003](https://issues.apache.org/jira/browse/HADOOP-18003). As we discussed here [#3635](https://github.com/apache/hadoop/pull/3635#discussion_r746873078) In some cases,

[jira] [Updated] (HADOOP-18003) Add a method appendIfAbsent for CallerContext

2021-11-10 Thread tomscut (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18003?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] tomscut updated HADOOP-18003: - Description: As we discussed here [#3635.|#discussion_r746873078] In some cases, when we need to add

[jira] [Created] (HADOOP-18003) Add a method appendIfAbsent for CallerContext

2021-11-10 Thread tomscut (Jira)
tomscut created HADOOP-18003: Summary: Add a method appendIfAbsent for CallerContext Key: HADOOP-18003 URL: https://issues.apache.org/jira/browse/HADOOP-18003 Project: Hadoop Common Issue Type:

[jira] [Work logged] (HADOOP-17975) Fallback to simple auth does not work for a secondary DistributedFileSystem instance

2021-11-10 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-17975?focusedWorklogId=680104=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-680104 ] ASF GitHub Bot logged work on HADOOP-17975: --- Author: ASF GitHub Bot

[GitHub] [hadoop] symious commented on pull request #3579: HADOOP-17975 Fallback to simple auth does not work for a secondary DistributedFileSystem instance.

2021-11-10 Thread GitBox
symious commented on pull request #3579: URL: https://github.com/apache/hadoop/pull/3579#issuecomment-965916214 @fapifta Sorry for not making it clear before. Thank you for the test : ) -- This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [hadoop] tomscut opened a new pull request #3643: HDFS-16315. Add metrics related to Transfer and NativeCopy for DataNode

2021-11-10 Thread GitBox
tomscut opened a new pull request #3643: URL: https://github.com/apache/hadoop/pull/3643 JIRA: [HDFS-16315](https://issues.apache.org/jira/browse/HDFS-16315). Datanodes already have `Read`, `Write`, `Sync` and `Flush` metrics. We should add `NativeCopy` and `Transfer` as well.

[GitHub] [hadoop] tomscut commented on a change in pull request #3635: HDFS-16310. RBF: Add client port to CallerContext for Router

2021-11-10 Thread GitBox
tomscut commented on a change in pull request #3635: URL: https://github.com/apache/hadoop/pull/3635#discussion_r747086158 ## File path: hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/ipc/CallerContext.java ## @@ -228,6 +228,26 @@ public Builder

[GitHub] [hadoop] hadoop-yetus commented on pull request #3642: YARN-10760. Number of allocated OPPORTUNISTIC containers can dip below 0

2021-11-10 Thread GitBox
hadoop-yetus commented on pull request #3642: URL: https://github.com/apache/hadoop/pull/3642#issuecomment-965838202 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0

[jira] [Work logged] (HADOOP-17975) Fallback to simple auth does not work for a secondary DistributedFileSystem instance

2021-11-10 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-17975?focusedWorklogId=679951=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-679951 ] ASF GitHub Bot logged work on HADOOP-17975: --- Author: ASF GitHub Bot

[GitHub] [hadoop] fapifta commented on pull request #3579: HADOOP-17975 Fallback to simple auth does not work for a secondary DistributedFileSystem instance.

2021-11-10 Thread GitBox
fapifta commented on pull request #3579: URL: https://github.com/apache/hadoop/pull/3579#issuecomment-965758819 @symious ah I see what you are thinking about, and what I have assumed about the proposal and its impact is not correct. Actually after looking into this further, and

[GitHub] [hadoop] symious commented on pull request #3638: HDFS-16313. Add metrics for each subcluster

2021-11-10 Thread GitBox
symious commented on pull request #3638: URL: https://github.com/apache/hadoop/pull/3638#issuecomment-965729191 @goiri I got the idea to use "subcluster" because of https://hadoop.apache.org/docs/stable/hadoop-project-dist/hadoop-hdfs-rbf/HDFSRouterFederation.html. I chose

[jira] [Work logged] (HADOOP-15566) Support OpenTelemetry

2021-11-10 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-15566?focusedWorklogId=679914=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-679914 ] ASF GitHub Bot logged work on HADOOP-15566: --- Author: ASF GitHub Bot

[GitHub] [hadoop] ndimiduk commented on pull request #3445: HADOOP-15566 Opentelemetry changes using java agent

2021-11-10 Thread GitBox
ndimiduk commented on pull request #3445: URL: https://github.com/apache/hadoop/pull/3445#issuecomment-965717655 @ArkenKiran where do you get the opentelemetry-agent jar for your testing? I've built this patch and it appears that the agent jar is not included. ``` $ mvn clean

[GitHub] [hadoop] hadoop-yetus commented on pull request #3618: YARN-11000. Replace queue resource calculation logic in updateClusterResource

2021-11-10 Thread GitBox
hadoop-yetus commented on pull request #3618: URL: https://github.com/apache/hadoop/pull/3618#issuecomment-965716914 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0

[GitHub] [hadoop] afchung commented on a change in pull request #3642: YARN-10760. Number of allocated OPPORTUNISTIC containers can dip below 0

2021-11-10 Thread GitBox
afchung commented on a change in pull request #3642: URL: https://github.com/apache/hadoop/pull/3642#discussion_r746917343 ## File path:

[GitHub] [hadoop] hadoop-yetus commented on pull request #3642: YARN-10760. Number of allocated OPPORTUNISTIC containers can dip below 0

2021-11-10 Thread GitBox
hadoop-yetus commented on pull request #3642: URL: https://github.com/apache/hadoop/pull/3642#issuecomment-965691903 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0

[GitHub] [hadoop] afchung commented on pull request #3642: YARN-10760. Number of allocated OPPORTUNISTIC containers can dip below 0

2021-11-10 Thread GitBox
afchung commented on pull request #3642: URL: https://github.com/apache/hadoop/pull/3642#issuecomment-965673846 @goiri thanks for the review! Pushed a new commit to address your comments, please have another look. -- This is an automated message from the Apache Git Service. To respond

[GitHub] [hadoop] afchung commented on a change in pull request #3642: YARN-10760. Number of allocated OPPORTUNISTIC containers can dip below 0

2021-11-10 Thread GitBox
afchung commented on a change in pull request #3642: URL: https://github.com/apache/hadoop/pull/3642#discussion_r746917343 ## File path:

[jira] [Work logged] (HADOOP-17979) Interface EtagSource to allow FileStatus subclasses to provide etags

2021-11-10 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-17979?focusedWorklogId=679845=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-679845 ] ASF GitHub Bot logged work on HADOOP-17979: --- Author: ASF GitHub Bot

[GitHub] [hadoop] hadoop-yetus commented on pull request #3633: HADOOP-17979. Add Interface EtagSource to allow FileStatus subclasses to provide etags

2021-11-10 Thread GitBox
hadoop-yetus commented on pull request #3633: URL: https://github.com/apache/hadoop/pull/3633#issuecomment-965658701 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0

[jira] [Resolved] (HADOOP-17992) Disable JIRA plugin for YETUS on Hadoop

2021-11-10 Thread Gautham Banasandra (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-17992?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gautham Banasandra resolved HADOOP-17992. - Fix Version/s: 2.10.2 Resolution: Fixed > Disable JIRA plugin for YETUS

[jira] [Work logged] (HADOOP-17992) Disable JIRA plugin for YETUS on Hadoop

2021-11-10 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-17992?focusedWorklogId=679827=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-679827 ] ASF GitHub Bot logged work on HADOOP-17992: --- Author: ASF GitHub Bot

[GitHub] [hadoop] GauthamBanasandra merged pull request #3623: HADOOP-17992. Disable JIRA plugin for YETUS on Hadoop

2021-11-10 Thread GitBox
GauthamBanasandra merged pull request #3623: URL: https://github.com/apache/hadoop/pull/3623 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

[GitHub] [hadoop] afchung commented on a change in pull request #3642: YARN-10760. Number of allocated OPPORTUNISTIC containers can dip below 0

2021-11-10 Thread GitBox
afchung commented on a change in pull request #3642: URL: https://github.com/apache/hadoop/pull/3642#discussion_r746877782 ## File path:

[jira] [Work logged] (HADOOP-17997) RBF: Namespace usage of mount table with multi subclusters can exceed quota

2021-11-10 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-17997?focusedWorklogId=679818=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-679818 ] ASF GitHub Bot logged work on HADOOP-17997: --- Author: ASF GitHub Bot

[GitHub] [hadoop] goiri commented on a change in pull request #3634: HADOOP-17997. RBF: Namespace usage of mount table with multi subclust…

2021-11-10 Thread GitBox
goiri commented on a change in pull request #3634: URL: https://github.com/apache/hadoop/pull/3634#discussion_r746875360 ## File path: hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/Quota.java ## @@ -50,7 +50,7 @@ public static Counts

[GitHub] [hadoop] goiri commented on pull request #3638: HDFS-16313. Add metrics for each subcluster

2021-11-10 Thread GitBox
goiri commented on pull request #3638: URL: https://github.com/apache/hadoop/pull/3638#issuecomment-965625228 > @goiri @ferhui Thanks for the review. Have aligned the terminology to use "subclusterName", please have a check. I have the feeling that `nsId` is the most standard term

[GitHub] [hadoop] goiri commented on a change in pull request #3635: HDFS-16310. RBF: Add client port to CallerContext for Router

2021-11-10 Thread GitBox
goiri commented on a change in pull request #3635: URL: https://github.com/apache/hadoop/pull/3635#discussion_r746873078 ## File path: hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/ipc/CallerContext.java ## @@ -228,6 +228,26 @@ public Builder

[GitHub] [hadoop] goiri commented on a change in pull request #3642: YARN-10760. Number of allocated OPPORTUNISTIC containers can dip below 0

2021-11-10 Thread GitBox
goiri commented on a change in pull request #3642: URL: https://github.com/apache/hadoop/pull/3642#discussion_r746869941 ## File path:

[GitHub] [hadoop] jojochuang commented on pull request #3640: HDFS-16187. SnapshotDiff behaviour with Xattrs and Acls is not consistent across NN restarts with checkpointing (#3340)

2021-11-10 Thread GitBox
jojochuang commented on pull request #3640: URL: https://github.com/apache/hadoop/pull/3640#issuecomment-965565971 TestDirectoryScanner not reproucible in my local tree. Will merge later. -- This is an automated message from the Apache Git Service. To respond to the message, please log

[jira] [Work logged] (HADOOP-18002) abfs rename idempotency broken -remove recovery

2021-11-10 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18002?focusedWorklogId=679776=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-679776 ] ASF GitHub Bot logged work on HADOOP-18002: --- Author: ASF GitHub Bot

[GitHub] [hadoop] hadoop-yetus commented on pull request #3641: HADOOP-18002. abfs rename idempotency broken -remove recovery

2021-11-10 Thread GitBox
hadoop-yetus commented on pull request #3641: URL: https://github.com/apache/hadoop/pull/3641#issuecomment-965546840 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0

[GitHub] [hadoop] hadoop-yetus commented on pull request #3618: YARN-11000. Replace queue resource calculation logic in updateClusterResource

2021-11-10 Thread GitBox
hadoop-yetus commented on pull request #3618: URL: https://github.com/apache/hadoop/pull/3618#issuecomment-965527037 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0

[GitHub] [hadoop] jianghuazhu commented on pull request #3629: HDFS-16305.Record the remote NameNode address when the rolling log is triggered.

2021-11-10 Thread GitBox
jianghuazhu commented on pull request #3629: URL: https://github.com/apache/hadoop/pull/3629#issuecomment-965508886 Can you help review this pr again, @virajjasani @tomscut . Thank you very much. -- This is an automated message from the Apache Git Service. To respond to the message,

[GitHub] [hadoop] afchung opened a new pull request #3642: YARN-10760. Number of allocated OPPORTUNISTIC containers can dip below 0

2021-11-10 Thread GitBox
afchung opened a new pull request #3642: URL: https://github.com/apache/hadoop/pull/3642 ### Description of PR * Checks that we do not release containers that do not exist * Checks that we increment recovered allocated OPPORTUNISTIC containers on RM restart * Adds

[jira] [Resolved] (HADOOP-17981) Support etag-assisted renames in FileOutputCommitter

2021-11-10 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-17981?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran resolved HADOOP-17981. - Resolution: Won't Fix closing as WONTFIX as even though the PR is complete, we don't

[jira] [Commented] (HADOOP-17979) Interface EtagSource to allow FileStatus subclasses to provide etags

2021-11-10 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-17979?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17441808#comment-17441808 ] Steve Loughran commented on HADOOP-17979: - once this is in MAPREDUCE-7341 will use it to

[jira] [Work logged] (HADOOP-18002) abfs rename idempotency broken -remove recovery

2021-11-10 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18002?focusedWorklogId=679716=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-679716 ] ASF GitHub Bot logged work on HADOOP-18002: --- Author: ASF GitHub Bot

[GitHub] [hadoop] steveloughran commented on pull request #3641: HADOOP-18002. abfs rename idempotency broken -remove recovery

2021-11-10 Thread GitBox
steveloughran commented on pull request #3641: URL: https://github.com/apache/hadoop/pull/3641#issuecomment-965437620 full test run ok, overloaded system triggered ``` ITestAzureBlobFileSystemFileStatus.testLastModifiedTime:144->Assert.assertTrue:42->Assert.fail:89 lastModifiedTime

[jira] [Work logged] (HADOOP-17979) Interface EtagSource to allow FileStatus subclasses to provide etags

2021-11-10 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-17979?focusedWorklogId=679695=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-679695 ] ASF GitHub Bot logged work on HADOOP-17979: --- Author: ASF GitHub Bot

[GitHub] [hadoop] steveloughran commented on pull request #3633: HADOOP-17979. Add Interface EtagSource to allow FileStatus subclasses to provide etags

2021-11-10 Thread GitBox
steveloughran commented on pull request #3633: URL: https://github.com/apache/hadoop/pull/3633#issuecomment-965421031 tested: azure cardiff. all good except failures addressed in #3641 -- This is an automated message from the Apache Git Service. To respond to the message, please log on

[jira] [Work logged] (HADOOP-17979) Interface EtagSource to allow FileStatus subclasses to provide etags

2021-11-10 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-17979?focusedWorklogId=679692=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-679692 ] ASF GitHub Bot logged work on HADOOP-17979: --- Author: ASF GitHub Bot

[jira] [Work logged] (HADOOP-17979) Interface EtagSource to allow FileStatus subclasses to provide etags

2021-11-10 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-17979?focusedWorklogId=679691=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-679691 ] ASF GitHub Bot logged work on HADOOP-17979: --- Author: ASF GitHub Bot

[GitHub] [hadoop] hadoop-yetus removed a comment on pull request #3633: HADOOP-17979. Add Interface EtagSource to allow FileStatus subclasses to provide etags

2021-11-10 Thread GitBox
hadoop-yetus removed a comment on pull request #3633: URL: https://github.com/apache/hadoop/pull/3633#issuecomment-964274996 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:|

[GitHub] [hadoop] hadoop-yetus removed a comment on pull request #3633: HADOOP-17979. Add Interface EtagSource to allow FileStatus subclasses to provide etags

2021-11-10 Thread GitBox
hadoop-yetus removed a comment on pull request #3633: URL: https://github.com/apache/hadoop/pull/3633#issuecomment-964212085 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:|

[jira] [Work logged] (HADOOP-18002) abfs rename idempotency broken -remove recovery

2021-11-10 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18002?focusedWorklogId=679690=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-679690 ] ASF GitHub Bot logged work on HADOOP-18002: --- Author: ASF GitHub Bot

[GitHub] [hadoop] steveloughran commented on pull request #3641: HADOOP-18002. abfs rename idempotency broken -remove recovery

2021-11-10 Thread GitBox
steveloughran commented on pull request #3641: URL: https://github.com/apache/hadoop/pull/3641#issuecomment-965412044 @snvijaya @joshelser @mehakmeet @mukund-thakur modified tests all happy; running full suite against azure cardiff -- This is an automated message from the Apache

[jira] [Work logged] (HADOOP-18002) abfs rename idempotency broken -remove recovery

2021-11-10 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18002?focusedWorklogId=679688=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-679688 ] ASF GitHub Bot logged work on HADOOP-18002: --- Author: ASF GitHub Bot

[jira] [Updated] (HADOOP-18002) abfs rename idempotency broken -remove recovery

2021-11-10 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18002?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated HADOOP-18002: Labels: pull-request-available (was: ) > abfs rename idempotency broken -remove

[GitHub] [hadoop] steveloughran opened a new pull request #3641: HADOOP-18002. abfs rename idempotency broken -remove recovery

2021-11-10 Thread GitBox
steveloughran opened a new pull request #3641: URL: https://github.com/apache/hadoop/pull/3641 Cut modtime-based rename recovery as it doesn't work. Applications will have to use etag API of HADOOP-17979 and implement it themselves. Why not do the HEAD and etag recovery in ABFS

[jira] [Commented] (HADOOP-17989) ITestAzureBlobFileSystemDelete failing "Operations has null HTTP response"

2021-11-10 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-17989?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17441787#comment-17441787 ] Steve Loughran commented on HADOOP-17989: - going to fix in HADOOP-18002 >

[jira] [Assigned] (HADOOP-17989) ITestAzureBlobFileSystemDelete failing "Operations has null HTTP response"

2021-11-10 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-17989?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran reassigned HADOOP-17989: --- Assignee: Steve Loughran (was: Josh Elser) > ITestAzureBlobFileSystemDelete

[jira] [Assigned] (HADOOP-18002) abfs rename idempotency broken -remove recovery

2021-11-10 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18002?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran reassigned HADOOP-18002: --- Component/s: fs/azure Target Version/s: 3.4.0 Affects Version/s:

[jira] [Created] (HADOOP-18002) abfs

2021-11-10 Thread Steve Loughran (Jira)
Steve Loughran created HADOOP-18002: --- Summary: abfs Key: HADOOP-18002 URL: https://issues.apache.org/jira/browse/HADOOP-18002 Project: Hadoop Common Issue Type: Bug Reporter:

[jira] [Work logged] (HADOOP-17979) Interface EtagSource to allow FileStatus subclasses to provide etags

2021-11-10 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-17979?focusedWorklogId=679662=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-679662 ] ASF GitHub Bot logged work on HADOOP-17979: --- Author: ASF GitHub Bot

[GitHub] [hadoop] hadoop-yetus removed a comment on pull request #3633: HADOOP-17979. Add Interface EtagSource to allow FileStatus subclasses to provide etags

2021-11-10 Thread GitBox
hadoop-yetus removed a comment on pull request #3633: URL: https://github.com/apache/hadoop/pull/3633#issuecomment-963644661 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:|

[jira] [Commented] (HADOOP-17017) S3A client retries on SSL Auth exceptions triggered by "." bucket names

2021-11-10 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-17017?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17441782#comment-17441782 ] Steve Loughran commented on HADOOP-17017: - yes, that works until AWS remove path access. which

[jira] [Updated] (HADOOP-17998) Enable get command run with multi-thread

2021-11-10 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-17998?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran updated HADOOP-17998: Component/s: fs > Enable get command run with multi-thread >

[jira] [Updated] (HADOOP-17998) Enable get command run with multi-thread

2021-11-10 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-17998?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran updated HADOOP-17998: Target Version/s: 3.4.0 (was: 3.3.1, 3.4.0) > Enable get command run with multi-thread

[jira] [Updated] (HADOOP-17998) Enable get command run with multi-thread

2021-11-10 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-17998?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran updated HADOOP-17998: Affects Version/s: 3.3.1 > Enable get command run with multi-thread >

[jira] [Commented] (HADOOP-17998) Enable get command run with multi-thread

2021-11-10 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-17998?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17441773#comment-17441773 ] Steve Loughran commented on HADOOP-17998: - can you submit this as a github PR? its how we

[GitHub] [hadoop] hadoop-yetus commented on pull request #3618: YARN-11000. Replace queue resource calculation logic in updateClusterResource

2021-11-10 Thread GitBox
hadoop-yetus commented on pull request #3618: URL: https://github.com/apache/hadoop/pull/3618#issuecomment-965269670 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0

[jira] [Commented] (HADOOP-17996) UserGroupInformation#unprotectedRelogin sets the last login time before logging in

2021-11-10 Thread Ravuri Sushma sree (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-17996?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17441761#comment-17441761 ] Ravuri Sushma sree commented on HADOOP-17996: - Uploaded a patch correcting the set last

[jira] [Updated] (HADOOP-17996) UserGroupInformation#unprotectedRelogin sets the last login time before logging in

2021-11-10 Thread Ravuri Sushma sree (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-17996?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ravuri Sushma sree updated HADOOP-17996: Attachment: HADOOP-17996.001.patch > UserGroupInformation#unprotectedRelogin

[GitHub] [hadoop] hadoop-yetus commented on pull request #3638: HDFS-16313. Add metrics for each subcluster

2021-11-10 Thread GitBox
hadoop-yetus commented on pull request #3638: URL: https://github.com/apache/hadoop/pull/3638#issuecomment-965132725 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0

[jira] [Work logged] (HADOOP-17995) Stale record should be remove when DataNodePeerMetrics#dumpSendPacketDownstreamAvgInfoAsJson

2021-11-10 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-17995?focusedWorklogId=679573=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-679573 ] ASF GitHub Bot logged work on HADOOP-17995: --- Author: ASF GitHub Bot

[GitHub] [hadoop] haiyang1987 commented on a change in pull request #3630: HADOOP-17995. Stale record should be remove when DataNodePeerMetrics#dumpSendPacketDownstreamAvgInfoAsJson

2021-11-10 Thread GitBox
haiyang1987 commented on a change in pull request #3630: URL: https://github.com/apache/hadoop/pull/3630#discussion_r746505180 ## File path: hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/metrics2/lib/MutableRollingAverages.java ## @@ -167,7 +167,7 @@

[GitHub] [hadoop] symious commented on pull request #3638: HDFS-16313. Add metrics for each subcluster

2021-11-10 Thread GitBox
symious commented on pull request #3638: URL: https://github.com/apache/hadoop/pull/3638#issuecomment-965041151 @goiri @ferhui Thanks for the review. Have aligned the terminology to use "subclusterName", please have a check. -- This is an automated message from the Apache Git Service.

[GitHub] [hadoop] tomscut commented on pull request #3615: HDFS-16298. Improve error msg for BlockMissingException

2021-11-10 Thread GitBox
tomscut commented on pull request #3615: URL: https://github.com/apache/hadoop/pull/3615#issuecomment-965025197 Thanks @tasanuma for your review and merge. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL

[GitHub] [hadoop] tomscut commented on pull request #3636: HDFS-16311. Metric metadataOperationRate calculation error in DataNod…

2021-11-10 Thread GitBox
tomscut commented on pull request #3636: URL: https://github.com/apache/hadoop/pull/3636#issuecomment-965024494 Thanks @tasanuma for your review. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to

[GitHub] [hadoop] hadoop-yetus commented on pull request #3629: HDFS-16305.Record the remote NameNode address when the rolling log is triggered.

2021-11-10 Thread GitBox
hadoop-yetus commented on pull request #3629: URL: https://github.com/apache/hadoop/pull/3629#issuecomment-965023293 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0

[jira] [Commented] (HADOOP-17998) Enable get command run with multi-thread

2021-11-10 Thread Chengwei Wang (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-17998?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17441639#comment-17441639 ] Chengwei Wang commented on HADOOP-17998: Test case: 1 dir 240 files  13G  *1. Get with single

[GitHub] [hadoop] hadoop-yetus commented on pull request #3640: HDFS-16187. SnapshotDiff behaviour with Xattrs and Acls is not consistent across NN restarts with checkpointing (#3340)

2021-11-10 Thread GitBox
hadoop-yetus commented on pull request #3640: URL: https://github.com/apache/hadoop/pull/3640#issuecomment-965005422 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0

[jira] [Work logged] (HADOOP-15327) Upgrade MR ShuffleHandler to use Netty4

2021-11-10 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-15327?focusedWorklogId=679550=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-679550 ] ASF GitHub Bot logged work on HADOOP-15327: --- Author: ASF GitHub Bot

[GitHub] [hadoop] ayushpal24 commented on pull request #3259: HADOOP-15327. Upgrade MR ShuffleHandler to use Netty4

2021-11-10 Thread GitBox
ayushpal24 commented on pull request #3259: URL: https://github.com/apache/hadoop/pull/3259#issuecomment-964993855 Hi @szilard-nemeth ,this is my first patch and i am not able to find any detail report to understand the reason of failures, any suggestions? Thanks -- This is an

[GitHub] [hadoop] tasanuma commented on pull request #3637: HDFS-16312. Fix typo for DataNodeVolumeMetrics and ProfilingFileIoEvents

2021-11-10 Thread GitBox
tasanuma commented on pull request #3637: URL: https://github.com/apache/hadoop/pull/3637#issuecomment-964991009 Thanks, @tomscut and @ferhui. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to

[GitHub] [hadoop] tasanuma merged pull request #3637: HDFS-16312. Fix typo for DataNodeVolumeMetrics and ProfilingFileIoEvents

2021-11-10 Thread GitBox
tasanuma merged pull request #3637: URL: https://github.com/apache/hadoop/pull/3637 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

[jira] [Commented] (HADOOP-17457) intermittent ITestS3AInconsistency.testGetFileStatus failure.

2021-11-10 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-17457?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17441628#comment-17441628 ] Steve Loughran commented on HADOOP-17457: - will be cut in HADOOP-17409 > intermittent

[jira] [Commented] (HADOOP-17409) Remove S3Guard - no longer needed

2021-11-10 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-17409?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17441624#comment-17441624 ] Steve Loughran commented on HADOOP-17409: - yeah, really speeds up tests. I'd like to get this

[GitHub] [hadoop] tasanuma commented on pull request #3615: HDFS-16298. Improve error msg for BlockMissingException

2021-11-10 Thread GitBox
tasanuma commented on pull request #3615: URL: https://github.com/apache/hadoop/pull/3615#issuecomment-964986691 Thanks for your contribution, @tomscut. Thanks for your review, @ferhui. -- This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [hadoop] tasanuma merged pull request #3615: HDFS-16298. Improve error msg for BlockMissingException

2021-11-10 Thread GitBox
tasanuma merged pull request #3615: URL: https://github.com/apache/hadoop/pull/3615 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

[jira] [Work logged] (HADOOP-17990) Failing concurrent FS.initialize commands when fs.azure.createRemoteFileSystemDuringInitialization is enabled on hadoop-azure ABFS

2021-11-10 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-17990?focusedWorklogId=679543=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-679543 ] ASF GitHub Bot logged work on HADOOP-17990: --- Author: ASF GitHub Bot

[GitHub] [hadoop] hadoop-yetus commented on pull request #3620: HADOOP-17990. Fix failing concurrent FS.initialize commands when fs.azure.createRemoteFileSystemDuringInitialization is enabled.

2021-11-10 Thread GitBox
hadoop-yetus commented on pull request #3620: URL: https://github.com/apache/hadoop/pull/3620#issuecomment-964984599 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0

[jira] [Work logged] (HADOOP-17981) Support etag-assisted renames in FileOutputCommitter

2021-11-10 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-17981?focusedWorklogId=679541=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-679541 ] ASF GitHub Bot logged work on HADOOP-17981: --- Author: ASF GitHub Bot

  1   2   >