[GitHub] [hadoop] tomscut edited a comment on pull request #2668: HDFS-15808. Add metrics for FSNamesystem read/write lock warnings

2021-02-08 Thread GitBox
tomscut edited a comment on pull request #2668: URL: https://github.com/apache/hadoop/pull/2668#issuecomment-775605267 @xkrogen Hi Erik, could you please review the code? Thanks a lot! This is an automated message from

[jira] [Work logged] (HADOOP-17511) Add an Audit plugin point for S3A auditing/context

2021-02-08 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-17511?focusedWorklogId=550054=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-550054 ] ASF GitHub Bot logged work on HADOOP-17511: --- Author: ASF GitHub Bot

[GitHub] [hadoop] aimiebell commented on a change in pull request #2675: HADOOP-17511. Add audit/telemetry logging to S3A connector

2021-02-08 Thread GitBox
aimiebell commented on a change in pull request #2675: URL: https://github.com/apache/hadoop/pull/2675#discussion_r572623035 ## File path: hadoop-tools/hadoop-aws/src/main/java/org/apache/hadoop/fs/s3a/S3AFileSystem.java ## @@ -1789,15 +1949,22 @@ public boolean

[GitHub] [hadoop] bolerio opened a new pull request #2693: Hadoop 16524 - resubmission following some unit test fixes

2021-02-08 Thread GitBox
bolerio opened a new pull request #2693: URL: https://github.com/apache/hadoop/pull/2693 This is a follow up on this PR which was rejected due to some failed unit tests https://github.com/apache/hadoop/pull/2470 that were missed initially. Most of the failures reported here appears

[jira] [Work logged] (HADOOP-17338) Intermittent S3AInputStream failures: Premature end of Content-Length delimited message body etc

2021-02-08 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-17338?focusedWorklogId=550046=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-550046 ] ASF GitHub Bot logged work on HADOOP-17338: --- Author: ASF GitHub Bot

[GitHub] [hadoop] hadoop-yetus commented on pull request #2692: HADOOP-17338 Intermittent S3AInputStream failures: Premature end of C…

2021-02-08 Thread GitBox
hadoop-yetus commented on pull request #2692: URL: https://github.com/apache/hadoop/pull/2692#issuecomment-775665531 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 11m

[GitHub] [hadoop] bshashikant commented on pull request #2677: HDFS-15817. Rename snapshots while marking them deleted.

2021-02-08 Thread GitBox
bshashikant commented on pull request #2677: URL: https://github.com/apache/hadoop/pull/2677#issuecomment-775663629 Thanks @mukul1987 and @bshashikant for the review. This is an automated message from the Apache Git

[GitHub] [hadoop] bshashikant merged pull request #2677: HDFS-15817. Rename snapshots while marking them deleted.

2021-02-08 Thread GitBox
bshashikant merged pull request #2677: URL: https://github.com/apache/hadoop/pull/2677 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go

[jira] [Work logged] (HADOOP-17038) Support disabling buffered reads in ABFS positional reads

2021-02-08 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-17038?focusedWorklogId=550042=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-550042 ] ASF GitHub Bot logged work on HADOOP-17038: --- Author: ASF GitHub Bot

[GitHub] [hadoop] anoopsjohn commented on a change in pull request #2646: HADOOP-17038 Support disabling buffered reads in ABFS positional reads.

2021-02-08 Thread GitBox
anoopsjohn commented on a change in pull request #2646: URL: https://github.com/apache/hadoop/pull/2646#discussion_r572594171 ## File path: hadoop-tools/hadoop-azure/src/test/java/org/apache/hadoop/fs/azurebfs/ITestAbfsInputStreamStatistics.java ## @@ -399,6 +406,89 @@ public

[jira] [Work logged] (HADOOP-17038) Support disabling buffered reads in ABFS positional reads

2021-02-08 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-17038?focusedWorklogId=550041=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-550041 ] ASF GitHub Bot logged work on HADOOP-17038: --- Author: ASF GitHub Bot

[GitHub] [hadoop] anoopsjohn commented on a change in pull request #2646: HADOOP-17038 Support disabling buffered reads in ABFS positional reads.

2021-02-08 Thread GitBox
anoopsjohn commented on a change in pull request #2646: URL: https://github.com/apache/hadoop/pull/2646#discussion_r572594015 ## File path: hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azurebfs/AzureBlobFileSystemStore.java ## @@ -634,12 +638,15 @@ public

[jira] [Work logged] (HADOOP-17338) Intermittent S3AInputStream failures: Premature end of Content-Length delimited message body etc

2021-02-08 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-17338?focusedWorklogId=550040=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-550040 ] ASF GitHub Bot logged work on HADOOP-17338: --- Author: ASF GitHub Bot

[GitHub] [hadoop] yzhangal commented on pull request #2692: HADOOP-17338 Intermittent S3AInputStream failures: Premature end of C…

2021-02-08 Thread GitBox
yzhangal commented on pull request #2692: URL: https://github.com/apache/hadoop/pull/2692#issuecomment-775654938 HI @steveloughran , Thanks for reviewing and merging HADOOP-16767. This is the PR for HADOOPs-17338 on branch-2.10, the test was done previously together with

[jira] [Commented] (HADOOP-16767) S3AInputStream reopening does not handle non IO exceptions properly

2021-02-08 Thread Yongjun Zhang (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-16767?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17281519#comment-17281519 ] Yongjun Zhang commented on HADOOP-16767: Thanks [~ste...@apache.org]! > S3AInputStream

[jira] [Work logged] (HADOOP-17338) Intermittent S3AInputStream failures: Premature end of Content-Length delimited message body etc

2021-02-08 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-17338?focusedWorklogId=550038=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-550038 ] ASF GitHub Bot logged work on HADOOP-17338: --- Author: ASF GitHub Bot

[GitHub] [hadoop] yzhangal opened a new pull request #2692: HADOOP-17338 Intermittent S3AInputStream failures: Premature end of C…

2021-02-08 Thread GitBox
yzhangal opened a new pull request #2692: URL: https://github.com/apache/hadoop/pull/2692 …ontent-Length delimited message body etc ## NOTICE Please create an issue in ASF JIRA before opening a pull request, and you need to set the title of the pull request which starts

[GitHub] [hadoop] yzhangal opened a new pull request #2691: Feature/hadoop 17338

2021-02-08 Thread GitBox
yzhangal opened a new pull request #2691: URL: https://github.com/apache/hadoop/pull/2691 ## NOTICE Please create an issue in ASF JIRA before opening a pull request, and you need to set the title of the pull request which starts with the corresponding JIRA issue number. (e.g.

[GitHub] [hadoop] hadoop-yetus commented on pull request #2690: HDFS-15821. Add metrics for in-service datanodes

2021-02-08 Thread GitBox
hadoop-yetus commented on pull request #2690: URL: https://github.com/apache/hadoop/pull/2690#issuecomment-775627526 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0

[GitHub] [hadoop] tangzhankun merged pull request #2665: YARN-9650. Set thread names for CapacityScheduler AsyncScheduleThread

2021-02-08 Thread GitBox
tangzhankun merged pull request #2665: URL: https://github.com/apache/hadoop/pull/2665 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go

[GitHub] [hadoop] tomscut commented on pull request #2668: HDFS-15808. Add metrics for FSNamesystem read/write lock warnings

2021-02-08 Thread GitBox
tomscut commented on pull request #2668: URL: https://github.com/apache/hadoop/pull/2668#issuecomment-775605267 @xkrogen Hi Erik Krogen, could you please review the code? Thanks a lot! This is an automated message from

[GitHub] [hadoop] tomscut commented on pull request #2676: Make some parameters configurable for DataNodeDiskMetrics

2021-02-08 Thread GitBox
tomscut commented on pull request #2676: URL: https://github.com/apache/hadoop/pull/2676#issuecomment-775602310 @hanishakoneru Hi Hanisha Koneru, could you please review the code for me? Thanks! This is an automated

[GitHub] [hadoop] LeonGao91 commented on a change in pull request #2639: HDFS-15785. Datanode to support using DNS to resolve nameservices to IP addresses to get list of namenodes.

2021-02-08 Thread GitBox
LeonGao91 commented on a change in pull request #2639: URL: https://github.com/apache/hadoop/pull/2639#discussion_r572426138 ## File path: hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/DFSUtil.java ## @@ -647,6 +634,58 @@ public static String

[GitHub] [hadoop] LeonGao91 commented on a change in pull request #2639: HDFS-15785. Datanode to support using DNS to resolve nameservices to IP addresses to get list of namenodes.

2021-02-08 Thread GitBox
LeonGao91 commented on a change in pull request #2639: URL: https://github.com/apache/hadoop/pull/2639#discussion_r572419061 ## File path: hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/DFSConfigKeys.java ## @@ -1557,6 +1557,17 @@ public static final

[jira] [Work logged] (HADOOP-16906) Add some Abortable.abort() interface for streams etc which can be terminated

2021-02-08 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-16906?focusedWorklogId=549939=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-549939 ] ASF GitHub Bot logged work on HADOOP-16906: --- Author: ASF GitHub Bot

[GitHub] [hadoop] hadoop-yetus commented on pull request #2684: HADOOP-16906. Abortable

2021-02-08 Thread GitBox
hadoop-yetus commented on pull request #2684: URL: https://github.com/apache/hadoop/pull/2684#issuecomment-775499774 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0

[GitHub] [hadoop] fengnanli commented on a change in pull request #2639: HDFS-15785. Datanode to support using DNS to resolve nameservices to IP addresses to get list of namenodes.

2021-02-08 Thread GitBox
fengnanli commented on a change in pull request #2639: URL: https://github.com/apache/hadoop/pull/2639#discussion_r572411463 ## File path: hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/DFSConfigKeys.java ## @@ -1557,6 +1557,17 @@ public static final

[jira] [Work logged] (HADOOP-17520) S3A to downgraded to unguarded if the DDB table cannot be found

2021-02-08 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-17520?focusedWorklogId=549921=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-549921 ] ASF GitHub Bot logged work on HADOOP-17520: --- Author: ASF GitHub Bot

[GitHub] [hadoop] hadoop-yetus commented on pull request #2689: HADOOP-17520. Downgrade missing DDB table to unguarded client

2021-02-08 Thread GitBox
hadoop-yetus commented on pull request #2689: URL: https://github.com/apache/hadoop/pull/2689#issuecomment-775489652 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0

[GitHub] [hadoop] zehaoc2 opened a new pull request #2690: HDFS-15821. Add metrics for in-service datanodes

2021-02-08 Thread GitBox
zehaoc2 opened a new pull request #2690: URL: https://github.com/apache/hadoop/pull/2690 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to

[jira] [Resolved] (HADOOP-17517) Non-randomized password used

2021-02-08 Thread Kihwal Lee (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-17517?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kihwal Lee resolved HADOOP-17517. - Resolution: Invalid > Non-randomized password used > > >

[GitHub] [hadoop] hadoop-yetus commented on pull request #2677: HDFS-15817. Rename snapshots while marking them deleted.

2021-02-08 Thread GitBox
hadoop-yetus commented on pull request #2677: URL: https://github.com/apache/hadoop/pull/2677#issuecomment-775468282 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0

[jira] [Work logged] (HADOOP-17520) S3A to downgraded to unguarded if the DDB table cannot be found

2021-02-08 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-17520?focusedWorklogId=549859=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-549859 ] ASF GitHub Bot logged work on HADOOP-17520: --- Author: ASF GitHub Bot

[jira] [Updated] (HADOOP-17520) S3A to downgraded to unguarded if the DDB table cannot be found

2021-02-08 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-17520?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated HADOOP-17520: Labels: pull-request-available (was: ) > S3A to downgraded to unguarded if the DDB

[GitHub] [hadoop] steveloughran opened a new pull request #2689: HADOOP-17520. Downgrade missing DDB table to unguarded client

2021-02-08 Thread GitBox
steveloughran opened a new pull request #2689: URL: https://github.com/apache/hadoop/pull/2689 If an s3a client starts up and cannot find/read the DDB table then it downgrades to being unguarded This is an automated

[jira] [Commented] (HADOOP-17520) S3A to downgraded to unguarded if the DDB table cannot be found

2021-02-08 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-17520?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17281349#comment-17281349 ] Steve Loughran commented on HADOOP-17520: - this is related to dynamic creation of role DT

[jira] [Commented] (HADOOP-17520) S3A to downgraded to unguarded if the DDB table cannot be found

2021-02-08 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-17520?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17281347#comment-17281347 ] Steve Loughran commented on HADOOP-17520: - Mostly straightforward. One troublespot relates to

[jira] [Work logged] (HADOOP-16906) Add some Abortable.abort() interface for streams etc which can be terminated

2021-02-08 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-16906?focusedWorklogId=549836=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-549836 ] ASF GitHub Bot logged work on HADOOP-16906: --- Author: ASF GitHub Bot

[GitHub] [hadoop] hadoop-yetus commented on pull request #2684: HADOOP-16906. Abortable

2021-02-08 Thread GitBox
hadoop-yetus commented on pull request #2684: URL: https://github.com/apache/hadoop/pull/2684#issuecomment-775397931 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0

[jira] [Work logged] (HADOOP-17497) refactoring the signature of S3ClientFactory breaks hboss compile

2021-02-08 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-17497?focusedWorklogId=549825=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-549825 ] ASF GitHub Bot logged work on HADOOP-17497: --- Author: ASF GitHub Bot

[GitHub] [hadoop] hadoop-yetus commented on pull request #2654: HADOOP-17497. refactoring the signature of S3ClientFactory breaks hboss compile.

2021-02-08 Thread GitBox
hadoop-yetus commented on pull request #2654: URL: https://github.com/apache/hadoop/pull/2654#issuecomment-775390596 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0

[jira] [Work logged] (HADOOP-13551) Collect AwsSdkMetrics in S3A FileSystem IOStatistics

2021-02-08 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-13551?focusedWorklogId=549817=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-549817 ] ASF GitHub Bot logged work on HADOOP-13551: --- Author: ASF GitHub Bot

[GitHub] [hadoop] hadoop-yetus commented on pull request #2687: HADOOP-13551. Wire up AWS SDK metrics to IOStatistics

2021-02-08 Thread GitBox
hadoop-yetus commented on pull request #2687: URL: https://github.com/apache/hadoop/pull/2687#issuecomment-775382658 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0

[jira] [Work logged] (HADOOP-16948) ABFS: Support single writer dirs

2021-02-08 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-16948?focusedWorklogId=549799=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-549799 ] ASF GitHub Bot logged work on HADOOP-16948: --- Author: ASF GitHub Bot

[GitHub] [hadoop] steveloughran commented on a change in pull request #1925: HADOOP-16948. Support single writer dirs.

2021-02-08 Thread GitBox
steveloughran commented on a change in pull request #1925: URL: https://github.com/apache/hadoop/pull/1925#discussion_r572291660 ## File path: hadoop-tools/hadoop-azure/src/site/markdown/abfs.md ## @@ -877,6 +877,21 @@ enabled for your Azure Storage account." The directories

[GitHub] [hadoop] LeonGao91 commented on pull request #2625: HDFS-15683. Allow configuring DISK/ARCHIVE capacity for individual volumes.

2021-02-08 Thread GitBox
LeonGao91 commented on pull request #2625: URL: https://github.com/apache/hadoop/pull/2625#issuecomment-775370895 thx @Jing9 for the review! I will open a jira for the javac issues. This is an automated message from the

[GitHub] [hadoop] Jing9 merged pull request #2625: HDFS-15683. Allow configuring DISK/ARCHIVE capacity for individual volumes.

2021-02-08 Thread GitBox
Jing9 merged pull request #2625: URL: https://github.com/apache/hadoop/pull/2625 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to

[GitHub] [hadoop] Jing9 commented on pull request #2625: HDFS-15683. Allow configuring DISK/ARCHIVE capacity for individual volumes.

2021-02-08 Thread GitBox
Jing9 commented on pull request #2625: URL: https://github.com/apache/hadoop/pull/2625#issuecomment-775369725 The javac warning looks unrelated. It reported "generated 14 new + 580 unchanged - 14 fixed = 594 total (was 594)" and the warnings are not caused by this PR. But we can use this

[jira] [Work logged] (HADOOP-16767) S3AInputStream reopening does not handle non IO exceptions properly

2021-02-08 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-16767?focusedWorklogId=549791=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-549791 ] ASF GitHub Bot logged work on HADOOP-16767: --- Author: ASF GitHub Bot

[GitHub] [hadoop] yzhangal commented on pull request #2685: HADOOP-16767 Handle non-IO exceptions in reopen()

2021-02-08 Thread GitBox
yzhangal commented on pull request #2685: URL: https://github.com/apache/hadoop/pull/2685#issuecomment-775367808 https://github.com/apache/hadoop/pull/2688 Seems the build failed, maybe because I uploaded the diff before your merge thus the base is different. As of now, I don't see your

[jira] [Work logged] (HADOOP-16948) ABFS: Support single writer dirs

2021-02-08 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-16948?focusedWorklogId=549785=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-549785 ] ASF GitHub Bot logged work on HADOOP-16948: --- Author: ASF GitHub Bot

[GitHub] [hadoop] steveloughran commented on a change in pull request #1925: HADOOP-16948. Support single writer dirs.

2021-02-08 Thread GitBox
steveloughran commented on a change in pull request #1925: URL: https://github.com/apache/hadoop/pull/1925#discussion_r572289020 ## File path: hadoop-tools/hadoop-azure/src/test/java/org/apache/hadoop/fs/azurebfs/ITestAzureBlobFileSystemLease.java ## @@ -0,0 +1,295 @@ +/** +

[jira] [Work logged] (HADOOP-16767) S3AInputStream reopening does not handle non IO exceptions properly

2021-02-08 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-16767?focusedWorklogId=549783=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-549783 ] ASF GitHub Bot logged work on HADOOP-16767: --- Author: ASF GitHub Bot

[GitHub] [hadoop] steveloughran commented on pull request #2685: HADOOP-16767 Handle non-IO exceptions in reopen()

2021-02-08 Thread GitBox
steveloughran commented on pull request #2685: URL: https://github.com/apache/hadoop/pull/2685#issuecomment-775362444 Has yetus started on the PR? as that's what really matters. You can always just link the JIRA to the PR by hand

[jira] [Work logged] (HADOOP-17498) Add a MkdirOperation for chained S3 operations during mkdir

2021-02-08 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-17498?focusedWorklogId=549781=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-549781 ] ASF GitHub Bot logged work on HADOOP-17498: --- Author: ASF GitHub Bot

[GitHub] [hadoop] steveloughran commented on pull request #2671: HADOOP-17498 mkdir operation.

2021-02-08 Thread GitBox
steveloughran commented on pull request #2671: URL: https://github.com/apache/hadoop/pull/2671#issuecomment-775360729 Needs: ITest, could do a unit test for completeness, it being isolated enough. With some counters of probes/createFakeDir we could verify that the operation did dir

[jira] [Work logged] (HADOOP-16767) S3AInputStream reopening does not handle non IO exceptions properly

2021-02-08 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-16767?focusedWorklogId=549767=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-549767 ] ASF GitHub Bot logged work on HADOOP-16767: --- Author: ASF GitHub Bot

[GitHub] [hadoop] yzhangal commented on pull request #2685: HADOOP-16767 Handle non-IO exceptions in reopen()

2021-02-08 Thread GitBox
yzhangal commented on pull request #2685: URL: https://github.com/apache/hadoop/pull/2685#issuecomment-775353597 Thanks a lot @steveloughran ! I just uploaded HADOOP-17338 PR for 2.10 branch, but somehow it's not visible in the jira yet. Wonder if you were able to see. Thanks.

[jira] [Work started] (HADOOP-17520) S3A to downgraded to unguarded if the DDB table cannot be found

2021-02-08 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-17520?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Work on HADOOP-17520 started by Steve Loughran. --- > S3A to downgraded to unguarded if the DDB table cannot be found >

[GitHub] [hadoop] hadoop-yetus commented on pull request #2688: Branch 2.10

2021-02-08 Thread GitBox
hadoop-yetus commented on pull request #2688: URL: https://github.com/apache/hadoop/pull/2688#issuecomment-775347943 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 0m

[GitHub] [hadoop] yzhangal opened a new pull request #2688: Branch 2.10

2021-02-08 Thread GitBox
yzhangal opened a new pull request #2688: URL: https://github.com/apache/hadoop/pull/2688 ## NOTICE Please create an issue in ASF JIRA before opening a pull request, and you need to set the title of the pull request which starts with the corresponding JIRA issue number. (e.g.

[jira] [Work logged] (HADOOP-17497) refactoring the signature of S3ClientFactory breaks hboss compile

2021-02-08 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-17497?focusedWorklogId=549753=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-549753 ] ASF GitHub Bot logged work on HADOOP-17497: --- Author: ASF GitHub Bot

[GitHub] [hadoop] steveloughran commented on a change in pull request #2654: HADOOP-17497. refactoring the signature of S3ClientFactory breaks hboss compile.

2021-02-08 Thread GitBox
steveloughran commented on a change in pull request #2654: URL: https://github.com/apache/hadoop/pull/2654#discussion_r572265444 ## File path: hadoop-tools/hadoop-aws/src/main/java/org/apache/hadoop/fs/s3a/S3ClientFactory.java ## @@ -30,6 +30,9 @@ /** * Factory for

[jira] [Assigned] (HADOOP-17520) S3A to downgraded to unguarded if the DDB table cannot be found

2021-02-08 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-17520?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran reassigned HADOOP-17520: --- Assignee: Steve Loughran > S3A to downgraded to unguarded if the DDB table cannot

[jira] [Created] (HADOOP-17520) S3A to downgraded to unguarded if the DDB table cannot be found

2021-02-08 Thread Steve Loughran (Jira)
Steve Loughran created HADOOP-17520: --- Summary: S3A to downgraded to unguarded if the DDB table cannot be found Key: HADOOP-17520 URL: https://issues.apache.org/jira/browse/HADOOP-17520 Project:

[jira] [Created] (HADOOP-17519) Decommission S3Guard

2021-02-08 Thread Steve Loughran (Jira)
Steve Loughran created HADOOP-17519: --- Summary: Decommission S3Guard Key: HADOOP-17519 URL: https://issues.apache.org/jira/browse/HADOOP-17519 Project: Hadoop Common Issue Type: Improvement

[jira] [Work logged] (HADOOP-13551) Collect AwsSdkMetrics in S3A FileSystem IOStatistics

2021-02-08 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-13551?focusedWorklogId=549750=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-549750 ] ASF GitHub Bot logged work on HADOOP-13551: --- Author: ASF GitHub Bot

[jira] [Work logged] (HADOOP-13551) Collect AwsSdkMetrics in S3A FileSystem IOStatistics

2021-02-08 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-13551?focusedWorklogId=549748=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-549748 ] ASF GitHub Bot logged work on HADOOP-13551: --- Author: ASF GitHub Bot

[jira] [Updated] (HADOOP-13551) Collect AwsSdkMetrics in S3A FileSystem IOStatistics

2021-02-08 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-13551?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated HADOOP-13551: Labels: pull-request-available (was: ) > Collect AwsSdkMetrics in S3A FileSystem

[GitHub] [hadoop] steveloughran commented on pull request #2687: HADOOP-13551. Wire up AWS SDK metrics to IOStatistics

2021-02-08 Thread GitBox
steveloughran commented on pull request #2687: URL: https://github.com/apache/hadoop/pull/2687#issuecomment-775337429 @mehakmeet FYI This is an automated message from the Apache Git Service. To respond to the message, please

[GitHub] [hadoop] steveloughran opened a new pull request #2687: HADOOP-13551. Wire up AWS SDK metrics to IOStatistics

2021-02-08 Thread GitBox
steveloughran opened a new pull request #2687: URL: https://github.com/apache/hadoop/pull/2687 tests which fail on the new builder API. Need to understand its config better *this PR doesn't fix the problem; it's the starting point to fixing it*

[jira] [Work logged] (HADOOP-17514) Remove trace subcommand from hadoop CLI

2021-02-08 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-17514?focusedWorklogId=549732=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-549732 ] ASF GitHub Bot logged work on HADOOP-17514: --- Author: ASF GitHub Bot

[GitHub] [hadoop] hadoop-yetus commented on pull request #2680: HADOOP-17514. Remove trace subcommand from hadoop CLI.

2021-02-08 Thread GitBox
hadoop-yetus commented on pull request #2680: URL: https://github.com/apache/hadoop/pull/2680#issuecomment-775319496 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0

[jira] [Work logged] (HADOOP-17516) Upgrade ant to 1.10.9

2021-02-08 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-17516?focusedWorklogId=549728=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-549728 ] ASF GitHub Bot logged work on HADOOP-17516: --- Author: ASF GitHub Bot

[GitHub] [hadoop] hadoop-yetus commented on pull request #2683: HADOOP-17516. Upgrade ant to 1.10.9.

2021-02-08 Thread GitBox
hadoop-yetus commented on pull request #2683: URL: https://github.com/apache/hadoop/pull/2683#issuecomment-775316840 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0

[jira] [Commented] (HADOOP-13551) Collect AwsSdkMetrics in S3A FileSystem IOStatistics

2021-02-08 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-13551?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17281221#comment-17281221 ] Steve Loughran commented on HADOOP-13551: - Stack {code} [ERROR]

[jira] [Commented] (HADOOP-17518) Usage of incorrect regex range A-z

2021-02-08 Thread Kihwal Lee (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-17518?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17281189#comment-17281189 ] Kihwal Lee commented on HADOOP-17518: - It is only referenced by a HttpFS test. We could simply

[jira] [Work logged] (HADOOP-17038) Support disabling buffered reads in ABFS positional reads

2021-02-08 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-17038?focusedWorklogId=549681=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-549681 ] ASF GitHub Bot logged work on HADOOP-17038: --- Author: ASF GitHub Bot

[GitHub] [hadoop] steveloughran commented on a change in pull request #2646: HADOOP-17038 Support disabling buffered reads in ABFS positional reads.

2021-02-08 Thread GitBox
steveloughran commented on a change in pull request #2646: URL: https://github.com/apache/hadoop/pull/2646#discussion_r572178766 ## File path: hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azurebfs/AzureBlobFileSystemStore.java ## @@ -634,12 +638,15 @@ public

[jira] [Work logged] (HADOOP-17038) Support disabling buffered reads in ABFS positional reads

2021-02-08 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-17038?focusedWorklogId=549677=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-549677 ] ASF GitHub Bot logged work on HADOOP-17038: --- Author: ASF GitHub Bot

[GitHub] [hadoop] steveloughran commented on a change in pull request #2646: HADOOP-17038 Support disabling buffered reads in ABFS positional reads.

2021-02-08 Thread GitBox
steveloughran commented on a change in pull request #2646: URL: https://github.com/apache/hadoop/pull/2646#discussion_r572175379 ## File path: hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azurebfs/services/AbfsInputStream.java ## @@ -135,6 +145,41 @@ public

[jira] [Commented] (HADOOP-16767) S3AInputStream reopening does not handle non IO exceptions properly

2021-02-08 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-16767?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17281164#comment-17281164 ] Steve Loughran commented on HADOOP-16767: - thanks, merged back > S3AInputStream reopening does

[jira] [Updated] (HADOOP-16767) S3AInputStream reopening does not handle non IO exceptions properly

2021-02-08 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-16767?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran updated HADOOP-16767: Fix Version/s: 2.10.2 > S3AInputStream reopening does not handle non IO exceptions

[jira] [Work logged] (HADOOP-16767) S3AInputStream reopening does not handle non IO exceptions properly

2021-02-08 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-16767?focusedWorklogId=549672=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-549672 ] ASF GitHub Bot logged work on HADOOP-16767: --- Author: ASF GitHub Bot

[GitHub] [hadoop] steveloughran merged pull request #2685: HADOOP-16767 Handle non-IO exceptions in reopen()

2021-02-08 Thread GitBox
steveloughran merged pull request #2685: URL: https://github.com/apache/hadoop/pull/2685 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to

[GitHub] [hadoop] hadoop-yetus commented on pull request #2686: HDFS-15826. Solve the problem of incorrect progress of delegation tokens when loading FsImage.

2021-02-08 Thread GitBox
hadoop-yetus commented on pull request #2686: URL: https://github.com/apache/hadoop/pull/2686#issuecomment-775178247 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0

[jira] [Work logged] (HADOOP-13327) Add OutputStream + Syncable to the Filesystem Specification

2021-02-08 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-13327?focusedWorklogId=549544=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-549544 ] ASF GitHub Bot logged work on HADOOP-13327: --- Author: ASF GitHub Bot

[GitHub] [hadoop] steveloughran commented on pull request #2587: HADOOP-13327 Output Stream Specification.

2021-02-08 Thread GitBox
steveloughran commented on pull request #2587: URL: https://github.com/apache/hadoop/pull/2587#issuecomment-775113631 FWIW, here are the IOStats logged on the runs ABFS (which supports hsync) ```

[jira] [Commented] (HADOOP-13887) Encrypt S3A data client-side with AWS SDK (S3-CSE)

2021-02-08 Thread Mehakmeet Singh (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-13887?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17280997#comment-17280997 ] Mehakmeet Singh commented on HADOOP-13887: -- I have been trying to add support for CSE using

[GitHub] [hadoop] hadoop-yetus commented on pull request #2677: HDFS-15817. Rename snapshots while marking them deleted.

2021-02-08 Thread GitBox
hadoop-yetus commented on pull request #2677: URL: https://github.com/apache/hadoop/pull/2677#issuecomment-775102419 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0

[jira] [Work logged] (HADOOP-13327) Add OutputStream + Syncable to the Filesystem Specification

2021-02-08 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-13327?focusedWorklogId=549510=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-549510 ] ASF GitHub Bot logged work on HADOOP-13327: --- Author: ASF GitHub Bot

[GitHub] [hadoop] steveloughran commented on a change in pull request #2587: HADOOP-13327 Output Stream Specification.

2021-02-08 Thread GitBox
steveloughran commented on a change in pull request #2587: URL: https://github.com/apache/hadoop/pull/2587#discussion_r571955452 ## File path: hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/RawLocalFileSystem.java ## @@ -137,8 +139,13 @@ public void

[jira] [Work logged] (HADOOP-16906) Add some Abortable.abort() interface for streams etc which can be terminated

2021-02-08 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-16906?focusedWorklogId=549508=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-549508 ] ASF GitHub Bot logged work on HADOOP-16906: --- Author: ASF GitHub Bot

[GitHub] [hadoop] steveloughran commented on pull request #2684: HADOOP-16906. Abortable

2021-02-08 Thread GitBox
steveloughran commented on pull request #2684: URL: https://github.com/apache/hadoop/pull/2684#issuecomment-775058042 Thanks @mukund-thakur Had some thoughts over the w/e 1. The call should return something, `AbortOutcome` which can include a list of non-critical exceptions

[jira] [Work logged] (HADOOP-17038) Support disabling buffered reads in ABFS positional reads

2021-02-08 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-17038?focusedWorklogId=549449=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-549449 ] ASF GitHub Bot logged work on HADOOP-17038: --- Author: ASF GitHub Bot

[GitHub] [hadoop] hadoop-yetus commented on pull request #2646: HADOOP-17038 Support disabling buffered reads in ABFS positional reads.

2021-02-08 Thread GitBox
hadoop-yetus commented on pull request #2646: URL: https://github.com/apache/hadoop/pull/2646#issuecomment-775003443 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0

[GitHub] [hadoop] jianghuazhu opened a new pull request #2686: HDFS-15826. Solve the problem of incorrect progress of delegation tokens when loading FsImage.

2021-02-08 Thread GitBox
jianghuazhu opened a new pull request #2686: URL: https://github.com/apache/hadoop/pull/2686 …ens when loading FsImage. ## NOTICE Please create an issue in ASF JIRA before opening a pull request, and you need to set the title of the pull request which starts with the

[GitHub] [hadoop] hadoop-yetus commented on pull request #2639: HDFS-15785. Datanode to support using DNS to resolve nameservices to IP addresses to get list of namenodes.

2021-02-08 Thread GitBox
hadoop-yetus commented on pull request #2639: URL: https://github.com/apache/hadoop/pull/2639#issuecomment-774974914 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0

[jira] [Work logged] (HADOOP-17038) Support disabling buffered reads in ABFS positional reads

2021-02-08 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-17038?focusedWorklogId=549427=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-549427 ] ASF GitHub Bot logged work on HADOOP-17038: --- Author: ASF GitHub Bot

[GitHub] [hadoop] anoopsjohn commented on pull request #2646: HADOOP-17038 Support disabling buffered reads in ABFS positional reads.

2021-02-08 Thread GitBox
anoopsjohn commented on pull request #2646: URL: https://github.com/apache/hadoop/pull/2646#issuecomment-774959132 > Production code looks good. Added some comments in tests. > Also pending checkstyle >

  1   2   >