[jira] [Updated] (HADOOP-17087) Add EC flag to stat commands

2020-06-24 Thread Hongbing Wang (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-17087?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hongbing Wang updated HADOOP-17087: --- Description: We currently do not have a brief way to judge an ec file.  {{hdfs fsck}}  can

[GitHub] [hadoop] hadoop-yetus commented on pull request #2038: HADOOP-17022 Tune S3AFileSystem.listFiles() api.

2020-06-24 Thread GitBox
hadoop-yetus commented on pull request #2038: URL: https://github.com/apache/hadoop/pull/2038#issuecomment-648744924 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 0m

[jira] [Updated] (HADOOP-17088) Failed to load Xinclude files with relative path in case of loading conf via URI

2020-06-24 Thread Yushi Hayasaka (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-17088?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yushi Hayasaka updated HADOOP-17088: Description: When we create a configuration file, which load a external XML file with

[jira] [Reopened] (HADOOP-17015) ABFS: Make PUT and POST operations idempotent

2020-06-24 Thread Thomas Marqardt (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-17015?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Marqardt reopened HADOOP-17015: -- We should revisit PR 2021 and try to find a better solution for rename.  Users expect

[GitHub] [hadoop] sunchao commented on a change in pull request #2080: HDFS-15417. RBF: Get the datanode report from cache for federation WebHDFS operations

2020-06-24 Thread GitBox
sunchao commented on a change in pull request #2080: URL: https://github.com/apache/hadoop/pull/2080#discussion_r444673117 ## File path: hadoop-hdfs-project/hadoop-hdfs-rbf/src/main/java/org/apache/hadoop/hdfs/server/federation/router/RouterRpcServer.java ## @@ -361,6 +380,23

[GitHub] [hadoop] NickyYe commented on a change in pull request #2080: HDFS-15417. RBF: Get the datanode report from cache for federation WebHDFS operations

2020-06-24 Thread GitBox
NickyYe commented on a change in pull request #2080: URL: https://github.com/apache/hadoop/pull/2080#discussion_r444685702 ## File path: hadoop-hdfs-project/hadoop-hdfs-rbf/src/main/java/org/apache/hadoop/hdfs/server/federation/router/RouterRpcServer.java ## @@ -1748,4

[GitHub] [hadoop] ishaniahuja commented on pull request #2072: HADOOP-17058. ABFS: Support for AppendBlob in Hadoop ABFS Driver

2020-06-24 Thread GitBox
ishaniahuja commented on pull request #2072: URL: https://github.com/apache/hadoop/pull/2072#issuecomment-648638038 namespace, rest version:= 2018-11-09 Tests run: 84, Failures: 0, Errors: 0, Skipped: 0 Tests run: 443, Failures: 0, Errors: 0, Skipped: 42 Tests run: 207,

[jira] [Resolved] (HADOOP-16659) ABFS: add missing docs for configuration

2020-06-24 Thread Bilahari T H (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-16659?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bilahari T H resolved HADOOP-16659. --- Resolution: Fixed > ABFS: add missing docs for configuration >

[jira] [Commented] (HADOOP-16659) ABFS: add missing docs for configuration

2020-06-24 Thread Bilahari T H (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-16659?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17143595#comment-17143595 ] Bilahari T H commented on HADOOP-16659: --- Duplicate task

[jira] [Assigned] (HADOOP-16659) ABFS: add missing docs for configuration

2020-06-24 Thread Bilahari T H (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-16659?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bilahari T H reassigned HADOOP-16659: - Assignee: Bilahari T H > ABFS: add missing docs for configuration >

[GitHub] [hadoop] Hexiaoqiao commented on pull request #2080: HDFS-15417. RBF: Get the datanode report from cache for federation WebHDFS operations

2020-06-24 Thread GitBox
Hexiaoqiao commented on pull request #2080: URL: https://github.com/apache/hadoop/pull/2080#issuecomment-648685825 Thanks @sunchao involve me here. In my internal version I try to turn off `getDatanodeReport` at Router side and not turn on webhdfs feature, The `getDatanodeReport` is very

[jira] [Created] (HADOOP-17087) Add EC flag to stat commands

2020-06-24 Thread Hongbing Wang (Jira)
Hongbing Wang created HADOOP-17087: -- Summary: Add EC flag to stat commands Key: HADOOP-17087 URL: https://issues.apache.org/jira/browse/HADOOP-17087 Project: Hadoop Common Issue Type:

[jira] [Commented] (HADOOP-17087) Add EC flag to stat commands

2020-06-24 Thread Hadoop QA (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-17087?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17143828#comment-17143828 ] Hadoop QA commented on HADOOP-17087: | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote ||

[GitHub] [hadoop] crossfire opened a new pull request #2097: HADOOP-17088. Failed to load Xinclude files with relative path in cas…

2020-06-24 Thread GitBox
crossfire opened a new pull request #2097: URL: https://github.com/apache/hadoop/pull/2097 …e of loading conf via URI https://issues.apache.org/jira/browse/HADOOP-17088 This is an automated message from the Apache Git

[jira] [Updated] (HADOOP-17088) Failed to load Xinclude files with relative path in case of loading conf via URI

2020-06-24 Thread Yushi Hayasaka (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-17088?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yushi Hayasaka updated HADOOP-17088: Status: Patch Available (was: Open) PR: [https://github.com/apache/hadoop/pull/2097] >

[jira] [Created] (HADOOP-17089) WASB: Update azure-storage-java SDK

2020-06-24 Thread Thomas Marqardt (Jira)
Thomas Marqardt created HADOOP-17089: Summary: WASB: Update azure-storage-java SDK Key: HADOOP-17089 URL: https://issues.apache.org/jira/browse/HADOOP-17089 Project: Hadoop Common Issue

[jira] [Created] (HADOOP-17088) Failed to load Xinclude files with relative path in case of loading conf via URI

2020-06-24 Thread Yushi Hayasaka (Jira)
Yushi Hayasaka created HADOOP-17088: --- Summary: Failed to load Xinclude files with relative path in case of loading conf via URI Key: HADOOP-17088 URL: https://issues.apache.org/jira/browse/HADOOP-17088

[jira] [Updated] (HADOOP-17087) Add EC flag to stat commands

2020-06-24 Thread Hongbing Wang (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-17087?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hongbing Wang updated HADOOP-17087: --- Attachment: HADOOP-17087.001.patch > Add EC flag to stat commands >

[jira] [Resolved] (HADOOP-17050) S3A to support additional token issuers

2020-06-24 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-17050?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran resolved HADOOP-17050. - Resolution: Fixed > S3A to support additional token issuers >

[jira] [Commented] (HADOOP-17087) Add EC flag to stat commands

2020-06-24 Thread Ayush Saxena (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-17087?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17143759#comment-17143759 ] Ayush Saxena commented on HADOOP-17087: --- Thanx [~wanghongbing] for the report with Ls command do

[jira] [Commented] (HADOOP-17087) Add EC flag to stat commands

2020-06-24 Thread Hongbing Wang (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-17087?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17143769#comment-17143769 ] Hongbing Wang commented on HADOOP-17087: {quote} Ls with -e option {quote} Sorry i didn't

[jira] [Resolved] (HADOOP-17054) ABFS: Fix idempotency test failures when SharedKey is set as AuthType

2020-06-24 Thread Thomas Marqardt (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-17054?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Marqardt resolved HADOOP-17054. -- Resolution: Fixed Accidentally reactivated HADOOP-17015 but meant to reactivate

[jira] [Commented] (HADOOP-17077) S3A delegation token binding to support secondary binding list

2020-06-24 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-17077?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17143848#comment-17143848 ] Steve Loughran commented on HADOOP-17077: - testing this this highlights that fetchdt and dtutil

[jira] [Updated] (HADOOP-17087) Add EC flag to stat commands

2020-06-24 Thread Ayush Saxena (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-17087?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ayush Saxena updated HADOOP-17087: -- Status: Patch Available (was: Open) > Add EC flag to stat commands >

[jira] [Commented] (HADOOP-17087) Add EC flag to stat commands

2020-06-24 Thread Hongbing Wang (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-17087?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17143771#comment-17143771 ] Hongbing Wang commented on HADOOP-17087: Stat is a comprehensive description of the file. Maybe

[jira] [Commented] (HADOOP-17087) Add EC flag to stat commands

2020-06-24 Thread Ayush Saxena (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-17087?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17143782#comment-17143782 ] Ayush Saxena commented on HADOOP-17087: --- For just EC policy you can use getErasureCodingPolicy()

[jira] [Commented] (HADOOP-17087) Add EC flag to stat commands

2020-06-24 Thread Hongbing Wang (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-17087?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17143785#comment-17143785 ] Hongbing Wang commented on HADOOP-17087: ok, I understand. I agree that no change is best.

[GitHub] [hadoop] aajisaka opened a new pull request #2098: HDFS-15424. Javadoc failing with "cannot find symbol com.google.protobuf.GeneratedMessageV3 implements"

2020-06-24 Thread GitBox
aajisaka opened a new pull request #2098: URL: https://github.com/apache/hadoop/pull/2098 This PR is to test https://issues.apache.org/jira/browse/YETUS-972 * Apply #2094 * Use https://github.com/apache/yetus/pull/112

[jira] [Assigned] (HADOOP-17086) Parsing errors in ABFS Driver with creation Time (being returned in ListPath)

2020-06-24 Thread Bilahari T H (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-17086?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bilahari T H reassigned HADOOP-17086: - Assignee: Bilahari T H > Parsing errors in ABFS Driver with creation Time (being

[GitHub] [hadoop] hadoop-yetus commented on pull request #2072: HADOOP-17058. ABFS: Support for AppendBlob in Hadoop ABFS Driver

2020-06-24 Thread GitBox
hadoop-yetus commented on pull request #2072: URL: https://github.com/apache/hadoop/pull/2072#issuecomment-648679739 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 0m

[GitHub] [hadoop] NickyYe opened a new pull request #2095: HDFS-15417. Apply umask when creating directory by WebHDFS

2020-06-24 Thread GitBox
NickyYe opened a new pull request #2095: URL: https://github.com/apache/hadoop/pull/2095 ## NOTICE Please create an issue in ASF JIRA before opening a pull request, and you need to set the title of the pull request which starts with the corresponding JIRA issue number. (e.g.

[GitHub] [hadoop] NickyYe closed pull request #2095: HDFS-15417. Apply umask when creating directory by WebHDFS

2020-06-24 Thread GitBox
NickyYe closed pull request #2095: URL: https://github.com/apache/hadoop/pull/2095 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to

[GitHub] [hadoop] hadoop-yetus commented on pull request #2080: HDFS-15417. RBF: Get the datanode report from cache for federation WebHDFS operations

2020-06-24 Thread GitBox
hadoop-yetus commented on pull request #2080: URL: https://github.com/apache/hadoop/pull/2080#issuecomment-648693848 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 21m

[GitHub] [hadoop] sunchao commented on a change in pull request #2080: HDFS-15417. RBF: Get the datanode report from cache for federation WebHDFS operations

2020-06-24 Thread GitBox
sunchao commented on a change in pull request #2080: URL: https://github.com/apache/hadoop/pull/2080#discussion_r444664594 ## File path: hadoop-hdfs-project/hadoop-hdfs-rbf/src/main/java/org/apache/hadoop/hdfs/server/federation/router/RouterRpcServer.java ## @@ -868,6 +904,50

[GitHub] [hadoop] NickyYe commented on pull request #2080: HDFS-15417. RBF: Get the datanode report from cache for federation WebHDFS operations

2020-06-24 Thread GitBox
NickyYe commented on pull request #2080: URL: https://github.com/apache/hadoop/pull/2080#issuecomment-648622048 > Thanks @NickyYe for the update. Yes we can separate the report cache consolidation as a follow-up. Could you create a JIRA for that? Thanks. Filed:

[jira] [Updated] (HADOOP-17086) Parsing errors in ABFS Driver with creation Time (being returned in ListPath)

2020-06-24 Thread Ishani (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-17086?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ishani updated HADOOP-17086: Description: I am seeing errors while running ABFS Driver against stg75 build in canary. This is

[jira] [Comment Edited] (HADOOP-16913) ABFS: Support for OAuth v2.0 endpoints

2020-06-24 Thread Bilahari T H (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-16913?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17143587#comment-17143587 ] Bilahari T H edited comment on HADOOP-16913 at 6/24/20, 7:17 AM: - This

[jira] [Resolved] (HADOOP-16913) ABFS: Support for OAuth v2.0 endpoints

2020-06-24 Thread Bilahari T H (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-16913?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bilahari T H resolved HADOOP-16913. --- Resolution: Fixed > ABFS: Support for OAuth v2.0 endpoints >

[jira] [Commented] (HADOOP-16913) ABFS: Support for OAuth v2.0 endpoints

2020-06-24 Thread Bilahari T H (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-16913?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17143587#comment-17143587 ] Bilahari T H commented on HADOOP-16913: --- This has been checked in as part of the following PR:

[GitHub] [hadoop] NickyYe commented on a change in pull request #2080: HDFS-15417. RBF: Get the datanode report from cache for federation WebHDFS operations

2020-06-24 Thread GitBox
NickyYe commented on a change in pull request #2080: URL: https://github.com/apache/hadoop/pull/2080#discussion_r444668178 ## File path: hadoop-hdfs-project/hadoop-hdfs-rbf/src/main/java/org/apache/hadoop/hdfs/server/federation/router/RouterRpcServer.java ## @@ -361,6 +380,23

[jira] [Created] (HADOOP-17086) Parsing errors in ABFS Driver with creation Time (being returned in ListPath)

2020-06-24 Thread Ishani (Jira)
Ishani created HADOOP-17086: --- Summary: Parsing errors in ABFS Driver with creation Time (being returned in ListPath) Key: HADOOP-17086 URL: https://issues.apache.org/jira/browse/HADOOP-17086 Project:

[GitHub] [hadoop] hadoop-yetus commented on pull request #2096: HDFS-15312. Apply umask when creating directory by WebHDFS

2020-06-24 Thread GitBox
hadoop-yetus commented on pull request #2096: URL: https://github.com/apache/hadoop/pull/2096#issuecomment-648694692 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 0m

[GitHub] [hadoop] mukund-thakur commented on pull request #2038: HADOOP-17022 Tune S3AFileSystem.listFiles() api.

2020-06-24 Thread GitBox
mukund-thakur commented on pull request #2038: URL: https://github.com/apache/hadoop/pull/2038#issuecomment-648702503 Latest commit fixes the above test failures. After multiple debugging efforts I thought I should run the tests without s3guard as well. So when I did that I see 8 tests

[jira] [Resolved] (HADOOP-16818) ABFS: Combine append+flush calls for blockblob & appendblob

2020-06-24 Thread Ishani (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-16818?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ishani resolved HADOOP-16818. - Release Note: it was decided to drop the usage of feature/API in the driver. (Combined Calls). There

[jira] [Commented] (HADOOP-17083) Update guava to 27.0-jre in hadoop branch-2.10

2020-06-24 Thread Hadoop QA (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-17083?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17143712#comment-17143712 ] Hadoop QA commented on HADOOP-17083: | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote ||

[GitHub] [hadoop] hadoop-yetus commented on pull request #2093: HDFS-15416. DataStorage#addStorageLocations() should add more reasonable information verification.

2020-06-24 Thread GitBox
hadoop-yetus commented on pull request #2093: URL: https://github.com/apache/hadoop/pull/2093#issuecomment-648620311 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 1m

[GitHub] [hadoop] NickyYe opened a new pull request #2096: HDFS-15312. Apply umask when creating directory by WebHDFS

2020-06-24 Thread GitBox
NickyYe opened a new pull request #2096: URL: https://github.com/apache/hadoop/pull/2096 ## NOTICE Please create an issue in ASF JIRA before opening a pull request, and you need to set the title of the pull request which starts with the corresponding JIRA issue number. (e.g.

[jira] [Commented] (HADOOP-17088) Failed to load Xinclude files with relative path in case of loading conf via URI

2020-06-24 Thread Yushi Hayasaka (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-17088?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17144036#comment-17144036 ] Yushi Hayasaka commented on HADOOP-17088: - [~jeagles] Thanks for the comment. But I think it is

[jira] [Commented] (HADOOP-17083) Update guava to 27.0-jre in hadoop branch-2.10

2020-06-24 Thread Ahmed Hussein (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-17083?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17143945#comment-17143945 ] Ahmed Hussein commented on HADOOP-17083: *findbugs Errors:* *===*   *FindBugs

[GitHub] [hadoop] hadoop-yetus commented on pull request #2097: HADOOP-17088. Failed to load Xinclude files with relative path in cas…

2020-06-24 Thread GitBox
hadoop-yetus commented on pull request #2097: URL: https://github.com/apache/hadoop/pull/2097#issuecomment-648860632 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 28m

[jira] [Comment Edited] (HADOOP-17088) Failed to load Xinclude files with relative path in case of loading conf via URI

2020-06-24 Thread Yushi Hayasaka (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-17088?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17144036#comment-17144036 ] Yushi Hayasaka edited comment on HADOOP-17088 at 6/24/20, 4:53 PM: ---

[jira] [Commented] (HADOOP-17088) Failed to load Xinclude files with relative path in case of loading conf via URI

2020-06-24 Thread Jonathan Turner Eagles (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-17088?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17143908#comment-17143908 ] Jonathan Turner Eagles commented on HADOOP-17088: - One important security feature is to

[jira] [Commented] (HADOOP-17091) Javadoc failing with "cannot find symbol com.google.protobuf.GeneratedMessageV3 implements"

2020-06-24 Thread Akira Ajisaka (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-17091?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17144112#comment-17144112 ] Akira Ajisaka commented on HADOOP-17091: Moved to Hadoop common. > Javadoc failing with

[jira] [Moved] (HADOOP-17091) Javadoc failing with "cannot find symbol com.google.protobuf.GeneratedMessageV3 implements"

2020-06-24 Thread Akira Ajisaka (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-17091?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Akira Ajisaka moved HDFS-15424 to HADOOP-17091: --- Component/s: (was: build) build

[jira] [Updated] (HADOOP-17090) Increase precommit job timeout from 5 hours to 20 hours

2020-06-24 Thread Akira Ajisaka (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-17090?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Akira Ajisaka updated HADOOP-17090: --- Description: Now we frequently increase the timeout for testing and undo the change before

[jira] [Created] (HADOOP-17090) Increase precommit job timeout from 5 hours to 20 hours

2020-06-24 Thread Akira Ajisaka (Jira)
Akira Ajisaka created HADOOP-17090: -- Summary: Increase precommit job timeout from 5 hours to 20 hours Key: HADOOP-17090 URL: https://issues.apache.org/jira/browse/HADOOP-17090 Project: Hadoop Common

[jira] [Updated] (HADOOP-17083) Update guava to 27.0-jre in hadoop branch-2.10

2020-06-24 Thread Ahmed Hussein (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-17083?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ahmed Hussein updated HADOOP-17083: --- Attachment: HADOOP-17083-branch-2.10.003.patch > Update guava to 27.0-jre in hadoop

[GitHub] [hadoop] crossfire commented on pull request #2097: HADOOP-17088. Failed to load Xinclude files with relative path in cas…

2020-06-24 Thread GitBox
crossfire commented on pull request #2097: URL: https://github.com/apache/hadoop/pull/2097#issuecomment-648868600 Hmm, it seems to be caused by https://issues.apache.org/jira/browse/HDFS-15424. This is an automated message

[jira] [Resolved] (HADOOP-17015) ABFS: Make PUT and POST operations idempotent

2020-06-24 Thread Thomas Marqardt (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-17015?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Marqardt resolved HADOOP-17015. -- Resolution: Fixed Sneha and I discussed this.  The common Hadoop scenario is a case

[GitHub] [hadoop] xiaoyuyao commented on a change in pull request #2085: HADOOP-17079. Optimize UGI#getGroups by adding UGI#getGroupsSet.

2020-06-24 Thread GitBox
xiaoyuyao commented on a change in pull request #2085: URL: https://github.com/apache/hadoop/pull/2085#discussion_r445130674 ## File path: hadoop-hdfs-project/hadoop-hdfs-httpfs/src/test/java/org/apache/hadoop/lib/service/security/DummyGroupMapping.java ## @@ -47,4 +48,9 @@

[GitHub] [hadoop] xiaoyuyao commented on a change in pull request #2085: HADOOP-17079. Optimize UGI#getGroups by adding UGI#getGroupsSet.

2020-06-24 Thread GitBox
xiaoyuyao commented on a change in pull request #2085: URL: https://github.com/apache/hadoop/pull/2085#discussion_r445128418 ## File path: hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/security/NullGroupsMapping.java ## @@ -31,6 +33,19 @@ public void

[GitHub] [hadoop] xiaoyuyao commented on a change in pull request #2085: HADOOP-17079. Optimize UGI#getGroups by adding UGI#getGroupsSet.

2020-06-24 Thread GitBox
xiaoyuyao commented on a change in pull request #2085: URL: https://github.com/apache/hadoop/pull/2085#discussion_r445128078 ## File path: hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/security/JniBasedUnixGroupsMapping.java ## @@ -19,9 +19,9 @@ package

[GitHub] [hadoop] hadoop-yetus commented on pull request #2080: HDFS-15417. RBF: Get the datanode report from cache for federation WebHDFS operations

2020-06-24 Thread GitBox
hadoop-yetus commented on pull request #2080: URL: https://github.com/apache/hadoop/pull/2080#issuecomment-649027770 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 0m

[GitHub] [hadoop] xiaoyuyao commented on a change in pull request #2085: HADOOP-17079. Optimize UGI#getGroups by adding UGI#getGroupsSet.

2020-06-24 Thread GitBox
xiaoyuyao commented on a change in pull request #2085: URL: https://github.com/apache/hadoop/pull/2085#discussion_r445127216 ## File path: hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/security/Groups.java ## @@ -345,28 +373,28 @@ public long read() {

[GitHub] [hadoop] hadoop-yetus commented on pull request #2099: HADOOP-17089: WASB: Update azure-storage-java SDK

2020-06-24 Thread GitBox
hadoop-yetus commented on pull request #2099: URL: https://github.com/apache/hadoop/pull/2099#issuecomment-649038099 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 0m

[jira] [Commented] (HADOOP-17083) Update guava to 27.0-jre in hadoop branch-2.10

2020-06-24 Thread Hadoop QA (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-17083?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17144254#comment-17144254 ] Hadoop QA commented on HADOOP-17083: | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote ||

[GitHub] [hadoop] xiaoyuyao commented on pull request #2085: HADOOP-17079. Optimize UGI#getGroups by adding UGI#getGroupsSet.

2020-06-24 Thread GitBox
xiaoyuyao commented on pull request #2085: URL: https://github.com/apache/hadoop/pull/2085#issuecomment-649038401 Thanks @Hexiaoqiao fro the review and positive results of similar changes in your deployment. I've addressed the feedback in the new commit.

[jira] [Updated] (HADOOP-17079) Optimize UGI#getGroups by adding UGI#getGroupsSet

2020-06-24 Thread Xiaoyu Yao (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-17079?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiaoyu Yao updated HADOOP-17079: Attachment: HADOOP-17079.002.patch > Optimize UGI#getGroups by adding UGI#getGroupsSet >

[jira] [Commented] (HADOOP-17079) Optimize UGI#getGroups by adding UGI#getGroupsSet

2020-06-24 Thread Xiaoyu Yao (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-17079?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17144288#comment-17144288 ] Xiaoyu Yao commented on HADOOP-17079: - Attach the patch file to trigger Jenkins. The PR link

[jira] [Updated] (HADOOP-17083) Update guava to 27.0-jre in hadoop branch-2.10

2020-06-24 Thread Ahmed Hussein (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-17083?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ahmed Hussein updated HADOOP-17083: --- Attachment: HADOOP-17083-branch-2.10.004.patch > Update guava to 27.0-jre in hadoop

[GitHub] [hadoop] sunchao commented on pull request #2080: HDFS-15417. RBF: Get the datanode report from cache for federation WebHDFS operations

2020-06-24 Thread GitBox
sunchao commented on pull request #2080: URL: https://github.com/apache/hadoop/pull/2080#issuecomment-648979719 Thanks @Hexiaoqiao . Similarly, in our case we also disabled this for router metrics but need webhdfs on router. Relying on `getDatanodeReport` is not the ideal approach but

[GitHub] [hadoop] hadoop-yetus commented on pull request #2083: HADOOP-17077. S3A delegation token binding to support secondary binding list

2020-06-24 Thread GitBox
hadoop-yetus commented on pull request #2083: URL: https://github.com/apache/hadoop/pull/2083#issuecomment-648981862 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 1m

[GitHub] [hadoop] ThomasMarquardt opened a new pull request #2099: HADOOP-17089: WASB: Update azure-storage-java SDK

2020-06-24 Thread GitBox
ThomasMarquardt opened a new pull request #2099: URL: https://github.com/apache/hadoop/pull/2099 HADOOP-17089: WASB: Update azure-storage-java SDK DETAILS: WASB depends on the Azure Storage Java SDK. There is a concurrency bug in the Azure Storage Java SDK that can cause the

[jira] [Commented] (HADOOP-17090) Increase precommit job timeout from 5 hours to 20 hours

2020-06-24 Thread Ayush Saxena (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-17090?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17144207#comment-17144207 ] Ayush Saxena commented on HADOOP-17090: --- Thanx [~aajisaka] for putting this up. Indeed this is an

[GitHub] [hadoop] xiaoyuyao commented on a change in pull request #2085: HADOOP-17079. Optimize UGI#getGroups by adding UGI#getGroupsSet.

2020-06-24 Thread GitBox
xiaoyuyao commented on a change in pull request #2085: URL: https://github.com/apache/hadoop/pull/2085#discussion_r445128651 ## File path: hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/security/authorize/AccessControlList.java ## @@ -20,10 +20,7 @@

[GitHub] [hadoop] xiaoyuyao commented on a change in pull request #2085: HADOOP-17079. Optimize UGI#getGroups by adding UGI#getGroupsSet.

2020-06-24 Thread GitBox
xiaoyuyao commented on a change in pull request #2085: URL: https://github.com/apache/hadoop/pull/2085#discussion_r445133102 ## File path: hadoop-hdfs-project/hadoop-hdfs-rbf/src/test/java/org/apache/hadoop/hdfs/server/federation/router/TestRouterUserMappings.java ## @@

[GitHub] [hadoop] xiaoyuyao commented on a change in pull request #2085: HADOOP-17079. Optimize UGI#getGroups by adding UGI#getGroupsSet.

2020-06-24 Thread GitBox
xiaoyuyao commented on a change in pull request #2085: URL: https://github.com/apache/hadoop/pull/2085#discussion_r445132957 ## File path: hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSPermissionChecker.java ## @@ -549,7 +549,6 @@

[jira] [Commented] (HADOOP-17072) Add getClusterRoot and getClusterRoots methods to FileSystem and ViewFilesystem

2020-06-24 Thread Virajith Jalaparti (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-17072?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=1712#comment-1712 ] Virajith Jalaparti commented on HADOOP-17072: - Thanks for the feedback!

[jira] [Commented] (HADOOP-17089) WASB: Update azure-storage-java SDK

2020-06-24 Thread Thomas Marqardt (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-17089?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17144571#comment-17144571 ] Thomas Marqardt commented on HADOOP-17089: -- Updated with test at

[GitHub] [hadoop] hadoop-yetus commented on pull request #2099: HADOOP-17089: WASB: Update azure-storage-java SDK

2020-06-24 Thread GitBox
hadoop-yetus commented on pull request #2099: URL: https://github.com/apache/hadoop/pull/2099#issuecomment-649165774 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 0m

[GitHub] [hadoop] ThomasMarquardt commented on a change in pull request #2099: HADOOP-17089: WASB: Update azure-storage-java SDK

2020-06-24 Thread GitBox
ThomasMarquardt commented on a change in pull request #2099: URL: https://github.com/apache/hadoop/pull/2099#discussion_r445274808 ## File path: hadoop-tools/hadoop-azure/src/test/java/org/apache/hadoop/fs/azure/ITestNativeAzureFileSystemConcurrencyLive.java ## @@ -130,15

[GitHub] [hadoop] virajith opened a new pull request #2100: HDFS-15436. Default mount table name used by ViewFileSystem should be configurable

2020-06-24 Thread GitBox
virajith opened a new pull request #2100: URL: https://github.com/apache/hadoop/pull/2100 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to

[jira] [Commented] (HADOOP-17079) Optimize UGI#getGroups by adding UGI#getGroupsSet

2020-06-24 Thread Hadoop QA (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-17079?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17144610#comment-17144610 ] Hadoop QA commented on HADOOP-17079: | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote ||

[jira] [Assigned] (HADOOP-17093) ABFS: GetAccessToken unrecoverable failures are being retried

2020-06-24 Thread Sneha Vijayarajan (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-17093?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sneha Vijayarajan reassigned HADOOP-17093: -- Assignee: Sneha Vijayarajan > ABFS: GetAccessToken unrecoverable failures

[jira] [Assigned] (HADOOP-17092) ABFS: Long waits and unintended retries when multiple threads try to fetch token using ClientCreds

2020-06-24 Thread Sneha Vijayarajan (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-17092?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sneha Vijayarajan reassigned HADOOP-17092: -- Assignee: Sneha Vijayarajan > ABFS: Long waits and unintended retries when

[jira] [Commented] (HADOOP-17083) Update guava to 27.0-jre in hadoop branch-2.10

2020-06-24 Thread Hadoop QA (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-17083?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17144615#comment-17144615 ] Hadoop QA commented on HADOOP-17083: | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote ||

[GitHub] [hadoop] umamaheswararao commented on pull request #2092: HDFS-15429. mkdirs should work when parent dir is an internalDir and fallback configured.

2020-06-24 Thread GitBox
umamaheswararao commented on pull request #2092: URL: https://github.com/apache/hadoop/pull/2092#issuecomment-649195125 Thanks a lot @rakeshadr for the discussion offline. I relaxed the restriction on creating parent dirs in the case of fallback does not have the directory structure same

[jira] [Created] (HADOOP-17092) ABFS: Long waits and unintended retries when multiple threads try to fetch token using ClientCreds

2020-06-24 Thread Sneha Vijayarajan (Jira)
Sneha Vijayarajan created HADOOP-17092: -- Summary: ABFS: Long waits and unintended retries when multiple threads try to fetch token using ClientCreds Key: HADOOP-17092 URL:

[jira] [Created] (HADOOP-17093) ABFS: GetAccessToken unrecoverable failures are being retried

2020-06-24 Thread Sneha Vijayarajan (Jira)
Sneha Vijayarajan created HADOOP-17093: -- Summary: ABFS: GetAccessToken unrecoverable failures are being retried Key: HADOOP-17093 URL: https://issues.apache.org/jira/browse/HADOOP-17093 Project:

[GitHub] [hadoop] DadanielZ commented on a change in pull request #2099: HADOOP-17089: WASB: Update azure-storage-java SDK

2020-06-24 Thread GitBox
DadanielZ commented on a change in pull request #2099: URL: https://github.com/apache/hadoop/pull/2099#discussion_r445266567 ## File path: hadoop-tools/hadoop-azure/src/test/java/org/apache/hadoop/fs/azure/ITestNativeAzureFileSystemConcurrencyLive.java ## @@ -130,15 +131,55

[jira] [Updated] (HADOOP-17092) ABFS: Long waits and unintended retries when multiple threads try to fetch token using ClientCreds

2020-06-24 Thread Sneha Vijayarajan (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-17092?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sneha Vijayarajan updated HADOOP-17092: --- Description: Issue reported by DB: we recently experienced some problems with

[jira] [Commented] (HADOOP-17079) Optimize UGI#getGroups by adding UGI#getGroupsSet

2020-06-24 Thread Hadoop QA (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-17079?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17144653#comment-17144653 ] Hadoop QA commented on HADOOP-17079: | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote ||

[GitHub] [hadoop] hadoop-yetus commented on pull request #2098: HDFS-15424. Javadoc failing with "cannot find symbol com.google.protobuf.GeneratedMessageV3 implements"

2020-06-24 Thread GitBox
hadoop-yetus commented on pull request #2098: URL: https://github.com/apache/hadoop/pull/2098#issuecomment-649236373 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 0m

[jira] [Commented] (HADOOP-17089) WASB: Update azure-storage-java SDK

2020-06-24 Thread Hudson (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-17089?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17144661#comment-17144661 ] Hudson commented on HADOOP-17089: - SUCCESS: Integrated in Jenkins build Hadoop-trunk-Commit #18378 (See

[GitHub] [hadoop] hadoop-yetus commented on pull request #2099: HADOOP-17089: WASB: Update azure-storage-java SDK

2020-06-24 Thread GitBox
hadoop-yetus commented on pull request #2099: URL: https://github.com/apache/hadoop/pull/2099#issuecomment-649219878 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 0m

[GitHub] [hadoop] asfgit merged pull request #2099: HADOOP-17089: WASB: Update azure-storage-java SDK

2020-06-24 Thread GitBox
asfgit merged pull request #2099: URL: https://github.com/apache/hadoop/pull/2099 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to

[GitHub] [hadoop] umamaheswararao commented on pull request #2092: HDFS-15429. mkdirs should work when parent dir is an internalDir and fallback configured.

2020-06-24 Thread GitBox
umamaheswararao commented on pull request #2092: URL: https://github.com/apache/hadoop/pull/2092#issuecomment-649224805 For some reason Yetus is failing at last and not posting results. Here is the link: https://builds.apache.org/job/hadoop-multibranch/job/PR-2092/2/console No test

[jira] [Resolved] (HADOOP-17089) WASB: Update azure-storage-java SDK

2020-06-24 Thread Thomas Marqardt (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-17089?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Marqardt resolved HADOOP-17089. -- Fix Version/s: 3.3.1 Release Note: Azure WASB bug fix that can cause list

[GitHub] [hadoop] ThomasMarquardt commented on pull request #2099: HADOOP-17089: WASB: Update azure-storage-java SDK

2020-06-24 Thread GitBox
ThomasMarquardt commented on pull request #2099: URL: https://github.com/apache/hadoop/pull/2099#issuecomment-649235262 The javadoc issue is tracked by HADOOP-17085. This is an automated message from the Apache Git