[GitHub] [hadoop] hadoop-yetus commented on pull request #2564: YARN-10538: Add RECOMMISSIONING nodes to the list of updated nodes returned to the AM

2021-01-05 Thread GitBox
hadoop-yetus commented on pull request #2564: URL: https://github.com/apache/hadoop/pull/2564#issuecomment-755142839 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0

[jira] [Work logged] (HADOOP-17191) ABFS: Run the integration tests with various combinations of configurations and publish a consolidated results

2021-01-05 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-17191?focusedWorklogId=531732=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-531732 ] ASF GitHub Bot logged work on HADOOP-17191: --- Author: ASF GitHub Bot

[GitHub] [hadoop] sumangala-patki closed pull request #2596: HADOOP-17191. ABFS: Run the tests with various combinations of configurations and publish a consolidated results

2021-01-05 Thread GitBox
sumangala-patki closed pull request #2596: URL: https://github.com/apache/hadoop/pull/2596 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above

[jira] [Work logged] (HADOOP-17191) ABFS: Run the integration tests with various combinations of configurations and publish a consolidated results

2021-01-05 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-17191?focusedWorklogId=531727=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-531727 ] ASF GitHub Bot logged work on HADOOP-17191: --- Author: ASF GitHub Bot

[GitHub] [hadoop] sumangala-patki opened a new pull request #2596: HADOOP-17191. ABFS: Run the tests with various combinations of configurations and publish a consolidated results

2021-01-05 Thread GitBox
sumangala-patki opened a new pull request #2596: URL: https://github.com/apache/hadoop/pull/2596 - Contributed by Bilahari T H (cherry picked from commit

[jira] [Updated] (HADOOP-17457) Seeing test ITestS3AInconsistency.testGetFileStatus failure.

2021-01-05 Thread Mukund Thakur (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-17457?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mukund Thakur updated HADOOP-17457: --- Affects Version/s: 3.3.1 > Seeing test ITestS3AInconsistency.testGetFileStatus failure. >

[jira] [Updated] (HADOOP-17457) Seeing test ITestS3AInconsistency.testGetFileStatus failure.

2021-01-05 Thread Mukund Thakur (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-17457?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mukund Thakur updated HADOOP-17457: --- Component/s: test fs/s3 > Seeing test

[jira] [Created] (HADOOP-17457) Seeing test ITestS3AInconsistency.testGetFileStatus failure.

2021-01-05 Thread Mukund Thakur (Jira)
Mukund Thakur created HADOOP-17457: -- Summary: Seeing test ITestS3AInconsistency.testGetFileStatus failure. Key: HADOOP-17457 URL: https://issues.apache.org/jira/browse/HADOOP-17457 Project: Hadoop

[jira] [Updated] (HADOOP-17457) Seeing test ITestS3AInconsistency.testGetFileStatus failure.

2021-01-05 Thread Mukund Thakur (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-17457?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mukund Thakur updated HADOOP-17457: --- Description: [*ERROR*] *Tests* *run: 3*, *Failures: 1*, Errors: 0, Skipped: 0, Time

[GitHub] [hadoop] nilotpalnandi opened a new pull request #2595: HDFS-15619 Add snapshot related metrics

2021-01-05 Thread GitBox
nilotpalnandi opened a new pull request #2595: URL: https://github.com/apache/hadoop/pull/2595 https://issues.apache.org/jira/browse/HDFS-15619 Metrics for - 1. Number of snapshot GC runs 2. Number of empty snapshot GC runs 3. Number of successful snapshot GC runs 4.

[jira] [Work logged] (HADOOP-17454) [s3a] Disable bucket existence check - set fs.s3a.bucket.probe to 0

2021-01-05 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-17454?focusedWorklogId=531717=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-531717 ] ASF GitHub Bot logged work on HADOOP-17454: --- Author: ASF GitHub Bot

[GitHub] [hadoop] mukund-thakur commented on pull request #2593: HADOOP-17454. [s3a] Disable bucket existence check - set fs.s3a.bucket.probe to 0

2021-01-05 Thread GitBox
mukund-thakur commented on pull request #2593: URL: https://github.com/apache/hadoop/pull/2593#issuecomment-755115007 We changed the default behaviour of bucket probe from 2 to 0 which was present from long time. Won't this cause problem for the downstream components which used to expect

[GitHub] [hadoop] aajisaka commented on pull request #2592: YARN-10560. Upgrade node.js to 10.23.1 and yarn to 1.22.5 in Web UI v2.

2021-01-05 Thread GitBox
aajisaka commented on pull request #2592: URL: https://github.com/apache/hadoop/pull/2592#issuecomment-755048566 Thank you @iwasakims This is an automated message from the Apache Git Service. To respond to the message,

[GitHub] [hadoop] aajisaka merged pull request #2592: YARN-10560. Upgrade node.js to 10.23.1 and yarn to 1.22.5 in Web UI v2.

2021-01-05 Thread GitBox
aajisaka merged pull request #2592: URL: https://github.com/apache/hadoop/pull/2592 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to

[GitHub] [hadoop] hadoop-yetus commented on pull request #2581: YARN-10553. Refactor TestDistributedShell

2021-01-05 Thread GitBox
hadoop-yetus commented on pull request #2581: URL: https://github.com/apache/hadoop/pull/2581#issuecomment-755045715 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0

[jira] [Work logged] (HADOOP-17451) intermittent failure of S3A huge file upload tests: count of bytes uploaded == 0

2021-01-05 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-17451?focusedWorklogId=531582=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-531582 ] ASF GitHub Bot logged work on HADOOP-17451: --- Author: ASF GitHub Bot

[GitHub] [hadoop] hadoop-yetus commented on pull request #2594: HADOOP-17451. IOStatistics test failures in S3A code.

2021-01-05 Thread GitBox
hadoop-yetus commented on pull request #2594: URL: https://github.com/apache/hadoop/pull/2594#issuecomment-754968448 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0

[jira] [Assigned] (HADOOP-17430) Restore ability to set Text to empty byte array

2021-01-05 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-17430?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran reassigned HADOOP-17430: --- Assignee: gaozhan ding > Restore ability to set Text to empty byte array >

[jira] [Resolved] (HADOOP-17430) Restore ability to set Text to empty byte array

2021-01-05 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-17430?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran resolved HADOOP-17430. - Fix Version/s: 3.3.1 Resolution: Fixed > Restore ability to set Text to empty

[jira] [Updated] (HADOOP-17430) Restore ability to set Text to empty byte array

2021-01-05 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-17430?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran updated HADOOP-17430: Summary: Restore ability to set Text to empty byte array (was: There is no way to

[jira] [Work logged] (HADOOP-17430) There is no way to clear Text bytes now

2021-01-05 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-17430?focusedWorklogId=531494=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-531494 ] ASF GitHub Bot logged work on HADOOP-17430: --- Author: ASF GitHub Bot

[GitHub] [hadoop] steveloughran merged pull request #2545: HADOOP-17430. Add clear bytes logic for hadoop Text

2021-01-05 Thread GitBox
steveloughran merged pull request #2545: URL: https://github.com/apache/hadoop/pull/2545 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to

[jira] [Work logged] (HADOOP-17430) There is no way to clear Text bytes now

2021-01-05 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-17430?focusedWorklogId=531491=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-531491 ] ASF GitHub Bot logged work on HADOOP-17430: --- Author: ASF GitHub Bot

[GitHub] [hadoop] steveloughran commented on pull request #2545: HADOOP-17430. Add clear bytes logic for hadoop Text

2021-01-05 Thread GitBox
steveloughran commented on pull request #2545: URL: https://github.com/apache/hadoop/pull/2545#issuecomment-754899716 Javadocs look good, and test failures are the OOM problems we've started having. +1, merging to trunk and backporting to 3.3. Thanks!

[jira] [Work logged] (HADOOP-17451) intermittent failure of S3A huge file upload tests: count of bytes uploaded == 0

2021-01-05 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-17451?focusedWorklogId=531486=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-531486 ] ASF GitHub Bot logged work on HADOOP-17451: --- Author: ASF GitHub Bot

[jira] [Updated] (HADOOP-17451) intermittent failure of S3A huge file upload tests: count of bytes uploaded == 0

2021-01-05 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-17451?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated HADOOP-17451: Labels: pull-request-available (was: ) > intermittent failure of S3A huge file upload

[GitHub] [hadoop] steveloughran opened a new pull request #2594: HADOOP-17451. IOStatistics test failures in S3A code.

2021-01-05 Thread GitBox
steveloughran opened a new pull request #2594: URL: https://github.com/apache/hadoop/pull/2594 Fixing tests which fail intermittently based on configs and in the case of the HugeFile tests, only in bulk runs when existing FS instances meant statistic probes sometimes ended up

[jira] [Commented] (HADOOP-17408) Optimize NetworkTopology while sorting of block locations

2021-01-05 Thread Jim Brennan (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-17408?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17259194#comment-17259194 ] Jim Brennan commented on HADOOP-17408: -- [~ahussein] thanks for the PR!  Can you please separate

[jira] [Updated] (HADOOP-17455) [s3a] Intermittent failure of ITestS3ADeleteCost.testDeleteSingleFileInDir

2021-01-05 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-17455?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran updated HADOOP-17455: Parent Issue: HADOOP-16830 (was: HADOOP-16829) > [s3a] Intermittent failure of

[jira] [Commented] (HADOOP-17455) [s3a] Intermittent failure of ITestS3ADeleteCost.testDeleteSingleFileInDir

2021-01-05 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-17455?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17259198#comment-17259198 ] Steve Loughran commented on HADOOP-17455: - Happening to me in a test run (not IDE tho'), where

[jira] [Work logged] (HADOOP-17404) ABFS: Piggyback flush on Append calls for short writes

2021-01-05 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-17404?focusedWorklogId=531468=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-531468 ] ASF GitHub Bot logged work on HADOOP-17404: --- Author: ASF GitHub Bot

[GitHub] [hadoop] DadanielZ commented on a change in pull request #2509: HADOOP-17404. ABFS: Small write - Merge append and flush

2021-01-05 Thread GitBox
DadanielZ commented on a change in pull request #2509: URL: https://github.com/apache/hadoop/pull/2509#discussion_r552173130 ## File path: hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azurebfs/constants/ConfigurationKeys.java ## @@ -55,6 +55,7 @@ public

[jira] [Commented] (HADOOP-17451) intermittent failure of S3A huge file upload tests: count of bytes uploaded == 0

2021-01-05 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-17451?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17259121#comment-17259121 ] Steve Loughran commented on HADOOP-17451: - I see the problem. The StorageStatistics stuff is

[jira] [Work logged] (HADOOP-17430) There is no way to clear Text bytes now

2021-01-05 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-17430?focusedWorklogId=531383=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-531383 ] ASF GitHub Bot logged work on HADOOP-17430: --- Author: ASF GitHub Bot

[GitHub] [hadoop] hadoop-yetus commented on pull request #2545: HADOOP-17430. Add clear bytes logic for hadoop Text

2021-01-05 Thread GitBox
hadoop-yetus commented on pull request #2545: URL: https://github.com/apache/hadoop/pull/2545#issuecomment-754795541 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0

[jira] [Work logged] (HADOOP-16080) hadoop-aws does not work with hadoop-client-api

2021-01-05 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-16080?focusedWorklogId=531363=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-531363 ] ASF GitHub Bot logged work on HADOOP-16080: --- Author: ASF GitHub Bot

[GitHub] [hadoop] sunchao commented on pull request #2575: HADOOP-16080. hadoop-aws does not work with hadoop-client-api

2021-01-05 Thread GitBox
sunchao commented on pull request #2575: URL: https://github.com/apache/hadoop/pull/2575#issuecomment-754776995 @steveloughran eh I only tested this in Spark (verified that the failure [here](https://github.com/apache/spark/pull/29843#issuecomment-733932857) was fixed while was

[jira] [Commented] (HADOOP-17451) intermittent failure of S3A huge file upload tests: count of bytes uploaded == 0

2021-01-05 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-17451?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17259055#comment-17259055 ] Steve Loughran commented on HADOOP-17451: - put byte count == 0 {code} [ERROR]

[jira] [Work logged] (HADOOP-16080) hadoop-aws does not work with hadoop-client-api

2021-01-05 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-16080?focusedWorklogId=531346=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-531346 ] ASF GitHub Bot logged work on HADOOP-16080: --- Author: ASF GitHub Bot

[GitHub] [hadoop] steveloughran commented on pull request #2575: HADOOP-16080. hadoop-aws does not work with hadoop-client-api

2021-01-05 Thread GitBox
steveloughran commented on pull request #2575: URL: https://github.com/apache/hadoop/pull/2575#issuecomment-754765770 Usual due diligence query: which s3 endpoint did you run the integration tests against? (I'll expect some tests failures there from HADOOP-16380 stabilisation; if

[jira] [Updated] (HADOOP-17456) S3A ITestPartialRenamesDeletes.testPartialDirDelete[bulk-delete=true] failure

2021-01-05 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-17456?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran updated HADOOP-17456: Parent Issue: HADOOP-16830 (was: HADOOP-16829) > S3A

[jira] [Updated] (HADOOP-17454) [s3a] Disable bucket existence check - set fs.s3a.bucket.probe to 0

2021-01-05 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-17454?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran updated HADOOP-17454: Release Note: S3A bucket existence check is disabled (fs.s3a.bucket.probe is 0), so

[jira] [Work logged] (HADOOP-17414) Magic committer files don't have the count of bytes written collected by spark

2021-01-05 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-17414?focusedWorklogId=531291=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-531291 ] ASF GitHub Bot logged work on HADOOP-17414: --- Author: ASF GitHub Bot

[GitHub] [hadoop] steveloughran commented on pull request #2530: HADOOP-17414. Magic committer files don't have the count of bytes written collected by spark

2021-01-05 Thread GitBox
steveloughran commented on pull request #2530: URL: https://github.com/apache/hadoop/pull/2530#issuecomment-754696434 Rebased, full test run. Failures of HADOOP-17403 and HADOOP-17451; both unrelated This is an automated

[jira] [Created] (HADOOP-17456) S3A ITestPartialRenamesDeletes.testPartialDirDelete[bulk-delete=true] failure

2021-01-05 Thread Steve Loughran (Jira)
Steve Loughran created HADOOP-17456: --- Summary: S3A ITestPartialRenamesDeletes.testPartialDirDelete[bulk-delete=true] failure Key: HADOOP-17456 URL: https://issues.apache.org/jira/browse/HADOOP-17456

[jira] [Updated] (HADOOP-17403) S3A ITestPartialRenamesDeletes.testRenameDirFailsInDelete failure: missing directory marker

2021-01-05 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-17403?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran updated HADOOP-17403: Summary: S3A ITestPartialRenamesDeletes.testRenameDirFailsInDelete failure: missing

[jira] [Resolved] (HADOOP-17403) S3A ITestPartialRenamesDeletes failure: missing directory marker

2021-01-05 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-17403?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran resolved HADOOP-17403. - Fix Version/s: 3.3.1 Assignee: Steve Loughran Resolution: Workaround >

[jira] [Commented] (HADOOP-17403) S3A ITestPartialRenamesDeletes failure: missing directory marker

2021-01-05 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-17403?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17258986#comment-17258986 ] Steve Loughran commented on HADOOP-17403: - seen during some backporting and made go away. I

[jira] [Commented] (HADOOP-17455) [s3a] Intermittent failure of ITestS3ADeleteCost.testDeleteSingleFileInDir

2021-01-05 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-17455?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17258982#comment-17258982 ] Steve Loughran commented on HADOOP-17455: - object_delete_objects expected:<2> but was:<3>

[jira] [Updated] (HADOOP-17455) [s3a] Intermittent failure of ITestS3ADeleteCost.testDeleteSingleFileInDir

2021-01-05 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-17455?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran updated HADOOP-17455: Parent: HADOOP-16829 Issue Type: Sub-task (was: Improvement) > [s3a]

[jira] [Resolved] (HADOOP-16995) ITestS3AConfiguration proxy tests fail when bucket probes == 0

2021-01-05 Thread Gabor Bota (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-16995?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gabor Bota resolved HADOOP-16995. - Resolution: Fixed got +1 from Steve on the PR, committed to trunk > ITestS3AConfiguration

[jira] [Resolved] (HADOOP-17454) [s3a] Disable bucket existence check - set fs.s3a.bucket.probe to 0

2021-01-05 Thread Gabor Bota (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-17454?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gabor Bota resolved HADOOP-17454. - Resolution: Fixed got +1 from Steve on PR, committed to trunk > [s3a] Disable bucket

[jira] [Work logged] (HADOOP-17454) [s3a] Disable bucket existence check - set fs.s3a.bucket.probe to 0

2021-01-05 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-17454?focusedWorklogId=531268=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-531268 ] ASF GitHub Bot logged work on HADOOP-17454: --- Author: ASF GitHub Bot

[GitHub] [hadoop] bgaborg merged pull request #2593: HADOOP-17454. [s3a] Disable bucket existence check - set fs.s3a.bucket.probe to 0

2021-01-05 Thread GitBox
bgaborg merged pull request #2593: URL: https://github.com/apache/hadoop/pull/2593 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to

[jira] [Work logged] (HADOOP-17454) [s3a] Disable bucket existence check - set fs.s3a.bucket.probe to 0

2021-01-05 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-17454?focusedWorklogId=531264=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-531264 ] ASF GitHub Bot logged work on HADOOP-17454: --- Author: ASF GitHub Bot

[GitHub] [hadoop] bgaborg commented on pull request #2593: HADOOP-17454. [s3a] Disable bucket existence check - set fs.s3a.bucket.probe to 0

2021-01-05 Thread GitBox
bgaborg commented on pull request #2593: URL: https://github.com/apache/hadoop/pull/2593#issuecomment-754676479 thanks Steve. created https://issues.apache.org/jira/browse/HADOOP-17455 please check the release notes for this issue on jira- I think I filled it the right way

[jira] [Created] (HADOOP-17455) [s3a] Intermittent failure of ITestS3ADeleteCost.testDeleteSingleFileInDir

2021-01-05 Thread Gabor Bota (Jira)
Gabor Bota created HADOOP-17455: --- Summary: [s3a] Intermittent failure of ITestS3ADeleteCost.testDeleteSingleFileInDir Key: HADOOP-17455 URL: https://issues.apache.org/jira/browse/HADOOP-17455 Project:

[jira] [Work logged] (HADOOP-17454) [s3a] Disable bucket existence check - set fs.s3a.bucket.probe to 0

2021-01-05 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-17454?focusedWorklogId=531207=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-531207 ] ASF GitHub Bot logged work on HADOOP-17454: --- Author: ASF GitHub Bot

[GitHub] [hadoop] hadoop-yetus commented on pull request #2593: HADOOP-17454. [s3a] Disable bucket existence check - set fs.s3a.bucket.probe to 0

2021-01-05 Thread GitBox
hadoop-yetus commented on pull request #2593: URL: https://github.com/apache/hadoop/pull/2593#issuecomment-754636231 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0

[jira] [Work logged] (HADOOP-17338) Intermittent S3AInputStream failures: Premature end of Content-Length delimited message body etc

2021-01-05 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-17338?focusedWorklogId=531168=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-531168 ] ASF GitHub Bot logged work on HADOOP-17338: --- Author: ASF GitHub Bot

[GitHub] [hadoop] steveloughran commented on pull request #2497: HADOOP-17338. Intermittent S3AInputStream failures: Premature end of …

2021-01-05 Thread GitBox
steveloughran commented on pull request #2497: URL: https://github.com/apache/hadoop/pull/2497#issuecomment-754614490 yes, use unbuffer if you can. It frees up the HTTPS connection. And while AWS S3 won't have problems, its probably good for other S3 stores as it will reduce the #of open

[jira] [Work logged] (HADOOP-16202) Stabilize openFile() and adopt internally

2021-01-05 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-16202?focusedWorklogId=531165=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-531165 ] ASF GitHub Bot logged work on HADOOP-16202: --- Author: ASF GitHub Bot

[GitHub] [hadoop] steveloughran commented on pull request #2584: HADOOP-16202. Enhance openFile()

2021-01-05 Thread GitBox
steveloughran commented on pull request #2584: URL: https://github.com/apache/hadoop/pull/2584#issuecomment-754613375 MR client not compiling; not seeing useful information from yetus. This is an automated message from the

[jira] [Work logged] (HADOOP-17454) [s3a] Disable bucket existence check - set fs.s3a.bucket.probe to 0

2021-01-05 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-17454?focusedWorklogId=531158=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-531158 ] ASF GitHub Bot logged work on HADOOP-17454: --- Author: ASF GitHub Bot

[GitHub] [hadoop] steveloughran edited a comment on pull request #2593: HADOOP-17454. [s3a] Disable bucket existence check - set fs.s3a.bucket.probe to 0

2021-01-05 Thread GitBox
steveloughran edited a comment on pull request #2593: URL: https://github.com/apache/hadoop/pull/2593#issuecomment-754605704 +1 pending successful Yetus build. I've turned off existence checks for a long time and makes things faster, especially creation of a new FS instance in test

[jira] [Work logged] (HADOOP-17454) [s3a] Disable bucket existence check - set fs.s3a.bucket.probe to 0

2021-01-05 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-17454?focusedWorklogId=531157=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-531157 ] ASF GitHub Bot logged work on HADOOP-17454: --- Author: ASF GitHub Bot

[GitHub] [hadoop] steveloughran commented on pull request #2593: HADOOP-17454. [s3a] Disable bucket existence check - set fs.s3a.bucket.probe to 0

2021-01-05 Thread GitBox
steveloughran commented on pull request #2593: URL: https://github.com/apache/hadoop/pull/2593#issuecomment-754605704 +1. Do update the release notes though, to say "you'll get told of missing/unreadable bucket on first operation against it". Test failure is going to be related to

[jira] [Commented] (HADOOP-16492) Support HuaweiCloud Object Storage as a Hadoop Backend File System

2021-01-05 Thread Junping Du (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-16492?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17258866#comment-17258866 ] Junping Du commented on HADOOP-16492: - Thanks for comments, [~ste...@apache.org]. The refactoring

[jira] [Work logged] (HADOOP-17454) [s3a] Disable bucket existence check - set fs.s3a.bucket.probe to 0

2021-01-05 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-17454?focusedWorklogId=531146=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-531146 ] ASF GitHub Bot logged work on HADOOP-17454: --- Author: ASF GitHub Bot

[jira] [Work logged] (HADOOP-17454) [s3a] Disable bucket existence check - set fs.s3a.bucket.probe to 0

2021-01-05 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-17454?focusedWorklogId=531149=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-531149 ] ASF GitHub Bot logged work on HADOOP-17454: --- Author: ASF GitHub Bot

[jira] [Work logged] (HADOOP-17454) [s3a] Disable bucket existence check - set fs.s3a.bucket.probe to 0

2021-01-05 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-17454?focusedWorklogId=531148=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-531148 ] ASF GitHub Bot logged work on HADOOP-17454: --- Author: ASF GitHub Bot

[GitHub] [hadoop] bgaborg edited a comment on pull request #2593: HADOOP-17454. [s3a] Disable bucket existence check - set fs.s3a.bucket.probe to 0

2021-01-05 Thread GitBox
bgaborg edited a comment on pull request #2593: URL: https://github.com/apache/hadoop/pull/2593#issuecomment-754598161 tested against ireland. Testing: I had one failing test during the CLI run, but it was ok after a rerun: ``` [ERROR]

[GitHub] [hadoop] bgaborg commented on pull request #2593: HADOOP-17454. [s3a] Disable bucket existence check - set fs.s3a.bucket.probe to 0

2021-01-05 Thread GitBox
bgaborg commented on pull request #2593: URL: https://github.com/apache/hadoop/pull/2593#issuecomment-754598161 Testing: I had one failing test during the CLI run, but it was ok after a rerun: ``` [ERROR]

[jira] [Commented] (HADOOP-16995) ITestS3AConfiguration proxy tests fail when bucket probes == 0

2021-01-05 Thread Gabor Bota (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-16995?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17258861#comment-17258861 ] Gabor Bota commented on HADOOP-16995: - https://github.com/apache/hadoop/pull/2593 >

[jira] [Commented] (HADOOP-17454) [s3a] Disable bucket existence check - set fs.s3a.bucket.probe to 0

2021-01-05 Thread Gabor Bota (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-17454?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17258860#comment-17258860 ] Gabor Bota commented on HADOOP-17454: - https://github.com/apache/hadoop/pull/2593 > [s3a] Disable

[jira] [Updated] (HADOOP-17454) [s3a] Disable bucket existence check - set fs.s3a.bucket.probe to 0

2021-01-05 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-17454?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated HADOOP-17454: Labels: pull-request-available (was: ) > [s3a] Disable bucket existence check - set

[GitHub] [hadoop] bgaborg opened a new pull request #2593: HADOOP-17454. [s3a] Disable bucket existence check - set fs.s3a.bucket.probe to 0

2021-01-05 Thread GitBox
bgaborg opened a new pull request #2593: URL: https://github.com/apache/hadoop/pull/2593 Also fixes HADOOP-16995. ITestS3AConfiguration proxy tests failures when bucket probes == 0 The improvement should include the fix, ebcause the test would fail by default otherwise.

[GitHub] [hadoop] hadoop-yetus commented on pull request #2564: YARN-10538: Add RECOMMISSIONING nodes to the list of updated nodes returned to the AM

2021-01-05 Thread GitBox
hadoop-yetus commented on pull request #2564: URL: https://github.com/apache/hadoop/pull/2564#issuecomment-754595737 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0

[jira] [Updated] (HADOOP-16995) ITestS3AConfiguration proxy tests fail when bucket probes == 0

2021-01-05 Thread Gabor Bota (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-16995?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gabor Bota updated HADOOP-16995: Priority: Major (was: Minor) > ITestS3AConfiguration proxy tests fail when bucket probes == 0 >

[jira] [Work started] (HADOOP-17454) [s3a] Disable bucket existence check - set fs.s3a.bucket.probe to 0

2021-01-05 Thread Gabor Bota (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-17454?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Work on HADOOP-17454 started by Gabor Bota. --- > [s3a] Disable bucket existence check - set fs.s3a.bucket.probe to 0 >

[jira] [Updated] (HADOOP-17454) [s3a] Disable bucket existence check - set fs.s3a.bucket.probe to 0

2021-01-05 Thread Gabor Bota (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-17454?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gabor Bota updated HADOOP-17454: Description: Set the value of fs.s3a.bucket.probe to 0 by default. Bucket existence checks are

[jira] [Work started] (HADOOP-16995) ITestS3AConfiguration proxy tests fail when bucket probes == 0

2021-01-05 Thread Gabor Bota (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-16995?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Work on HADOOP-16995 started by Gabor Bota. --- > ITestS3AConfiguration proxy tests fail when bucket probes == 0 >

[GitHub] [hadoop] hadoop-yetus commented on pull request #2592: YARN-10560. Upgrade node.js to 10.23.1 and yarn to 1.22.5 in Web UI v2.

2021-01-05 Thread GitBox
hadoop-yetus commented on pull request #2592: URL: https://github.com/apache/hadoop/pull/2592#issuecomment-754564555 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0

[jira] [Assigned] (HADOOP-16995) ITestS3AConfiguration proxy tests fail when bucket probes == 0

2021-01-05 Thread Gabor Bota (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-16995?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gabor Bota reassigned HADOOP-16995: --- Assignee: Gabor Bota (was: Mukund Thakur) > ITestS3AConfiguration proxy tests fail when

[jira] [Updated] (HADOOP-17454) [s3a] Disable bucket existence check - set fs.s3a.bucket.probe to 0

2021-01-05 Thread Gabor Bota (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-17454?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gabor Bota updated HADOOP-17454: Release Note: S3A bucket existence check is disabled (fs.s3a.bucket.probe is 0), so there will

[jira] [Commented] (HADOOP-17454) [s3a] Disable bucket existence check - set fs.s3a.bucket.probe to 0

2021-01-05 Thread Gabor Bota (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-17454?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17258817#comment-17258817 ] Gabor Bota commented on HADOOP-17454: - cc [~ste...@apache.org] > [s3a] Disable bucket existence

[jira] [Updated] (HADOOP-17454) [s3a] Disable bucket existence check - set fs.s3a.bucket.probe to 0

2021-01-05 Thread Gabor Bota (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-17454?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gabor Bota updated HADOOP-17454: Affects Version/s: 3.3.0 > [s3a] Disable bucket existence check - set fs.s3a.bucket.probe to 0 >

[jira] [Created] (HADOOP-17454) [s3a] Disable bucket existence check - set fs.s3a.bucket.probe to 0

2021-01-05 Thread Gabor Bota (Jira)
Gabor Bota created HADOOP-17454: --- Summary: [s3a] Disable bucket existence check - set fs.s3a.bucket.probe to 0 Key: HADOOP-17454 URL: https://issues.apache.org/jira/browse/HADOOP-17454 Project: Hadoop

[GitHub] [hadoop] hadoop-yetus commented on pull request #2591: YARN-10561. Upgrade node.js to 10.23.1 and yarn to 1.22.5 in YARN application catalog webapp

2021-01-05 Thread GitBox
hadoop-yetus commented on pull request #2591: URL: https://github.com/apache/hadoop/pull/2591#issuecomment-754553184 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0

[jira] [Updated] (HADOOP-17452) Upgrade guice to 4.2.3

2021-01-05 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-17452?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang updated HADOOP-17452: - Description: Upgrade guice to 4.2.3 to fix compatibility issue: {noformat} Exception in thread

[jira] [Updated] (HADOOP-17452) Upgrade guice to 4.2.3

2021-01-05 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-17452?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang updated HADOOP-17452: - Summary: Upgrade guice to 4.2.3 (was: Upgrade guice to 4.1.0) > Upgrade guice to 4.2.3 >

[GitHub] [hadoop] hadoop-yetus commented on pull request #2592: YARN-10560. Upgrade node.js to 10.23.1 and yarn to 1.22.5 in Web UI v2.

2021-01-05 Thread GitBox
hadoop-yetus commented on pull request #2592: URL: https://github.com/apache/hadoop/pull/2592#issuecomment-754511076 (!) A patch to the testing environment has been detected. Re-executing against the patched versions to perform further tests. The console is at

[GitHub] [hadoop] aajisaka opened a new pull request #2592: YARN-10560. Upgrade node.js to 10.23.1 and yarn to 1.22.5 in Web UI v2.

2021-01-05 Thread GitBox
aajisaka opened a new pull request #2592: URL: https://github.com/apache/hadoop/pull/2592 JIRA: https://issues.apache.org/jira/browse/YARN-10560 This is an automated message from the Apache Git Service. To respond to the

[GitHub] [hadoop] aajisaka opened a new pull request #2591: YARN-10561. Upgrade node.js to 10.23.1 in YARN application catalog webapp

2021-01-05 Thread GitBox
aajisaka opened a new pull request #2591: URL: https://github.com/apache/hadoop/pull/2591 JIRA: https://issues.apache.org/jira/browse/YARN-10561 Upgrade node.js and yarn to the latest versions. This is an automated