[jira] [Work logged] (HADOOP-18065) ExecutorHelper.logThrowableFromAfterExecute() is too noisy.

2022-01-05 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18065?focusedWorklogId=704414=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-704414 ] ASF GitHub Bot logged work on HADOOP-18065: --- Author: ASF GitHub Bot

[GitHub] [hadoop] mukund-thakur commented on pull request #3860: HADOOP-18065 ExecutorHelper.logThrowableFromAfterExecute() is too noisy.

2022-01-05 Thread GitBox
mukund-thakur commented on pull request #3860: URL: https://github.com/apache/hadoop/pull/3860#issuecomment-1006294876 Yetus failure is just because no new tests are added. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub

[jira] [Work logged] (HADOOP-18065) ExecutorHelper.logThrowableFromAfterExecute() is too noisy.

2022-01-05 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18065?focusedWorklogId=704413=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-704413 ] ASF GitHub Bot logged work on HADOOP-18065: --- Author: ASF GitHub Bot

[GitHub] [hadoop] mukund-thakur merged pull request #3860: HADOOP-18065 ExecutorHelper.logThrowableFromAfterExecute() is too noisy.

2022-01-05 Thread GitBox
mukund-thakur merged pull request #3860: URL: https://github.com/apache/hadoop/pull/3860 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

[GitHub] [hadoop] hadoop-yetus commented on pull request #3155: HDFS-16095. Add lsQuotaList command and getQuotaListing api for hdfs …

2022-01-05 Thread GitBox
hadoop-yetus commented on pull request #3155: URL: https://github.com/apache/hadoop/pull/3155#issuecomment-1006289619 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0

[jira] [Created] (HADOOP-18071) ABFS: Set driver global timeout for ITestAzureBlobFileSystemBasics

2022-01-05 Thread Sumangala Patki (Jira)
Sumangala Patki created HADOOP-18071: Summary: ABFS: Set driver global timeout for ITestAzureBlobFileSystemBasics Key: HADOOP-18071 URL: https://issues.apache.org/jira/browse/HADOOP-18071

[jira] [Created] (HADOOP-18070) libhdfspp authentication failed

2022-01-05 Thread wondertx (Jira)
wondertx created HADOOP-18070: - Summary: libhdfspp authentication failed Key: HADOOP-18070 URL: https://issues.apache.org/jira/browse/HADOOP-18070 Project: Hadoop Common Issue Type: Bug

[jira] (HADOOP-17971) Exclude IBM Java security classes from being shaded/relocated

2022-01-05 Thread Michail Giannakopoulos (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-17971 ] Michail Giannakopoulos deleted comment on HADOOP-17971: - was (Author: miccagiann): Can we backport this fix to hadoop-client 2.10.0 stream. I have the same issue when trying to

[jira] [Commented] (HADOOP-17971) Exclude IBM Java security classes from being shaded/relocated

2022-01-05 Thread Michail Giannakopoulos (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-17971?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17469651#comment-17469651 ] Michail Giannakopoulos commented on HADOOP-17971: - Can we backport this fix to

[GitHub] [hadoop] cndaimin commented on pull request #3842: HDFS-16403. Improve FUSE IO performance by supporting FUSE parameter max_background

2022-01-05 Thread GitBox
cndaimin commented on pull request #3842: URL: https://github.com/apache/hadoop/pull/3842#issuecomment-1006249942 @sodonnel I have removed the unrelated code, could you please take a look again? Thanks. -- This is an automated message from the Apache Git Service. To respond to the

[GitHub] [hadoop] hadoop-yetus commented on pull request #3861: HDFS-16316.Improve DirectoryScanner: add regular file check related block.

2022-01-05 Thread GitBox
hadoop-yetus commented on pull request #3861: URL: https://github.com/apache/hadoop/pull/3861#issuecomment-1006241356 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0

[GitHub] [hadoop] hadoop-yetus commented on pull request #3859: YARN-10632. Make auto queue creation maximum allowed depth configurable

2022-01-05 Thread GitBox
hadoop-yetus commented on pull request #3859: URL: https://github.com/apache/hadoop/pull/3859#issuecomment-1006237897 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0

[GitHub] [hadoop] tomscut commented on a change in pull request #3827: HDFS-16396. Reconfig slow peer parameters for datanode

2022-01-05 Thread GitBox
tomscut commented on a change in pull request #3827: URL: https://github.com/apache/hadoop/pull/3827#discussion_r779258490 ## File path: hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/DataNode.java ## @@ -642,13 +656,76 @@ public String

[GitHub] [hadoop] hadoop-yetus commented on pull request #3863: HDFS-16413. Reconfig dfs usage parameters for datanode

2022-01-05 Thread GitBox
hadoop-yetus commented on pull request #3863: URL: https://github.com/apache/hadoop/pull/3863#issuecomment-1006223517 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0

[GitHub] [hadoop] tomscut commented on pull request #3753: HDFS-16371. Exclude slow disks when choosing volume

2022-01-05 Thread GitBox
tomscut commented on pull request #3753: URL: https://github.com/apache/hadoop/pull/3753#issuecomment-1006187746 Thanks @tasanuma for your review and merging this. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the

[GitHub] [hadoop] tasanuma commented on pull request #3753: HDFS-16371. Exclude slow disks when choosing volume

2022-01-05 Thread GitBox
tasanuma commented on pull request #3753: URL: https://github.com/apache/hadoop/pull/3753#issuecomment-1006186602 Merged it. Thanks for your contribution, @tomscut! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the

[GitHub] [hadoop] tasanuma merged pull request #3753: HDFS-16371. Exclude slow disks when choosing volume

2022-01-05 Thread GitBox
tasanuma merged pull request #3753: URL: https://github.com/apache/hadoop/pull/3753 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

[GitHub] [hadoop] hadoop-yetus commented on pull request #3753: HDFS-16371. Exclude slow disks when choosing volume

2022-01-05 Thread GitBox
hadoop-yetus commented on pull request #3753: URL: https://github.com/apache/hadoop/pull/3753#issuecomment-1006166618 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0

[jira] [Commented] (HADOOP-18026) Fix default value of Magic committer

2022-01-05 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18026?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17469596#comment-17469596 ] Steve Loughran commented on HADOOP-18026: - done. > Fix default value of Magic committer >

[jira] [Assigned] (HADOOP-18026) Fix default value of Magic committer

2022-01-05 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18026?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran reassigned HADOOP-18026: --- Assignee: guophilipse > Fix default value of Magic committer >

[GitHub] [hadoop] hadoop-yetus commented on pull request #3862: HDFS-11107.TestStartup#testStorageBlockContentsStaleAfterNNRestart fails intermittently.

2022-01-05 Thread GitBox
hadoop-yetus commented on pull request #3862: URL: https://github.com/apache/hadoop/pull/3862#issuecomment-1006055417 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0

[jira] [Commented] (HADOOP-18024) SocketChannel is not closed when IOException happens in Server$Listener.doAccept

2022-01-05 Thread Haoze Wu (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18024?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17469527#comment-17469527 ] Haoze Wu commented on HADOOP-18024: --- [~ayushtkn]  Sorry for the delay. I should have fixed the issue

[jira] [Created] (HADOOP-18069) CVE-2021-0341 in okhttp@2.7.5 detected in hdfs-client

2022-01-05 Thread Eugene Shinn (Truveta) (Jira)
Eugene Shinn (Truveta) created HADOOP-18069: --- Summary: CVE-2021-0341 in okhttp@2.7.5 detected in hdfs-client Key: HADOOP-18069 URL: https://issues.apache.org/jira/browse/HADOOP-18069

[jira] [Work logged] (HADOOP-18068) upgrade AWS SDK to 1.12.x

2022-01-05 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18068?focusedWorklogId=704168=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-704168 ] ASF GitHub Bot logged work on HADOOP-18068: --- Author: ASF GitHub Bot

[jira] [Updated] (HADOOP-18068) upgrade AWS SDK to 1.12.x

2022-01-05 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18068?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated HADOOP-18068: Labels: pull-request-available (was: ) > upgrade AWS SDK to 1.12.x >

[GitHub] [hadoop] steveloughran opened a new pull request #3864: HADOOP-18068. upgrade AWS SDK to 1.12.132

2022-01-05 Thread GitBox
steveloughran opened a new pull request #3864: URL: https://github.com/apache/hadoop/pull/3864 ### Description of PR move to latest AWS SDK ### How was this patch tested? itests without s3guard ``` -Dparallel-tests -DtestsThreadCount=6 -Dmarkers=keep -Dscale

[jira] [Commented] (HADOOP-18024) SocketChannel is not closed when IOException happens in Server$Listener.doAccept

2022-01-05 Thread Ayush Saxena (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18024?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17469486#comment-17469486 ] Ayush Saxena commented on HADOOP-18024: --- The introduced test is failing in the daily build. Was

[jira] [Updated] (HADOOP-18031) Build arm64 (aarch64) and x86_64 image with the same Dockerfile

2022-01-05 Thread Chao Sun (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18031?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chao Sun updated HADOOP-18031: -- Target Version/s: 3.4.0, 3.3.3 (was: 3.4.0, 3.3.2, 3.3.3) > Build arm64 (aarch64) and x86_64 image

[jira] [Updated] (HADOOP-18065) ExecutorHelper.logThrowableFromAfterExecute() is too noisy.

2022-01-05 Thread Chao Sun (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18065?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chao Sun updated HADOOP-18065: -- Target Version/s: 3.3.3 (was: 3.3.2) > ExecutorHelper.logThrowableFromAfterExecute() is too noisy.

[GitHub] [hadoop] hadoop-yetus commented on pull request #3842: HDFS-16403. Improve FUSE IO performance by supporting FUSE parameter max_background

2022-01-05 Thread GitBox
hadoop-yetus commented on pull request #3842: URL: https://github.com/apache/hadoop/pull/3842#issuecomment-1005977842 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0

[jira] [Commented] (HADOOP-18056) DistCp: Filter duplicates in the source paths

2022-01-05 Thread Ayush Saxena (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18056?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17469472#comment-17469472 ] Ayush Saxena commented on HADOOP-18056: --- Committed to trunk and branch-3.3 Thanx Everyone for

[jira] [Resolved] (HADOOP-18056) DistCp: Filter duplicates in the source paths

2022-01-05 Thread Ayush Saxena (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18056?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ayush Saxena resolved HADOOP-18056. --- Fix Version/s: 3.4.0 3.3.3 Hadoop Flags: Reviewed

[jira] [Work logged] (HADOOP-18056) DistCp: Filter duplicates in the source paths

2022-01-05 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18056?focusedWorklogId=704150=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-704150 ] ASF GitHub Bot logged work on HADOOP-18056: --- Author: ASF GitHub Bot

[GitHub] [hadoop] ayushtkn merged pull request #3825: HADOOP-18056. DistCp: Filter duplicates in the source paths.

2022-01-05 Thread GitBox
ayushtkn merged pull request #3825: URL: https://github.com/apache/hadoop/pull/3825 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

[GitHub] [hadoop] sunchao commented on pull request #3854: HDFS-16410. Fixing Insecure Xml parsing in OfflineEditsXmlLoader

2022-01-05 Thread GitBox
sunchao commented on pull request #3854: URL: https://github.com/apache/hadoop/pull/3854#issuecomment-1005952115 @steveloughran sure, I've cherry-picked it to branch-3.3 and branch-3.3.2 -- This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [hadoop] hadoop-yetus commented on pull request #3861: HDFS-16316.Improve DirectoryScanner: add regular file check related block.

2022-01-05 Thread GitBox
hadoop-yetus commented on pull request #3861: URL: https://github.com/apache/hadoop/pull/3861#issuecomment-1005940570 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0

[jira] [Updated] (HADOOP-17077) S3A delegation token binding to support secondary binding list

2022-01-05 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-17077?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran updated HADOOP-17077: Parent: HADOOP-18067 (was: HADOOP-17566) > S3A delegation token binding to support

[jira] [Updated] (HADOOP-13294) Test hadoop fs shell against s3a; fix problems

2022-01-05 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-13294?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran updated HADOOP-13294: Parent: HADOOP-18067 (was: HADOOP-17566) > Test hadoop fs shell against s3a; fix

[jira] [Updated] (HADOOP-15479) s3guard bucket-info command to add a verify-property =

2022-01-05 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-15479?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran updated HADOOP-15479: Parent: HADOOP-18067 (was: HADOOP-17566) > s3guard bucket-info command to add a

[jira] [Updated] (HADOOP-16800) ITestDynamoDBMetadataStore.testTableVersioning failure -DDB deleteItem consistency?

2022-01-05 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-16800?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran updated HADOOP-16800: Parent: HADOOP-18067 (was: HADOOP-17566) >

[jira] [Updated] (HADOOP-14707) AbstractContractDistCpTest to test attr preservation with -p, verify blobstores downgrade

2022-01-05 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-14707?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran updated HADOOP-14707: Parent: HADOOP-18067 (was: HADOOP-17566) > AbstractContractDistCpTest to test attr

[jira] [Commented] (HADOOP-18068) upgrade AWS SDK to 1.12.x

2022-01-05 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18068?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17469441#comment-17469441 ] Steve Loughran commented on HADOOP-18068: -

[jira] [Created] (HADOOP-18068) upgrade AWS SDK to 1.12.x

2022-01-05 Thread Steve Loughran (Jira)
Steve Loughran created HADOOP-18068: --- Summary: upgrade AWS SDK to 1.12.x Key: HADOOP-18068 URL: https://issues.apache.org/jira/browse/HADOOP-18068 Project: Hadoop Common Issue Type:

[jira] [Updated] (HADOOP-15057) s3guard bucket-info command to include default bucket encryption info

2022-01-05 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-15057?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran updated HADOOP-15057: Parent: HADOOP-18067 (was: HADOOP-17566) > s3guard bucket-info command to include

[jira] [Updated] (HADOOP-15870) S3AInputStream.remainingInFile should use nextReadPos

2022-01-05 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-15870?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran updated HADOOP-15870: Parent: HADOOP-18067 (was: HADOOP-17566) > S3AInputStream.remainingInFile should use

[jira] [Updated] (HADOOP-16017) Add some S3A-specific create file options

2022-01-05 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-16017?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran updated HADOOP-16017: Parent: HADOOP-18067 (was: HADOOP-17566) > Add some S3A-specific create file options >

[jira] [Updated] (HADOOP-15503) strip s3.amazonaws.com off hostnames before making s3a calls

2022-01-05 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-15503?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran updated HADOOP-15503: Parent: HADOOP-18067 (was: HADOOP-17566) > strip s3.amazonaws.com off hostnames before

[jira] [Updated] (HADOOP-14132) Filesystem discovery to stop loading implementation classes

2022-01-05 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-14132?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran updated HADOOP-14132: Parent: HADOOP-18067 (was: HADOOP-17566) > Filesystem discovery to stop loading

[jira] [Resolved] (HADOOP-17566) Über-jira: S3A Hadoop 3.3.2 features

2022-01-05 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-17566?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran resolved HADOOP-17566. - Fix Version/s: 3.3.2 Resolution: Done > Über-jira: S3A Hadoop 3.3.2 features >

[jira] [Updated] (HADOOP-14639) test YARN log collection works to s3a

2022-01-05 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-14639?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran updated HADOOP-14639: Parent: HADOOP-18067 (was: HADOOP-17566) > test YARN log collection works to s3a >

[jira] [Updated] (HADOOP-17566) Über-jira: S3A Hadoop 3.3.2 features

2022-01-05 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-17566?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran updated HADOOP-17566: Issue Type: Improvement (was: Bug) > Über-jira: S3A Hadoop 3.3.2 features >

[jira] [Updated] (HADOOP-13293) add a special 0 byte input stream for empty blobs

2022-01-05 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-13293?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran updated HADOOP-13293: Parent: HADOOP-18067 (was: HADOOP-17566) > add a special 0 byte input stream for empty

[jira] [Updated] (HADOOP-13585) shell rm command to not rename to ~/.Trash in object stores

2022-01-05 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-13585?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran updated HADOOP-13585: Parent: HADOOP-18067 (was: HADOOP-17566) > shell rm command to not rename to ~/.Trash

[jira] [Updated] (HADOOP-12020) Support AWS S3 reduced redundancy storage class

2022-01-05 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-12020?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran updated HADOOP-12020: Parent: HADOOP-18067 (was: HADOOP-17566) > Support AWS S3 reduced redundancy storage

[jira] [Updated] (HADOOP-16308) ITestS3AContractSeek teardown closes test FS before superclass can do its cleanup

2022-01-05 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-16308?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran updated HADOOP-16308: Parent: HADOOP-18067 (was: HADOOP-17566) > ITestS3AContractSeek teardown closes test FS

[jira] [Updated] (HADOOP-14776) clean up ITestS3AFileSystemContract

2022-01-05 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-14776?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran updated HADOOP-14776: Parent: HADOOP-18067 (was: HADOOP-17566) > clean up ITestS3AFileSystemContract >

[jira] [Updated] (HADOOP-16633) Tune hadoop-aws parallel test surefire/failsafe settings

2022-01-05 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-16633?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran updated HADOOP-16633: Parent: HADOOP-18067 (was: HADOOP-17566) > Tune hadoop-aws parallel test

[jira] [Updated] (HADOOP-13846) S3A to implement rename(final Path src, final Path dst, final Rename... options)

2022-01-05 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-13846?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran updated HADOOP-13846: Parent: HADOOP-18067 (was: HADOOP-17566) > S3A to implement rename(final Path src,

[jira] [Updated] (HADOOP-15915) Report problems w/ local S3A buffer directory meaningfully

2022-01-05 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-15915?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran updated HADOOP-15915: Parent: HADOOP-18067 (was: HADOOP-17566) > Report problems w/ local S3A buffer

[jira] [Updated] (HADOOP-15347) S3ARetryPolicy to handle AWS 500 responses/error code TooBusyException with the throttle backoff policy

2022-01-05 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-15347?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran updated HADOOP-15347: Parent: HADOOP-18067 (was: HADOOP-17566) > S3ARetryPolicy to handle AWS 500

[jira] [Updated] (HADOOP-15575) Add a way for an FS instance to say "really, no trash interval at all"

2022-01-05 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-15575?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran updated HADOOP-15575: Parent: HADOOP-18067 (was: HADOOP-17566) > Add a way for an FS instance to say "really,

[jira] [Updated] (HADOOP-16538) S3AFilesystem trash handling should respect the current UGI

2022-01-05 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-16538?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran updated HADOOP-16538: Parent: HADOOP-18067 (was: HADOOP-17566) > S3AFilesystem trash handling should respect

[jira] [Updated] (HADOOP-15303) make s3a read fault injection configurable including "off"

2022-01-05 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-15303?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran updated HADOOP-15303: Parent: HADOOP-18067 (was: HADOOP-17566) > make s3a read fault injection configurable

[jira] [Updated] (HADOOP-16574) ITestS3AAWSCredentialsProvider tests fail if a bucket has DTs enabled

2022-01-05 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-16574?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran updated HADOOP-16574: Parent: HADOOP-18067 (was: HADOOP-17566) > ITestS3AAWSCredentialsProvider tests fail if

[jira] [Updated] (HADOOP-15555) S3AFileStatus to add a serialVersionUID; review & test serialization

2022-01-05 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-1?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran updated HADOOP-1: Parent: HADOOP-18067 (was: HADOOP-17566) > S3AFileStatus to add a serialVersionUID;

[jira] [Updated] (HADOOP-15807) ITestS3AContractRootDir failure on non-S3Guarded bucket

2022-01-05 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-15807?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran updated HADOOP-15807: Parent: HADOOP-18067 (was: HADOOP-17566) > ITestS3AContractRootDir failure on

[jira] [Updated] (HADOOP-15000) s3a new getdefaultblocksize be called in getFileStatus which has not been implemented in s3afilesystem yet

2022-01-05 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-15000?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran updated HADOOP-15000: Parent: HADOOP-18067 (was: HADOOP-17566) > s3a new getdefaultblocksize be called in

[jira] [Updated] (HADOOP-14603) S3A input stream to support ByteBufferReadable

2022-01-05 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-14603?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran updated HADOOP-14603: Parent: HADOOP-18067 (was: HADOOP-17566) > S3A input stream to support

[jira] [Updated] (HADOOP-16432) Review S3A documentation to make sure it is consistent with the current codebase

2022-01-05 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-16432?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran updated HADOOP-16432: Parent: HADOOP-18067 (was: HADOOP-17566) > Review S3A documentation to make sure it is

[jira] [Updated] (HADOOP-15069) support git-secrets commit hook to keep AWS secrets out of git

2022-01-05 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-15069?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran updated HADOOP-15069: Parent: HADOOP-18067 (was: HADOOP-17566) > support git-secrets commit hook to keep AWS

[jira] [Updated] (HADOOP-15249) review S3A translateException translation matches IBM CORS spec

2022-01-05 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-15249?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran updated HADOOP-15249: Parent: HADOOP-18067 (was: HADOOP-17566) > review S3A translateException translation

[jira] [Updated] (HADOOP-16155) S3AInputStream read(bytes[]) to not retry on read failure: pass action up

2022-01-05 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-16155?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran updated HADOOP-16155: Parent: HADOOP-18067 (was: HADOOP-17566) > S3AInputStream read(bytes[]) to not retry on

[jira] [Updated] (HADOOP-16154) ITestS3A select tests fail if user kinited in

2022-01-05 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-16154?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran updated HADOOP-16154: Parent: HADOOP-18067 (was: HADOOP-17566) > ITestS3A select tests fail if user kinited

[jira] [Updated] (HADOOP-15224) builld up md5 checksum as blocks are built in S3ABlockOutputStream; validate upload

2022-01-05 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-15224?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran updated HADOOP-15224: Parent: HADOOP-18067 (was: HADOOP-17566) > builld up md5 checksum as blocks are built

[jira] [Updated] (HADOOP-13695) S3A to use a thread pool for async path operations

2022-01-05 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-13695?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran updated HADOOP-13695: Parent: HADOOP-18067 (was: HADOOP-17566) > S3A to use a thread pool for async path

[jira] [Updated] (HADOOP-14302) Test MR split optimisation with recursive listing

2022-01-05 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-14302?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran updated HADOOP-14302: Parent: HADOOP-18067 (was: HADOOP-17566) > Test MR split optimisation with recursive

[jira] [Updated] (HADOOP-16522) Encrypt S3A buffered data on disk

2022-01-05 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-16522?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran updated HADOOP-16522: Parent: HADOOP-18067 (was: HADOOP-17566) > Encrypt S3A buffered data on disk >

[jira] [Updated] (HADOOP-17242) S3A (async) ObjectListingIterator to block in hasNext() for results

2022-01-05 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-17242?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran updated HADOOP-17242: Parent: HADOOP-18067 (was: HADOOP-17566) > S3A (async) ObjectListingIterator to block

[jira] [Updated] (HADOOP-13611) FileSystem/s3a processDeleteOnExit to skip the exists() check

2022-01-05 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-13611?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran updated HADOOP-13611: Parent: HADOOP-18067 (was: HADOOP-17566) > FileSystem/s3a processDeleteOnExit to skip

[jira] [Updated] (HADOOP-15570) s3guard uploads command to list date and initiator of outstanding uploads

2022-01-05 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-15570?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran updated HADOOP-15570: Parent: HADOOP-18067 (was: HADOOP-17566) > s3guard uploads command to list date and

[jira] [Updated] (HADOOP-15782) Clarify committers.md around v2 failure handling

2022-01-05 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-15782?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran updated HADOOP-15782: Parent: HADOOP-18067 (was: HADOOP-17566) > Clarify committers.md around v2 failure

[jira] [Updated] (HADOOP-14661) S3A to support Requester Pays Buckets

2022-01-05 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-14661?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran updated HADOOP-14661: Parent: HADOOP-18067 (was: HADOOP-17566) > S3A to support Requester Pays Buckets >

[jira] [Updated] (HADOOP-15651) NPE in S3AInputStream.read() in ITestS3AInconsistency.testOpenFailOnRead

2022-01-05 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-15651?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran updated HADOOP-15651: Parent: HADOOP-18067 (was: HADOOP-17566) > NPE in S3AInputStream.read() in

[jira] [Updated] (HADOOP-15628) S3A Filesystem does not check return from AmazonS3Client deleteObjects

2022-01-05 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-15628?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran updated HADOOP-15628: Parent: HADOOP-18067 (was: HADOOP-17566) > S3A Filesystem does not check return from

[jira] [Updated] (HADOOP-16567) S3A Secret access to fall back to XML if credential provider raises IOE.

2022-01-05 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-16567?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran updated HADOOP-16567: Parent: HADOOP-18067 (was: HADOOP-17566) > S3A Secret access to fall back to XML if

[jira] [Updated] (HADOOP-16134) Add S3AWriteOpContext for write ops; pass in statistics and other settings

2022-01-05 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-16134?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran updated HADOOP-16134: Parent: HADOOP-18067 (was: HADOOP-17566) > Add S3AWriteOpContext for write ops; pass in

[jira] [Updated] (HADOOP-16571) s3a to improve diags on s3a bad request message

2022-01-05 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-16571?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran updated HADOOP-16571: Parent: HADOOP-18067 (was: HADOOP-17566) > s3a to improve diags on s3a bad request

[jira] [Updated] (HADOOP-16066) S3a DelegationToken bindings to to support a "correlation ID" for the UA header

2022-01-05 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-16066?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran updated HADOOP-16066: Parent: HADOOP-18067 (was: HADOOP-17566) > S3a DelegationToken bindings to to support a

[jira] [Updated] (HADOOP-15650) Add custom InstanceProfileCredentialsProvider with more resilience to throttling

2022-01-05 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-15650?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran updated HADOOP-15650: Parent: HADOOP-18067 (was: HADOOP-17566) > Add custom

[jira] [Updated] (HADOOP-17063) S3A deleteObjects hanging/retrying forever

2022-01-05 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-17063?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran updated HADOOP-17063: Parent: HADOOP-18067 (was: HADOOP-17566) > S3A deleteObjects hanging/retrying forever >

[jira] [Updated] (HADOOP-16164) S3aDelegationTokens to add accessor for tests to get at the token binding

2022-01-05 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-16164?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran updated HADOOP-16164: Parent: HADOOP-18067 (was: HADOOP-17566) > S3aDelegationTokens to add accessor for

[jira] [Updated] (HADOOP-13704) S3A getContentSummary() to move to listFiles(recursive) to count children; instrument use

2022-01-05 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-13704?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran updated HADOOP-13704: Parent: HADOOP-18067 (was: HADOOP-17566) > S3A getContentSummary() to move to

[jira] [Updated] (HADOOP-17746) compatibility table in directory_markers.md doesn't render right

2022-01-05 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-17746?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran updated HADOOP-17746: Parent: HADOOP-18067 (was: HADOOP-17566) > compatibility table in directory_markers.md

[jira] [Updated] (HADOOP-12878) Impersonate hosts in s3a for better data locality handling

2022-01-05 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-12878?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran updated HADOOP-12878: Parent: HADOOP-18067 (was: HADOOP-17566) > Impersonate hosts in s3a for better data

[jira] [Updated] (HADOOP-13276) S3a operations keep retrying if the password is wrong

2022-01-05 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-13276?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran updated HADOOP-13276: Parent: HADOOP-18067 (was: HADOOP-17566) > S3a operations keep retrying if the password

[jira] [Updated] (HADOOP-14334) S3 SSEC tests to downgrade when running against a mandatory encryption object store

2022-01-05 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-14334?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran updated HADOOP-14334: Parent: HADOOP-18067 (was: HADOOP-17566) > S3 SSEC tests to downgrade when running

[jira] [Updated] (HADOOP-16940) ITestCustomSigner uses absolute paths off the bucket root rather than fork-relative

2022-01-05 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-16940?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran updated HADOOP-16940: Parent: HADOOP-18067 (was: HADOOP-17566) > ITestCustomSigner uses absolute paths off

[jira] [Updated] (HADOOP-16480) S3 Select Exceptions are not being converted to IOEs

2022-01-05 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-16480?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran updated HADOOP-16480: Parent: HADOOP-18067 (was: HADOOP-17566) > S3 Select Exceptions are not being converted

[jira] [Updated] (HADOOP-13728) S3A can support short user-friendly aliases for configuration of credential providers.

2022-01-05 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-13728?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran updated HADOOP-13728: Parent: HADOOP-18067 (was: HADOOP-17566) > S3A can support short user-friendly aliases

[jira] [Updated] (HADOOP-13636) s3a rm on the CLI generates deprecation warning on io.bytes.per.checksum

2022-01-05 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-13636?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran updated HADOOP-13636: Parent: HADOOP-18067 (was: HADOOP-17566) > s3a rm on the CLI generates deprecation

  1   2   >